How to Do Your First Ophthalmology Systematic Review: Tips for Foundation Doctors and Students

Zaid Farhan

Foundation Year 2 Doctor, Milton Keynes University Hospital NHS Foundation Trust, Milton Keynes, UK.

Introduction

Systematic reviews are often placed at the top of the evidence pyramid, and unlike primary research you do not need patients, a lab or ethics approval, but you do need rigorous methods (1). As a student or foundation doctor, you are usually invited to “help with” someone else’s review. You screen abstracts, with no view of the whole process or of how the review was designed. That leaves you unable to lead a question of your own, or to spot where reviews go wrong, which they often do. In one evaluation of 327 systematic reviews of interventions for retina and vitreous conditions, only 40.1% were judged reliable, and the commonest failing was no evidence of a comprehensive search (2).

Across the Cochrane Eyes and Vision database of 4,786 eligible reviews, the reliability of intervention reviews ranged from 66% (corneal disease) to 28% (refractive error) (3). Output has also outpaced the wider literature: systematic review publications rose 2,728% between 1991 and 2014, against 153% for all PubMed items, with many reviews redundant (4). Your first review must therefore be original in question and rigorous in method. This article is a companion to JFO’s audit guide (5). My own systematic review of public versus private ophthalmology provision and outcomes is in the final stages of publication with the British Journal of Ophthalmology. Over eight tips, I cover the process and end with a checklist (Box 1) and a realistic timeline (Table 1).

Tip #1 – Choose a Question You Can Actually Finish

Start with a clinical or service problem you have seen, then shape it with PICO: Population, Intervention, Comparator, Outcome (6,7). The Cochrane Handbook suggests framing the objective as “To assess the effects of (intervention) for (health problem) in (population)” (6).

Feasible beats ambitious. For a small team, the search should return hundreds to low thousands of records, not tens of thousands you will never screen.

Illustrative example only (not a registered review; a real team would first check for existing reviews on this comparison):

  • Population: adults with proliferative diabetic retinopathy
  • Intervention: intravitreal anti-VEGF injection
  • Comparator: panretinal photocoagulation
  • Outcomes: best-corrected visual acuity at 12 months (primary); need for vitrectomy; adverse events (secondary)
  • Study design: randomised controlled trials
  • Objective: “To assess the effects of intravitreal anti-VEGF injection compared with panretinal photocoagulation on best-corrected visual acuity at 12 months in adults with proliferative diabetic retinopathy.”

If a first search returns 40,000 records, shrink the question. Replace “retinal disease” with one named condition, narrow the population or comparator, or limit the study design.

Not every review is an intervention review. Prevalence, diagnostic accuracy, service-provision and adverse-event reviews adapt the framework (for example, exposure in place of intervention, or adding study design), and this is where many ophthalmology, AI-imaging and health-services reviews sit.

Before you commit, check that the review does not already exist. Search the Cochrane Eyes and Vision database of systematic reviews (https://eyes.cochrane.org/resources/cev-database-systematic-reviews-eyes-and-vision) (3), PROSPERO (Tip 2) and the published literature. If a recent, reliable review answers your question, change the angle (a new outcome, population or set of newer studies) or pick another question. Redundant reviews are a documented problem (4).

Tip #2 – Write and Register a Protocol Early

A protocol fixes your question, eligibility criteria, search plan, outcomes, risk of bias tools and analysis plan before you see any results. It stops you changing the goalposts to suit what you find. Write it with PRISMA-P, a 17-item checklist (8).

Register the protocol on PROSPERO, a free registry created by the UK Centre for Reviews and Dissemination to reduce duplication and reporting bias (9,10). Register as early as possible, ideally before you begin screening against your eligibility criteria. PROSPERO will not accept a review that has progressed beyond completion of data extraction, and scoping reviews and literature reviews are not eligible (10). Allow time for administrative checking; registration is not instant (10). Keep the registration number and quote it in your manuscript. If you change the protocol later, record the amendment and the reason.

Use the protocol to settle your analysis plan too. Decide which comparisons, subgroup analyses and sensitivity analyses you are prespecifying, and label anything added later as exploratory.

Tip #3 – Build and Validate a Multi-Database Search

The search is where many reviews fail: no evidence of a comprehensive search was the commonest reason retina and vitreous reviews were judged unreliable (2). Missed studies mean biased conclusions.

Involve a health-sciences librarian early. Most NHS trusts and universities have one, so ask your trust library or university library service.

Search at least Embase, MEDLINE, Web of Science Core Collection and Google Scholar. In one exploratory study, that combination gave 98.3% overall recall and 100% recall in 72% of reviews (11). Google Scholar results are hard to reproduce, so record the exact query and how many results you screened. Check which databases your NHS OpenAthens account or university gives you. For intervention reviews, add the Cochrane Central Register of Controlled Trials (CENTRAL) and trial registries such as ClinicalTrials.gov and WHO ICTRP, which are standard sources in Cochrane methods (1). Search grey literature and conference abstracts where relevant; ARVO, ESCRS and EURETINA meetings are examples.

Build the strategy from concept blocks: condition, intervention and study design. Combine controlled vocabulary (MeSH in MEDLINE, Emtree in Embase) with free-text synonyms, because ophthalmic terminology varies (“age-related macular degeneration”, “AMD”, “neovascular AMD”, “wet AMD”). Ask a colleague or your librarian to peer review the strategy with PRESS, which covers six elements: translation of the question; Boolean and proximity operators; subject headings; text words; spelling, syntax and line numbers; and limits and filters (12).

Test the strategy against five to ten papers you already know are eligible. If it misses any, fix it. Record the database, platform, search date, exact strategy and limits, and report them using PRISMA-S (13). You will need the search date for the flow diagram and any later update.

Tip #4 – Screen in Duplicate, Extract with a Piloted Form, Follow PRISMA

Deduplicate your records, then upload them to screening software. Rayyan is a free web and mobile app for semi-automated screening (14). Covidence is subscription software, so check whether your trust or university has access (15). AI-assisted screening prioritisation in Rayyan or Covidence can speed things up but should not replace a second human reviewer.

Screen titles and abstracts, then full texts, with two reviewers working independently and a third, typically senior, person to settle disagreements. In one methodological review, single screening missed a median of 5% of eligible studies (range 0 to 58%): 3% for experienced reviewers and 13% for less experienced ones (16). If you are the less experienced screener, that is your argument for double screening, which remains the conventional and safer default.

Calibrate first: Have every screener independently screen the same first 50 to 100 records, discuss the disagreements and tighten the eligibility wording. Log each full-text exclusion against a fixed list of reasons.

Build your extraction form before you extract. Pilot it on two or three included studies, refine it, then extract in duplicate or have a second reviewer check your work. For ophthalmology, include the unit of analysis (patient or eye; one or both eyes), follow-up time, visual acuity scale (logMAR, ETDRS letters or Snellen), imaging device or modality, and injection number or regimen.

The two-eyes pitfall deserves its own field. If a study analyses both eyes of the same patient as though they were independent, the eyes are correlated and the precision is overstated. Record how each study handled this, because it affects both risk of bias and any pooling.

Report with PRISMA 2020: a 27-item checklist, an abstract checklist and a flow diagram (17). Keep a running tally of records at every stage from day one, so the flow diagram takes an afternoon rather than a week.

Tip #5 – Match the Risk of Bias Tool to the Study Design, Then Grade the Evidence

Choose the risk of bias tool by the designs you will include. Use RoB 2 for randomised trials (18) and ROBINS-I for non-randomised studies of interventions, including cohort studies comparing treatments (19). The Newcastle-Ottawa Scale is widely used for observational cohort and case-control studies of exposures, although it is a quality scale rather than a domain-based risk of bias tool (20). Use QUADAS-2 for diagnostic accuracy studies, such as imaging or AI-diagnostic reviews (21). Fix the tool in your protocol, check that it fits the designs you will actually find, and have two reviewers assess each study independently.

Rate the certainty of evidence for your main outcomes with GRADE (22). A summary-of-findings table is the usual way to show it.

Meta-analysis is a decision, not a default. Pool only when studies are similar enough in population, intervention, comparator and outcome measurement. Otherwise a structured narrative synthesis serves readers better than a pooled number that misleads. Cochrane’s chapter on analysing data sets out the choices (23). If you do pool, prespecify the model in your protocol (random-effects is usual when studies differ clinically) rather than choosing it after seeing the results, report I² (24), and explore heterogeneity only through subgroup or sensitivity analyses planned in advance (23).

Think about your outcomes before promising a pooled result. Visual acuity may be reported as logMAR, ETDRS letters or Snellen. ETDRS letters and Snellen should be converted to a common scale, usually logMAR, before pooling as a mean difference; then ask whether the result means anything clinically. A review of ocular adverse events after a drug may read better as a narrative or as proportions.

Before submission, appraise your own review against AMSTAR 2 (25).

Tip #6 – Agree Roles and Authorship at the Start, and Be Realistic About Time

Assign roles in writing: content lead, methodologist or statistician, librarian, two screeners plus an arbiter, extraction lead, writer and guarantor. Reviews are team work; registered reviews had a mean of five authors (26). Two independent reviewers are the norm, so you need at least one co-reviewer.

Agree the authorship order at protocol stage, in writing, and revisit it if roles change. Use the four ICMJE criteria: a substantial contribution to conception or design, or to acquisition, analysis or interpretation; drafting or critically revising the work; final approval; and agreement to be accountable (27). Anyone who does not meet all four, such as someone who only screened records, should be acknowledged rather than listed as an author (27). The CRediT taxonomy describes each person’s contribution transparently (28). The January 2026 ICMJE update added guidance on AI, so declare any AI use as your target journal requires (27).

Now the time. Registered reviews took a mean of 67.3 weeks, just over 15 months, to complete and publish (26). Do not promise a supervisor a review in an eight-week student project or a four-month rotation. Table 1 shows how the phases might fit around rotations. Protocol writing and search building can be done in short bursts during a busy rotation, while screening and extraction can be batched into whatever time you have.

Tip #7 – Understand What a Review Is Worth for Ophthalmology ST1

The ST1 portfolio guide lists the Publications domain (maximum 6 points for the domain) as “peer-reviewed original research publications or high-quality systematic review articles” (29). As summarised there, first or joint first authorship earns 3 points, other publications up to 4th author earn 1 point, and 5th author or lower earns none. You need the first page of the published or “in press” paper, and the same work cannot be claimed in more than one domain (29). Authorship position matters, which is why Tip 6 says to settle the order early. A review in which you are fifth author scores nothing in that domain, while a first-author review scores 3 points but takes longer to lead.

Scoring changes between years, check the official applicant guide and criteria when they are published. Do not run a review only for points: a low-quality review is poor value, and the journal should be peer reviewed and have a Clarivate Journal Impact Factor (30).

Tip #8 – Use These Resources

  • Cochrane Handbook (1)
  • Reporting: PRISMA 2020 (17), PRISMA-P (8), PRISMA-S (13)
  • Registration and existing reviews: PROSPERO (10); Cochrane Eyes and Vision database (3)
  • Search and screening: PRESS (12); Rayyan (14); Covidence (15)
  • Appraisal: RoB 2 (18), ROBINS-I (19), QUADAS-2 (21), GRADE (22), AMSTAR 2 (25)
  • Authorship: ICMJE (27); CRediT (28)
  • Your trust or university library, for librarian support and database access

Keep a methods log from day one. Note every decision, search date, screening milestone and protocol amendment. It will save you hours when you write the manuscript.

Final Thoughts

A systematic review is a project management exercise as much as a literature exercise. What protects it is transparent methods, a check for existing reviews before you start, a librarian, dual screening and authorship agreed in writing. Write the protocol before you run your final search. Use Box 1 as your checklist and Table 1 to set realistic expectations with your supervisor. Aim for a small, well-specified review rather than a sprawling one. With a narrow question, clear methods and a committed team, a rigorous review is achievable alongside foundation training.

Box 1. First systematic review checklist

A. Question and feasibility

☐ Question written in PICO (or PECO/PICOS) form; objective sentence drafted

☐ Cochrane Eyes and Vision database, PROSPERO and literature checked for existing reviews

☐ Scope feasible for team size and time

☐ Peer-reviewed target journal chosen

☐ Review type chosen (not scoping or narrative if you want PROSPERO)

B. Protocol and registration

☐ Protocol written using PRISMA-P

☐ Eligibility, outcomes, risk of bias tool and synthesis plan fixed

☐ Registered on PROSPERO, ideally before screening begins

☐ Registration number recorded; amendments logged with reasons

C. Search

☐ Librarian involved

☐ Embase, MEDLINE, Web of Science, Google Scholar (plus CENTRAL and registries for interventions)

☐ Strategy tested against known eligible papers and peer reviewed (PRESS)

☐ Full strategies and search dates saved (PRISMA-S)

☐ Searches rerun if more than 6–12 months old at submission

D. Screening and extraction

☐ Records deduplicated; Rayyan or Covidence set up

☐ Calibration round done; two independent screeners plus arbiter

☐ Full-text exclusion reasons recorded

☐ Extraction form piloted; second reviewer checks extraction

☐ Handling of two eyes per patient recorded for each study

E. Appraisal and synthesis

☐ Risk of bias tool matched to design, done in duplicate

☐ Decision on meta-analysis justified; model prespecified; I² reported if pooled

☐ GRADE certainty assessed for main outcomes

F. Team and writing

☐ Roles and authorship order agreed in writing (ICMJE four criteria)

☐ CRediT statement drafted

☐ PRISMA 2020 checklist and flow diagram completed

☐ Own review checked against AMSTAR 2

☐ AI use declared as the target journal requires

Table 1. Suggested 15-month timeline for a first systematic review alongside foundation training

PhaseMonthsMain tasksApprox. h/wkRotation-friendly tip
1. Scope and team1Refine PICO; check for existing reviews; recruit co-reviewers and librarian; agree roles and authorship2–3Start before a lighter rotation; book an hour with the librarian early
2. Protocol and registration2–3Write protocol (PRISMA-P); pilot eligibility criteria; register on PROSPERO3–4Write in short weekly blocks in a shared document
3. Search3–4Build, peer review (PRESS) and run searches; export and deduplicate2–3Ask the librarian to run the final searches; save dates and strategies
4. Screening4–7Calibration; title and abstract; full text; resolve conflicts3–5Screen in 20 to 30 minute batches on Rayyan mobile; agree weekly targets with your co-screener
5. Extraction and appraisal7–10Pilot form; extract; risk of bias in duplicate4–6Book joint sessions with your co-reviewer; use study leave (if approved) for heavy days
6. Synthesis10–12Narrative or meta-analysis; GRADE; summary-of-findings table3–5Book statistician support early, not in the final month
7. Writing and submission12–14Draft; PRISMA 2020 checklist and flow diagram; AMSTAR 2 self-check; confirm authorship; choose journal4–6Set internal deadlines with a named owner per section
8. Peer review and revision14 onwardsRespond to reviewers; update search if needed2–4Expect delays, as journal turnaround alone often takes months; keep search dates and logs ready for updates

References

  1. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions. Version 6.5. Cochrane; 2024. Available from: https://www.cochrane.org/handbook
  2. Le JT, Qureshi R, Twose C, Rosman L, Han G, Fapohunda K, et al. Evaluation of systematic reviews of interventions for retina and vitreous conditions. JAMA Ophthalmol. 2019;137(12):1399-405.
  3. Lê JT, Qureshi R, Rouse B, Twose C, Rosman L, Lindsley K, et al. Development and content of a database of systematic reviews for eyes and vision. Eye (Lond). 2022;36(4):883-5.
  4. Ioannidis JPA. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q. 2016;94(3):485-514.
  5. Shahnazari S. How to publish your ophthalmology audit: tips using modern quality improvement tools. J Found Ophthalmol. 2022 Dec 23. doi:10.48089/jfo7688099. Available from: https://www.jfophth.com/how-to-publish-your-ophthalmology-audit-tips-using-modern-quality-improvement-tools/
  6. Thomas J, Kneale D, McKenzie JE, Brennan SE, Bhaumik S. Chapter 2: Determining the scope of the review and the questions it will address. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions. Version 6.5. Cochrane; 2024.
  7. Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: a key to evidence-based decisions. ACP J Club. 1995;123(3):A12-3.
  8. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1.
  9. Booth A, Clarke M, Dooley G, Ghersi D, Moher D, Petticrew M, et al. The nuts and bolts of PROSPERO: an international prospective register of systematic reviews. Syst Rev. 2012;1:2.
  10. Centre for Reviews and Dissemination, University of York. Registering a review on PROSPERO. York: CRD; (cited 2026 Sep 30). Available from: https://www.crd.york.ac.uk/prospero/
  11. Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev. 2017;6:245.
  12. McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40-6.
  13. Rethlefsen ML, Kirtley S, Waffenschmidt S, Ayala AP, Moher D, Page MJ, et al. PRISMA-S: an extension to the PRISMA Statement for reporting literature searches in systematic reviews. Syst Rev. 2021;10:39.
  14. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016;5:210.
  15. Covidence systematic review software. Melbourne: Veritas Health Innovation; (cited 2026 Sep 30). Available from: https://www.covidence.org
  16. Waffenschmidt S, Knelangen M, Sieben W, Bühn S, Pieper D. Single screening versus conventional double screening for study selection in systematic reviews: a methodological systematic review. BMC Med Res Methodol. 2019;19:132.
  17. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.
  18. Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898.
  19. Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919.
  20. Wells GA, Shea B, O’Connell D, Peterson J, Welch V, Losos M, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Ottawa: Ottawa Hospital Research Institute; (cited 2026 Sep 30). Available from: https://www.ohri.ca/programs/clinical_epidemiology/oxford.asp
  21. Whiting PF, Rutjes AWS, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529-36.
  22. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924-6.
  23. Deeks JJ, Higgins JPT, Altman DG, McKenzie JE, Veroniki AA, editors. Chapter 10: Analysing data and undertaking meta-analyses. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions. Version 6.5. Cochrane; 2024.
  24. Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557-60.
  25. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008.
  26. Borah R, Brown AW, Capers PL, Kaiser KA. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open. 2017;7(2):e012545.
  27. International Committee of Medical Journal Editors. Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals. Updated January 2026 (cited 2026 Sep 30). Available from: https://www.icmje.org/recommendations/
  28. Brand A, Allen L, Altman M, Hlava M, Scott J. Beyond authorship: attribution, contribution, collaboration, and credit. Learn Publ. 2015;28(2):151-5.
  29. Jafari M. Ophthalmology ST1 2026 portfolio: evidence folder updates and practical guidance. J Found Ophthalmol. 2025 Oct 11. Available from: https://www.jfophth.com/ophthalmology-st1-2026-portfolio-evidence-folder-updates-and-practical-guidance/
  30. NHS England. Ophthalmology ST1: self-assessment and supporting evidence. Medical Hub; 2026. Available from: https://medical.hee.nhs.uk/medical-training-recruitment/medical-specialty-training/ophthalmology/ophthalmology-st1/overview-of-ophthalmology-st1/self-assessment-and-supporting-evidencehttps://medical.hee.nhs.uk/medical-training-recruitment/medical-specialty-training/ophthalmology/ophthalmology-st1/overview-of-ophthalmology-st1/self-assessment-and-supporting-evidence

Leave a Reply