Skip to Main Content

Homepage: How to Conduct a Review

"The society that separates its scholars from its warriors will have its thinking done by cowards and its fighting done by fools." -- Thucydides

How to Conduct a Review

  • Develop Research Question
    • Goal:
      • Answer Specific Question
      • Assess Intervention
      • Synthesize existing evidence
    • Choose Framework
      • PICO- Population, Intervention, Comparison, Outcome. Best for quantitative research. Looking for Effectiveness of ____
      • PEO- Population, Exposure, Outcome. Best for qualitative research. Etiology, risk or benefit.
      • SPIDER- Sample, Phenomenon, Design, Evaluation, Research Type. Best for research on samples instead of populations. how prevalent something is.
      • SPICE- Setting, Perspective, Intervention, Comparison, Evaluation. Best for qualitative research evaluating the outcome of a service. 
      • ECLIPSE- Expectation, Client group, Location, Impact, Professionals, Service. Best for qualitative research evaluating the outcome of a service or policy.
      • PIRD- Population, Index Test, Reference Test, Diagnosis of interest. Looking for diagnostic test accuracy.
      • PTM- Population, Type of measurement instrument, Measurement properties. Psychometric
  • Complete initial review of literature to determine need for review.
    • Check if this has already been done.
    • if there are enough primary studies available. If not, you may need to switch to Scoping Review.
    • identify the scope of the studies that match your question topics. Who is the key author, Studies and Search terms?
  • Determine eligibility criteria (Be wary of bias.):
    • Participants
    • Study design
    • Intervention type
    • Outcomes
    • Publication type
    • Date range
    • Languages. 
    • Geographic Location
    • Peer Review
    • Setting
  • Build a team. 
    • Consider interdisciplinary support.
      • Ex: Librarian to develop search string and select optimal databases or a Statistician for expert review of data.
    • Define roles and expectations from the beginning to as to head off issues as the project develops.
    • Size of team:
      • odd number of reviewers (tie-breaker)
      • 3-12 member team dependent on the scope of the systematic review
    • Test Screener or Pilot screener- Chooses a random sample of selected citations for inclusion to ensure consistency across the responses. Creates less errors during the screening process as screeners are applying inclusion and exclusion criteria properly and results in fewer conflicts due to misunderstandings.
  • Choose citation manager and determine if you will use systematic review software such as Covidence.
    • Citation managers: Endnote (BAMC), or free options such as Zotero
  • Standards and Guidelines
  • Continue to define criteria.
  • Develop search string and databases to be used.
  • Choose assessment tools.
  • Determine method of extracting data
  • PRISMA-P
  • Make Protocol public in a website or registry.
    • Protocol should include: rationale, hypothesis and methods you plan to use.
      • Where can I publish my protocol?
        • PROSPERO
        • Open Science Framework
        • Campbell Systematic Reviews
        • Cochrane Database of Systematic Reviews
        • Systematic Reviews (BMC)
        • Joanna Briggs Institute (JBI)
      • Include at least:
        • Team Members
        • Rational and Objectives
        • Eligibility Criteria
        • Databases and key terms- full search string if possible
        • Process and tools for screening, assessment of quality and data extraction
        • Data items to be extracted.
        • Data Synthesis methods
  • Define Main Concepts - (identify facets)
  • Identify applicable databases and choose at least three.
  • work your terms into facets and identify controlled vocabulary such as MeSH terms
    • Remember to use synonyms for natural language terms
    • Terms with similar meanings
    • different spellings (US vs UK or hyphenated terms)
    • Acronyms
    • inconsistent terms
    • Broad and specific terms as applicable
    • Be aware of outdated terms
    • proprietary terms vs generic terms
  • choose field tags
  • Combine terms then concepts into facets and link with Boolean Operators (AND, OR, NOT) remember to use nesting. 
    • "AND" Narrows the search.
    • "OR" Widens the search.
    • "NOT" Excludes terms (used very rarely, must be careful to not exclude relevant information)
  • Search string needs to be translated into search subsequent database using their formatting standards. 
  • Hand Searching- used to supplement a Traditional search. Can be completed by reviewing the references in the traditional search for articles to include in the review. A researcher can also browse important journals relevant to the field of study to ensure that no articles were missed. Clinical trial registries, white papers, pharmaceutical reports, corporate reports, conference papers, theses, dissertations and Guidelines. 
  • Grey Literature: While many Systematic Review Criteria exclude Grey Literature or non-Peer Reviewed resources. The object is to synthesize all available evidence. It should be reviewed if your research topic can benefit. Searching Clinical Trials, Theses and Dissertations can be especially useful. *Grey Literature Databases are generally less robust and search strategies will likely need to be distilled down.
  • SAVE. SAVE. SAVE. Make a file on your computer with your search string, limiters, date search run, exported files, any notes, and if you used deduplication functions and what functions.
  • Document the search using PRISMA-S or Cochrane Handbook Section 4.5
  • Validate and peer review the search.

  • Contact a Librarian to help you with this process! No need to struggle on your own. 

  • Export search results and Upload into citation manager *double check what file format your citation manager required before you begin*
    • Export from PubMed to EndNote
      • Start at your Search Results to be exported
      • click "send to" under the search bar.
      • Choose "citation Manager"
      • Select "all results from selection drop down.
      • Click Create file.
      • Save to your computer.
      • In EndNote click on Files
      • Click "Import"
      • Select "Import Option- PubMed (NLM)"
      • Choose the file saved earlier.
  • De-duplicate
    • be wary of false positives and negative when using function in database or citation manager.
  • Move articles into systematic review software (if using) for screening.
  • Remember PRISMA requires you to report the number of duplicates removed.
  • Determine if you will use a Systematic review tool to organize articles and streamline the review process.
    • Popular Tools
      • Covidence
      • Rayyan
      • Excel
      • Colandr
      • SRDR+ (AHRQ
      • Systematic Review Toolbox
  • Read title and abstract for every article and vote for inclusion or exclusion based on criteria.
    • No: Does not meet criteria- Do not include
    • Maybe: not enough information to decide yet- Move to full text screening.
    • Yes: appears to meet criteria- Move to full text screening.
  • Resolve any disputes using predetermined method.
  • Obtain full text for remaining articles.
  • Read remaining full text.
    • Include: Does meet Criteria include in Systematic Review
    • Exclude: Does not meet Criteria Do Not Include.
  • Choose and Use assessment tools to grade each article and check for Bias.
  • Strength of Evidence
  • Create a summary of the quality of the literature in your review.
  • Be on the lookout for bias! Use a tool such as Cochrane Risk of Bias tool or ROBIS
  • Critically appraise studies included. 
    • Relevance
    • Reliability
    • Validity
    • Applicability
  • Choose information to be collected from each study.
  • Choose Extraction Method
  • Create Data Extraction Table- Test if necessary.
  • Extract Data
  • Review data for errors-
    • Should be done by 2 or more people. -As per Cochrane Handbook (Chap 5)
    • Can be done by hand or using a computer program dependent of level of difficulty.
  • Review the standards you are using (ex: PRISMA)
    • Cochrane Handbook- Chap 15
    • JBI Manual for Evidence Synthesis- Chap 12.3
    • PRISMA 2020
  • Gather Completed Data tables and PRISMA chart.
  • Complete PRISMA Flow Diagram (if using)
  • Write introduction to topic and study, then Methods, then Results (include PRISMA diagram) and then Discussion.
  • Now Write your Abstract and Conclusion
  • Include entire search strategy in appendix 
  • Cite the studies included in your systematic review and those used in the paper but not included.
  • Time to publish. Identify journals you are interested in publishing and review their publication criteria.

Cornell University Library. (n.d.). Libguides: A guide to evidence synthesis. A Guide to Evidence Synthesis - LibGuides at Cornell University. https://guides.library.cornell.edu/evidence-synthesis/translate

Eden, J. (2011). Finding what works in Health Care Standards for Systematic Reviews. National Academies Press.

University of North Carolina at Chapel Hill. (n.d.). Libguides: Systematic reviews: Home. Home - Systematic Reviews - LibGuides at University of North Carolina at Chapel Hill. Retrieved April 5, 2023, from https://guides.lib.unc.edu/systematic-reviews/overview

University of York Centre for Reviews and Dissemination Guidance

google-site-verification=Fypw_UyiYoHXN-EGgpIee-9aIghb8Wt4Up5y7wwsQ8k