Posted on July 18, 2017
Systematic reviews sit amongst the top of the evidence hierarchy. This is all well and good, provided they are conducted appropriately. To ensure a review can provide high quality, reliable evidence, they must be completed meticulously, following reporting guidelines such as the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and the Cochrane Handbook (1, 2) A key element of this is a robust search strategy.
Systematic reviews take existing literature and synthesise it, either qualitatively or quantitatively. In order to be ‘systematic’ and minimise selection bias, the search must be comprehensive, encompassing all of the relevant research. Bias will exist if the authors fail to include ALL of the pertinent research. “How can you be sure your conclusions are true if you have not explored all of the evidence?”
Of course, some articles may be excluded (for instance if they are methodologically flawed) once you come to screening the evidence, especially if this is a meta-analysis. However, the authors must ensure that initially all of the relevant evidence is captured by the search.
Performing an in-depth search is often overwhelming for authors setting out with a systematic review. It can seem confusing or difficult and is often substituted for a more manageable, less rigorous search. The following checklist should provide a clear framework for those wanting to ensure their search is truly systematic.
All illustrative example of database searching will be performed on Ovid Medline. Please note that this is a basic worked example to illustrate some of the key principles involved in searching. Using the search outlined below, you are likely to pick up key studies. However, to minimise your chances of missing relevant articles then do consultant a librarian who will be able to assist you with advanced search techniques. This is the best way to maximise sensitivity and reduce your risk of missing relevant articles.
Firstly, it is important that systematic review protocols (including search strategies) are prospectively registered to prevent deviation from the intended methods and minimise bias. A popular registration database is the International Prospective Register of Systematic Reviews (PROSPERO).
Also, it is a good idea to carry out a search for any similar systematic reviews. It may be pointless to repeat an existing systematic review. Although if an existing systematic review is outdated or features flawed methodology, then a new systematic review on that topic can be justified.
Before a search can be performed, it is crucial that the research question is explicitly defined. There are many ways to do this, but the most common method is to divide your question according to the Patients, Interventions, Comparisons and Outcomes (PICO) model.
Patients: Which patient population do you want to explore?
Adults? Elderly? Paediatrics? Males? Females? Certain ethnicities? Inpatients? Community patients? Certain co-morbidities?
Interventions: What intervention(s) do you want to explore?
A certain medication? A certain surgical procedure?
Comparisons: What are you going to compare the intervention(s) against? (Some reviews may not have a comparison so this section is optional).
Are you going to compare the intervention with no intervention? A control? Usual treatment? A different medication? A different surgical procedure?
Outcomes: What outcomes do you want to explore?
Mortality rate? Morbidity or the development of a certain disease? Change in baseline parameters such as blood pressure, weight or cholesterol?
Once a well-defined research question has been established, it is important to outline where you will search for the evidence. Systematic searches should aim to search as many different sources as possible. This can be broken down into the following:
Online databases* (this will be the major area for medical literature searches)
Grey literature* (this refers to unpublished material/published in a non-commercial form)
Ongoing trials (clinicaltrials.gov)
Types of articles
Randomized controlled trials (RCTs)
Case series studies
How far back do you wish to explore?
Perhaps the intervention you want to investigate was only developed in the last 10 years
Other languages too? Ideally you should be willing to translate these in order to fully understand their findings
*The following databases are useful for clinical systematic reviews:
PsychINFO – key database for mental health literature
MEDLINE – large medical database
EMBASE – large medical database
SCOPUS – includes many scientific disciplines
Cochrane Library – high-quality evidence
Web of Science – includes many scientific disciplines
CINAHL – includes biomedicine, healthcare, nursing and allied health articles
When searching online databases, the terms and their synonyms for each of the components of the PICO model must be written out, including abbreviations. It is also important to use alternate spellings and word endings. This can be done using a number of strategies within the database:
Firstly, truncation involves putting an asterisk (*) in a word with a variable ending e.g. toxic* will search for toxic, toxicity and toxicology.
Secondly, using a question mark (?) in place of a single letter for words that have alternate spellings (such as American vs British English) will search for both spellings, e.g. p?ediatric will search for both paediatric and pediatric. The question mark replaces a single character or no character, allowing you to pick up alternate spellings. It should be noted that these terms can vary between databases, and so should always be checked prior to conducting a search.
You should also search for hyphenated terms with and without the hyphen as different authors may title their work differently.
Let’s imagine we want to conduct a systematic review to assess the effect of primary percutaneous coronary intervention (PCI) versus streptokinase on mortality in myocardial infarction (MI) patients. (Clicking on the screenshots below will enlarge them).
Patients: myocardial infarction, MI, myocardial necrosis, heart attack
As you can see, each line represents a synonymous term that is searched for, with the corresponding number of articles found in the results column. These terms can be combined using ‘OR’ to ensure the database captures all of the articles relating to these terms.
For the final screenshot, all of the groups of terms are combined with ‘AND’ to ensure the database only displays results that are related to the combination of terms (i.e. about PCI and streptokinase in MI patients). In this example, it yielded a total of 254 results.
Medical Subject Headings (or MeSH terms) are terms predefined by the database using human indexers in concordance with thorough protocols. MeSH terms encompass ‘Headings’, ‘Subheadings’, ‘Supplementary Concept Records’ and ‘Publication Characteristics’. Further definitions of these categories and relevant examples can be found on the National Library of Medicine’s website (https://www.nlm.nih.gov/bsd/disted/meshtutorial/introduction/04.html). The same website also provides detailed descriptions of how the MeSH system is structured and how this can be searched to further refine or expand your literature search. Using Medline as an example, you must select ‘Map term to subject heading’ to search for MeSH terms.’
If your search yields no results then it is important to re-evaluate your PICO model and broaden the search terms. Conversely, if the results are very expansive this will take considerable effort to screen the relevant information. It may be worth narrowing the search terms in order to achieve a more focused search.
Once the formal search has been completed, you must now remove the duplicates and screen the titles/abstracts of the remaining results. This is a large task in itself, and information on how this can be done is beyond the scope of this blog.
Once competed, a further exploration for relevant studies can begin. There are a number of strategies that should be carried out at this stage.
At this point it is crucial that you finalise your list of included studies and document your search well so that you can refer to this in the future and during the rest of the review process.
Congratulations! If you have followed the various steps in this article, you are well on your way to completing a comprehensive systematic review! If you need further help, consult your librarian.