TITLE: How many systematic reviews include outcomes in their search strategy and acknowledge the limitation? A meta-epidemiological review
AUTHORS: Yasushi Tsujimoto, Yusuke Tsutsumi, Yuki Kataoka, Masahiro Banno, Toshi A Furukawa

[DESCRIPTION]
<div class = "text-blocks"><div class = "text-block">MAIN PROTOCOL</div></div>

[STEPS]
?. How many systematic reviews include outcomes in their search strategy and acknowledge the limitation? A meta-epidemiological review
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							Yasushi Tsujimoto, MD, MPH1,2,3,4, Yusuke Tsutsumi, MD, MPH, DrPH4,5,6, Yuki Kataoka, MD, MPH, DrPH4,5,7, Masahiro Banno, MD, PhD4,8,9, Toshi A Furukawa, MD, PhD1
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							1.Departments of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine / School of Public Health, Yoshida Konoe cho, Sakyo-ku, Kyoto, Japan.
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							2.Department of nephrology and dialysis, Kyoritsu Hospital, Chuo-cho 16-5, Kawanishi, Hyogo, Japan
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							3.Cochrane Japan, Akashi Cho 10-1, Chuo-ku, Tokyo, Japan
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							4.Systematic Review Peer Support Group, Koraibashi, Chuo-ku, Osaka, Japan
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							5.Department of Healthcare Epidemiology, School of Public Health in the Graduate School of Medicine, Kyoto University, Yoshida Konoe cho, Sakyo-ku, Kyoto, Japan.
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							6.Department of Emergency Medicine, National Hospital Organization Mito Medical Center, Ibaraki, Japan
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							7.Department of Respiratory Medicine, Hyogo Prefectural Amagasaki General Medical Center, Higashi Naniwa-cho 2-17-77, Amagasaki, Hyogo, Japan
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							8.Department of Psychiatry, Seichiryo Hospital, Tsurumai 4-16-27, Showa-ku, Nagoya, Aichi, Japan
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							9.Department of Psychiatry, Nagoya University Graduate School of Medicine, Tsurumai-cho 65, Showa-ku, Nagoya, Aichi, Japan
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							Background
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							Rigorous search strategy is essential to conducting systematic reviews. In general, the structure of search strategies in bibliographic databases, such as MEDLINE or EMBASE, will typically consist of three sets of terms: (i) the population; (ii) the interventions evaluated; and (iii) terms to search for the types of study design(1). It has been shown that the inclusion of the outcome terms leads to lower retrieval of relevant studies and thus it is recommended not to include outcomes in the search strategy, except in some special cases such as reviews of specific adverse effects(1, 2). Moreover, such systematic reviews may also assess other outcomes than those that they had used in the search strategy. Collecting articles that have the certain outcome in the title or abstract but then assessing outcomes that were not in the search strategy may lead to selective outcome reporting.
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							To the best of our knowledge, there has been no investigation of this topic in the literature. We will use the recent Cochrane reviews as examples and assess i) what proportion of them include the outcome in their search term, ii) how often the limitations of such practices were discussed/acknowledged, and iii) what proportion of such Cochrane review assessed outcomes other than those that they used in the search strategy.
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							Methods
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							Eligibility criteria
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							We will include all the Cochrane reviews of interventions published in 2020 which included specific outcomes in the search strategy for MEDLINE. We will exclude the reviews if the terms related to review outcomes are in their search strategy but are not combined with terms related to participants or interventions by “AND”. We will also exclude empty reviews, reviews without Summary of Findings (SoF) tables, or reviews without search strategy for MEDLINE in the appendices.
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							Search strategy and study selection
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							We will search the Cochrane Database of Systematic Reviews for the eligible reviews using the filter that specifies the time period as January 1, 2020 to December 31, 2020 and the types of reviews as intervention. To exclude reviews without SoF or MEDLINE search strategy in the appendices, we will scrape them from Cochrane library by Python using digital object identifier (DOI). For reviews that we could not scrape the SoF or MEDLINE search strategy, one researcher will visually check and confirm the eligibility. For the rest of reviews, two investigators will independently screen titles and search strategy in the appendices. Any discrepancy will be resolved by discussion and if this fails a third investigator will act as an arbiter.
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							Outcomes of the study
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							The main outcomes of interest are i) proportion of Cochrane reviews that included outcomes in their search terms as a necessary term, ii) proportion of Cochrane reviews in which the limitations of searching for outcomes or assessing outcomes that were not included in the search strategy were acknowledged, and iii) proportion of Cochrane reviews that searched for outcomes but assessed outcomes other than those in the MEDLINE search strategy among Cochrane reviews searched for outcomes. The proportion of Cochrane reviews that included outcomes in their search terms is calculated as the number of included reviews divided by the number of hits in the initial search. We defined the acknowledgement of the limitations above as any mention of searching for outcomes in the Discussion or downgrading of publication bias domain in GRADE assessment with an explanation of searching for outcomes.
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							Data extraction
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							Two investigators will extract the data independently and in duplicate. We will extract text on how the limitations of searching for outcomes or assessing outcomes that were not included in the search strategy were acknowledged in the Discussion or SoF table. We will extract the characteristics of reviews including the number of articles screened, the number of included studies, and the number of excluded studies with the reason of wrong outcome from the flow chart. Cochrane review group, the number of Cochrane reviews that the first author of the review published so far, involvement of trial search coordinator from Cochrane review group, types of study design to be included, whether the outcome in the title is related to adverse effect, and the number of outcomes in the "Types of Outcomes" section separately for those in the search strategy and those not, will be also sought. For the outcomes listed in the first SoF table in each Cochrane review, we will abstract whether they are in the search strategy as outcome terms, the number of studies reported them, statistical significance, direction of the effect (intervention favored or not), and the GRADE certainty of evidence.
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							Statistical analysis
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							We will narratively summarize how the limitations of searching for outcomes or assessing outcomes that were not included in the search strategy are acknowledged. We will compare the characteristics of reviews that evaluated outcomes other than the search strategy with those that did not. For the outcomes listed in the SoF tables, we will compare the characteristics of the outcomes in the search strategy and those not. All analyses will be performed using STATA 14.2 (StataCorp LP, Texas).
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							References:
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }
							1.Higgins JPT TJ, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.0: Cochrane; 2019 [updated July 2019. Available from:www.training.cochrane.org/handbook.2.Frandsen TF, Bruun Nielsen MF, Lindhardt CL, Eriksen MB. Using the full PICO model as a search tool for systematic reviews resulted in lower recall for some PICO elements. J Clin Epidemiol. 2020;127:69-75.
							  .justify:after {
							    content: "";
							    display:inline-block;
							    width: 100%;
							  }