TITLE: Generic Protocol for Environmental Health Systematic Reviews Based on COSTER Recommendations
AUTHORS: Paul Whaley

[DESCRIPTION]
<div class = "text-blocks"><div class = "text-block"><span>A protocol template to help researchers follow the </span><a href="https://www.sciencedirect.com/science/article/pii/S016041202031881X" style = "text-decoration:underline;color:blue;cursor:pointer;"><span style = ":;">COSTER recommendations</span></a><span> for conduct of systematic reviews. This instance covers the planning steps of a systematic review and will help with writing up the systematic review protocol.</span></div><div class = "text-block">The intent is to convert COSTER from a checklist of things which need to be done into a sequence of actions which can be followed by a research team.</div><div class = "text-block"><span>When completing the protocol and either registering it or submitting it to a journal, please cite this instance of the protocol template and the parent manuscript, DOI </span><a href="https://doi.org/10.1016/j.envint.2020.105926" style = "text-decoration:underline;color:blue;cursor:pointer;"><span style = ":;">10.1016/j.envint.2020.105926</span></a><span style = ":;">.</span></div></div>

[STEPS]
?. [Securing capacity, competencies, and tools]
Assess the team's combined competence in conduct of a systematic review. Recommendation 1.1.1.   AB1CompetencyTeam member(s) (initials)2Information science (for e.g. search strategies)3Evidence appraisal methods (i.e. risk of bias assessment)4Statistical methods5Domain or subject expertise6Systematic review methods
AB1CompetencyTeam member(s) (initials)2Information science (for e.g. search strategies)3Evidence appraisal methods (i.e. risk of bias assessment)4Statistical methods5Domain or subject expertise6Systematic review methods
?. [Securing capacity, competencies, and tools]
Identify information management practices and tools for each stage of the review. Recommendation 1.1.2.   AB1Information management componentTools or packages2Reference manager3Knowledge management tool4Systematic review software5Statistics software and packages6Aritifical Intelligence support tools (e.g. for screening)
AB1Information management componentTools or packages2Reference manager3Knowledge management tool4Systematic review software5Statistics software and packages6Aritifical Intelligence support tools (e.g. for screening)
?. [Securing capacity, competencies, and tools]
List the potential conflicts of interest of the authors. Recommendations 1.1.3 and 1.1.4. This should include both financial and non-financial interests which readers should be aware of in order to understand the motivations of the authors of the review.By listing the interests as potential, you are confirming that they are not apparent conflicts of interest, i.e. they cannot reasonably be expected to compromise the integrity of the systematic review. People with apparent conflicts of interest should be excluded from decision-making roles in the review. Interests should be declared using the ICMJE Conflict of Interest Disclosure Forms, as attached. The summary statements generated by the forms for each author can be copy-pasted into the table.  AB1AuthorICMJE COI Summary2345
AB1AuthorICMJE COI Summary2345
?. [Setting the research question ("problem formulation")]
Demonstrate the need for a new review. Recommendation 1.2.1
?. [Setting the research question ("problem formulation")]
Describe the scientific value of the question(s), i.e. why it is important that it be investigated.
?. [Setting the research question ("problem formulation")]
Describe the importance to stakeholders of the question(s) being asked.
?. [Setting the research question ("problem formulation")]
Summarise relevant existing primary research and evidence syntheses to justify conducting a new systematic review.
?. [Setting the research question ("problem formulation")]
Articulate the scientific rationale for each question via development of a theoretical framework. Recommendation 1.2.2. For example, this would describe how the exposure is related to the outcomes of interest if the systematic review is an investigation of an exposure-outcome relationship. The theoretical framework should include discussion of the biological plausibility of the relationship being investigated.
?. [Setting the research question ("problem formulation")]
For each research question to be answered by the review, prospectively define a statement of the research objective in terms of Population, Exposure or Intervention, Comparator, Outcome, Study Design, and Target Condition, selected as appropriate. Recommendation 1.2.3.
?. [Setting the research question ("problem formulation")]
Define the target Population of interest. These are the objects of investigation, i.e. the entities to which exposures or interventions happen.  AB1Species2Sex3Age4Health status5Additional characteristics
AB1Species2Sex3Age4Health status5Additional characteristics
?. [Setting the research question ("problem formulation")]
Define the target Exposure or Intervention of interest. This concerns the administered or observed change in conditions of the objects of investigation. It should include timing, duration and dose.  AB1Exposure or interventionWhat is the exposure or intervention?2TimingWhen does the exposure or intervention happen?3DurationFor how long does the exposure or intervention last?4DoseWhat is the dose regimen (amount, frequency)?
AB1Exposure or interventionWhat is the exposure or intervention?2TimingWhen does the exposure or intervention happen?3DurationFor how long does the exposure or intervention last?4DoseWhat is the dose regimen (amount, frequency)?
?. [Setting the research question ("problem formulation")]
Define the target Comparator of interest. This concerns the characteristics of the exposure or intervention being used as the comparator to which the target exposure or intervention is being compared.  AB1ComparatorWhat is the comparator exposure or intervention?2TimingWhen does the comparator happen?3DurationFor how long is the comparator administered?4DoseWhat is the dose of the comparator (amount, frequency)?
AB1ComparatorWhat is the comparator exposure or intervention?2TimingWhen does the comparator happen?3DurationFor how long is the comparator administered?4DoseWhat is the dose of the comparator (amount, frequency)?
?. [Setting the research question ("problem formulation")]
Define the target Outcome(s) of interest. This concerns the change being measured in the exposure or intervention group. These should be the primary outcomes of interest to the systematic review which form the hypothesis or hypotheses being tested. Secondary outcomes can also be listed.  AB1Primary outcome 12Primary outcome 234  AB1Secondary outcome 12Secondary outcome 234
AB1Primary outcome 12Primary outcome 234
AB1Secondary outcome 12Secondary outcome 234
?. [Setting the research question ("problem formulation")]
Define the Target Condition. This is the object of a test method for diagnosis or detection. It is only necessary for a systematic review of a diagnostic or detection test method.  AB1Target condition characteristic 12Target condition characteristic 23
AB1Target condition characteristic 12Target condition characteristic 23
?. [Defining the eligibility criteria and designing the process for screening evidence for inclusion]
Define and justify unambiguous and appropriate eligibility criteria for each component of the objective statement. Recommendation 1.3.1, 1.3.3, 1.3.4, 1.3.5  AB1PECO elementDescription of eligibility criteria2Eligible populationsInclude e.g. age, sex, health status, socioeconomic status, occupation etc.3Eligible exposuresInclude timing, methods for measurement exposure4Eligible comparatorsThe populations and exposures against which the exposed populations are being compared5Eligible primary outcomesSpecify the outcome, whether the outcome is apical (whole organism) or intermediate (is a marker of an apical outcome); the acceptable outcome measures (diagnostic criteria, scales, etc.) and timing of outcome measurement6Eligible secondary outcomesSpecify the outcome, whether the outcome is apical (whole organism) or intermediate (is a marker of an apical outcome); the acceptable outcome measures (diagnostic criteria, scales, etc.) and timing of outcome measurement7Eligible study designsDefine eligibile study designs by design features rather than design labels.  ABC1PECO elementDescription of exclusion criteriaReasons for exclusion2Excluded populations3Excluded exposures4Excluded comparators5Excluded outcomes
AB1PECO elementDescription of eligibility criteria2Eligible populationsInclude e.g. age, sex, health status, socioeconomic status, occupation etc.3Eligible exposuresInclude timing, methods for measurement exposure4Eligible comparatorsThe populations and exposures against which the exposed populations are being compared5Eligible primary outcomesSpecify the outcome, whether the outcome is apical (whole organism) or intermediate (is a marker of an apical outcome); the acceptable outcome measures (diagnostic criteria, scales, etc.) and timing of outcome measurement6Eligible secondary outcomesSpecify the outcome, whether the outcome is apical (whole organism) or intermediate (is a marker of an apical outcome); the acceptable outcome measures (diagnostic criteria, scales, etc.) and timing of outcome measurement7Eligible study designsDefine eligibile study designs by design features rather than design labels.
ABC1PECO elementDescription of exclusion criteriaReasons for exclusion2Excluded populations3Excluded exposures4Excluded comparators5Excluded outcomes
?. [Defining the eligibility criteria and designing the process for screening evidence for inclusion]
Define the points at which screening for eligibility will take place. Recommendation 1.3.2. Will there be screening at title and abstract, full text, or both?   A1Points at which screening will take place2Describe whether there will be screening at title and abstract, full text, or both
A1Points at which screening will take place2Describe whether there will be screening at title and abstract, full text, or both
?. [Defining the eligibility criteria and designing the process for screening evidence for inclusion]
Include all relevant, publicly-available evidence, except for research for which there is insufficient methodological information to allow appraisal of internal validity. Recommendation 1.3.6. Exclude evidence which is not publicly available. Recommendation 1.3.9  A1Policy on eligibility of grey literature and unpublished evidence2Describe how grey literature will be handled in the systematic review. If some or all grey literature is to be excluded, explain why and anticipate its implications as a limitation of review methods.
A1Policy on eligibility of grey literature and unpublished evidence2Describe how grey literature will be handled in the systematic review. If some or all grey literature is to be excluded, explain why and anticipate its implications as a limitation of review methods.
COSTER recommends that grey literature (i.e. studies that have not been published in peer-reviewed journals) should be included in systematic reviews. This is because the relevance of evidence is determined by the SR objectives, not by the publication status of that evidence, the language the evidence is in, nor its compatibility with the analyses planned by the reviewers.Only publicly available information about a study should be eligible for inclusion. If the planned SR will bring into the public domain evidence which was previously inaccessible, this makes the evidence eligible for inclusion.Studies for which there is insufficient information for risk of bias to be evaluated should be excluded from a SR, to prevent the inclusion in a SR of evidence that is potentially misleading but cannot be identified as such by the reviewers.
?. [Defining the eligibility criteria and designing the process for screening evidence for inclusion]
Include evidence which is relevant to review objectives irrespective of whether its results are in a usable form. Recommendation 1.3.7  A1Policy on eligibility of studies with unusable data2Describe how studies which report their results in a manner incompatible with planned analyses will be handled in the systematic review.
A1Policy on eligibility of studies with unusable data2Describe how studies which report their results in a manner incompatible with planned analyses will be handled in the systematic review.
COSTER recommends that documents be included in a SR regardless of whether their data fit the analysis plan of the reviewers or they are in a language in which the reviewers are fluent. This is to ensure that study documents which may contain information of potential relevance to the SR’s research objectives are not excluded from the data extraction step of the SR; however, they may be excluded from specific synthesis steps such as meta-analysis.
?. [Defining the eligibility criteria and designing the process for screening evidence for inclusion]
Include relevant evidence irrespective of language. Recommendation 1.3.8.  A1Policy on eligibility of studies based on language2State the language/s in which the systematic review will be written, and how studies not written in that language will be handled.  A1Languages to be included in the systematic review2345
A1Policy on eligibility of studies based on language2State the language/s in which the systematic review will be written, and how studies not written in that language will be handled.
A1Languages to be included in the systematic review2345
?. [Defining the strategy for searching for evidence relevant to the review objectives]
Design sufficiently sensitive search criteria, so that studies which meet the eligibility criteria of the review are not inadvertently excluded. Document the search methods in sufficient detail to render them transparent and reproducible. Recommendations 1.4.1, 2.6
?. [Defining the eligibility criteria and designing the process for screening evidence for inclusion]
Pilot test the screening process. Recommendation 1.4.7
A generic protocol for piloting the screening stage of a systematic review is available here:https://www.protocols.io/view/a-general-protocol-for-pilot-testing-the-screening-bkc9ksz6
?. [Defining the strategy for searching for evidence relevant to the review objectives]
Search all the key scientific databases for the topic, including national, regional and subject-specific databases. Recommendation 2.1  A1List of databases2Database 13Database 24Database 3
A1List of databases2Database 13Database 24Database 3
?. [Defining the strategy for searching for evidence relevant to the review objectives]
Structure search strategies for each database, electronic and other source, using appropriate controlled vocabulary, free-text terms and logical operators in a manner which prioritises sensitivity. Document the search methods and results in sufficient detail to render them transparent and reproducible. Recommendations 2.3, 2.6  AB1DatabaseSearch strategy2345
AB1DatabaseSearch strategy2345
?. [Defining the strategy for searching for evidence relevant to the review objectives]
Define reproducible strategies for identifying and searching sources of grey literature (databases, websites etc.). Document the search methods and results in sufficient detail to render them transparent and reproducible. Recommendations 2.2 and 2.6  ABCD1Grey literature sourceSearch strategyDate of searchNo. of results234
ABCD1Grey literature sourceSearch strategyDate of searchNo. of results234
?. [Defining the strategy for searching for evidence relevant to the review objectives]
Plan for re-running all searches and screen the results for potentially eligible studies within 12 months prior to publication of the review (screening at least at the level of title plus abstract). Recommendation 2.7  AB1TimingWhen will the searches be updated prior to publication of the review?2SourcesWhich sources will be searched again?3Level of screeningWhat level of screening will be conducted?4Updating findingsHow will review findings be updated in context of new studies?
AB1TimingWhen will the searches be updated prior to publication of the review?2SourcesWhich sources will be searched again?3Level of screeningWhat level of screening will be conducted?4Updating findingsHow will review findings be updated in context of new studies?
?. [Defining the strategy for searching for evidence relevant to the review objectives]
Search within the reference lists of included studies and other reviews relevant to the topic (“hand-searching”) and consider searching in the reference lists of documents which have cited included studies. Search by contacting relevant individuals and organisations. Recommendations 2.4 and 2.5  AB1Supplementary search strategiesIndicate if will be used2Hand search references of included studies3Hand search references of relevant reviews4Hand search references of studies cited by included studies5By contacting individuals and organisations6Other
AB1Supplementary search strategiesIndicate if will be used2Hand search references of included studies3Hand search references of relevant reviews4Hand search references of studies cited by included studies5By contacting individuals and organisations6Other
?. [Defining the eligibility criteria and designing the process for screening evidence for inclusion]
Screening of each piece of evidence for inclusion to be conducted by at least two people working independently, with an appropriate process (e.g. third-party arbitration) for identifying and settling disputes. Recommendation 3.1  AB1Team members who will conduct screeningMethod for resolving disputes2
AB1Team members who will conduct screeningMethod for resolving disputes2
?. [Defining the eligibility criteria and designing the process for screening evidence for inclusion]
Do not exclude multiple reports of the same research (e.g. multiple publications, conference abstracts etc.); instead collate the methodological information from each of the reports as part of the data extraction process for each unit of evidence. Recommendation 3.4  A1Multiple publications policy2Describe how multiple publications derived from the same study will be aggregated.
A1Multiple publications policy2Describe how multiple publications derived from the same study will be aggregated.
?. [Methods for synthesising and evaluating the evidence]
Design the "characteristics of included studies" table. Recommendation 1.4.2
?. [Defining the eligibility criteria and designing the process for screening evidence for inclusion]
Design the PRISMA flow chart for presentation of the results of the screening process. Recommendation 3.2
?. [Methods for synthesising and evaluating the evidence]
Define the risk of bias assessment methods to be used for evaluating the internal validity of the included research. If observational studies are included, this should cover identification of plausible confounders. Recommendation 1.4.3
Review teams may find the FEAT (Focus-Extent-Application-Transparency) mnemonic to be useful in defining their risk of bias assessment methods.Focus: The focus of the tool should be exclusively the internal validity of a study. If other quality constructs are of interest, each should be assessed in a separate process.Extent: All the important threats to internal validity should be covered by the tool. If observational studies are being appraised, the threats should include all important confounders.Application: The appraisal process should produce consistent, accurate descriptions of the extent to which a study is vulnerable to each identified threat to internal validity. The judgements should be in a form which can be logically incorporated into the evidence synthesis.Transparency: The reason for each judgement should be documented, quoting as justification relevant text from the study documentation.Extent: All the important threats to internal validity should be covered by the tool. If observational studies are being appraised, the threats should include all important confounders.Application: The appraisal process should produce consistent, accurate descriptions of the extent to which a study is vulnerable to each identified threat to internal validity. The judgements should be in a form which can be logically incorporated into the evidence synthesis.Transparency: The reason for each judgement should be documented, quoting as justification relevant text from the study documentation.Application: The appraisal process should produce consistent, accurate descriptions of the extent to which a study is vulnerable to each identified threat to internal validity. The judgements should be in a form which can be logically incorporated into the evidence synthesis.Transparency: The reason for each judgement should be documented, quoting as justification relevant text from the study documentation.Refer to Section 5 of the COSTER recommendations for detail on how the risk of bias assessment process should be conducted.
?. [Methods for synthesising and evaluating the evidence]
Design the methods for synthesising the included studies, to cover: qualitative and quantitative methods (with full consideration given to synthesis methods to be used when meta-analysis is not possible); assessment of heterogeneity; choice of effect measure (e.g. RR, OR etc.); methods for meta-analysis and other quantitative synthesis; pre-defined, appropriate effect modifiers for sub-group analyses. Recommendations 1.4.4, 6.1  AB1Synthesis ComponentPlanned Methods2Qualitative or narrative methods3Quantitative methods4Conditions for combining studies in overall and subgroup analyses5Choice of effect measure6Assessment of heterogeneity (6.3) and consequences of developing summary results (6.4)7Effect modifiers for subgroup analysis8Transformation of scales into common measures (6.2)9Assessment of publication bias (6.5)10Impact of the risk of bias assessment on the synthesis (6.6)11Sensitivity analyses (6.7)12Other methods
AB1Synthesis ComponentPlanned Methods2Qualitative or narrative methods3Quantitative methods4Conditions for combining studies in overall and subgroup analyses5Choice of effect measure6Assessment of heterogeneity (6.3) and consequences of developing summary results (6.4)7Effect modifiers for subgroup analysis8Transformation of scales into common measures (6.2)9Assessment of publication bias (6.5)10Impact of the risk of bias assessment on the synthesis (6.6)11Sensitivity analyses (6.7)12Other methods
Refer to section 6 of COSTER for detailed recommendations for how evidence should be synthesised in systematic reviews. Popay et al. (2006) attached provides very useful guidance on how to approach the non-quantitive components of the synthesis.
?. [Methods for synthesising and evaluating the evidence]
Define the methods for determining how, given strengths and limitations of the overall body of evidence, confidence in the results of the synthesis of the evidence for each outcome is to be captured and expressed. (For reviews which include multiple streams of evidence, this may need to be defined for each stream.) Recommendation 1.4.5
The components of assessment of confidence or certainty in the evidence are described in section 7 of COSTER.
?. [Methods for synthesising and evaluating the evidence]
Design and pilot the data extraction forms. Recommendation 1.4.7
?. [Methods for synthesising and evaluating the evidence]
Risk of bias assessment is to be conducted by at least two people working independently, with an appropriate process (e.g. third-party arbitration) for identifying and settling disputes. Recommendation 5.3  AB1Team members conducting risk of bias assessmentMethod for identifying and settling disputes2
AB1Team members conducting risk of bias assessmentMethod for identifying and settling disputes2
?. [Methods for synthesising and evaluating the evidence]
Define the tool selection and modification process (how will a suitable tool be identified, and what process will be followed to identify and validate any necessary modifications?) Recommendation 1.4.3  ABCD1Tool selectedStudies to which it is appliedModifications madeMethod for validating modifications23
ABCD1Tool selectedStudies to which it is appliedModifications madeMethod for validating modifications23
?. [Methods for synthesising and evaluating the evidence]
Define the training and piloting process for the risk of bias assessment (how will the review team be trained in use of the tool, and what are the conditions under which the piloting process will be determined satisfactory?) Recommendation 1.4.7
?. [Methods for synthesising and evaluating the evidence]
For reviews which include multiple streams of evidence (e.g. animal and human studies), define the methods for integrating the individual streams into an overall result. Recommendation 1.4.6This should include a description of the relative relevance of populations (e.g. species, age, comorbidities etc.), exposures (e.g. timing, dose), and outcomes (direct or surrogate, acute or chronic model of disease, etc.), as appropriate, per which inferences about predicted effects in target populations can be made from observed effects in study populations.
?. [Methods for synthesising and evaluating the evidence]
Pilot the process for the assessment of confidence in the results of the synthesis of the evidence. How will the review team be trained in use of the tool, and what are the conditions under which the piloting process will be determined satisfactory? Recommendation 1.4.7
?. [Registering and publishing the protocol]
Create a permanent public record of intent to conduct the review (e.g. by registering the protocol in an appropriate registry) prior to conducting the literature search. Recommendation 1.5.1
?. [Registering and publishing the protocol]
As appropriate for review planning and question formulation, secure peer-review and public feedback on a draft version of the protocol, incorporating comments into the final version of the protocol. Recommendation 1.5.2
?. [Registering and publishing the protocol]
Publish the final version of the protocol in a public archive, prior to screening studies for inclusion in the review. Recommendation 1.5.3
Publication of the protocol in a journal is equivalent to publication in a public archive.