Guidelines for reporting meta-epidemiological methodology research

Published research should be reported to evidence users with clarity and transparency that facilitate optimal appraisal and use of evidence and allow replication by other researchers. Guidelines for such reporting are available for several types of studies but not for meta-epidemiological methodology studies. Metaepidemiological studies adopt a systematic review or meta-analysis approach to examine the impact of certain characteristics of clinical studies on the observed effect and provide empirical evidence for hypothesised associations. The unit of analysis in metaepidemiological studies is a study, not a patient. The outcomes of meta-epidemiological studies are usually not clinical outcomes. In this guideline, we adapt items from the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) to fit the context of meta-epidemiological studies.
The goal is generally broad but often focuses on examining the impact of certain characteristics of clinical studies on the observed effect, describing the distribution of research evidence in a specific setting, examining heterogeneity and exploring its causes, identifying and describing plausible biases and providing empirical evidence for hypothesised associations. Unlike classic epidemiology, the unit of analysis for meta-epidemiological studies is a study, not a patient. The outcomes of meta-epidemiological studies are usually not clinical outcomes. [bib_ref] I-01 Meta-epidemiology: building the bridge from research evidence to clinical practice, Zhang [/bib_ref] [bib_ref] Statistical methods for assessing the influence of study characteristics on treatment effects..., Sterne [/bib_ref] [bib_ref] Meta-analysis and the meta-epidemiology of clinical research, Naylor [/bib_ref] In this guide, we adapt the items used in the PRISMA statement [bib_ref] The PRISMA statement for reporting systematic reviews and meta-analyses of studies that..., Liberati [/bib_ref] for reporting systematic reviews and meta-analysis to fit the setting of meta-epidemiological studies. The rationale for starting with PRISMA is that meta-epidemiological research requires a systematic approach (ie, a systematic review) to select data (the input for methodological analysis). The overarching goal of this guideline is to improve the transparency and clarity of healthcare research.

## Examples of meta-epidemiological studies
Analysis of Cochrane systematic reviews showed that trials with inadequate or unclear allocation concealment yielded larger estimates of effect relative to controlled trials with adequate allocation concealment. [bib_ref] Randomisation to protect against selection bias in healthcare trials, Odgaard-Jensen [/bib_ref] In a second meta-epidemiological study, non-blinded trials of complementary medicine exaggerated the effect size by an average of 0.56 SD (95% CI -0.71 to -0.41) compared with blinded trials. [bib_ref] Bias due to lack of patient blinding in clinical trials. A systematic..., Hrobjartsson [/bib_ref] A third example is a study that explored the effect of trials stopped early for benefit on future research. [bib_ref] Randomized trials addressing a similar question are commonly published after a trial..., Murad [/bib_ref] In a fourth example, it was shown that in almost 30% of endocrinology studies, the largest treatment effect was observed in the first or second published studies (ie, earlier studies, when studies were ordered chronologically using the date of publication). 13

## Proposed reporting guidelines
We have adapted items of the PRISMA statement to fit a meta-epidemiological study. The items are categorised according to the title, abstract, introduction, methods, results, conclusion and funding and are supported by rationale and examples.

# Title and abstract
Both the title and abstract should identify the report as a meta-epidemiological study, for example, 'Early studies reported extreme findings with large variability: a meta-epidemiologic study in the field of endocrinology' . [bib_ref] Early studies reported extreme findings with large variability: a meta-epidemiologic study in..., Wang [/bib_ref] The rationale for this identification is to improve indexing and facilitate retrieval. It also helps clinicians searching for systematic reviews about clinical topics to quickly find relevant evidence and avoid reviewing additional non-clinical meta-epidemiological

# Introduction
The introduction should include the rationale and objectives of the study. This requirement is similar to PRISMA. However, it is more important to explicitly state the hypothesis being tested in meta-epidemiological studies compared with a systematic review summarising evidence about the effectiveness of an intervention (which may or may not have a clear hypothesis and primarily aim at empirically evaluating a clinical outcome).

# Methods
The methods section should specify if the meta-epidemiological study follows a priori established protocol and clarify the eligibility criteria, information sources, search methods, study selection and data collection process, data items collected, summary measures and synthesis methods (including assumptions and computational approximations and methods of additional analyses such as sensitivity, subgroup or meta-regression analyses). These requirements are similar to PRISMA. However, registration of protocols may not be needed, particularly since systematic reviews registration sites are geared more towards clinical questions. Search strategy in a meta-epidemiological study is different because it is not driven by a clinical question. Evaluating and reporting the risk of bias in individual studies may or may not be relevant to the question at hand. The type of bias can also be different. Data extraction is often more complex in meta-epidemiological studies and should be described in details sufficient to allow replication. The methods section of a meta-epidemiological study should describe in details the effect measure used in statistical analysis because it is likely that a clinical audience will not be familiar with such measure (eg, ratio of OR, standardised measures and regression coefficients). Such measures are not commonly used in studies that report on treatment effects (ie, clinical audience would be familiar with OR but not familiar with ratio of OR). Quantifying interagreement between investigators is more critical in a meta-epidemiological study than in a clinical systematic review because of the judgements commonly made to classify data in a meta-epidemiological study. For example, a study evaluated the association between industry affiliation and researchers' position on the cardiovascular risk of rosiglitazone. Reviewers had to assess how favourable was the researchers' opinion about rosiglitazone using an ordinal scale. 14 This task is fairly subjective and requires judgement. In this case, agreement between reviewers is critical to report and affects the reliability of this meta-epidemiological study (arguably more critical than in a systematic review of clinical outcomes).

# Results
The results section should include the results of the study selection process (how many studies, how many patients, etc). Different from PRISMA, study and patient characteristics need to be tailored to the meta-epidemiological question at hand. Therefore, it may not be important to know patients' age, comorbidities and settings in some questions (although such characteristics may be important in other questions). For example, the meta-epidemiological study showing that trials with inadequate allocation concealment yielded larger estimates of effects (relative to trials with adequate allocation concealment) does not necessarily have to describe patients' characteristics or comorbidities in each individual trial. [bib_ref] Randomisation to protect against selection bias in healthcare trials, Odgaard-Jensen [/bib_ref] Publication bias should be evaluated and reported similar to PRISMA recommendations.

# Discussion and conclusions
As with PRISMA, the discussion section should include summary of the main findings and comparison with existing knowledge and should also include the limitations and strengths of the meta-epidemiological study.
Limitations at a study level may not be important (as in clinical systematic reviews); however, limitations at the level of the meta-epidemiological study are important. The quality of evidence is a highly important step in a systematic review reporting on a treatment effect [bib_ref] How to read a systematic review and meta-analysis and apply the results..., Murad [/bib_ref] and is critical for decision making but also may not be relevant to a meta-epidemiological study. Investigators should, however, inform readers of their certainty in the results, even if this was determined narratively 16 because formal approaches 15 are intended to rate the certainty of evidence about interventions. Lastly, systematic reviews of clinical questions should include in their discussion the implication of their findings on future research and current clinical practice. Meta-epidemiological studies will have profound implications for future research and should report such implications. Meta-epidemiological studies will likely have minimal direct impact on clinical practice. Nevertheless, indirect impact on clinical practice is possible and should be reported when present. For example, a meta-epidemiological study showed that in endocrinology, the largest treatment effect was observed in the first or second published studies. [bib_ref] Early studies reported extreme findings with large variability: a meta-epidemiologic study in..., Wang [/bib_ref] Clinically, this should make endocrinologists sceptical about new drugs, laboratory tests or genetic markers and not rush into recommending them to their patients until more studies confirm findings observed in these earlier studies. [bib_ref] Early studies reported extreme findings with large variability: a meta-epidemiologic study in..., Wang [/bib_ref] In table 1, we present PRISMA items with adaptations to fit the setting of meta-epidemiological studies.

# Discussion and limitations
Reporting guidelines have three goals: to enhance the applicability and use of research results, to allow readers to critically appraise the results and to permit reproducibility and replication of the work, which is essential for accountability and transparency purposes. Meta-epidemiological studies should have similar standards and require reporting guidelines that achieve these three goals.
In this guideline, we propose adaptation to PRISMA items to fit meta-epidemiological studies. Meta-epidemiological studies are inherently heterogeneous in terms

# Introduction

## Rationale
Describe the rationale for the meta-epidemiological study in the context of what is already known

## Objectives
Provide an explicit statement of the goal of the meta-epidemiological study and the hypothesis being empirically tested

# Methods

## Protocol

## Ebm primer
of their approach and execution, which is a challenge for current reporting guidelines of systematic reviews (such as PRISMA). For example, a meta-epidemiological study used trial registries as a source of its data. [bib_ref] Patient-important outcomes in registered diabetes trials, Gandhi [/bib_ref] Registries are not searched using standard search terms and methods and may not fit the reporting requirement needed for systematic reviews. Entries in trial registries are also dynamic, just like Internet searches. In this case, authors of such study should make every possible effort to describe their work in a reproducible fashion but acknowledge these limitations. [bib_ref] Patient-important outcomes in registered diabetes trials, Gandhi [/bib_ref] In addition, reporting guidelines are frequently underused and are not always emphasised by journal editors. A descriptive analysis of published randomised trials in the four highest impact general medical journals shows inconsistencies and non-adherence to the CONSORT statement (eg, allocation concealment and details of blinding were reported only in 12% and 21%, respectively). [bib_ref] Assessment of adherence to the CONSORT statement for quality of reports on..., Ghimire [/bib_ref] Adherence across all reporting guidelines was estimated to be 86% in another study. [bib_ref] A systematic scoping review of adherence to reporting guidelines in health care..., Samaan [/bib_ref] Evaluation of the instructions given to authors shows that PRISMA guidance was referenced only in 13% of gastroenterology and hepatology journals. [bib_ref] Evaluation of the endorsement of the preferred reporting items for systematic reviews..., Panic [/bib_ref] Here, we have proposed a reporting guideline to be used when publishing meta-epidemiological research. We hope that these guidelines will be used, modified and enhanced and lead to more clarity and transparency of healthcare research.