Publication Bias in registry-based non-randomized studies
Creators: name: Starke, Paula
name: Mathes, Tim
Publisher: name: Open Science Framework
Year: 2022
Subjects: subject: Medicine and Health Sciences

description: Main question and relevance: Publication bias refers to the association between the estimated treatment effect and the likelihood of publication. Raising awareness for the topic among researchers is important as publication bias can reduce the reliability of synthesized research when positive findings are overrepresented and therefore effect estimates are overestimated. Much research has shown that publication bias is a major problem in clinical research [1]. However, to our knowledge, there is no meta-research study that has assessed the risk of publication bias in studies that are based on registries. Our aim is to assess how much publication bias exists in registry-based studies that compare an intervention. Literature search: We will conduct a meta-research study on 9 registries from different medical specialties. The identification of registries will follow a two-step approach. Step One: First, to identify potentially eligible registries the PubMed database will be searched for empirical studies that are based on data from a patient registry. The patient registry should contain data from individual patients. Screening will be performed until we have identified 9 registries from the following medical fields (one for each field ): • Accident and emergency medicine • Cardiology • Endocrinology diabetes and metabolism • General surgery • Infectious diseases • Oncology • Orthopedics • Pediatrics • Psychiatry Health system registries, national registries, mortality registries, registries of residents and other registries that include entire populations of a country, region, specific institution etc. are excluded. We search PubMed with two different search filters. Both filters consist of the two components #cohort type study and #registry-based study. Additionally, the search is supplemented by a either the mesh term “Therapeutics” or “surgical procedures, operative" to capture registries from all 9 medical fields. For the #cohort type study component a search filter developed by Waffenschmidt et al. [2] is used: (cohort[all] OR (control[all] AND study[all]) OR (control[tw] AND group*[tw]) OR epidemiologic studies[mh] OR program[tw] OR clinical trial[pt] OR comparative stud*[all] OR evaluation studies[all] OR statistics as topic[mh] OR survey*[tw] OR follow-up*[all] ORtime factors[all] OR ci[tw]) NOT ((animals[mh:noexp] NOT humans[mh:noexp]) OR comment[pt] OR editorial[pt] OR review[pt] OR meta analysis[pt] OR case report[tw] OR consensus[mh] OR guideline[pt] OR history[sh])mesh) The “registry-based” component is based on a search filter developed by Mathes et al. [3]: ((Registries[mesh] OR registry[tiab] OR registries[tiab] OR register[tiab] OR database[tiab]) NOT (("Trials Registry"[tiab] OR "Trial Registry"[tiab] OR "registry number"[tiab] OR "registry:"[tiab] OR registry;[tiab] OR "Trial Register"[tiab] OR "register number"[tiab] OR "register:"[tiab] OR register;[tiab] OR "number:"[tiab] OR number;[tiab] OR "database:"[tiab] OR database;[tiab] OR "Trials database"[tiab] OR "Trial database"[tiab) NOT (registry[ti] OR register[ti] OR "registry based"[tiab] OR "registry trial"[tiab] OR "national registry"[tiab] OR "national register"[tiab] OR "national registries"[tiab] OR "national database"[tiab] OR "patient registry"[tiab] OR "patient register"[tiab] OR "diseases registry"[tiab] OR "outcomes registry"[tiab] OR "clinical registry"[tiab] OR "clinical register"[tiab] OR "clinical data registry"[tiab]))) We will sort the search from newest to oldest to ensure that we only identify registries based on which studies have been performed in the last years. We will start with screening results published in the last 3 month and extend the time range if necessary. References (titles and abstracts) identified by the PubMed search will be checked against the inclusion criteria independently by two reviewers. Step two: In the second step, we will search for studies that are based on data from the identified registries. Inclusion criteria for the studies are: 1) Comparison of two interventions 2) Binary outcome reported 3) Data available to calculate a relative effect measure 4) Cohort type study Publications from a certain registry will be searched in PubMed, Google Scholar and the webpages of the registries if available. The search will be continued until we have identified at least 10 studies for each registry that have assessed the same outcome. In the case that we could not identify these number, we will repeat this search procedure, continuing with step one. Data Extraction and Synthesis Extraction: Data will be extracted in piloted standardized forms. In the first step, data extraction will include the name of the registry and clinical specialty. In the second step, data extraction for each registry will be extended to the definition of the registry population, primary aim of the registry (e.g. quality assurance), country/region, and sample size of the registry. Data extraction for each study will include number of patients per group, patient characteristics, description of intervention and control, endpoint (definition), study design, missing values per group, and number of patients and events per group (or effect estimates with precision). One reviewer will extract data directly into the forms, which are checked by a second reviewer. Synthesizes: All effect measures will be pooled so that standard (more established care) is the reference group. Publication bias will be analyzed within each registry. As a preliminary method to detect possible publication bias, funnel plots will be prepared. We will use Copa’s model-based test suggested by Duan et al. to test if publication bias is statistically significant [4]. A p-value of &lt; 0.05 will be considered statistically significant. References: [1] Dwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., &amp; Reporting Bias Group. (2013). Systematic review of the empirical evidence of study publication bias and outcome reporting bias—an updated review. PloS one, 8(7), e66844. [2] Waffenschmidt, S.,Navarro-Ruan, T., Hobson, N., Hausner, E. ,Sauerland, S., Haynes, R.B. (2020). Development and validation of study filters for identifying controlled non-randomized studies in PubMed and Ovid MEDLINE. Res Syn Meth, 11:617–626. doi: 10.1002/jrsm.1425. [3] Mathes, T., Buehn, S., Prengel, P., Pieper, D. (2017). Registry-based randomized controlled trials merged the strength of randomized controlled trials and observational studies and give rise to more pragmatic trials, Journal of Clinical Epidemiology, doi: 10.1016/j.jclinepi.2017.09.017. [4] Duan, R., Piao, J., Marks-Anglin, A., Tong, J., Lin, L., Chu, H., ... &amp; Chen, Y. (2020). Testing for publication bias in meta-analysis under Copas selection model. arXiv preprint arXiv:2007.00836.