Metabias: A Challenge for Comparative Effectiveness Research
Authors: Steven Goodman, Kay Dickersin
Journal: Annals of Internal Medicine (2011)

Editorials5 July 2011Metabias: A Challenge for Comparative Effectiveness ResearchSteven Goodman, MD, MHS, PhD and Kay Dickersin, MA, PhDSteven Goodman, MD, MHS, PhDFrom Johns Hopkins University, Baltimore, MD 21205.Search for more papers by this author and Kay Dickersin, MA, PhDFrom Johns Hopkins University, Baltimore, MD 21205.Search for more papers by this authorAuthor, Article, and Disclosure Informationhttps://doi.org/10.7326/0003-4819-155-1-201107050-00010 SectionsAboutFull TextPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissions ShareFacebookTwitterLinkedInRedditEmail Comparative effectiveness research encompasses both individual primary research studies and syntheses of the primary research, typically systematic reviews and meta-analyses. Before accepting the results of either form of study, decision makers must critically assess their methods to identify sources of potential bias.For primary research, critical appraisal involves close examination of research methods, including design, data, execution, analysis, and interpretation. For meta-analyses, individual studies are examined in the same way, but the collection of studies is also examined for heterogeneity. Studies are deemed heterogeneous if their methods or results differ from one another so much that the studies cannot be ...References1. Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess. 2003;7:1-76. [PMID: 12583822] CrossrefMedlineGoogle Scholar2. Dickersin K, Chalmers I. Recognising, investigating and dealing with incomplete and biased reporting of clinical research: from Francis Bacon to the World Health Organisation. Accessed at www.jameslindlibrary.org/essays/biased_reporting/biased_reporting.html on 23 May 2011. Google Scholar3. Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev. 2009;:MR000006. [PMID: 19160345] MedlineGoogle Scholar4. Dwan K, Altman DG, Cresswell L, Blundell M, Gamble CL, Williamson PR. Comparison of protocols and registry entries to published reports for randomised controlled trials. Cochrane Database Syst Rev. 2011;:MR000031. [PMID: 21249714] MedlineGoogle Scholar5. Song F, Parekh S, Hooper L, Loke YK, Ryder J, Sutton AJ, et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess. 2010;14:iii, ix-xi, 1-193. [PMID: 20181324] CrossrefMedlineGoogle Scholar6. Vedula SS, Bero L, Scherer RW, Dickersin K. Outcome reporting in industry-sponsored trials of gabapentin for off-label use. N Engl J Med. 2009;361:1963-71. [PMID: 19907043] CrossrefMedlineGoogle Scholar7. Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291:2457-65. [PMID: 15161896] CrossrefMedlineGoogle Scholar8. Fontanarosa PB, Flanagin A, DeAngelis CD. Reporting conflicts of interest, financial aspects of research, and role of sponsors in funded studies [Editorial]. JAMA. 2005;294:110-1. [PMID: 15998899] CrossrefMedlineGoogle Scholar9. Eyding D, Lelgemann M, Grouven U, Härter M, Kromp M, Kaiser T, et al. Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials. BMJ. 2010;341:c4737. [PMID: 20940209] CrossrefMedlineGoogle Scholar10. McGauran N, Wieseler B, Kreis J, Schüler YB, Kölsch H, Kaiser T. Reporting bias in medical research—a narrative review. Trials. 2010;11:37. [PMID: 20388211] CrossrefMedlineGoogle Scholar11. Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ. 2003;326:1167-70. [PMID: 12775614] CrossrefMedlineGoogle Scholar12. Als-Nielsen B, Chen W, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events? JAMA. 2003;290:921-8. [PMID: 12928469] CrossrefMedlineGoogle Scholar13. Gøtzsche PC. Research integrity and pharmaceutical industry sponsorship [Editorial]. Med J Aust. 2005;182:549-50. [PMID: 15938676] CrossrefMedlineGoogle Scholar14. Dickersin K, Rennie D. Registering clinical trials. JAMA. 2003;290:516-23. [PMID: 12876095] CrossrefMedlineGoogle Scholar15. ClinicalTrials.gov. Protocol registration system. Accessed at prsinfo.clinicaltrials.gov/fdaaa.html on 23 May 2011. Google Scholar16. World Health Organization. International Clinical Trials Registry Platform (ICTRP). Accessed at www.who.int/ictrp/en/ on 23 May 2011. Google Scholar17. Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The ClinicalTrials.gov results database—update and key issues. N Engl J Med. 2011;364:852-60. [PMID: 21366476] CrossrefMedlineGoogle Scholar18. DeAngelis CD, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al; International Committee of Medical Journal Editors. Clinical trial registration: a statement from the International Committee of Medical Journal Editors [Editorial]. JAMA. 2004;292:1363-4. [PMID: 15355936] CrossrefMedlineGoogle Scholar19. Dechartres A, Boutron I, Trinquart L, Charles P, Ravaud P. Single-center trials show larger treatment effects than multicenter trials: evidence from a meta-epidemiologic study. Ann Intern Med. 2011;155:39-51. LinkGoogle Scholar20. Dickersin K, Min YI. Publication bias: the problem that won't go away. Ann NY Acad Sci. 1993;703:135-46. [PMID: 8192291] CrossrefMedlineGoogle Scholar Author, Article, and Disclosure InformationAuthors: Steven Goodman, MD, MHS, PhD; Kay Dickersin, MA, PhDAffiliations: From Johns Hopkins University, Baltimore, MD 21205.Disclosures: None disclosed. Forms can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M11-1186.Corresponding Author: Steven Goodman, MD, MHS, PhD, Johns Hopkins University, Oncology Biostatistics, 550 North Broadway, Suite 1103, Baltimore, MD 21205; e-mail, [email protected]edu.Current Author Addresses: Dr. Goodman: Johns Hopkins University, Oncology Biostatistics, 550 North Broadway, Suite 1103, Baltimore, MD 21205.Dr. Dickersin: Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, Baltimore, MD 21205. PreviousarticleNextarticle Advertisement FiguresReferencesRelatedDetailsSee AlsoSingle-Center Trials Show Larger Treatment Effects Than Multicenter Trials: Evidence From a Meta-epidemiologic Study Agnes Dechartres , Isabelle Boutron , Ludovic Trinquart , Pierre Charles , and Philippe Ravaud Metrics Cited byNon-Alcoholic Fatty Liver Disease in Patients with Polycystic Ovary Syndrome: A Systematic Review, Meta-Analysis, and Meta-RegressionInitial combination therapy for hypertension in patients of African ancestry: a systematic review and meta-analysisEffect of Intermittent Pneumatic Compression in Addition to Pharmacologic Prophylaxis for Thromboprophylaxis in Hospitalized Adult Patients: A Systematic Review and Meta-AnalysisCharacterisation of MS phenotypes across the age span using a novel data set integrating 34 clinical trials (NO.MS cohort): Age is a key contributor to presentationToward more rigorous and informative nutritional epidemiology: The rational space between dismissal and defense of the status quoAgreement of Risk-of -Bias varied in systematic reviews on acupuncture and was associated with methodological qualityEvaluating the quality of evidence for gaming disorder: A summary of systematic reviews of associations between gaming disorder and depression or anxietyTen questions to consider when interpreting results of a meta‐epidemiological study—the MetaBLIND study as a caseThe surgical treatment of cerebrospinal fistula: Qualitative and quantitative analysis of indications and resultsEvaluation of Systematic Reviews of Interventions for Retina and Vitreous ConditionsReporting randomised trials of social and psychological interventions: the CONSORT-SPI 2018 ExtensionConflicts of interest in infection prevention and control research: no smoke without fire. A narrative reviewThe Importance of Reporting Biases in Patient Care: Can We Trust the Evidence From Either Individual Studies or Systematic Reviews?Kay Dickersin, MA, PhD and Riaz Qureshi, MScA meta-analysis of the incidence of complications associated with groin access after the use of resuscitative endovascular balloon occlusion of the aorta in trauma patientsConservative treatment for advanced T3–T4 laryngeal cancer: meta-analysis of key oncological outcomesAddressing culture and context in humanitarian response: preparing desk reviews to inform mental health and psychosocial supportThe influence of the team in conducting a systematic reviewIndustry sponsorship and research outcomeAssociation between trial registration and treatment effect estimates: a meta-epidemiological studyEvaluation of the Cochrane tool for assessing risk of bias in randomized clinical trials: overview of published comments and analysis of user practice in Cochrane and non-Cochrane reviewsLaryngeal preneoplastic lesions and cancer: challenging diagnosis. Qualitative literature review and meta-analysis.Scientific ReportingEmpirical evaluation of which trial characteristics are associated with treatment effect estimatesSensitivity subgroup analysis based on single-center vs. multi-center trial status when interpreting meta-analyses pooled estimates: the logical way forwardIntegrating multiple data sources (MUDS) for meta-analysis to improve patient-centered outcomes research: a protocol for a systematic reviewMaking personalized medicine more affordableRelationship between Decrease in Serum Sodium Level and Bone Mineral Density in Osteoporotic Fracture PatientsComparison of intervention effects in split-mouth and parallel-arm randomized controlled trials: a meta-epidemiological studyComparative Effectiveness in TransplantationThe evolution of assessing bias in Cochrane systematic reviews of interventions: celebrating methodological contributions of the Cochrane CollaborationAssessing Risk of Bias in Randomised Clinical Trials Included in Cochrane Reviews: The why is Easy, the how is a ChallengeIndustry sponsorship and research outcomeRehabilitation for ankle fractures in adultsInfluence of Reported Study Design Characteristics on Intervention Effect Estimates From Randomized, Controlled TrialsJelena Savović, PhD, Hayley E. Jones, PhD, Douglas G. Altman, DSc, Ross J. Harris, MSc, Peter Jüni, MD, Julie Pildal, MD, PhD, Bodil Als-Nielsen, MD, PhD, Ethan M. Balk, MD, MPH, Christian Gluud, DrSciMed, Lise Lotte Gluud, DrSciMed, John P.A. Ioannidis, MD, DSc, Kenneth F. Schulz, PhD, MBA, Rebecca Beynon, MA, Nicky J. Welton, PhD, Lesley Wood, PhD, David Moher, PhD, Jonathan J. Deeks, PhD, and Jonathan A.C. Sterne, PhDWhat Comparative Effectiveness Research Is Needed? A Framework for Using Guidelines and Systematic Reviews to Identify Evidence Gaps and Research PrioritiesTianjing Li, MD, MHS, PhD, S. Swaroop Vedula, MD, MPH, Roberta Scherer, PhD, and Kay Dickersin, MA, PhDClassification systems to improve assessment of risk of biasActivity of alemtuzumab monotherapy in treatment-naive, relapsed, and refractory severe acquired aplastic anemiaNever Mind the Bollocks: Chance, Noise, Skepticism, and Statistics 5 July 2011Volume 155, Issue 1Page: 61-62KeywordsBiostatisticsClinical trialsCohort studiesComparative effectiveness researchHealth carePrevention, policy, and public healthResearch and reporting methodsResearch designSystematic reviewsTreatment guidelines ePublished: 5 July 2011 Issue Published: 5 July 2011 Copyright & PermissionsCopyright © 2011 by American College of Physicians. All Rights Reserved.PDF downloadLoading ...