Need for a Quantitative Meta-Analysis of Trials of Adjuvant Interferon in Melanoma
Authors: Keith Wheatley, Natalie Ives, Barry Hancock, Martin Gore, M.B. Lens, M. Dawes
Journal: Journal of Clinical Oncology (2002)

Article Tools SPECIAL DEPARTMENTS Article Tools OPTIONS & TOOLS Export Citation Track Citation Add To Favorites Rights & Permissions COMPANION ARTICLES No companion articles ARTICLE CITATION DOI: 10.1200/JCO.2002.02.101 Journal of Clinical Oncology - published online before print September 21, 2016 PMID: 12351611 Need for a Quantitative Meta-Analysis of Trials of Adjuvant Interferon in Melanoma Keith WheatleyxKeith WheatleySearch for articles by this author , Natalie IvesxNatalie IvesSearch for articles by this author , Barry HancockxBarry HancockSearch for articles by this author , Martin GorexMartin GoreSearch for articles by this author M.B. LensxM.B. LensSearch for articles by this author , M. DawesxM. DawesSearch for articles by this author Show More University of Birmingham, Birmingham, United KingdomUniversity of Sheffield, Sheffield, United KingdomRoyal Marsden Hospital, London, United KingdomUniversity of Oxford, John Radcliffe Hospital, Oxford, United Kingdom https://doi.org/10.1200/JCO.2002.02.101 First Page Full Text PDF Figures and Tables © 2002 by American Society of Clinical OncologyjcoJ Clin OncolJournal of Clinical OncologyJCO0732-183X1527-7755American Society of Clinical OncologyResponse01102002In Reply:Although we appreciate what can be achieved by meta-analysis using appropriate methods, Wheatley et al published the results of their meta-analysis only in the abstract form,1 which did not allow us—while performing our systematic review—to critically appraise the methodology of their study or to assess whether it was conducted according to the Quality of Reporting of Meta-Analyses (QUOROM) statement.2Investigating whether heterogeneity of treatment effect exists between trials and exploring the possible reasons for heterogeneity between studies is an important aspect of conducting a meta-analysis. Many reviewers when conducting a meta-analysis fail to investigate appropriately the sources of heterogeneity.3 It is necessary to distinguish the concept of clinical heterogeneity from the statistical one.The possibility of excess variability between the results of the different trials is examined by the test of homogeneity (occasionally described as a test for heterogeneity).4 In their letter, Wheatley et al state that there is no clear evidence of heterogeneity of treatment effect between the 11 trials included in their meta-analysis, while the test of homogeneity was nonsignificant. In many meta-analyses, however, such statistical evidence is lacking and the test for homogeneity is nonsignificant. Yet this cannot be interpreted as evidence of homogeneity of the results of all the trials included.5 In addition, the test of homogeneity has low power to detect excess variation, and the possibility of a type II (false-negative) error must always be considered.However, although Wheatley et al quantified the absence of statistical heterogeneity, we cannot ignore clinical heterogeneity between trials. Many epidemiologists consider the testing for heterogeneity as largely irrelevant when the studies in the meta-analysis are clinically heterogenous.6 Wheatley et al performed a stratified meta-analysis, dividing trials in two groups (low-dose and high-dose groups) according to the interferon dose used in the study. A separate analysis carried out within each subgroup failed to demonstrate any difference in the benefit of interferon between two groups of trials. Wheatley et al did not consider other clinical characteristics of the included trials as an important source of heterogeneity that may moderate treatment efficacy and mislead us about the overall summaries.7 Also, there is no described analysis that can compensate biases introduced through poor trial design and execution. The achievement of allocation concealment is crucial when reporting randomized trials.8 In our review, only two studies had sufficient description of the study design to suggest that adequate concealment of allocation had taken place.9 Furthermore, all included trials contained significant methodologic bias.We agree with Wheatley et al that systematic reviews that use individual patient data are regarded as more reliable than other forms of systematic review. However, the first step must be having the data from properly designed and reported randomized trials. Pooling the data from different trials with considerable bias may underestimate clinically important evidence from individual studies, thus doing more harm than good.1. Wheatley K, Hancock B, Gore M, et al: Interferon-alpha as adjuvant therapy for melanoma: A meta-analysis of the randomised trials. Proc Am Soc Clin Oncol 20:: 349a,,2001 (abstr 1394) Google Scholar2. Moher D, Cock DJ, Eastwood S, et al: Improving the quality of reports of meta-analyses of randomised controlled trials: The QUOROM statement—Quality of Reporting of Meta-Analyses. Lancet 354:: 1896,1999-1900, Crossref, Medline, Google Scholar3. Thompson SG: Why sources of heterogeneity in meta-analysis should be investigated. BMJ 309:: 1351,1994-1355, Crossref, Medline, Google Scholar4. Deeks JJ, Altman DG, Bradburn MJ: Statistical methods for examining heterogeneity and combining results from several studies in meta-analysis, in Egger M, Smith GD, Altman DG (eds): Systematic Reviews in Health Care: Meta-Analysis in Context ( ed 2 ). London United Kingdom, BMJ Books, 2001 Google Scholar5. Thompson SG: Why and how sources of heterogeneity should be investigated, in Egger M, Smith GD, Altman DG (eds): Systematic Reviews in Health Care: Meta-Analysis in Context ( ed 2 ). London United Kingdom, BMJ Books, 2001 Google Scholar6. Hardy RJ, Thompson SG: Detecting and describing heterogeneity in meta-analysis. Stat Med 17:: 841,1998-856, Crossref, Medline, Google Scholar7. Eggermont AMM: The role interferon-alpha in malignant melanoma remains to be defined. Eur J Cancer 37:: 2147,2001-2153, Crossref, Medline, Google Scholar8. Moher D, Schulz KF, Altman DG: The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 357:: 1191,2001-1194, Crossref, Medline, Google Scholar9. Lens MB, Dawes M: Interferon alfa therapy for malignant melanoma: A systematic review of randomized controlled trials. J Clin Oncol 20:: 1818,2002-1825, Link, Google Scholar