On Lumping, Splitting, and the Nosology of Clinical Trial Populations and End Points

Published Ahead of Print on February 18, 2014 as 10.1200/JCO.2013.54.4429 The latest version is at http://jco.ascopubs.org/cgi/doi/10.1200/JCO.2013.54.4429 J OURNAL OF C LINICAL O NCOLOGY On Lumping, Splitting, and the Nosology of Clinical Trial Populations and End Points T O THE E DITOR : Wildiers et al 1 espouse conventional wisdom in reaching the conclusion that “all clinical trials in oncology should be without an upper age limit.” We agree greater represen- tation of elderly patients in clinical trials is needed but disagree with this conclusion, and feel two premises should be challenged: 1. “Clinical Trials Need to be Representative of the Whole Population in Whom the Treatment Will be Used Later” 1(p3715) Ideally, clinical trial populations should represent those in which its findings are applied; however, this is not an unqualified need. Clinical trials naturally create selective, idealized conditions that ne- cessitate extrapolating their findings to a wider population than rep- resented by their participants. 2 Furthermore, evidence that outcomes differ as a function of cancer trial participation is weak. 3 The authors emphasize that excluding elderly patients limits generalizability of treatment, and extrapolating findings from younger populations risks overutilization of expensive, ineffective treatments. However, under- utilization of effective treatments is as much a concern as overuti- lization of ineffective treatments. They consider relaxed inclusion/ exclusion criteria “the price society has to pay if it wants to ensure that older patients are not subjected to toxic therapies that provide no tangible clinical benefit.” 1(p3712) Although this seems correct, it side- steps an inconvenient truth: this can substantially increase trial costs, without greatly augmenting knowledge. Including more patients unlikely to benefit from treatment increases sample size, without necessarily increasing power. 4-8 In resource-constrained settings, op- timizing allocation strategies becomes paramount. 9 Using age as a selection factor does not imply exclusion from clinical trials generally. Different trials can be tailored to address specific patients’ needs, which might be more consistent with personalized medicine. Rather than liberalizing entry criteria, investigators should define populations most likely to benefit from treatment, to maximize efficiency and value of information gained. Reproducing trials in elderly populations assuming that results from other trials are nongeneralizable is mis- guided. Forgoing the assurance from such studies might be considered the price we pay to increase the dimensionality of scientific knowledge. 2. “Overall Survival is Considered the Gold Standard in Clinical Trials, Especially When Evaluating the Superiority of New Treatments” 1(p3712) Wildiers et al 1 do not acknowledge that overall survival (OS) is itself a composite end point, comprised of competing causes of death. They admit composite end points are justified when “the expected effects on each component are similar based on clinical/biologic plausibility.” 1(p3713 ) This condition is generally unrealistic with re- spect to OS, because typically, if a cancer therapy is to be beneficial, it Journal of Clinical Oncology, Vol 32, 2014 C O R R E S P O N D E N C E should reduce mortality from cancer, not other causes, and we hope this benefit is not offset by increased treatment-related or competing mortality. Previously, we have shown that effects on a composite end point can be interpreted as the weighted average of effects on cause- specific events. 4 As such, it is problematic when composite effects, but not cause-specific effects, are not designated explicitly, since an effect on cancer mortality or noncancer mortality tells us nothing about the effect on either outcome. Contrary to Wildiers et al’s statement that using composite end points increases statistical efficiency, their use can predispose studies to both type I 10-11 and type II error 4-8 : type I error, because a positive result may be attributable in whole or part to an effect that is inconsistent with the treatment mechanism; type II error, because a negative result may be attributable to overestimating power in the presence of competing risks. Predisposition to both type I and type II error is the calling card of a bad model, leading us to think “OS as the gold standard” 1(p3712) should go the way of the actual gold standard, formally abandoned in 1976. 12 An approach the authors briefly address, which we advocate, is to consider coprimary end points in competing risks settings. Whether type I and/or II error must be adjusted for multiple testing depends on the framework applied. For example, coprimary end points can be aggregated, assigning type I and II error for the composite end point, with cause-specific effects designated explicitly. 4,13 Wildiers et al 1 im- ply that for this approach to be valid, a precondition “is that cause of death can be reliably ascertained.” If ascertainment were biased, we agree that would be a problem, but if it were just imprecise, we are not convinced this precondition is necessary, 14 and fear it will discourage investigators from determining cause of death when it might be chal- lenging, but not impossible. A concern with composite end points is that effects on the end point may be driven in part or whole by one of its components. 15 We agree that disease-specific event probabilities should be reported, but would add that competing event probabilities, and effects on competing events should be reported, to safeguard against publication bias. Since it is not typically reasonable to hypoth- esize that a cancer therapy will favorably affect noncancer mortality, we should view claims that treatments improve survival in the elderly with utmost scrutiny, particularly when the mechanism for this ben- efit cannot be evinced using clinical data. The authors list increased sample size among the limitations of coprimary end points; in con- trast, we consider this a justifiable cost to verify the mechanism by which beneficial effects are achieved. Specificity is the hallmark of every science, and one we should aspire to in clinical oncology. In general, we believe readers should be wary of rules for defining trial populations and end points without reference to a specific question. Age limits should not be off-limits for clinical trials. If one’s hypothesis is that effects are homogeneous with respect to age, then it makes no difference if elderly patients are included, save concerns regarding efficiency. If one’s hypothesis is that effects are heterogeneous, then elderly subpopulations could be stud- ied separately. Generalizability in geriatric subpopulations is most likely to be a significant concern when there is reason to expect the cause-specific treatment effects (not composite effects) would differ © 2014 by American Society of Clinical Oncology Information downloaded from jco.ascopubs.org and provided by at University of Ca San Diego on February 19, 2014 from Copyright © 2014 American Society of Clinical Oncology. All rights reserved. Copyright 2014 by American Society of Clinical Oncology