When Bad Things Happen to Good Studies
Authors: Judith B. Manola, Robert J. Gray
Journal: Journal of Clinical Oncology (2011)

Watchingoveraclinicaltrialduringitsconceptionanddevelopment is not unlike being a parent. The principal investigator is constantly surveying the horizon for potentially hazardous situations, seeking to protect delicate offspring from peril. The perils that might assault a clinical trial and prevent it from developing into a fully mature publication include poor accrual, lack of power, and unseen factors that bias the results toward the null. Preventing these bad thingsfromhappeningtogoodstudiesinthecontextofheadandneck cancer is particularly challenging. Inthearticlethataccompaniesthiseditorial,Roseetal 1 reportthe results of a Surveillance, Epidemiology, and End Results/Medicare analysis that points out one of these perils: biasing results toward the null. The study included 34,568 patients with nonmetastatic squamous cell carcinoma of the head and neck diagnosed between 1994 and 2003. The authors point out that the impact of competing mortality can tip the balance of an otherwise feasible trial toward negative results. As improvements in cancer therapy lead to increasingly prolonged survival outcomes and patients with earlier stage disease are studied, the issue of competing risks becomes more significant. Areviewofrandomizedstudiesinheadandneckcancer 2-19 with afocusonphaseIIIstudieswithoverallsurvivalendpointsprovidesa senseoftheextenttowhichtheseparentalconcernsaremanifested.In Table 1, we see three studies 3,4,7 that were stopped early as a result of pooraccrual.Thedifficultiesofenrollingpatientsontocomplex,multimodalitytherapies,thefrustratinglyslowpaceofstudyapprovaland activation, and the relative scarcity of patients with disease all make it essential to design studies that are as parsimonious as possible with respect to patient resources. Additionally, although publication bias may prevent us from providing a comprehensive list, we note studies that were powered at 80% and failed to demonstrate a survival improvement. Perhaps with more robust statistical power, these would have demonstrated some activity—we will never know. We also note in several cases that the number of deaths unrelated to disease or treatmentcouldbeinfluencingthestatedeffectsize.Understatistically ideal circumstances, the death rate on the control arm will be lower than the death rate on the experimental arm because the new treatment is superior. However, if patients on both arms are dying from othercausesataratethatisunrelatedtotreatment,theobserveddeath rate will be a mixture of deaths from cancer and deaths from other causes, and the hazard ratio will be a mixture of the cancer-specific hazard ratio and the deaths from other causes hazard ratio (presumably a hazard ratio of 1). We have several recommendations for improvement. First, we must be thoughtful in designing studies. We must make the best possible effort to understand the population under study. Rose et al 1 provide us with valuable insight into the risks of competing comorbidities in patients with head and neck cancer who are comprised in our study populations. We also need to make sure we are designing studies using realistic effect sizes and well-documented hazard rates forcontrolarms.Then,wecanusemethodssuchasthosedescribedby Lachin and Foulkes 20 to account for what can be thought of as crossoverofpatientsontheexperimentalarmwithcomorbidconditionsto the control arm. By adjusting power and sample sizes to account for these competing risks, we can protect against having these events swamp a true treatment effect.