Randomized trials in surgery

IN 1996, THE LANCET PUBLISHED A COMMENTARY “Surgical Research or Comic Opera: Questions, but Few Answers.”1 The author reviewed the first issue of 9 general surgery journals in 1996 and found that only 7% of papers reported data derived from a randomized trial, whereas 46% were based on case series. A decade later (2006), we conducted a similar survey and found that the relative percentage of case series, operationally defined as a single institution’s experience with a single procedure, had decreased to 34%. At the same time, we found that the relative number of cohort studies (51%) and the relative percentage of articles reporting data derived from a randomized trial (10%) remained relatively similar. Such a review raises several interesting questions about contemporary surgical research. The main issue that stands out is this one: Why the number of randomized trials, which had been identified as being low compared with other fields of medicine in the mid-1990s, has not increased over time in an environment where the demands for rigorous evidence have grown? In addressing this question, it is clear that different research designs---ranging from case series to randomized trials to meta-analyses---are needed for the evaluation of surgical procedures. The appropriate choice of research design depends on the effect size of the intervention, the specific clinical circumstances, the relevant clinical question, and the stage of the life cycle in which a surgical procedure is evaluated. Case series, for example, are valuable in rare diseases, whereas prospective single-arm studies can provide important insights if the natural history of a disease is known and the outcome is unequivocal. Moreover, the first stage of clinical evaluation for a novel procedure is usually a single-arm prospective study in a diseased population to assess the feasibility of its use in humans. These studies, which are generally small in size, provide preliminary evidence about short-term safety and efficacy. The major value of these studies is that they offer easily obtainable prima facie evidence for designing large-scale, pivotal trials that are randomized or otherwise controlled. However, our review suggests that such large-scale confirmatory trials are not always carried out, especially randomized studies. The advantage of randomized trials is well known; they offer a powerful means to evaluate the efficacy of procedures in that comparison groups are equally constituted with respect to known and unknown predictors of outcome. As such, these trials offer an unbiased assessment of the impact of a procedure. So why are such trials not more prevalent in surgery? We believe that such trials can be more difficult to design and conduct for surgical procedures than for new pharmaceutical agents. Moreover, because surgical innovation is characterized by a high degree of incremental change and heavy dependence on technical skill and experience, statistical inference from trial data is often more challenging. In this analysis, we first explore the methodological, ethical, and logistic challenges posed by conducting randomized trials of novel surgical procedures. We review several options for addressing these challenges in terms of trial design and analytic techniques. We then discuss issues involved in the interpretation of clinical trials and the value of so-called pragmatic randomized trials. We conclude by evaluating some of the larger infrastructure issues that need to be addressed before a robust, surgical clinical trial enterprise can be created.