Apply sound scientific methodology to the evaluation of new medical treatments
Authors: Rachel A. Lindor, Keith D. Lindor
Journal: Hepatology (2011)

We appreciate the response to our January 2011 editorial in HEPATOLOGY1 and welcome the opportunity to continue the discussion about the relative roles of observational studies and randomized clinical trials (RCTs). Importantly, we would like to reiterate our support for broader use of observational research as a complement to traditional clinical trials, not as a replacement. We agree with Wiesner that there are some issues, such as questions of basic efficacy and safety, that are best evaluated in the setting of an RCT.2 However, we also recognize that the controlled setting inherent to RCTs limits their utility in evaluating measures of real-world effectiveness, which can be significantly affected by patients' social circumstances and biological variation. In addition, steadfast adherence to traditional RCTs hinders the evaluation of a number of rare diseases, for which logistical barriers make it difficult to accrue sufficient numbers of patients within the confines of a traditional clinical trial. Wiesner's response includes a number of examples in which treatments based on observational studies were later demonstrated through clinical trials to be ineffective or harmful. However, in each of these cases—including high dose chemotherapy with bone marrow transplant for breast cancer, estrogen replacement for menopause, erythropoietin for anemia of chronic kidney disease, and early stage hemodialysis—initial questions about the effectiveness of the interventions were clouded by conflicting RCT data as well. Therefore, these examples demonstrate the potential fallibility of both types of trials and speak more to the need for careful trial design rather than the superiority of a given trial type. Similarly, the criticism of recalls by the Food and Drug Administration warrants further scrutiny. Wiesner is correct in noting that more than a hundred devices were recalled by the Food and Drug Administration after having been cleared for marketing without undergoing clinical trials. However, the same study referenced by the article he cites actually shows that devices evaluated by RCTs prior to marketing were far more likely to be recalled than devices that went through a less stringent evaluation pathway, again illustrating the limitations of traditional clinical trials.3 Together, the two sets of examples provided by Wiesner reflect many of the common criticisms of observational studies. At the same time, however, these examples also illustrate the limitations of traditional clinical trials, which often arrive at conflicting conclusions and fail to capture sufficient data to ensure patient safety. We do not try to identify here where RCTs or observational studies are more appropriate, but rather hope to prompt further discussion about the need to reconsider the evidence base on which we make clinical decisions. In the future, both observational studies and RCTs are sure to influence clinical practice, and the sooner we understand their respective strengths and weaknesses, the more effective our care will be. Rachel A. Lindor*, Keith D. Lindor M.D.*, * Division of Gastroenterology and Hepatology, Mayo Clinic, Rochester, MN.