The value of observational research in liver diseases
Authors: Rachel A. Lindor, Keith D. Lindor
Journal: Hepatology (2010)

Hepatology recently published a study by Feuerstadt et al.,1 who reported a 3.3% effectiveness rate for hepatitis C treatment in an observational study; this is a stark contrast to the 56% to 63% efficacy rates reported for the same treatment regimen in the setting of randomized controlled trials (RCTs). Such comparisons have contributed to the growing awareness that real-world data derived from observational studies often vary widely from those data derived from the controlled settings of clinical trials. Contemporary methods for producing this type of real-world data can be performed with data from specialized registries or existing administrative and claims information and include a variety of designs, such as case-control, cohort, and cross-sectional studies, in which patients are not randomly assigned to treatment groups. A confluence of factors over the last decade has elevated the legitimacy of these observational studies, especially for the purpose of understanding how real patients are affected by our interventions. Specific provisions in the recent health care reform bill highlight the importance of this type of information and suggest that observational data will play an increasingly important role in the shift toward the production of more efficient and relevant evidence. RCT, randomized controlled trial. RCTs, first introduced to the medical literature in 1948, have long been considered the gold standard of clinical research.2 The rapid ascent of RCTs to the top of the evidence hierarchy was aided by statisticians such as Archie Cochrane, namesake of the Cochrane Collaboration, who recognized the ability of highly controlled, prospective trials to avoid certain biases inherent to the traditional chart review–based research methods. For example, randomization of patients eliminates allocation bias, and blinding of participants avoids ascertainment bias. Despite these advantages, RCTs do have a number of well-recognized limitations. Most prominently, the results of these trials are often not generalizable because of their reliance on rigid protocols and strict exclusion criteria. As a result of artificially intense follow-up or broad exclusion criteria, RCTs provide limited information on how treatments will perform when they are applied in real-world settings to diverse groups of patients. As we begin to understand the effects of individual variations and social circumstances on health outcomes, the inability of RCTs to explore these influences will further limit their utility. In addition to the issue of generalizability, many RCTs suffer from being excessively slow and expensive: they often require half a decade and tens of millions of dollars. The delays required for enrollment and data collection prevent patients from being able to take advantage of new treatments and leave product sponsors with less time to recoup their development costs; this contributes to the high prices of new drugs and devices. As a result, the reliance on RCTs often means that fewer patients have access to more expensive treatments. Despite these shortcomings, RCTs are still touted as the preferred form of research by a number of influential bodies, such as the US Preventive Services Task Force, the Agency for Healthcare Research and Quality, and the World Health Organization.3, 4 The preference for RCT data over observational data can be at least partially attributed to a number of large-scale comparisons of the two trial types in the 1980s. The disparate results reached by these two methods led researchers to conclude that observational data were "irretrievably skewed" because of their vulnerability to the biases that RCTs are designed to avoid.5 However, today's observational studies have little in common with those early predecessors. Advances in statistical modeling enable researchers to correct for the biases that were once thought to be inseparable from observational studies.6 In addition, the concurrent expansion of electronic medical records has given researchers the ability to comb increasingly large pools of data with more efficiency than was possible with paper records. Furthermore, because observational studies do not apply the same exclusion criteria as RCTs, they can be used in more diverse populations and settings, and they deliver more generalizable data as a result. Observational methods also have the potential to enhance researchers' understanding of rare diseases. By pooling data from multiple sources, observational studies can provide more information about a broader spectrum of patients, even when those patients are separated by space and time. Within the field of hepatology, this has the potential to improve our understanding and treatment of patients affected by a number of uncommon conditions, such as Wilson's disease, hemochromatosis, and primary biliary cirrhosis. Observational studies are also potentially faster and less expensive than traditional trials. More than 90% of industry-sponsored RCTs are slowed by enrollment delays, which have contributed directly to a time-to-approval increase of 70% over the last decade.7 In contrast, observational research is often able to rely on retrospective data analysis, which completely bypasses the enrollment hurdle. Once a trial is underway, observational methods are able to take advantage of larger data pools and, therefore, reach the statistical power to draw conclusions much sooner. The consequences of this ability can be profound; it is estimated that the problems leading to the withdrawal of Vioxx, which occurred 65 months after the drug was placed on the market, could have been identified within just 3 months with the observational tools currently being developed by the Food and Drug Administration.8 The concept that observational studies should play a larger role in research is not a new one, and much of the infrastructure is already in place. In 2003, for example, the Agency for Healthcare Research and Quality invested in the development of the Developing Evidence to Inform Decisions About Effectiveness (DEcIDE) network, a computerized record system designed to provide observational data for a number of common conditions. In 2007, the Food and Drug Administration called for the creation of the Sentinel System, a computerized surveillance tool intended to quickly identify medication-related adverse events through the observation of outcomes in real-world patients. The Veterans Affairs Health Care System and the HMO Research Network maintain databases with more than 8 million and 10 million patient records, respectively.9 In addition, Medicare provides incentives for providers to adopt electronic medical records by 2011, and this promises further expansion of the capabilities of administrative databases.10 The need to recognize the advantages of observational research and to embrace new applications for observational data is magnified by the recent establishment of the Patient-Centered Outcomes Research Institute, which will soon be directing an estimated 500 million federal dollars annually to comparative effectiveness research.11 Under this umbrella, policy makers will determine, among other things, what types of questions can be explored by observational studies, whether those questions are best answered with administrative databases or clinical registries, and whether the studies can be based on retrospective data or will require prospective collection. These determinations cannot be made without the support and involvement of clinicians in each field. While these investments are waiting to be made, the broader use of observational methods can and should be embraced now. Such reliance on observational data would not be premature because observational studies already account for the majority of published research. Notably, payers in the United States have recognized the value of these studies and are beginning to use observational data in making coverage determinations. Medicare's ability to make decisions after treatments are observed in practice, officially outlined in 2006, is known as "coverage with evidence development." Under this policy, Medicare decisions can be based on either the enrollment of patients in prospective clinical trials or patients' participation in registries. Although this policy has been controversial and has been used infrequently to date, it has recently been advanced by former administrators of Medicare, promoted by the Institute of Medicine, and proposed to the Medicare Payment Advisory Commission; this suggests that this policy and the real-world data on which it relies will play a larger role in the future of research.12-14 The need to reconsider the status of RCTs as the gold standard of clinical evidence will grow more pressing as new treatments are developed, resources become increasingly limited, and patients continue to demand more personalized care. As a field, hepatology stands to gain significantly by embracing the use of observational studies. Not only do observational methods enable researchers to cast a wider net in the study of rare diseases, these methods also give them an opportunity to understand why treatments for even the most common diseases often fail to work as promised. Only by revealing the discrepancies between real-world and controlled data will we be able to identify the patients most likely to be left behind and to begin to address the factors underlying these differences. Although restructuring the traditional evidence hierarchy and building the infrastructure for observational studies will take a concerted effort from multiple parties, the result will be one of the greatest rewards of clinical medicine: better care for our patients.