How Systematic Reviews Can Foster Evidence‐Based Clinical Decisions

Field: Health Professions (Health Sciences)

Health care providers increasingly recognize the importance of making decisions based on the bestpossible evidence. This evidence includes both research- and practice-generated data. Despite substantial financial investment in extensive research (National Institutes of Health 2007), some areas of practice face the problem of having little or no research to guide decision-making, while others are inundated with articles reporting on research developments. Growth in health-related research has led to 20,000–30,000 health and medical journals that annually publish about two million scientific articles and add 500,000 citations to Medline (Pang et al. 2004). On top of keeping up with the sheer volume of reports, other difficulties arise. For example, a multiplicity of methods to summarize and synthesis study results exists, making reliable overall conclusions difficult. This contributes to practitioners, who seek an overall picture of what the best available research supports or refutes, encountering difficulty finding or interpreting existing research. In the next two columns, we will address how to use information from high-quality systematic reviews in making clinical decisions. For a single practitioner to find and organize all that is known about a topic is impossible. Clinicians need a method of gaining a synopsis of what is known about a topic; therefore, organizing and synthesizing these data is imperative. A librarian named Gould painted his vision of such organization and synthesis this way, “I look forward to such an organization of the literary records of medicine that a puzzled worker in any part of the world shall in an hour be able to gain the knowledge pertaining to a subject of the experience of every other person in the world” (Clarke 2005). This vision was described in 1898; however, only recently have the techniques developed that make it possible to both locate the research findings quickly and then make them available to anyone in any part of the world. While the advent of the Internet has made huge amounts of information available, it still must be translated into more user-friendly formats that busy clinicians can use practically. In combination with technological advances, methodologies, such as systematic reviews, have evolved to address all of what we know about an issue, making research results available in an accurate, concise, and timely fashion. The administration of antiarrhythmic drugs after a heart attack demonstrates one of the benefits of systematic reviews. This class of drugs was used widely during the 1970s and 1980s. Several clinical trials were carried out on their effectiveness and safety. Some trials showed more benefit than others; other trials raised concerns about adverse effects. Debate raged among different medical experts, federal regulators, and pharmaceutical companies (Moore 1995). How should the drugs be regulated and for which patients should they be prescribed? Each focused on controlled studies that supported one or the other side of the argument. Each had reasons to give more or less weight to other studies with opposing findings. Understandably, each had a view and found evidence to support it. What was needed was a systematic review of all the evidence to provide an objective summary of effectiveness. Such a systematic review of the 20 clinical trials to date was published in 1983 by Furberg. When all the studies were examined together, more than 6,000 patients had participated. Not only did the review show that there was no evidence of the claimed benefits of these drugs, but it also highlighted an increase in mortality among patients taking the experimental drugs. However, since all the patients had suffered a heart attack prior to being given the drugs, they were already at risk of further heart attacks that could lead to death. It was not clear if the drugs were to blame for these deaths. While further research was still needed, action was required on the knowledge generated by this synthesis. Systematic reviews summarize the research already conducted. They also play an important role in directing future research. Furberg's (1983) systematic review of antiarrhythmic drugs pointed out a number of limitations in the individual studies that had been conducted to date. It also demonstrated that a placebo-controlled trial was urgently needed to demonstrate whether the deaths were due to the research subjects' underlying heart problems or due to the drugs. Tragically, such a placebo-controlled study was not reported until 1989. Before that, researchers had conducted more than 50 clinical trials involving over 23,000 subjects and doctors had prescribed the drugs to hundreds of thousands of patients. That study clearly showed that more people died in the groups receiving the antiarrhythmic drugs than died in the control groups (Cardiac Arrhythmia Suppression Trial Investigators 1989). One estimate claimed that more people in the United States died from these drugs than were killed in combat during the Vietnam War (Moore 1995). Part of the problem was that each individual clinical trial did not systematically review previous literature. If an “all we know” section (i.e., a systematic review) had been included in these studies, growing trends of serious adverse events could have been identified. Governmental and private agencies have identified the need to readily provide this kind of synthesis to enable clinicians to make clinical decisions on the best available evidence. The Agency for Healthcare Quality & Research (AHRQ) in the United States has established evidence-based practice centers (EPC) whose mission is “synthesizing scientific evidence to improve quality and effectiveness in health care” (Agency for Healthcare Quality & Research 2007). Since 1997, AHRQ has launched 25 EPCs and has produced over 150 evidence reports using systematic review methodology. These evidence reports are available free of charge and can be accessed at http://www.ahrq.gov/clinic/epc/epcseries.htm. In 1996, the Joanna Briggs Institute (JBI) began in Australia with the mission to improve “the health status of the global population through the delivery of health care that is based on the best available evidence” ( Joanna Briggs Institute 2007). Among other initiatives, JBI develops “methods to appraise and synthesise evidence, conducting systematic reviews and analyses of the research literature (evidence translation)” (Joanna Briggs Institute 2007). JBI and its associated collaborating centers and evidence synthesis groups use both the JBI and the Cochrane Collaboration's systematic review methodologies for their review process and have produced over 40 systematic reviews. A subscription to JBI is required to access these systematic reviews, although many have been published in the International Journal of Evidence-Based Healthcare. The Cochrane Collaboration is the grandfather of these types of organizations. It is named after A.L. Cochrane, a British epidemiologist who criticized the medical profession in the 1970s for not having a systematic method of combining the results of similar randomized controlled trials (Cochrane 1972). Officially launched in 1993, the Cochrane Collaboration is an international, non-profit, independent organization with a mission “to help people make well-informed decisions about health care by preparing, maintaining and promoting the accessibility of systematic reviews of the effects of healthcare interventions” (Cochrane Collaboration 2007). The Cochrane Library includes the Cochrane Database of Systematic Reviews, which houses over 3,000 systematic reviews and almost 2,000 protocols (reviews in the making) conducted by The Cochrane Collaboration. The methodology used to conduct these reviews has been the gold standard for systematic reviews of randomized controlled trials and is described in the Cochrane Handbook (Higgins & Green 2006). Methods for reviewing other types of studies are also being developed for Cochrane reviews. Access to these reviews requires a subscription to the Cochrane Library, although many organizations and even countries are taking out subscriptions to make the information more readily available (see Table 1). A systematic review is a way of summarizing the results of research studies in a format that gives a critical assessment of the efficacy and safety of the issue being reviewed. The objective of a systematic review is to provide summary information that helps clinicians to make decisions about health care interventions based on the best evidence available. Cook and colleagues (1997) indicate that systematic reviews are designed to assist clinicians to determine what can be reliably known from the health care literature by synthesizing bodies of evidence. The benefits of the systematic review methodology include limitation of bias and increased confidence in conclusions across all existing studies that address a specific clinical question. Too many examples exist where interventions continued to be used when sufficient evidence existed to demonstrate that those interventions were harmful (the antiarrhythmic drug example) or not effective. In other cases, practitioners delayed introducing interventions into their practices because they were unaware of the results of several studies supporting their effectiveness and safety. The Cochrane Collaboration has developed a rigorous method for conducting systematic reviews of health care interventions and promotes the use of systematic reviews in practice. The organization also encourages further development of systematic review methodology to improve its reliability and applicability. Cochrane defines a systematic review as: A review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyse data from the studies that are included in the review. Statistical methods (meta-analysis) may or may not be used to analyse and summarise the results of the included studies. (Kitchener 2005, p. 45) This definition assists in distinguishing systematic reviews from narrative reviews. A narrative review often develops when an expert in the field is asked to summarize what is known in the area. These reviews can be helpful, but they also have some limitations (i.e., potential bias) that systematic reviews seek to overcome. For example, narrative reviewers typically do not describe how they conducted their searches or how widely they searched for research results. Their criteria for including or not including certain studies are usually not given. Also missing can be their criteria for judging the quality of included studies. The steps in a systematic review (see Table 2) are designed to reduce the ways that different biases can influence study findings and the reviewers' conclusions. Another term that frequently arises in the context of systematic reviews is “meta-analysis,” which is the statistical combination of the results from two or more separate studies (Higgins & Green 2006). A systematic review may include a meta-analysis, but the terms are not synonymous. A systematic review always evaluates the data from primary studies. However, it may or may not be possible to combine the data from different studies. If the data can be combined, the results can be similar to producing results from a larger study. When the reviewers conclude that it is appropriate to combine the data from different studies, several statistical techniques can be used depending on how the data from the studies are reported. When data are not able to be statistically analyzed, systematic evaluation of the studies (often called integrative reviews) can lead to valuable conclusions as well. It is important for educators to know how to assist clinicians to be able to distinguish among the types of reviews of research and to best choose what will help them to answer their clinical questions. The Cochrane Handbook describes both meta-analysis and integrative review techniques designed to minimize bias and produce the highest quality systematic reviews. Some of these statistics and techniques will be described in more detail in the next column, which will address the process of conducting a systematic review and critical appraisal of systematic reviews.