Reviews: Making Sense of an Often Tangled Skein of Evidence

Medical science is an intricately detailed tapestry of evidence, woven over many decades, from tens of thousands of hypotheses and observations. Astute observers rarely discern the tapestry's pattern from single studies; they know that a single study seldom allows people to see a pattern that changes insight and clinical practice. Far more often, medical practice changes only after a body of evidence accumulates to the point of revealing meaningful patterns. Recognizing the central role of evidence synthesis in shaping standards for clinical practice, Annals places a high priority on publishing up-to-date summative articles. This editorial offers advice for review artisans who hope to see their wares on display in Annals. In an accompanying supplement sponsored by the Agency for Healthcare Research and Quality (part 2 of this issue), authors from the North American Evidence-based Practice Centers offer detailed advice on how to do systematic reviews, how to compile evidence reports, and how to handle the many methodologic challenges involved in writing systematic reviews. Reviews that appear in Annals are either systematic or narrative. Systematic reviews usually address a small number of focused questions and use rigorous, structured methods to search, select, evaluate, and synthesize the existing evidence to answer these questions. Some systematic reviews use meta-analysis to combine the findings of individual studies to estimate the true relationships between the exposures or interventions and outcomes of interest. The accompanying supplement further defines systematic reviews. Narrative reviews are more difficult to define, but we know one when we see it. These reviews can be easy to read but also deceptively simplistic in their treatment of the evidence. They usually address broad topic areas rather than a few tightly formulated questions. Narrative reviews often do not have a Methods section to describe the process of selecting the studies that the author discusses in the text. Weak narrative reviews focus on the results of cited studies without describing their design, strengths, or weaknesses. The primary aim of a review article is to educate. Systematic reviews educate by describing the evidence to the reader. Narrative reviews tend to lecture. In light of our goal to publish unbiased summaries of current scientific truth, Annals editors like systematic reviews. We believe that an unsystematic approach to selecting, evaluating, and synthesizing the evidence diminishes a review article's chances of including relevant evidence, recognizing shortcomings of the evidence, or distinguishing between recommendations based on evidence and those based largely on opinion. Systematic reviews are more transparent and more replicable than narrative reviews, primarily because of the description of the methods. We appreciate systematic reviewers' use of explicit methods for searching the literature, careful selection and appraisal of the studies, and attention to the quality of evidence as the authors synthesize patterns of evidence into conclusions. However, even systematic reviews can fall short in execution. We are disappointed when systematic reviews simply list the characteristics and findings of a series of single studies without attempting, in a sophisticated and clinically meaningful manner, to discover the pattern in a body of evidence. Although we greatly value meta-analyses, we look askance if they seem to be mechanistically produced without careful consideration of the appropriateness of pooling results or little attempt to integrate the findings into the contextual background. We want all reviews, including meta-analyses, to include rich qualitative synthesis. Both types of reviews are valuable, and as much as we like systematic reviews, we recognize that not every topic is ripe for this approach. Narrative reviews are particularly useful for topics with a fragmentary evidence base, such as emerging clinical issues, rare diseases, or new technologies. So, we advise authors of narrative reviews to strive for a more systematic, more transparent approach to their task. We also urge authors of systematic reviews to strive for a richer narrative synthesis that will speak clearly to clinicians. Although we've drawn rather sharp distinctions between narrative reviews and systematic reviews, we believe that authors of either type of review can benefit from the following guidelines. Ask important, well-formulated questions, and organize the review around answering them. When framing important questions, think about relevant patient populations, exposures (whether it is an intervention, a risk or prognostic factor, or a diagnostic test), comparison groups, and outcomes. Resist the urge to review everything ever published on a topic. A convincing discussion of a few facts is far more valuable than a superficial discussion of a dizzying number of features of a topic. Include a Methods section. It should contain details on bibliographic sources, dates of literature searches, types of studies that are most appropriate to address each question, and the critical appraisal and synthesis processes for evaluating and summarizing studies. Design the search and selection of articles so that the process yields the types of evidence best suited to answer the questions that are the focus of the review. Disclose potential conflicts of interest. Bias due to conflicts of interest is particularly hard to spot in a narrative review, since authors typically do not describe their methods for selecting evidence. All authors should strive for balance in forming conclusions. Use tables to show key features of studies and to facilitate comparisons. Critique the quality of the literature. Do so consistently throughout, so that readers have a clear understanding of the strengths and weaknesses of studies. Avoid using quality scores that aggregate several features of an article into a single measure. Aggregated quality scores can obfuscate rather than clarify. Strive for a judicious synthesis. Do not simply list the results of individual studies without taking great pains to synthesize the body of evidence. Take into account such variables as study design, study populations, tested strategies, and outcomes. Well-synthesized reviews summarize data with attention to amount, type, strength, and patterns of evidence (for example, 10 small cross-sectional studies conducted primarily in referral clinics showed that Although most studies among older adults showed positive associations, the magnitude of these associations were all small and of uncertain clinical significance). Good syntheses also explain data inconsistencies and conflicts. Take care when combining studies. Use clinical judgment and critical appraisal of studies to help determine whether quantitative methods are appropriate to help summarize data, estimate a common effect, or evaluate heterogeneity across studies. Clearly justify and describe any quantitative methods. Avoid the temptation to combine weak, heterogeneous studies solely to calculate a single, apparently precise but potentially misleading summary estimate: Faux summary estimates are potentially misleading and harmful. Summarize key take-home points carefully yet succinctly to help busy readers grasp the bottom line. When offering recommendations for clinical care, specify whether the recommendation is based on high-quality evidence, limited evidence, or expert opinion alone (for example, Few studies directly address this issue, but the authors' experience is that patients seem to do best when). Refrain from making specific recommendations that you cannot support with good evidence. We hope that authors of summative articles of any kind use the practices outlined in this editorial and that the accompanying supplement discusses in detail. We also hope that readers look for these features when deciding whether a review is sufficiently sound to justify applying its conclusions to their own practice. Clinicians, patients, and policymakers count on those who review the literature to weave the often tangled skein of evidence into a vivid tapestryor at least a usable piece of cloth.