Grading Quality of Evidence and Strength of Recommendations: A Perspective

Evidence-based practice requires translating research findings into clinical and policy decision making. Clinical practice guidelines (CPGs) serve this purpose by evaluating evidence and making recommendations about therapeutic and diagnostic interventions and clinical management strategies. Systematic reviews are considered the best available evidence and are often used in the development of CPGs [1,2]. Since guideline development involves an assessment of the overall quality of evidence and complex balancing of trade-offs between the important benefits and harms of any given intervention, arbitrariness, value judgements, and subjectivity ultimately come into play in the guideline development process and associated recommendations [3]. In order to minimize cognitive bias in interpreting evidence and make the inherently subjective process more transparent and consistent, CPGs have traditionally employed formal systems or frameworks to understand and grade the quality of the body of evidence and strength of recommendations [4,5]. One such framework is the grading quality of evidence and strength of recommendations (GRADE), which is commonly used by guideline panels in deriving health care recommendations. GRADE was developed to overcome some of the deficiencies of earlier efforts [6]. GRADE defines the quality of evidence as the collective level of confidence guideline developers have about the validity of estimates of benefits and harms for any given intervention, and the strength of guideline recommendation as the extent of collective confidence that adherence to the recommendation will do more good than harm [7]. It urges guideline developers to consider all important patient outcomes of benefit and harm, to systematically evaluate the quality of their estimates, and to assess the trade-offs between evidence of benefits and harms, the preferences and values placed by patients on outcomes, the opportunity cost associated with the recommendation, and the feasibility of recommendations given a clinical setting before formulating guideline recommendations. Details of the GRADE approach have been published elsewhere [8]. In a new Policy Forum published in this issue of PLoS Medicine, Kavanagh [9] questions the external consistency of the GRADE framework by comparing the Surviving Sepsis Campaign (SSC) guideline recommendations developed in 2004 and updated in 2008. Moreover, Kavanagh expresses his concerns on the processes of the GRADE development and its formal validation. Had we likened the GRADE approach to an instrument or a health profile built on discrete logic to capture evidence, we would have concurred with some of Kavanagh’s criticism of GRADE. However, we see GRADE as a framework uncovering implicit subjectivity and invoking a systematic, explicit, judicious, and transparent approach to interpreting, as opposed to ‘‘capturing’’ evidence. It reveals how values are assigned to judgments, but what values are assigned it does not dictate simply because it cannot dictate. Below we first present our concern about one aspect of the GRADE framework and then our perspective on the various criticisms of it.