Assessing Appropriateness of Coronary Angiography: Another Step in Improving Quality

Coronary angiography has been the principal means of assessing coronary disease in patients who may need coronary revascularization (1). Introduced in the 1960s, the test has become one of the most frequently performed procedures in the United States. The American Heart Association estimates that more than 1.3 million procedures were done in 2005, an increase of 342% since 1979 (2). With the phenomenal growth of this expensive test and the small but significant morbidity and mortality associated with it, many have questioned the appropriateness of the present rates of use of the procedure. Over the past 20 years, studies have shown that use of coronary angiography varies widely (3). Overuse is somewhat easy to determine, because registries of patients who have undergone catheterization provide the denominator for calculating the proportion that was appropriate. Underuse is more difficult to assess, because procedural databases do not record patients who need the procedure but do not get it. Measuring appropriateness became easier in 1986, when RAND Corporation investigators used a modified Delphi process to develop appropriateness criteria for medical and surgical procedures (4). The RANDUniversity of California at Los Angeles appropriateness method uses an expert panel to review the evidence for the benefits and risks of a procedure, rates the appropriateness of doing the procedure in many clinical situations, and develops consensus about which situations are the most appropriate indications for the procedure. Deciding on appropriateness criteria is similar to developing practice guidelines but differs in several important ways. The guideline process relies heavily on randomized clinical trial evidence and uses expert opinion only when good evidence is not available. The American College of Cardiology/American Heart Association (ACC/AHA) guidelines committee uses a particularly rigorous guideline development process (5, 6). However, translating an ACC/AHA guideline into a tool for assessing qualitya practice measurerequires the second step of developing performance criteria. Most cardiac conditions have just a few performance measures, because less than one third of the guideline recommendations are supported by randomized trial and registry data. Practice measures have another serious limitation: The supporting evidence comes from randomized trials that deal with cut-and-dried situations, whereas most real-life clinical scenarios are complex. In contrast to guidelines, appropriateness criteria rely much more on expert opinion and can take into account the myriad factors that influence clinical treatment decisions. Although appropriateness criteria are more applicable to a wide variety of clinical situations, they reflect the evidence as filtered by expert clinical opinion, incorporate the biases of the expert panel members, and lack a strong direct link to clinical trial data. These limitations might make them less reliable in predicting outcomes and reduce their credibility for assessing quality. In this issue, Hemingway and colleagues (7) assess the reliability and validity of patient-specific appropriateness criteria for coronary angiography in patients with suspected stable angina pectoris. This group previously presented results of the ACRE (Appropriateness of Coronary Revascularization) study (8), which showed that not having a revascularization procedure when it was indicated was associated with worse outcomes. In the current ACRE study report, Hemingway and colleagues measured agreement between appropriateness ratings developed independently by two 11-member panels. Each member of each panel judged appropriateness of coronary angiography for 338 indications on a 9-point scale. The panels then met independently and developed a consensus rating for each indication. The panels' ratings agreed only moderately well (= 0.58) and were worse for indications that did not include exercise electrocardiography (ECG) results. The authors then determined the appropriateness of angiography in 9356 patients with suspected angina pectoris done in 6 centers in England and measured outcomes at a mean of 3 years. More than half of the patients who had appropriate indications for angiography did not have the procedure. Not undergoing coronary angiography when indicated was associated with a 2.5-fold worse composite outcome (cardiac death, myocardial infarction, and acute coronary syndromes). The study provides valuable new insights into the validity and potential application of appropriateness ratings. First, the moderate agreement between 2 independent panels evaluating the same patients raises concerns about using this tool to assess quality. Second, the ratings predicted a subsequent adverse cardiovascular outcome, which confirms that the ratings are clinically relevant. Finally, the study determined that many patients deemed appropriate for angiography fail to receive it and have worse outcomes (7). This last finding is an important message in a climate of concern about overuse of coronary angiography, despite studies showing low rates of overuse and high rates of underuse. Even more disturbing is that the sickest and oldest patients are least likely to receive angiography, a finding that has also been shown for the use of angioplasty in acute coronary syndromes (9). Hemingway and colleagues' study has important limitations. First, the only test results used in the appropriateness criteria were exercise ECG results. Exercise ECG is not useful in many patients because they have conduction delays or resting ECG abnormalities that make interpretation of the exercise ECG unreliable, or they cannot adequately exercise. The ACC/AHA guidelines recommend some form of stress imaging in this setting (10). If the appropriateness criteria had used the results of more sensitive and specific stress imaging tests, they might have performed better. Second, the appropriateness guidelines did not take into account the baseline severity of angina. Coronary angiography is done for a variety of reasons (1, 11). In patients with stable angina that is incapacitating despite good medical management, the most common indication is to see whether revascularization could relieve symptoms. Less often, physicians recommend angiography because noninvasive test results raise suspicion of severe coronary disease. Studies done in the 1960s and 1970s demonstrated that, regardless of the severity of symptoms, patients with 3-vessel and left main disease have better long-term survival with revascularization. However, fewer than one third of patients with angina have 3-vessel or left main disease on coronary angiography, and only some of them undergo revascularization. Whether revascularization still confers a survival advantage given improved medical therapy is unclear. Indeed, the results of the recent COURAGE (Clinical Outcomes Utilizing Revascularization and Aggressive Drug Evaluation) trial, a randomized trial of intensive medical therapy versus percutaneous coronary intervention in patients with stable angina, showed that the 2 interventions had the same rates of death, myocardial infarction, or acute coronary syndromes, whereas percutaneous coronary intervention gave better angina relief (12). Because relief of angina was probably the primary indication for coronary angiography in most patients in Hemingway and colleagues' study, the composite end point should have included angina relief. Finally, several other factors contribute to the variability in the use of coronary angiography: regional practice patterns, socioeconomic factors, patient preference, availability of catheterization facilities, reimbursement, and insurance status, as well as race, ethnicity, and sex (13, 14). The appropriateness ratings that Hemingway and colleagues used took some but not all of these factors into account. To assess quality, the Joint Commission and Centers for Medicare & Medicaid Services adopted performance measures for certain medical conditions and recently introduced pay-for-performance programs based on those measures. Given the limitations of performance measures, should appropriateness ratings also be used? Hemingway and colleagues' study raises concerns about appropriateness measures, given the modest agreement between the 2 expert panels. To evaluate individual practitioners for pay-for-performance, appropriateness measures must be very reliable and widely accepted. On the other hand, appropriateness ratings have real meritparticularly because they take into account comorbid conditions and predict outcomesand they may perform better after further refinement. Another practical concern is difficulty in obtaining the clinical data needed to assess appropriateness. Gibbons and associates (1517) evaluated the American College of Cardiology Foundation appropriateness criteria for single-photon emission computed tomography and stress echocardiography at the Mayo Clinic. The appropriateness ratings of 2 experienced cardiovascular nurse chart abstractors agreed only moderately well, because they had to make subjective assessments and data were missing from the electronic medical record. As Gibbons and associates have suggested, appropriateness criteria may be more reliable and effective if used as criteria for ordering a test. The computerized ordering system could require the ordering physician to enter all necessary information, could match the data to appropriateness criteria, and might influence the decision to order angiography. The need to practice guideline-based medicine and to assess our performance has never been more urgent. When guidelines are followed, outcomes improve (18). Most clinicians know the guidelines but fail to use them in practice. When systems or tools are available to support their use in decision making, guideline adherence is better (19). Appropriateness criteria have great promise, especially if integrated into practice through th