Scorecard Cardiovascular Medicine: Its Impact and Future Directions

The convergence of concern about the cost of medical care, the availability of large amounts of clinical outcome data in computerized databases [1-3], and dramatic advances in the methods of assessing factors related to outcome [4] have ushered in a new era of accountability for physicians, hospitals, and health care systems. Previous systems of quality assurance, including conferences about morbidity and mortality and about physician peer review organizations, have proved to be inadequate to reassure the public, and especially sophisticated business managers, that the medical profession is committed to providing the best possible quality of care. Outcomes research (dating back to the observational analyses of Florence Nightingale about mortality after amputation related to sanitation [5]) has provided an empirical base on which to assess quality of care by measuring patient outcomes as a function of the severity of illness of the patient at baseline, the therapy received, and the system in which health care is provided. Cardiovascular practitioners are at the center of this issue because of the high cost of cardiac procedures and the large number of outcome studies delineating effective methods of controlling for baseline risk when assessing clinical outcome [6]. We focus on the process by which quality of care can be quantified and on the issues that will be raised as reports on quality become public knowledge and suggest directions for the future, with the expectation that the specific concerns of cardiovascular medicine will be generalizable to other procedure-rich specialties. Until the mid-1980s, outcome data for individual hospitals and physicians were rarely discussed and were not released to the public. In 1986, dissemination of cardiovascular outcome data to the public began with reports from the Health Care Financing Administration and Veterans Affairs hospitals for mortality associated with coronary artery bypass surgery [7]. The Medicare mortality data for bypass surgery by hospital site are now made available to the public each year [8]. The acceleration of outcomes-effectiveness research in this area became evident in 1991 with two studies [9, 10] of perioperative mortality for coronary artery bypass surgery in New England and Philadelphia; the studies found marked intersite, intrasite, interoperator, and geographic variability. In 1991, a landmark event that truly ushered in the era of scorecard medicine occurred when, for the first time, case load and risk-adjusted mortality data were published in the newspaper for every cardiac surgeon in New York State [11]. This publication of operator-specific data will undoubtedly occur in other states; thus, we need to accelerate the processes by which individual practitioners become familiar with the methods of outcomes research and to develop strategies that incorporate public knowledge of outcomes into practice at the individual and system level. Justification for Public Disclosure Initially, most practitioners react defensively to the concept that medical outcomes should be measured and made available to physician peer review organizations, insurers, patients, and the public. Concerns are generally expressed about the ability of risk stratification to adequately adjust for differences and the ability of current methods to measure outcomes reliably. Despite these reservations, the process of scorecard medicine has been initiated and, in our opinion, will not be deterred. The arguments in favor of the scorecard approach from the clinical medicine viewpoint are that the process of measuring outcome provides a formal approach to improving individual decision making and system-wide allocation of resources, as shown in other complex work environments with the adoption of the Deming method of continuous quality improvement. The arguments in favor of disclosure are that patient payments and tax support through Medicare and Medicaid put large amounts of money into health care, the results of medical care can be measured, and those who pay are entitled to know what they are getting for their money. Litigation has already paved the way for release of outcome data about individual physicians to the general public. Invoking the Freedom of Information Act, a newspaper (Newsday) sued the New York State Department of Health to obtain its database on bypass surgery and on cardiac surgeons. The Supreme Court of New York State ruled that the citizens were entitled to know these data in order to make informed decisions about their health care [11]. This entitlement has been popularized by the media with such video news magazine statements as, You wouldn't buy a car with bad ratings, so why put yourself in the hands of a doctor with a questionable record? and It's one of the quirks of human nature that most of us spend more time shopping for a car or a VCR than choosing our doctors. Tonight our hidden cameras are going to show you what a difference comparison shopping could make [12]. A newly published book guiding selection of the best top doctors and the top hospitals has quickly become a best seller [13]. The public clearly has the right and need to know more information about individual physicians' credentials, including outcome data, but the ideal and responsible way to provide this disclosure and the appropriate rules of disclosure remain uncertain. Issues Raised by the Availability of Scorecard Medicine As noted in Table 1, the release of adjusted outcome data presents several logistic issues that must be carefully considered and, in our opinion, have not been resolved [14, 15]. The most appropriate group of patients for constructing a scorecard must be decided on. The most widely publicized scorecards have been constructed for procedures (such as surgery or angioplasty), but in many cases a diagnostic category would be more appropriate (such as unstable angina) to prevent shifting of high-risk patients away from procedures. By operating or doing angioplasty on lower-risk patients and treating the high-risk patients medically, procedure-related mortality would decline, but mortality for the diagnosis may actually increase. For each patient group, whether identified by procedure or by diagnosis, the most appropriate risk-adjustment model must be developed. As our clinical and pathophysiologic understanding improves, these models must be updated. In our experience, most clinicians are uncomfortable with the methods of risk modeling and are skeptical about their accuracy. One notable example occurs when data important for risk adjustment are not available; most clinicians do not accept schemes of data imputation. Mortality may be a less important outcome than functional status, and, in some patients with terminal illness, earlier death may actually be desirable. Measurement of functional status outcomes is even more susceptible to clinician skepticism. Table 1. Clinical, Statistical, and Logistic Problems of Releasing Operator-Specific Data to the Public The statistical methods of outcome data analysis and reporting are evolving rapidly. Even among full-time clinical epidemiologic and statistical researchers, considerable controversy exists about the best modeling methods. The appropriate methods for the translation of large database analyses to individual practices and hospitals remain an area of uncertainty. The major issue in the statistical arena is closing the tremendous gap that has developed between researchers and clinicians and administrators who must make decisions based on the results of analyses. Making inferences from aggregated outcomes data without understanding the concepts of bias, confounding, and statistical power, in addition to other fundamental concepts of clinical epidemiology, could lead to erroneous inferences and policies. Potential Impact Preliminary evidence that the operator-specific scorecard approach can improve medical care recently came from New York State in a 3-year follow-up of the statewide, risk-adjusted mortality rate for coronary artery bypass surgery; this decreased from 4.3% to 2.7%, a 38% decrease [16, 17]. Interhospital shifting of high-risk patients to centers with the best record may, in part, have accounted for the improvement. For example, several cardiac surgeons refused to do high-risk bypass surgery on the father of a journalist in New York City soon after the dissemination of the Newsday report [18]. Eventually, after multiple consultations and refusals to accept the patient for surgery, this patient with poor left ventricular function had a successful bypass operation at a large-volume center. It is possible that surgeons might game (alter) the system by entering more high-risk data about their patients, thus improving the risk-adjusted mortality rate. Physicians or hospitals with poor scorecards should examine clinical practice organization and skills and should change them; physicians or hospitals with exceptionally good outcomes should be rewarded and imitated. However, it is harder to interpret numeric differences within the range of acceptable results, especially when those differences are within the range of chance variation. In our opinion, internal dissemination within the profession about specific numeric outcomes within the acceptable range could motivate continuous examination and improvement, but considerable opportunity for misinterpretation of differences due to the play of chance exists with public disclosure of such differences. One of the most important concepts of modern clinical trials is the critical importance of adequate sample size to ensure that observed differences are certain. Displaying confidence intervals of the point estimates, as in Table 2, provides relevant information. However, an approach with a lower chance for misinterpretation would be to define an acceptable range and to publish specific point estimates for outcomes only when the observed value for a physician or institution