The hazards of evidence-based medicine: assessing variations in care.

ASSESSING QUALITY OF CARE FREQUENTLY INVOLVES using measures of processes of care, such as Medicare’s 25 quality metrics. Adherence to these processes is thought to lead to improved outcomes. For example, the Surgical Care Improvement Project was introduced in 2006, with the goal of reducing surgical complications by 25% by 2010. Based on observational studies demonstrating associations between process and outcomes, experts concluded that adherence to this series of process measures would result in better care. Medicare adopted these and published them on its Hospital Compare Web site as measures of hospital quality. However, for some process measures, studies have shown that adherence to these measures is not necessarily associated with improved outcomes. This has been the case for perioperative antibiotic use and postoperative wound infection and for acute myocardial infarction, heart failure, and pneumonia. More worrisome is that in some cases, adherence to the prescribed process measure may be associated with considerable harm, such as with tight glucose control in critically ill patients. Hence, rewarding institutions with high quality of care as determined by process measures and penalizing systems such as those with poorer performance on these metrics might pose potential harm. For instance, an analysis of geographic variation in health care delivery showed that regional variation in chronic conditions among Medicare beneficiaries was inversely related to their case-fatality rate. In that analysis, patients in some areas, such as McAllen, Texas, were among those with the highest mean number of chronic conditions, whereas patients in other areas, such as Grand Junction, Colorado, were among those with the fewest. McAllen has been associated with very high Medicare expenditures and poor quality of care based on Hospital Compare, whereas the opposite association has been noted for Grand Junction. However, although the Medicare expenditures in McAllen are relatively high compared with Grand Junction, the case-fatality rate is lower for a group of patients with more chronic conditions. The definitive outcome of a lower case-fatality rate in McAllen suggests that the higher expenditures were appropriate, yet the intermediate outcomes from process measures as defined on Hospital Compare suggest the opposite. Provision of more or fewer health care services relative to the average may be either beneficial or harmful. For individual patients cared for by individual physicians, variation in care is sometimes desirable. One patient’s chronic illness is not another’s, and treating all patients the same would be clinical nonsense. However, maximizing variation for individuals does not extend to groups of patients cared for by groups of physicians. Physicians and patients arriving at the same decisions in reference to treatment risks and benefits should average out when outcomes are compared among large groups. Thus, studies of variation in care seek to find similarity between large regions. These studies are limited because the outcomes measured and information available for such comparisons are not always reliable. Before concluding that variation is undesirable, caution is needed when assessing data to ensure that such variation accurately reflects the entire scope of medical decision making and how these decisions affect outcomes. In this issue of JAMA, Haymart and colleagues report on variations in the administration of radioactive iodine (RAI) following operations for thyroid cancer. The incidence of thyroid cancer is increasing, but the reasons for the increase are unclear. Along with the increase in incidence, there has been a concomitant increase in thyroidectomy rates and an increase in the proportion of patients treated with RAI. Using data from a cohort of 189 219 patients with welldifferentiated thyroid cancer treated at 981 hospitals associated with the National Cancer Database, Haymart et al reported a significant increase in the proportion of patients treated with RAI between 1990 and 2008, although there was wide variation in RAI use, with variation occurring in both directions. On one hand, approximately one-fourth to one-third of patients with advanced-stage disease or large tumors were not treated with RAI despite general agreement that RAI is necessary for these patients. On the other