Composite Measures for Rating Hospital Quality with Major Surgery

With wide recognition that surgical outcomes vary across hospitals, information on surgical quality is in high demand. Patients, families, and referring physicians are looking for information to help them select the best hospital for specific procedures (Kizer 2001). Payers and large health care purchasers need accurate hospital ratings to inform their selective referral and value-based purchasing programs (Galvin 2001). To meet these demands, several organizations are publicly reporting measures of surgical quality. The Leapfrog Group, a large coalition of public and private purchasers, reports information on mortality and hospital volume with major surgical procedures (Dimick et al. 2009). As its primary measure of surgical quality, the Center for Medicare and Medicaid Services (CMS) publicly reports hospital compliance with several process measures on its Hospital Compare website (Stulberg et al. 2010). Whether such measures are optimal for helping patients and payers identify the safest hospitals for major procedures is uncertain, however. Measures based on structure, process, and outcomes all have distinct limitations (Birkmeyer, Dimick, and Birkmeyer 2004). Hospital volume, for example, is a useful predictor of outcomes for some operations, but it is a relatively weak proxy of quality for most procedures (Birkmeyer et al. 2002; Halm, Lee, and Chassin 2002). Outcome measures, such as mortality and morbidity, are often too “noisy” to reliably measure hospital quality due to small sample sizes and low event rates (Shahian et al. 2001; Dimick, Welch, and Birkmeyer 2004). Process measures, such as those of the Surgical Care Improvement Program (SCIP) reported by CMS on Hospital Compare, are only weakly related to hospital outcomes with surgery (Stulberg et al. 2010). Providing patients with all available information may be better than any single indicator, but they would still require guidance on how to weight various measures, particularly when they conflict. In previous work, we have evaluated several different approaches for improving hospital quality measurement for surgery. Using national Medicare data, we recently described the value of a simple composite measure for profiling hospital quality with surgery based on volume and mortality alone (Dimick et al. 2009). This measure has since become the quality standard for the Leapfrog Group and was recently endorsed by the National Quality Forum (NQF) for use with esophagectomy, pancreatectomy, and abdominal aortic aneurysm repair. In another recent publication, we demonstrated the value of empirical Bayes techniques for filtering out statistical “noise” in surgical mortality measurement (Dimick, Staiger, and Birkmeyer 2010). Finally, we have demonstrated the feasibility of a more comprehensive composite measure that takes into account a broader array of measures, albeit in a narrow clinical context (Staiger et al. 2009). In this article, we demonstrate the value of a comprehensive composite measure for a broad range of surgical conditions. We combine the insights from our prior work, and others, to create a composite measure that incorporates all sources of information, including quality indicators for other, related operations, and uses empirical Bayes techniques to filter out statistical “noise.” Given the manner in which hospital ratings are used, we examined the ability of these measures to forecast future hospital mortality, compared to several other existing quality measures, including hospital volume, risk-adjusted mortality, simple empirical Bayes methods, and the process measures reported by the Center for Medicare and Medicaid Services (CMS) on its Hospital Compare website.