Claims Model for Mortality Figure 3. Comparison of the state-level risk-standardized mortality rates generated by the medical record model and the administrative model. Risk-standardized mortality rates were generated with both models for the 50,858 patients in the chart model derivation cohort. The correlation coefficient for rates generated by the 2 models is 0.86 (Standard Error = 0.032). doi:10.1371/journal.pone.0017401.g003 chart abstraction model to determine if pneumonia was present on admission; this was not feasible based on claims. In addition, we did not try to distinguish between community-acquired and healthcare-associated pneumonia in our model, which is also consistent with the denominator population of the National Pneumonia Project, and consistent with the work of the Pneumonia PORT and a number of studies linking processes of care to patient mortality [14,20,22,23]. We chose 30 days after admission as the standard period for outcome assessment. This approach is consistent with previously National Quality Forum-endorsed mortality measures [11,12] and the time frame of assessment used by the Pneumonia PORT, as well as with prior studies of pneumonia processes and outcomes [14,20,22,23]. Evaluating mortality at 30 days (as opposed to evaluating in-hospital mortality) eliminates bias that might have occurred due to varying lengths of inpatient stay, and removes any incentive that might occur to discharge a patient from the hospital too early. A period of outcomes evaluation that is longer than the usual length of stay ensures that events early after discharge are captured, and places a premium on appropriate discharge planning [9]. We employed hierarchical modeling in the development of this model that accounts for clustering of patients within hospitals and permits separation of within- and between-hospital variation in observed outcomes. Use of hierarchical modeling reduces the chance that a hospital will falsely be characterized as an ‘‘outlier’’ for pneumonia mortality. For hospitals with low patient volume, predicted RSMRs will be at or near the national average. For those patients who were transferred from one hospital to another, we assigned responsibility for patient outcomes to the first hospital. This approach increases accountability for the index hospital to appropriately make decisions about transfer of patients, and avoids hospitals that receive transfer patients who are seriously ill from being inappropriately ‘‘penalized’’ for having higher mortality rates [13]. Finally, all details of the development and methodology used to generate this measure of hospital 30-day mortality for pneumonia are in the public domain and subject to ongoing critique and revision. The coefficients in our model were consistent with clinical expectations. Two exceptions were chronic obstructive pulmonary disease (HCC 108) and asthma (HCC 110), which among those patients with pneumonia were prognostically favorable. However, these results are consistent with what was found in the Pneumonia PORT. On an empirical basis, this appears to be a true relationship. The underlying mechanism is unknown, but it may be related to unmeasured factors related to pneumonia severity in these patients. There are several issues to consider about our methodology. The model is specific to Medicare fee-for-service patients and may not be generalizable to other data sources and patient populations. However, while the model was limited to Medicare patients, approximately two-thirds of all hospitalizations for pneumonia occur in this age group. Additionally, the only national data available on which to calculate the RSMRs are Medicare claims. The need to evaluate paid hospital claims and all claims from the year before the index hospitalization results in significant time lags between patient care and reported measure rates. Consistent with measure criteria from the National Quality Forum, the mortality model does not adjust for socioeconomic status, race, or ethnicity because risk-adjusting for socioeconomic status would hold hospitals with a large proportion of such patients to a different standard of care than hospitals treating a larger proportion of patients of higher socioeconomic status [24]. An additional issue is whether an area under the ROC curve of 0.70 for discriminating survivors from non-survivors is good enough to publicly report hospital mortality. The goal of the model is to produce estimates of hospital performance. The estimates from this model agree strongly with the estimates from a medical record model with a higher C-statistic. Moreover, interval estimates of RSMRs can be obtained using the bootstrap resampling method [25,26]. We believe that much of the unexplained variation in the RSMR derives from the latent variable of hospital quality. Finally, PLoS ONE | www.plosone.org 6 April 2011 | Volume 6 | Issue 4 | e17401