Claims Model for Mortality Figure 2. Distribution of hospital-level standardized 30-day pneumonia mortality rates. Risk-standardized mortality rates for the 4,684 hospitals are based on the administrative claims model using data for 449,296 Medicare patients discharged during calendar year 2000. doi:10.1371/journal.pone.0017401.g002 50th, and 75th percentiles for unadjusted mortality were 14.5%, 17.3%, and 20.7%, respectively. The distribution of RSMRs, however, ranged from 13.0% to 23.7%, with 25th, 50th, and 75th percentiles of 16.5%, 17.4%, and 18.3%, respectively. Medical Record Validation From a total of 75,616 cases for which administrative claims and medical record abstracted data were available, the final medical record derivation sample included 50,858 cases that met inclusion criteria (Table S1). The unadjusted 30-day mortality rate for this cohort was 14.5%. Twenty covariates based on the Pneumonia PORT risk factors were included in the final model (Table S4). The area under the ROC curve was 0.77 (Table S3). The observed mortality rate ranged from 1.9% in the lowest predicted decile to 45.9% in the highest. As expected, the explained variation was higher in the medical record model (R2 = 0.20) than in the administrative model. In this cohort of patients with medical record data, the administrative model had an area under the ROC curve of 0.71, an observed mortality rate ranging from 2.4% in the lowest decile to 36.9% in the highest predicted decile, and an adjusted R2 of 0.12 (Table S3). Estimated state-specific RSMRs derived from the medical record and administrative models are displayed in Figure 3. The slope of the weighted regression line of the state-specific RSMR estimated from the 2 models is 0.77 (SE = 0.06) with an intercept of close to 0 (intercept = 0.03; SE = 0.01) and a correlation coefficient of 0.86 (SE = 0.03). The mean difference in RSMR between the models is ,0.0001; the range was 20.0089 to 0.0088 so that both positive and negative differences were observed. Stability of the Administrative Model over Time The administrative model demonstrated good stability over time. In each of the administrative validation cohorts, the model fit was similar to the derivation cohort (Table S3). Unadjusted mortality ranged from 14.0% in 2003 to 15.0% in the 2001 dataset. The percent explained variation ranged from 0.12 to 0.13, while the area under the ROC curve was the same as in the derivation sample (0.72). Discussion We have demonstrated that an administrative claims-based model for calculating 30-day pneumonia mortality rates for Medicare fee-for-service patients results in risk-standardized estimates at the state level that are highly correlated with the estimates obtained from a medical record model. For the purposes of profiling outcomes, the claims model was a very good surrogate for the medical record model, and the model was very stable over time. While insufficient samples of medical record data prevented us from validating our administrative claims model against medical record data at the level of the individual hospital, the previously developed risk-stratified measure of 30-day mortality for acute myocardial infarction, which was developed using the same methodology, was validated at the hospital level and demonstrated similar findings from the administrative and medical record models [11]. Our approach to the development of this administrative claims model for profiling hospital mortality rates is consistent with the 7 preferred attributes identified in the American Heart Association Scientific Statement that defined standards for statistical models used for public reporting of health outcomes [13]. Several aspects of the development of this model warrant additional discussion. We explicitly defined the population of patients that are eligible for inclusion in the claims model and chose the ICD-9-CM codes that are consistent with those used in the National Pneumonia Project for patients with a principal diagnosis of pneumonia [4]. We specifically did not include in the sample those patients with a principal diagnosis of sepsis (038.xx) or respiratory failure (518.81 or 518.84) with a secondary diagnosis of pneumonia to avoid including cases where pneumonia could have been a complication of the hospitalization. Although this is not consistent with the denominator population of the National Pneumonia Project, a medical record data element is used in the PLoS ONE | www.plosone.org 5 April 2011 | Volume 6 | Issue 4 | e17401