# Presentation: 610423

## Evaluating the AHRQ Inpatient Mortality Indicators for Public Reporting in California

- AHRQ QI Users Meeting
**Joseph Parker, PhD**
**Brian Paciotti, PhD**
**Merry Holliday-Hanson, PhD**
- Healthcare Outcomes Center, OSHPD
- Rockville, MD
- September 10, 2008

## AHRQ Inpatient Mortality Indicators (IMIs): Background to CA Decision

- Per statute, OSHPD required to publish 9 risk-adjusted hospital outcome reports per year
- Traditional approach to producing reports costly, time-consuming, fraught with delays
- CA patient discharge data now available only 7 months after end of reporting year (inpatient mortality)
- State death file (necessary for 30-day mortality) not available until 15 months after end of reporting year
- POA now incorporated in APR-DRG risk adjustment algorithm
- Some IMIs have undergone NQF vetting process

## AHRQ Inpatient Mortality Indicators: California Plans for Public Reporting

| Conditions (7) | Procedures (8) |
| --- | --- |
| Acute Stroke
Gastrointestinal (GI) Hemorrhage
Hip Fracture*
AMI 
AMI without transfer cases 
Pneumonia*
Congestive Heart Failure*

 | Esophageal Resection*
Pancreatic Resection*
Craniotomy
Carotid Endarterectomy
Percutaneous Transluminal Coronary Angioplasty (PTCA)
Abdominal Aortic Aneurysm (AAA) Repair*
Hip Replacement
Coronary Artery Bypass Graft (CABG) Surgery
 |

**Indicators NOT planned for release in Gold**

- * **Endorsed by NQF May, 2008**

## Comparison of Traditional OSHPD Mortality Reports* with AHRQ IMIs 

- _OSHPD Mortality Reports_
- 30-day mortality
- Link with CA death file data (delay)
- Include POA information
- Detailed technical reports accompany results
- Data validation study performed
- Considered quality “measures”
- No national vetting/endorsement

- _AHRQ IMIs_
- Inpatient mortality
- No death file data used
- _Now_ include POA information
- Minimum documentation (Tech. Note with links to AHRQ)
- Data not formally validated in CA
- Considered quality “indicators”
- Six received National Quality Forum (NQF) endorsement

*** Not including CABG report based on clinical data**

## Planned and Published OSHPD Quality Metrics: Varying Levels of Validity 

|  | CA CABG Report | OSHPD Traditional Reports * | OSHPD Benchmark Reports** | AHRQ IMIs | AHRQ Volume & Utilization Indicators |
| --- | --- | --- | --- | --- | --- |
| Type of Data | Clinical Registry | Patient Discharge Data | Patient Discharge Data | Patient Discharge Data | Patient Discharge Data |
| Data Quality Checks | Extensive, ongoing changes | Automated; no changes past acceptance | Automated; no changes past acceptance | Automated; no changes past acceptance | Automated; no changes past acceptance |
| Medical Chart Audit | Yearly | With initial validation study | Limited: CHF, None: AAA | None | None |
| Risk Model Review | Yearly | Infrequent | Infrequent | Periodic by AHRQ | Periodic by AHRQ |
| Expert Panel Review | Continuous | With initial validation, TAC | TAC | Periodic by AHRQ & NQF | Periodic by AHRQ & NQF |
| Principal Source of Validation | National STS & associated literature | Initial validation study & literature | For CHF, CMS validation; for AAA, literature | Extensive literature review & NQF vetting | Extensive literature review & NQF vetting |

******* ****Includes two reports produced (heart  attack, pneumonia) and two in progress **

******** ****Two reports in progress (AAA repair & CHF) to be released without a formal validation study**

## AHRQ Message on IMI Validity

*Providers, policy makers, and researchers can use with inpatient data to *_*identify apparent variations*_* in the quality of inpatient care *
*Although quality assessments based on administrative data cannot be definitive, they can be used to *_*flag potential quality problems*_* and success stories, which can then be further investigated and studied *
*Hospital associations, individual hospitals, purchasers, regulators, and policymakers at the local, State, and Federal levels can use readily available hospital administrative data to *_*begin the assessment of quality of care*_

## OSHPD Message on IMI Validity

*  **OSHPD views these indicators as potentially useful starting points for examining hospital quality but *_*does not regard them as definitive measures of quality*_*. When this information is carefully considered, with its limitations, *_*alongside other reliable healthcare provider information*_*, it may be helpful to patients and purchasers when making decisions about healthcare treatment choices. Healthcare providers may also benefit from using this information in quality improvement activities. *

## OSHPD Implementation of IMIs

- Used 2007 CA patient Discharge Data
- Transformed data elements and values into formats for AHRQ software (Version 3.2)
- POA option utilized
- APR-DRG risk scores based only on conditions coded as pre-existing, not hospital-related complications
- Calculated risk-adjusted rates
- Used APR-DRG risk model with coefficients from CA & NY
- Logistic regression model (with random hospital effects)
- Calculated statistical outliers
**No hospital case volume limit applied**
**95% upper and lower CIs**
- “Better” and “Worse than Expected” labels used

## 2007 CA Statewide California IMI Results

| Procedure/Condition | #  Cases | # Hospitals | Rate | # Better | # Worse |
| --- | --- | --- | --- | --- | --- |
| Esophageal Resection |       190 | 59 |     6.5 | 0 | 2 |
| Pancreatic Resection |       623 | 121 |     4.5 | 0 | 5 |
| Craniotomy |  11,427 | 294 |     6.2 | 1 | 14 |
| Acute Stroke |  49,915 | 343 |   10.4 | 20 | 33 |
| GI Hemorrhage |  48,691 | 358 |     2.1 | 1 | 13 |
| Hip Fracture |  23,700 | 300 |     2.4 | 0 | 14 |
| PTCA |  52,152 | 154 |     1.3 | 2 | 11 |
| Carotid Endarterectomy |    8,132 | 238 |     0.4 | 1 | 7 |

## Are IMIs (Potentially) Biased in Ways That Can be Detected Using Internal Data?

- Do certain types of hospitals (teaching, public, profit, non-profit) have worse IMI rates than other types of hospitals?
- Do hospitals with a large percentage of DNR, palliative care, or SNF patients have worse IMI rates than other hospitals?
- Do hospitals with very poor POA coding quality (rarely coding complications) have better IMI rates than others?
- Do *high transfer-intensity* hospitals benefit from use of inpatient mortality compared to 30-day mortality?
- Does the AHRQ inclusion of POA improve validity of original IMI by bringing it closer to a gold standard proxy?

## Hospital 2007 IMI Rates by Type of Hospital

*** p<.05  **p<0.01 ***p<0.001 (ANOVA)**

|  | Hospital Type |  |  |  |
| --- | --- | --- | --- | --- |
|  | Non-Profit | Investor | Public | Teaching |
| Number of Hospitals(1) | 191 | 96 | 62 | 20 |
| IMIs |  |  |  |  |
| Craniotomy*** (%) | 5.4 | 4.7 | 11.7 | 6.9 |
| Stroke** (%) | 10.4 | 9.0 | 12.3 | 10.6 |
| GI Hemorrhage** (%) | 2.2 | 1.9 | 3 | 1.8 |
| Hip Fracture* (%) | 2.3 | 2.8 | 3.3 |  2.2 |
| PTCA** (%) | 1.2 | 1.0 | 1.5 | 1.6 |
| Carotid Endarterectomy (%) | 0.4 | 0.3 | 0.5 | 0.1 |

**(1) Includes all CA hospitals meeting minimum volume thresholds: **
**     ****number of actual hospitals reporting varies by IMI**

## Effect of DNR, Palliative Care, and SNF Patients on Hospital Performance

- For each hospital, across all patients, calculated the percent of total who had the following:
- DNR coded within 24 hours of admission
- ICD-9 code for palliative care (V66.7)
- Source of admission = SNF
- Correlated percent of patients with DNR, PC, & SNF with IMI rates
- Determined which 10% of hospitals had highest coding rates, separately, for DNR, PC & SNF
- Calculated and compared the IMI rates for the top 10% group with other 90%

## Correlations Between Hospital DNR, Palliative Care, and SNF Admission Rates and IMI Rates

| IMIs | # Hospitals | Pearson Correlation |  |  |
| --- | --- | --- | --- | --- |
|  |  | DNR Rate | Pall. Care  Rate | SNF Admit Rate |
| Esophageal Resection | 16 | 0.08 | 0.37 | 0.05 |
| Pancreatic Resection | 60 | 0.10 | 0.19 | -0.05 |
| Craniotomy | 159 | -0.13 | 0.02 | -0.05 |
| Acute Stroke | 273 | 0.22*** | 0.21*** | -0.08 |
| Gastrointestinal Hemorrhage | 292 | 0.00 | 0.02 | 0.05 |
| Hip Fracture | 277 | 0.10 | 0.08 | 0.04 |
| PTCA | 132 | 0.01 | 0.10 | 0.07 |
| Carotid Endarterectomy | 170 | 0.03 | -0.01 | -0.01 |
| Community-Acquired Pneumonia (OSHPD report) | 354 | .11* | 0.13* | 
--- |

***p<.05, **p<.01, ***p<.0001****	**

## Do Hospitals Coding Large Numbers of DNR, Palliative Care, or SNF Admissions Have Higher IMI Rates?

| IMIs | DNR |  | Palliative Care |  | SNF Admissions |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | High | Other | High | Other | High | Other |
| Craniotomy (%) | -- | -- | 5.2 | 6.3 | 4.3 | 6.2 |
| Stroke (%) | 15.8  | 10.0 | 11.9 | 10.1 | 8.8 | 10.5 |
| GI Hemorrhage (%) | 2.7 | 2.1 | 2.3 | 2.1 | 2.3 | 2.1 |
| Hip Fracture (%) | 1.7 | 2.6 | 2.3 | 2.6 | 1.8 | 2.6 |
| PTCA (%) | 1.8 | 1.3 | 1.3 | 1.3 | 1.5 | 1.3 |
| Carotid Endarterectomy  (%) | 0 | 0.3 | 0.7 | 0.3 | 1.2 | 0.3 |

## Impact of Hospital POA Coding Quality on Performance

- Collecting POA data element does not ensure reliably coded POA information across hospitals
- Calculated IMI rates using non-POA version and POA version of APR-DRG software
- Produced scatter plots and calculated correlations between both versions of indicators
- Created two POA coding quality metrics to understand impact of poor POA coding on IMI rates:
- OSHPD metric using percent of all hospital secondary diagnoses coded POA=‘yes’ (gross measure)
- POA coding quality metric published by Hughes et al. (2006)

## Do Hospitals with Poor POA Coding Have Better IMI Rates Than Other Hospitals?

| IMIs | Hughes et al. Metric |  | OSHPD Metric |  |
| --- | --- | --- | --- | --- |
|  | Other | Poor | Other | Poor |
| Craniotomy (%) | 5.7 | 9.4 | 6.0 | 9.1 |
| Stroke (%) | 10.6 | 9.3 | 10.4 | 8.7 |
| GI Hemorrhage (%) | 2.1 | 2.5 | 2.1 | 2.4 |
| Hip Fracture (%) | 2.6 | 2.2 | 2.6 | 2.0 |
| PTCA (%) | 1.2 | 1.6 | 1.3 | 3.6 |
| Carotid Endarterectomy (%) | 0.3 | 0.3 | 0.3 | 0 |

## Does the Use of Inpatient Mortality vs. 30-Day Mortality Bias Hospital Results?

- 30-day mortality is the preferred measure for outcomes assessment
- Not impacted by hospital discharge practices
- May result in less timely reports
- Transfer rates vary greatly by hospital
- Transfers/discharges to acute care (excluded in IMIs): 2.5% overall
- Transfers/discharges to SNF/IC/Other care: 14.5% overall
- Methods
- Created hospital “transfer intensity” measure
- Rates created for top 10%, middle 80%, and bottom 10% of hospitals according to ALL patient discharges to non-acute care (SNF/Intermediate Care, Other: e.g., psych., phys. Med., rehab.)
- Calculated crude inpatient and 30-day mortality rates for hospitals by transfer intensity group
- Calculated difference between hospital inpatient and 30-day crude mortality rates by IMI
- Calculated an “impact” measure  that describes how different transfer intensity groups are impacted by use of inpatient mortality

## How Does Hospital Transfer Intensity Impact 30-day Crude Mortality Rates Compared to Inpatient Rates? 

| IMI | Transfer Intensity | # Hospitals | Inpatient Rate (%) | 30-Day Rate (%) | Difference | Impact* |
| --- | --- | --- | --- | --- | --- | --- |
| Craniotomy | High | 6 | 14.3 | 19.0 | 4.7 | +2.2 |
|  | Moderate | 121 | 9.9 | 12.8 | 2.9 | -- |
|  | Low | 11 | 8.5 | 10.3 | 1.8 | -1.1 |
| Stroke | High | 18 | 7.2 | 13.7 | 6.5 | -0.6 |
|  | Moderate | 236 | 10.4 | 17.5 | 7.1 | -- |
|  | Low | 16 | 10.1 | 14.1 | 4.0 | -2.9 |
| GI Hemorrhage | High | 25 | 2.5 | 4.9 | 2.5 | -0.4 |
|  | Moderate | 246 | 2.5 | 5.4 | 2.9 | -- |
|  | Low | 20 | 2.8 | 4.6 | 1.7 | -1.2 |
| Hip Fracture | High | 21 | 3.2 | 8.1 | 4.9 | +0.5 |
|  | Moderate | 250 | 3.1 | 7.5 | 4.4 | -- |
|  | Low | 13 | 2.1 | 4.1 | 2.0 | -2.4 |
| Carotid
Endarter-ectomy | High | 5 | .32 | .63 | .31 | -0.2 |
|  | Moderate | 164 | .47 | .98 | .51 |  |
|  | Low | 6 | .39 | .39 | .00 | -0.51 |

- How Does Hospital Transfer Intensity Impact 30-day Crude Mortality Rates Compared to Inpatient Rates?

*** Impact = Difference for High or Low group minus difference for moderate group**

## Does Inclusion of POA Coding Improve the IMIs Relative to a Gold Standard Proxy?

- Generated 2006 risk-adjusted mortality rates (RAMRs) for hospitals using CABG gold standard proxy
- (Parker et al., Med Care 2006)
- For same patient cohort, generated RAMRs using AHRQ software with and without POA
- Used inpatient mortality for both outcomes
- Calculated correlation between AHRQ CABG IMI (with & without POA) with CABG gold standard proxy
- Correlation with gold standard proxy improved from .87 (without POA) to .93 (with POA): a 7% improvement

## Summary of Findings

- Public hospitals had highest mortality rates for 5 of 6 IMIs: Both Teaching and Investor hospitals had lowest rates for 3 IMIs.
- Some patient illness severity is not explained by APR-DRG risk model (as demonstrated by DNR, PC & SNF status) but this does not strongly bias results (stroke a possible exception)
- Hospitals that do a poor job of POA coding (complications credited as pre-existing conditions) do not appear to benefit in terms of lower IMI rates
- Hospitals that rarely transfer patients to SNF/IC/Other care perform more poorly when using inpatient mortality compared to 30-day mortality
- Transfer rates to SNF/IC/Other care by hospital type:
**	****Non-profit = 15%, Investor = 20%, Public= 15%, Teaching = 10%**
- Utilization of POA coding improves assessment of hospital quality relative to a CABG gold standard proxy