# Presentation: 501761

## Evaluating the AHRQ Inpatient Mortality Indicators for Public Reporting in California

- AHRQ Users Meeting
**Joseph Parker, PhD**
**Brian Paciotti, PhD**
**Merry Holliday-Hanson, PhD**
- Rockville, MD
- September 10, 2008

## AHRQ Inpatient Mortality Indicators (IMIs): Background to CA Decision

- Per statute, OSHPD should be publishing 9 risk-adjusted hospital mortality reports per year
- Traditional approach to producing reports costly, time-consuming, fraught with delays
- CA patient discharge data now available only 7 months after end of reporting year (inpatient mortality)
- State death file (necessary for 30-day mortality) not available until 15 months after end of reporting year
- APR-DRG risk model not ‘black box’ anymore
- POA now incorporated in APR-DRG risk adjustment algorithm
- Some IMIs have undergone NQF vetting process

## AHRQ Inpatient Mortality Indicators: California Plans for Public Reporting

| Conditions (7) | Procedures (8) |
| --- | --- |
| Acute Stroke
Gastrointestinal (GI) Hemorrhage
Hip Fracture*
AMI 
AMI without transfer cases 
Pneumonia*
Congestive Heart Failure*

 | Esophageal Resection*
Pancreatic Resection*
Craniotomy
Carotid Endarterectomy
Percutaneous Transluminal Coronary Angioplasty (PTCA)
Abdominal Aortic Aneurysm (AAA) Repair*
Hip Replacement
Coronary Artery Bypass Graft (CABG) Surgery
 |

**Indicators NOT planned for release in Gold**

- * **Endorsed by NQF May, 2008**

## Comparison of Traditional OSHPD Mortality Reports* with AHRQ IMIs 

- _OSHPD Mortality Reports_
- 30-day mortality
- Link with CA death file data (delay)
- Include POA information
- Detailed technical reports accompany results
- Data validation study performed
- Considered quality “measures”
- No national vetting/endorsement

- _AHRQ IMIs_
- Inpatient mortality
- No death file data used
- _Now_ include POA information
- Minimum documentation (Tech. Note with links to AHRQ)
- Data not formally validated in CA
- Considered quality “indicators”
- Six received National Quality Forum (NQF) endorsement (Hip replacement mortality withdrawn)

*** Not including CABG report based on clinical data**

## Planned and Published OSHPD Quality Metrics: Varying Levels of Validity 

|  | CA CABG Report | OSHPD Traditional Reports * | OSHPD Benchmark Reports** | AHRQ IMIs | AHRQ Volume & Utilization Indicators |
| --- | --- | --- | --- | --- | --- |
| Type of Data | Clinical Registry | Patient Discharge Data | Patient Discharge Data | Patient Discharge Data | Patient Discharge Data |
| Data Quality Checks | Extensive, ongoing changes | Automated; no changes past acceptance | Automated; no changes past acceptance | Automated; no changes past acceptance | Automated; no changes past acceptance |
| Medical Chart Audit | Yearly | With initial validation study | Limited: CHF, None: AAA | None | None |
| Risk Model Review | Yearly | Infrequent | Infrequent | Periodic by AHRQ | Periodic by AHRQ |
| Expert Panel Review | Continuous | With initial validation, TAC | TAC | Periodic by AHRQ & NQF | Periodic by AHRQ & NQF |
| Principal Source of Validation | National STS & associated literature | Initial validation study & literature | For CHF, CMS validation; for AAA, literature | Extensive literature review & NQF vetting | Extensive literature review & NQF vetting |

******* ****Includes two reports produced (heart  attack, pneumonia) and two in progress **

******** ****Two reports in progress (AAA repair & CHF) to be released without a formal validation study**

## AHRQ Message on IMI Validity

*Providers, policy makers, and researchers can use with inpatient data to *_*identify apparent variations*_* in the quality of inpatient care *
*Although quality assessments based on administrative data cannot be definitive, they can be used to *_*flag potential quality problems*_* and success stories, which can then be further investigated and studied *
*Hospital associations, individual hospitals, purchasers, regulators, and policymakers at the local, State, and Federal levels can use readily available hospital administrative data to *_*begin the assessment of quality of care*_

## OSHPD Message on IMI Validity

*  **OSHPD views these indicators as potentially useful starting points for examining hospital quality but *_*does not regard them as definitive measures of quality*_*. When this information is carefully considered, with its limitations, *_*alongside other reliable healthcare provider information*_*, it may be helpful to patients and purchasers when making decisions about healthcare treatment choices. Healthcare providers may also benefit from using this information in quality improvement activities. *

## OSHPD Implementation of IMIs

- Used 2007 CA patient Discharge Data
- Transformed data elements and values into formats for AHRQ software (Version 3.2)
- POA option utilized
- APR-DRG risk scores based only on conditions coded as pre-existing,  not hospital-related complications
- Calculated risk-adjusted rates
- Used APR-DRG risk model with coefficients from CA & NY
- Logistic regression model (with random hospital effects)
- Calculated statistical outliers
**No hospital case volume limit applied**
**95% upper and lower CIs**
- “Better” and “Worse than Expected” labels used

## 2007 CA Statewide California IMI Results

| Procedure/Condition | #  Cases | # Hospitals | Rate | # Better | # Worse |
| --- | --- | --- | --- | --- | --- |
| Esophageal Resection |       190 | 59 |     6.5 | 0 | 2 |
| Pancreatic Resection |       623 | 121 |     4.5 | 0 | 5 |
| Craniotomy |  11,427 | 294 |     6.2 | 1 | 14 |
| Acute Stroke |  49,915 | 343 |   10.4 | 20 | 33 |
| GI Hemorrhage |  48,691 | 358 |     2.1 | 1 | 13 |
| Hip Fracture |  23,700 | 300 |     2.4 | 0 | 14 |
| PTCA |  52,152 | 154 |     1.3 | 2 | 11 |
| Carotid Endarterectomy |    8,132 | 238 |     0.4 | 1 | 7 |

## Analyses to Detect Major Biases in IMIs Using Internal Data

- Do certain types of hospitals (teaching, public, profit, non-profit) have worse IMI rates than other types of hospitals?
- Do hospitals with a large percentage of DNR, palliative care, or SNF patients have worse IMI rates than other hospitals?
- Do hospitals with very poor POA coding quality (rarely coding complications) have better IMI rates than others?
- Do *high transfer-intensity* hospitals benefit from use of inpatient mortality compared to 30-day mortality?
- Does the AHRQ inclusion of POA improve validity of original IMI by bringing it closer to a gold standard proxy?

## Hospital 2007 IMI Rates by Type of Hospital

**p<.05  **p<0.01 ***p<0.001 (ANOVA)**

|  | Hospital Type |  |  |  |
| --- | --- | --- | --- | --- |
|  | Non-Profit | Investor | Public | Teaching |
| Number of Hospitals* | 191 | 96 | 62 | 20 |
| IMIs |  |  |  |  |
| Craniotomy*** | 5.4 | 4.7 | 11.7 | 6.9 |
| Stroke** | 10.4 | 9.0 | 12.3 | 10.6 |
| GI Hemorrhage** | 2.2 | 1.9 | 3 | 1.8 |
| Hip Fracture* | 2.3 | 2.8 | 3.3 |  2.2 |
| PTCA** | 1.2 | 1.0 | 1.5 | 1.6 |
| Carotid Endarterectomy | 0.4 | 0.3 | 0.5 | 0.1 |

**Includes all CA hospitals meeting minimum volume thresholds: **
**     ****most IMIs will have fewer hospitals reporting**

## Effect of DNR, Palliative Care, and SNF Patients on Hospital Performance

- For each hospital, across all patients, calculated the percent of total who had:
- DNR coded within 24 hours of admission
- ICD-9 code for palliative care (99.7)
- Source of admission = SNF
- Marker for unmeasured severity
- Correlated % of patients with DNR, PC, & SNF with 8 IMI rates
- For DNR, PC & SNF, determined which 10% of hospitals had highest coding rates
- Calculated and compared the IMI rates for the high 10% group with other 90%
- However, when comparing hospitals in top & bottom 5% of mortality performance, no consistent trend in DNR, PC, and SNF patient caseloads (even for CHF & stroke)

## Correlations Between Hospital DNR, Palliative Care, and SNF Admission Rates and IMI Rates

| IMIs | # Hospitals | Pearson Correlation |  |  |
| --- | --- | --- | --- | --- |
|  |  | DNR Rate | Pall. Care  Rate | SNF Admit Rate |
| Esophageal Resection | 16 | 0.08 | 0.37 | 0.05 |
| Pancreatic Resection | 60 | 0.10 | 0.19 | -0.05 |
| Craniotomy | 159 | -0.13 | 0.02 | -0.05 |
| Acute Stroke | 273 | 0.22*** | 0.21*** | -0.08 |
| Gastrointestinal Hemorrhage | 292 | 0.00 | 0.02 | 0.05 |
| Hip Fracture | 277 | 0.10 | 0.08 | 0.04 |
| PTCA | 132 | 0.01 | 0.10 | 0.07 |
| Carotid Endarterectomy | 170 | 0.03 | -0.01 | -0.01 |
| Community-acquired pneumonia (OSHPD report) | 354 | .11* | 0.13* | 
--- |

***p<.05, **p<.01, ***p<.0001****	**

## Do Hospitals Coding Large Numbers of DNR, Palliative Care, or SNF Admissions Have Higher IMI Rates?

| IMIs | DNR |  | Palliative Care |  | SNF Admissions |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | High | Other | High | Other | High | Other |
| Craniotomy (%) | -- | -- | 5.2 | 6.3 | 4.3 | 6.2 |
| Stroke (%) | 15.8  | 10.0 | 11.9 | 10.1 | 8.8 | 10.5 |
| GI Hemorrhage (%) | 2.7 | 2.1 | 2.3 | 2.1 | 2.3 | 2.1 |
| Hip Fracture (%) | 1.7 | 2.6 | 2.3 | 2.6 | 1.8 | 2.6 |
| PTCA (%) | 1.8 | 1.3 | 1.3 | 1.3 | 1.5 | 1.3 |
| Carotid Endarterectomy  (%) | 0 | 0.3 | 0.7 | 0.3 | 1.2 | 0.3 |

## Impact of Hospital POA Coding Quality on Performance

- POA analyses using % cases coded POA=‘yes’ (crude measure) and 3M POA coding quality metric
- No consistent correlation (direction or strength) between % hospital cases coded POA=“yes” and IMI rates
- No consistent difference between lowest & highest 5% of hospitals by mortality rate, on % cases coded POA=“yes”
- Using 3M metric, poor coding hospitals are not over-represented in lowest 5% mortality rate hospitals

## Do Hospitals with Bad POA Coding Have Better IMI Rates Than Other Hospitals?

| IMIs | Hughes et al. Metric |  | OSHPD Metric |  |
| --- | --- | --- | --- | --- |
|  | Other | Bad | Other | Bad |
| Craniotomy (%) | 5.7 | 9.4 | 6.0 | 9.1 |
| Stroke (%) | 10.6 | 9.3 | 10.4 | 8.7 |
| GI Hemorrhage (%) | 2.1 | 2.5 | 2.1 | 2.4 |
| Hip Fracture (%) | 2.6 | 2.2 | 2.6 | 2.0 |
| PTCA (%) | 1.2 | 1.6 | 1.3 | 3.6 |
| Carotid Endarterectomy (%) | 0.3 | 0.3 | 0.3 | 0 |

## Does the Use of Inpatient Mortality vs. 30-Day Mortality Bias Hospital Results?

- 30-day mortality is the preferred measure for outcomes assessment
- Not impacted by hospital discharge practices
- May result in less timely reports
- Transfer rates vary greatly by hospital
- Transfers to acute care (excluded in IMIs):
- Transfers to SNF/other care:
- Methods
- Created hospital “transfer intensity” measure
- Rates created for top 10%, middle 80%, and bottom 10% of hospitals according to ALL patient discharges to non-hospital care (SNF/intermediate care, other care)need to describe/research better
- Calculated crude inpatient and 30-day mortality rates for hospitals by transfer intensity group
- Calculated difference between hospital inpatient and 30-day crude mortality rates by IMI
- Calculated an “impact measure  that describes to what degree different transfer intensity groups are favorably impacted by use of inpatient mortality

## What Impact Does Hospital Transfer Intensity Have on 30-day Crude Mortality Rates Compared to Crude Inpatient Rates? 

| IMI | Transfer Intensity | # Hospitals | Inpatient Rate | 30-Day Rate | Difference | Impact* |
| --- | --- | --- | --- | --- | --- | --- |
| Craniotomy | High | 6 | 14.3 | 19.0 | 4.7 | +2.2 |
|  | Moderate | 121 | 9.9 | 12.8 | 2.9 | -- |
|  | Low | 11 | 8.5 | 10.3 | 1.8 | -1.1 |
| Stroke | High | 18 | 7.2 | 13.7 | 6.5 | -0.6 |
|  | Moderate | 236 | 10.4 | 17.5 | 7.1 | -- |
|  | Low | 16 | 10.1 | 14.1 | 4.0 | -2.9 |
| GI Hemorrhage | High | 25 | 2.5 | 4.9 | 2.5 | -0.4 |
|  | Moderate | 246 | 2.5 | 5.4 | 2.9 | -- |
|  | Low | 20 | 2.8 | 4.6 | 1.7 | -1.2 |
| Hip Fracture | High | 21 | 3.2 | 8.1 | 4.9 | +0.5 |
|  | Moderate | 250 | 3.1 | 7.5 | 4.4 | -- |
|  | Low | 13 | 2.1 | 4.1 | 2.0 | -2.4 |
| Carotid
Endarter-ectomy | High | 5 | .32 | .63 | .31 | -0.2 |
|  | Moderate | 164 | .47 | .98 | .51 |  |
|  | Low | 6 | .39 | .39 | .00 | -0.51 |

- What Impact Does Hospital Transfer Intensity Have on 30-day Crude Mortality Rates Compared to Crude Inpatient Rates?

*** Impact = Difference for High or Low group minus difference for moderate group**

## Does Inclusion of POA Coding Improve the IMIs Relative to a “Gold Standard”?

- Generated 2006 risk-adjusted mortality rates (RAMRs) for hospitals using CABG gold standard proxy
- (Parker et al., Med Care 2006)
- For same patient cohort, generated RAMRs using AHRQ software with and without POA
- Used inpatient mortality for both outcomes
- Calculated correlation between AHRQ CABG IMI (with & without POA) with CABG gold standard proxy
- Correlation with gold standard proxy improved from .87 (without POA) to .93 (with POA): a 7% improvement

## Summary of Findings

- Some patient illness severity is not explained by APR-DRG risk model (as demonstrated by DNR, PC & SNF status) but this does not strongly bias results (stroke a possible exception)
- Hospitals that do a poor job of POA coding do not appear to benefit in terms of their IMI results
- Public hospitals had higher mortality rates for 5 out 6 IMIs: Both Teaching and Investor hospitals had lowest rates for 3 IMIs.
- Hospitals that rarely transfer patients to SNFs & other care perform more poorly when using inpatient mortality compared to 30-day mortality
- Transfer to SNF/other care rates by hospital Type:
**	****Non-profit = 15%, Investor = 20%, Public= 15%, Teaching = 10%**
- Utilization of POA coding improves assessment of hospital quality relative to a CABG gold standard proxy

- BACKUP SLIDES