# Presentation: 454416

## AHRQ Inpatient Quality Indicators – Interpretative Guide

- Dallas-Fort Worth Hospital Council Data Initiative
- August 2002

## The purpose of this Interpretative Guide is to assist hospitals and other users of the AHRQ Inpatient Quality Indicators (IQI) understand and interpret the results derived from the application of the IQI software to their own data.  The Guide first provides an overview of the risk adjustment approach implemented in the IQI software.  It also provides a brief description of the differences between observed rates and risk adjusted rates of the IQI indicators, the calculation of “Expected Rates” as defined by the Texas Health Care Information Council (THCIC), and the use of statistical confidence intervals for the risk adjusted rates.  Through the use of tables showing examples of CABG Mortality Indicator analyses, the Guide illustrates the comparison of observed and risk adjusted rates, the calculation of THCIC Expected Rates, and the use of confidence intervals to assess the statistical significance of differences in risk adjusted rates.

##  Risk-adjusted rates are the estimated performance of hospitals if they had an ‘average’ casemix.  The average casemix was estimated from the AHRQ national database (data from 28 states).  

-  Estimates of the average case mix reflect the distribution in age, sex, and 3M APR-DRG categories among the hospitals in the national database.
-  Most indicators are risk-adjusted for age, sex and 3M APR-DRG’s.
-  Mortality measures use the 3M APR-DRG risk of mortality score and the utilization measures use the 3M APR-DRG severity score.
-  The measures Cesarean section, VBAC, and laparoscopic cholecystectomy rates are characterized by a single 3M APR-DRG and are only risk-adjusted by age.  Laparoscopic cholecystectomy is also adjusted by sex.

- AHRQ Inpatient Quality Indicators
**Interpretative Guide - Example **

- Risk-Adjustment

## Interpretative Guide - Example 

**Interpretative Guide - Example**** **

- Differences Between Hospital Observed Rates and Risk-Adjusted Rates

- Observed Rate > Risk-adjusted Rate
- Mortality indicators:  The hospital’s patient population for the condition or procedure has a higher risk of mortality due to its casemix (for example, older or a greater proportion of a higher-risk APR-DRG).
- Utilization indicators: The hospital’s population has a higher risk of receiving the procedure due to its demographic composition (for example, older or a greater proportion of a higher-risk gender).
- Observed Rate < Risk-adjusted Rate
- Mortality indicators: The hospital’s patient population for the condition or procedure has a lower risk of mortality due to its casemix (for example, younger or a greater proportion of a lower-risk APR-DRG).
- Utilization indicators: The hospital’s population has a lower risk of receiving the procedure due to its demographic composition (for example, younger or a greater proportion of a lower-risk gender).
- Observed Rate = Risk-adjusted Rate
- Mortality indicators: The hospital’s patient casemix for the condition or procedure is similar to other hospitals’, suggesting that patient composition is not a contributing factor to the hospital’s performance for the mortality indicator.
- Utilization indicators: The hospital’s population is similar to other hospitals in demographic composition.
- See example page 4.

## Interpretative Guide - Example 

**Interpretative Guide - Example**** **

## Interpretative Guide - Example 

- The Expected Rate represents a mathematical computation using the risk-adjusted rates. The formula used by  THCIC is:
- Hospital Expected Rate = (Hospital Observed Rate / Hospital Risk-Adjusted Rate) times the Comparative (State or Data Initiative) Average Risk Adjusted Rate
- The expected rate answers the question “What would the mortality rate be for this group of patients (the hospital’s patients) if the hospital had performed to the average standard of other hospitals in the state (expected rate) rather than how they actually performed (the hospital’s observed rate)?”  It is the predicted outcome when applying the state average casemix to a specific hospital.
- Observed rate < Expected Rate
- Mortality indicators: The hospital’s performance is good, better than average.
- Observed rate > Expected Rate
- Mortality indicators: The hospital’s performance is poor, worse than average.
- Observed rate = Expected Rate
- Mortality indicators: The hospital’s performance is average, consistent with the state average performance.
- See example on page 6.

- AHRQ Inpatient Quality Indicators
**Interpretative Guide - Example**** **

## Interpretative Guide - Example 

**The formula for Hospital Expected Rate is**:
- (Hospital Observed Rate / Hospital RA Rate) times Comparative Ave. RA Rate
- In this example: Expected Rate with THCIC Comparative  (3.18 / 3.06) * 4.2 = 4.36
- Observed Rate of 3.18 is lower than expected rate of 4.36.  Interpretation is this hospital’s performance is better than average.

- AHRQ Inpatient Quality Indicators
**Interpretative Guide - Example**** **

## Evaluation of Performance - Statistical Significance Testing

- Statistical Significance of Hospital Risk-Adjusted Rate
- The test is whether a specific hospital’s risk-adjusted rate is significantly higher or lower than the state average (or other comparative average) risk-adjusted rate with a 95% confidence interval.
- The risk-adjusted rate represents the likely outcome if a group of average patients (national average case mix) were cared for by the hospital.  _Or in other words, if the hospital’s case mix was adjusted to that of the national average how the hospital would be expected to perform compared to peers (state or other comparative group_).  This can be used to compare performance across hospitals.  The formula uses the AHRQ national mean square of error (these reference data values are provided in the AHRQ software).  The formula is: Hospital Risk-Adjusted Rate +/- (1.96) times the (square root of mean square of error/square root of number of cases in hospital denominator population).
- Comparative RA Rate > Hospital RA Rate, Upper Confidence Interval Value
- Mortality Indicators: Hospital’s risk-adjusted mortality rate is statistically significantly lower than the comparative group, performance is better.
- Comparative RA Rate < Hospital RA Rate, Lower Confidence Interval Value
- Mortality Indicators:  Hospital’s risk-adjusted mortality rate is statistically significantly higher than the comparative group, performance is worse.
- Comparative RA Rate = Hospital RA Rate, Within Values of Lower and Upper Confidence Intervals
- Mortality Indicators:  Hospital’s risk-adjusted mortality rate is not different than the comparative group.
- See example page 9.

- AHRQ Inpatient Quality Indicators
**Interpretative Guide - Example **

## Performance is Worse, Symbol used is   - 

- Comparative RA Rate > Hospital RA Rate, Upper Confidence Interval Value
- Mortality and Utilization Indicators: Hospital’s risk-adjusted is statistically significantly lower than the comparative group, performance is better.
- Performance is Better, Symbol used is   **+**
- Comparative RA Rate < Hospital RA Rate, Lower Confidence Interval Value
- Mortality and Utilization Indicators:  Hospital’s risk-adjusted rate is statistically significantly higher than the comparative group, performance is worse.
- Performance is Worse, Symbol used is   -
- Comparative RA Rate = Hospital RA Rate, Within Values of Lower and Upper Confidence Intervals
- Mortality and Utilization Indicators:  Hospital’s risk-adjusted rate is not different than the comparative group.
- Performance is Not Different or Average, Symbol used is ** **o
- See example page 9.

- AHRQ Inpatient Quality Indicators
**Interpretative Guide - Example**** **

## AHRQ Inpatient Quality Indicators

**Interpretative Guide - Example **