# Presentation: 016833

## The Effects of Critical Access Hospital Conversion on Patient Safety 

- Pengxiang Li, PhD, University of Pennsylvania
- John E. Schneider, PhD, University of Iowa
- Marcia M. Ward, PhD, University of Iowa

- Support for this work was funded by the Agency for Healthcare Research and Quality through grant # HS015009

## Background 

- Many of the smallest rural hospitals were not able to recover their Medicare costs under the prospective payment system (PPS) rates (Dalton, et al, 2005; Stensland, et al, 2004)
- Medicare Rural Hospital Flexibility Program of the 1997 Balanced Budget Act
- To protect small, financially vulnerable rural hospitals
- Allowed hospitals meeting certain criteria to convert to critical access hospitals (CAH)
- Changed Medicare reimbursement mechanism from prospective (PPS) to cost-based.
- One of the objectives of the policy was to increase the quality of care in these hospitals

## Patient Safety

- Definition
- “freedom from accidental injury due to medical care, or medical errors.” (IOM, 1999)
- “the failure of a planned action to be completed as intended or the use of a wrong plan to achieve an aim...[including] problems in practice, products, procedures, and systems.” (Quality Interagency Coordination Task Force, 2000)
- Each year, more than 44,000 Americans die in hospitals due to preventable medical errors (AHA, 1999).

**Notes:**
To err is human P 31

## Objective

- To examine the impact of critical access hospital (CAH) conversion on hospital patient safety in rural hospitals

## Study Sample and Unit of Analysis

- Eight year panel data for 89 Iowa rural hospitals (rural PPS hospitals and CAHs)
- Unit of analysis is hospital-year
**Figure 1: Time Frame of Iowa Rural Hospitals CAH Conversion**** **

## Outcome Measurements               AHRQ Patient Safety Indicators

- Created by AHRQ PSI SAS macro V3.0
- Each observed rate of patient safety indicator can be defined as “the outcome of interest in the population at risk”
- The risk-adjusted rate is the rate the provider would have if it had the same case-mix as the reference population (2003 state inpatient datasets from 38 states) given the provider’s actual performance.
- AHRQ recommends suppressing the estimates if fewer than 30 cases are in the denominator
- Only five patient safety indicators are able to provide PSI measures for all rural Iowa hospitals
- PSI-5: foreign body left during procedure
- PSI-6: iatrogenic pneumothorax
- PSI-7: selected infections due to medical care
- PSI-15: accidental puncture or laceration
- PSI-16: transfusion reaction
- Too rare to provide variability to differentiate hospitals in Iowa

## Outcome Measurements               AHRQ Patient Safety Indicators

- Four patient safety indicators (PSI-5, PSI-6, PSI-7 and PSI-15) are selected as the main measures for patient safety performance in Iowa rural hospitals.
- A composite patient safety indicator variable is the weighted average of each indicator
- The weights are based on the frequency of the numerator of each PSI in our sample (8-year inpatient discharges among 89 Iowa rural hospitals).
- Binary PSIs
- If the value of a PSI is higher than the median of the PSI in our sample, the binary variable for the PSI (poor performance) is equal to one; otherwise, it is equal to zero

## CAH Variables

- CAH: CAH binary indicator
- CAHit=1, if hospital i is in CAH status in year t
- CAHit=0, if hospital i is in PPS status in year t
- CAHmv: moving average CAH variable
- To examine the relatively long-term effects of CAH conversion
- CAHmvit=(CAHit+CAHi(t-1)+CAHi(t-2) )/3
- e.g. CAHmv=0.33 if the hospital is in the first year of conversion

## Models

- Panel data tobit model
- The distributions of PSIs are skewed
- The value of PSIs is non-negative with a mass at 0
- Tobit models have a conceptual advantage in analyzing data where the distribution of the dependent variable is normal above a limiting value
- PSIit*= β0 + β1CAHit+ ∑β2Xit + ∑β3Zt + εit
- PSIit= max (0, PSIit*)

## Control variables

- Xit: a vector of other explanatory variables for hospital i in year t
- % Medicare patient days
- % Medicaid days
- The hospital mean of the Charlson comorbidity score
- % surgical discharges
- market concentration (HHI)
- county-level per capita income
- county-level population density
- Zt is a vector of the year dummy variables which will adjust the effects of unmeasured, time-specific factors

## Models

- Problem with tobit model
- Assumption: truncated normal distribution
- A deviation to the assumption will lead to significant bias
- Using binary PSIs as dependent variables
- Generalized Estimating Equations (GEE) Logit Model (predict odds of bad performance)
- log(pit/1-pit)= b0 + b1CAHit+ ∑bnXit + ∑bmZt + eit
- The variance function is pit(1-pit)
- We assume the within-subject association among repeated measures is a first-order autoregressive correlation pattern

## Data Sources

- Iowa state inpatient datasets (SID) used to calculate PSIs for each hospital each year.
- Hospital case mix and HHI were calculated using Iowa SIDs.
- Other county variables were retrieved from Bureau of Health Professions Area Resource File (ARF)

## Table 1: Means, Standard Deviation and Sources of Variables, 1997 and 2004

| Variable | Data sources | 1997 |  | 2004 |  |
| --- | --- | --- | --- | --- | --- |
|  |  | Mean | Std. Dev. | Mean | Std. Dev. |
| PSI-5 (per 1000 discharges) | Iowa SID, AHRQ | 0.04 | 0.22 | 0.07 | 0.33 |
| PSI-6 (per 1000 discharges) | Iowa SID, AHRQ | 0.47 | 1.37 | 0.18 | 0.81 |
| PSI-7 (per 1000 discharges) | Iowa SID, AHRQ | 0.83 | 1.81 | 0.27 | 0.90 |
| PSI-15 (per 1000 discharges) | Iowa SID, AHRQ | 2.44 | 3.22 | 1.78 | 3.24 |
| Composite PSI (per 1000 discharges) | Iowa SID, AHRQ | 3.78 | 4.46 | 2.30 | 3.65 |
| CAH | IHA | 0 | 0 | 0.74 | 0.44 |
| CAHmv | IHA | 0 | 0 | 0.62 | 0.43 |
| % Medicare days | Iowa SID | 66.68% | 11.33% | 65.71% | 14.88% |
| % Medicaid days | Iowa SID | 6.72% | 4.49% | 7.20% | 4.72% |
| Hospital casemix | CMS | 1.06 | 0.09 | 0.99 | 0.12 |
| Market concentration (HHI) | Iowa SID | 8,936 | 1,969 | 8,925 | 2,008 |
| County per capita income | ARF | 21,180 | 1,846 | 26,401 | 4,338 |
| County population density | ARF | 31.80 | 18.51 | 31.80 | 19.34 |

## Table 2. Cross-sectional Comparison of Means of PSIs between Rural PPS and CAHs, 1997 to 2004 

| Year | Hospital categories | Number of Hospitals | PSI-5 | PSI-6 | PSI-7 | PSI-15 | Composite score of 4PSIs |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 1997 | Rural PPS | 89 | 0.04 | 0.47 | 0.83 | 2.44 | 1.93 |
| 1998 | Rural PPS | 89 | 0.05 | 0.27 | 0.36 | 2.72 | 2.04 |
| 1999 | Rural PPS | 88 | 0.02 | 0.3 | 0.69 | 2.99 | 2.29 |
|   | CAH | 1 | 0 | 0 | 0 | 0 | 0 |
| 2000 | Rural PPS | 78 | 0.06 | 0.36 | 0.6 | 3.09 | 2.35* |
|   | CAH | 11 | 0 | 0 | 0.41 | 2.12 | 1.58* |
| 2001 | Rural PPS | 57 | 0.07 | 0.21** | 0.69 | 2.65 | 2.03 |
|   | CAH | 32 | 0 | 0.07** | 1.17 | 2.24 | 1.8 |
| 2002 | Rural PPS | 45 | 0.04 | 0.23 | 0.64** | 2.00** | 1.56** |
|   | CAH | 44 | 0.21 | 0.34 | 0.14** | 1.89** | 1.41** |
| 2003 | Rural PPS | 34 | 0.01 | 0.46** | 0.88** | 2.38** | 1.89** |
|   | CAH | 55 | 0.06 | 0.26** | 0.41** | 1.89** | 1.45** |
| 2004 | Rural PPS | 23 | 0.12* | 0.29** | 0.54** | 2.68** | 2.04** |
|   | CAH | 66 | 0.06* | 0.14** | 0.17** | 1.46** | 1.09** |

- *  Significant difference in PSIs between rural PPS and CAH at p ≤ 0.10 (Wilcoxon rank sum test)
- **Significant difference in PSIs between rural PPS and CAH at p ≤ 0.05 (Wilcoxon rank sum test)

## Figure 2. Changes in Patient Safety Indicators after Conversion

## Table 3. GEE Logit Models of Binary PSIs (1=poor performance, 0=good performance)

|   | PSI-5 | PSI-6 | PSI-7 | PSI-15 | Composite score of 4PSIs |
| --- | --- | --- | --- | --- | --- |
| CAH | -0.80 | -1.19** | -1.26** | -0.92** | -0.70** |
| % Medicare days | -0.01 | -0.01 | -0.01* | -0.01 | -0.01 |
| % Medicaid days | -0.01 | 0.02 | 0.03 | 0.02 | 0.03 |
| % of surgical discharges | 0.09** | 0.05** | 0.01 | 0.18** | 0.16** |
| Charlson Index | 2.01** | 0.69 | 1.05 | 1.51** | 1.37** |
| Market concentration (HHI) | -2.11 | 0.37 | 0.24 | -0.21 | -0.15 |
| Per capita income ($1,000) | 0 | -0.03 | 0.09 | 0.06 | 0.07** |
| Population density | -0.01 | 0.02 | 0.01 | 0 | 0 |
| Intercept | -3.23 | -2.67* | -3.73** | -3.87** | -3.54** |
| observations | 712 | 712 | 712 | 712 | 712 |

- Note: Due to space limit, year dummy variables were omitted in this table.
- *  Statistically significant at 0.10 level.
- **Statistically significant at 0.05 level.

## Table 4. Sensitivity analysis: GEE models

| Models |   | PSI-5 | PSI-6 | PSI-7 | PSI-15 | Composite score of 4PSIs |
| --- | --- | --- | --- | --- | --- | --- |
| Models in Table 3 (89 hospitals, 1997 to 2004) | CAH‡ | -0.8 | -1.19** | -1.26** | -0.92** | -0.70** |
|  | CAHmv | -0.67 | -1.57** | -2.05** | -1.37** | -1.03** |
| Models adding proxy variable (the lag 1 year dependent variable) as covariates | CAH | -1.08 | -0.90** | -1.13** | -0.83** | -0.64** |
|  | CAHmv | -1.06 | -1.08* | -1.67** | -1.08** | -0.80** |
| Models using DRG-weight as risk adjustment (89 hospitals, 1997 to 2004) | CAH | -0.78 | -1.19** | -1.25** | -0.92** | -0.70** |
|  | CAHmv | -0.63 | -1.56** | -2.07** | -1.36** | -1.03** |
| Models excluding 8 hospitals which is in rural PPS in 2006 (81 hospitals, 1997 to 2004) | CAH | -0.94 | -1.11** | -1.41** | -0.85** | -0.63** |
|  | CAHmv | -0.79 | -1.45** | -2.28** | -1.27** | -0.94** |
| Models adjusting for hospital transfer behavior¥ | CAH | -0.84 | -1.24** | -1.28** | -0.93** | -0.72** |
|  | CAHmv | -0.76 | -1.68** | -2.12** | -1.43** | -1.12** |

- *   Statistically significant at 0.1 level. ** Statistically significant at 0.05 level.
- ‡  The same as the model in Table 3.
- ¥  Adding two variables (percentage of acute inpatient admission were transferred from other short-term hospital and percentage of acute inpatient patients were transferred to other short-term hospitals) into the models in Table II-6

## Table 5. Sensitivity Analyses: Tobit Models of Continuous PSIs 

- *   Statistically significant at 0.1 level. ** Statistically significant at 0.05 level.
- ++ Convergence was not achieved.
- +++   Add hospital dummy variables in cross-sectional Tobit model
- ‡   The results should be interpreted with caution, given that estimations for some coefficients were not stable under quadchk
- ∏  PSI-5 are observed rate.

## Results

- Cross-section and pre-post conversion comparisons showed that CAH hospitals had better performance of patient safety than rural PPS hospitals.
- The odds ratios of poor performance in CAH hospitals compared to rural PPS hospitals are 0.30 (CI: 0.14-0.64) for PSI-6, 0.29 (CI: 0.15-0.56) for PSI-7, 0.40 (CI: 0.24-0.67) for PSI-15, and 0.49 (CI: 0.31-0.80) for composite score of 4PSIs. CAH conversion had no significant impact on the observed rates of foreign body left during procedure.
- Moving average CAH indicator had larger effects than binary CAH scale.
- Sensitivity analyses using tobit models consistent results
- Findings were robust among sensitivity analyses using different samples and different methods

## Better PSI Performance Associated CAH Conversion: Alternative Explanations 

- Changes in coding behaviors
- Did not find any significant change in hospital-level mean number of DX, mean Charlson score, mean number of Elixhauser comorbidities
- Changes in referral patterns (get less severe patients)
- No significant change in
- percentage of patients transferred from other hospitals
- percentage of patients transferred to other hospitals
- Models adjusting for referral behaviors show consistent results
- It reflects a trend toward improvement in patient safety for all hospitals
- None-converted hospitals are comparison group
- Add year dummy variables
- Difference in difference

## Conclusion and implication

- CAH conversion in rural hospitals resulted in enhanced performance of patient safety.
- After conversion, CAHs have better perform in patient safety in the long run.
- We speculate that the likely mechanism involved an increase in financial resources following CAH conversion to cost-based reimbursement for Medicare patients
- Limitations of the study
- Administrative databases (missing codes, coding errors)
- We are not able to rule out the possibility of coding behavior change
- The five indicators may not reflect the whole picture of patient safety in rural hospitals
- CAH conversion variable is endogeneous.