# Presentation: 533406

## Comparison of Cost and Outcomes of Inpatient and Outpatient Treatment for Substance Use Disorders

- Paul Barnett PhD
- Mistry Gage MPH
- John Finney PhD

## Outline

- Background on Effect of Inpatient Setting on Substance Use Treatment
- Instrumental Variables (IV)
- VA Outcomes Monitoring Project

## Overview

- Assess effectiveness of outcomes and  cost in inpatient and outpatient substance use treatment while controlling for selection bias

## Background

- Treatment for substance use disorders has shifted from inpatient to outpatient setting
- Payers have shifted from inpatient treatment
- Extra cost of inpatient setting
- number of studies have not found any advantage to inpatient care

**Notes:**
number of studies that suggest that inpatient treatment is not anymore effective than outpatient treatment

## Clinical Trials

- No advantage to inpatient setting
- (Alterman et al. 1994; McKay et al. 1998; Rychtarik et al. 2000; Weithmann and Hoffman, 2005)
- Short-term advantages of inpatient treatment not sustained over the long-run
- (Greenwood et al. 2001;Schneider et al. 1996)

**Notes:**
In the randomized experiment, exogenous variation in treatment is provided by the random assignment of participants to the treatment and control conditions 
Randomized trial where people are randomly placed into the treatment and control groups have strong internal validity, it would be easy to estimate the treatment effect by comparing the mean outcomes of the treated and control groups. 

## Clinical Trials

- Limitations
- Exclusion criteria
- Substance use disorder severity
- Psychiatric co-morbidities
-                                    (Finney et al. 1996)
- Estimate treatment effect
- External validity

**Notes:**
Exclusion/Inclusion criteria
	co-morbidites

Effectiveness
	desired results obtained under optimal conditions but not necessarily under standard conditions
External Validity
	Population included in trials nort representative of the population under 	treatment. Results cannot be generalized
	Women and the elderly are often underrepresented

## Observational Studies

- Few studies show better outcomes from inpatient treatment
- 	(McKay et al. 2002; Mojtabai and Graff Zivin, 2003)
- Majority find no significance once patient characteristics are controlled
- (Harrison and Asche, 1998;McLellan et al. 1994)

**Notes:**
Observational studies allow for the ability to estimate treatment effect in real world conditions. 

In observational studies there is no control over treatment assignment
 How can we adjust for selection bias in observational studies) ? (go to next slide)

## Observational Studies

- Limitations
- Selection bias
- Non-randomized groups differ in observed and unobserved characteristics
- Sicker patients are assigned to inpatient care
- Differential selection into treatment group

**Notes:**
Observational studies allow for the ability to estimate treatment effect in real world conditions. 

In observational studies there is no control over treatment assignment

However, studies lack randomization of subjects into treatment groups and must address selection bias to estimate the effect of treatment.
Non randomized groups usuaully differ in observed and unobserved characteristics resulting in differential selection into treatment groups causing selection bias when evaluating the effect of treatment 

How does one adjust for selection bias in observational studies? (next slide)

## Instrumental Variables (IV)

- Predictor of treatment that has no independent effect on outcome
- Uses exogenous variables to control for selection bias

**Notes:**
Several ways to control for selection bias 
Econometric technique
IV produce a consistent estimator of a parameter when the explanatory variables (covariates) are correlated with the error terms. Such correlation can be caused by endogeneity, by omitted covariates, or by measurement errors in the covariates. 
In OMP some sites make greater use of residential and inpatient setting

## Instrumental Variables (IV)

**Y=****β****0**** + ****β****1****T**** ****+ ****β****2****X +****ε****i**
- Treatment (T) may be correlated with unobserved case-mix εi
- Violation of assumptions of OLS
- β1 may be biased
- Parameter may understate or overstate effect of T on Y

**Notes:**
Violation of OLS-Errors correlated with independent variables 

## Instrumental Variables (IV) Assumptions

- Relevance: Instrumental variable causes variation in the treatment variable
- Exogeneity: Instrumental variable has no direct effect on the outcome variable
- 					(Newhouse and McClellan, 1998)

## Instrumental Variables (IV) Examples

- Randomization to intervention group leads to more likely receipt of intervention
- Distance from patient residence to nearest cardiac facility predicts likelihood of coronary angioplasty
- Inclusion of drug in nearest pharmacy formulary predicts greater likelihood that drug will be dispensed to patient

## Instrumental Variables (IV)

- Instrumental                         Treatment
- variable(s)		          variable(s)
- 	Other 						Outcome
-  	variables 						measure
- 				(Adapted from Newhouse and McClellan, 1998)

## Instrumental Variables (IV) Estimation

- 2 Stage Least Square (2LS)
- Stage 1: T=α0 + α1Z + α2X + ε
-              ^
- Stage 2: Y=β0 + β1T + β2X + υ
- ^
- T: predicted value of T from the first
- stage
- Z: instrumental variable
- X: patient characteristics

**Notes:**
IV assumes you want to estimate a single equation from a system of equation 
Under the 2SLS approach, in a first stage, each endogenous covariate (predictor variable) is regressed on all valid instruments, including the full set of exogenous covariates in the main regression. Since the instruments are exogenous, these approximations of the endogenous covariates will not be correlated with the error term. So, intuitively they provide a way to analyze the relationship between the outcome variable and the endogenous covariates. 
In the second stage, the regression of interest is estimated as usual, except that in this each endogenous covariate is replaced with its approximation estimated in the first stage. 

## Testing Assumptions

- Good instrument
- statistical significance of the instrument in the first stage
- the instrument is not a direct determinant of outcomes

**Notes:**
For a good instrument, you are looking for statistical significance
of the instrument in the first stage. Also, intuitively, it should be clear that Z isn’t a direct
determinant of y.

## Outcomes Monitoring Project

- Evaluate performance of VA substance abuse programs
- Selection of study sites
- Enumerated and stratified
- 141 VA treatment sites
- Selection of study participants
- Up to 50 new patients at each site

## Outcomes Monitoring Program

- Enrollment
- 3 waves FY01-04
- 4,905 at baseline
- 3,286 at follow-up
- Outcomes Measure
- Improvement in substance use
- Abstinence

**Notes:**
To select study sites we enumerated all VA treatment programs and stratified by setting.   Sites were then randomly selected from each strata resulting in 136 study sites. At each program, up to 50 new patients were randomly selected and administered a brief, self-report questionnaire that assessed Addiction Severity Index. 
4,905 patients completed a self-report version of the Addiction Severity Index (ASI) at baseline and at follow-up, an average of 6.7 months later 3,286 patients completed the questionnaire.  

## Addiction Severity Index (ASI)

- Domains
- Alcohol
- Drug
- Psychiatric
- Measures problems/functioning in last 30 days
- Composite score ranges from 0 to 1

**Notes:**
The ASI consists of 7 domains (list domains we used).  ASI composite scores reflect patient functioning/problems over the past 30 days. Composite scores range from zero to one, with zero indicating no problems and one indicating severe problems. 

## Statistical Methods Outcomes

- Multiple Imputations
- To address missing ASI follow-up data and attrition bias
- (Schafer, 1997; Shafer & Graham, 2002)
- Estimated regressions for outcomes for each imputation and combined with Rubin’s method.

**Notes:**
Methods to control for confounding and measurement error 
Options for controlling selection bias in observational studies include:
(list those on slide)
Fixed effects (person as own control)
Propensity score
	constructing matched pairs or sets or strata that balance observed covariates
	does not balance unobserved covariates

## Statistical Methods Costs

- Cost adjusted to FY03
- Complete not imputated
- Estimated regressions with index and follow-up costs
- Pre-study cost
- Patient case-mix

**Notes:**
Methods to control for confounding and measurement error 
Options for controlling selection bias in observational studies include:
(list those on slide)
Fixed effects (person as own control)
Propensity score
	constructing matched pairs or sets or strata that balance observed covariates
	does not balance unobserved covariates

## Patient Characteristics by Setting Where Enrolled

|  | Inpatient Setting N=2,208 | Outpatient Setting
N=2,697 |
| --- | --- | --- |
| Psychiatric Diagnoses |  |  |
| Schizophrenia (%) | 7.73 | 7.12 |
| Bipolar Disorder (%) | 12.7** | 8.61 |
| Depression (%) | 42.5** | 35.6 |
| Other Non-Psychotic Disorder (%) | 68.7 | 71.6 |
| Post-Traumatic Stress Disorder (%) | 18.4* | 15.6 |
| Medical Diagnoses |  |  |
| AIDS (%) | 1.68 | 1.04 |
| Hepatitis C (%) | 18.5** | 8.87 |
| Hypertension (%) | 28.6** | 20.5 |
| Lower Back Pain | 14.8** | 10.2 |

-  * significant at 5%; ** significant at 1%

**Notes:**
For this analysis inpatient setting includes inpat, res, and dom
Outpatient setting includes IOP and outpt

## Patient Characteristics by Setting Where Enrolled

|  | Inpatient Setting N=2,208 | Outpatient Setting
N=2,697 |
| --- | --- | --- |
| Baseline Substance Use Diagnoses |  |  |
| Mean Days Intoxicated with Alcohol  | 7.75** | 5.81 |
| Mean Days with Drug Problems  | 8.06** | 5.43 |
| Mean Days with Alcohol Problems | 9.80** | 7.69 |
| Mean Days in a Controlled Environment  | 9.55** | 7.22 |
| Heroin Use (%) | 6.11* | 3.23 |
| Cocaine Use (%) | 34.7* | 24.6 |
| Amphetamine Use (%) | 6.11 | 4.30 |
| Stimulant Use (%) | 37.4* | 26.9 |

-  * significant at 5%; ** significant at 1%

## Challenge

- Selection bias
- Influence of unobserved covariates correlated with treatment
- Some medical centers make greater use of inpatient setting
- Program-level error terms
- Patients from same program correlated

## Why use Instrumental Variables (IV)?

- To take advantage of VA administrative data on facility level variation in the use of inpatient treatment

## Instrumental Variables (IV)

- Facility-Level Index
- Percentage of the facility's patients who received their substance abuse treatment in the inpatient setting
- Count of patients who had an
- inpatient substance use stay
- Total number of patients who received any type of
- substance use service inpatient or outpatient

**Notes:**
Facility level index captures policy that some sites more likely to use inpatient setting

## Instrumental Variables (IV)

- Instrumental                         Treatment
- variable(s)		          variable(s)
- 	Other 						Outcome
-  	variables 						measure
- 				(Adapted from Newhouse and McClellan, 1998)

## Instrumental Variables (IV) Relevance

- Treatment is correlated with error term
- Patients with severe co-morbidities in inpatient programs
- Some sites make greater use of residential and inpatient settings
- IV captures policy
- “natural experiment” caused by policy

## Characteristics of Patients by Type of Facility Where They Enrolled

|  | Centers More Likely to Use Inpatient Setting N=2,083 | Centers Less Likely to Use Inpatient Setting
N=2,822 |
| --- | --- | --- |
| Treatment |  |  |
| Inpatient Index Episode (%) | 67.3** | 26.2 |
| Facility-Level Index (IV) | 0.655 | 0.345 |
| Baseline Substance Use Diagnoses |  |  |
| Mean Days Intoxicated with Alcohol | 7.30 | 6.23 |
| Mean Days with Drug Problems | 7.77** | 5.76 |
| Mean Days with Alcohol Problems | 9.06 | 8.33 |
| Mean Days in a Controlled Environment | 8.34 | 8.23 |
| Heroin Use (%) | 6.58** | 3.01 |
| Cocaine Use (%) | 33.7* | 25.7 |
| Stimulant Use (%) | 36.3* | 28.2 |

-  * significant at 5%; ** significant at 1%

**Notes:**
IV only works is outpatient treatment at medical centers more likely to use inpatient setting

## Two Stage Least Square Regression 

- Stage 1: T=α0 + α1Z + α2X + ε
- T=Treatment (inpatient treatment 0/1)
- Z=Instrumental variable (percent of inpatient substance use treatment)
- X=Covariates (baseline substance use and psychiatric co-morbidities)
- ε=Error term

**Notes:**
IV assumes you want to estimate a single equation from a system of equation 
Under the 2SLS approach, in a first stage, each endogenous covariate (predictor variable) is regressed on all valid instruments, including the full set of exogenous covariates in the main regression. Since the instruments are exogenous, these approximations of the endogenous covariates will not be correlated with the error term. So, intuitively they provide a way to analyze the relationship between the outcome variable and the endogenous covariates. In the second stage, the regression of interest is estimated as usual, except that in this each endogenous covariate is replaced with its approximation estimated in the first stage. 

## Instrumental Variable Estimation: 1st Stage Regression Parameters

|  | Coefficient | Z Statistic |
| --- | --- | --- |
| Facility-Level Index (IV) | 12.5** | 14.68 |
| Baseline Mean Psychiatric ASI Problem Score | 0.417 | 0.99 |
| Baseline Mean Days Intoxicated with Alcohol | 0.012 | 1.08 |
| Baseline Mean Days with Drug Problems | 0.023* | 1.99 |
| Baseline Stimulant Use | -0.246 | -0.94 |
| Baseline Mean Days in a Controlled Environment | 0.024* | 2.15 |
| Baseline Heroin Use | -.430 | -0.74 |

-  * significant at 5%; ** significant at 1%

## Testing Assumptions

- Correlated with treatment
- First stage regression treatment significantly predicted by IV
- No direct effect on outcomes
- Construction/Assumption
- Outpatient treatment observed at sites with propensity to treat inpatient

## Two Stage Least Square Regression 

-        ^
- Stage 2: Y=β0 + β1T + β2X + υ
- Y=Outcome (improvement in substance use and abstinence)
- T=Treatment (inpatient treatment 0/1-instrumented)
- X=Covariates (baseline substance use and psychiatric co-morbidity)
- υ =Error term

**Notes:**
IV assumes you want to estimate a single equation from a system of equation 
Under the 2SLS approach, in a first stage, each endogenous covariate (predictor variable) is regressed on all valid instruments, including the full set of exogenous covariates in the main regression. Since the instruments are exogenous, these approximations of the endogenous covariates will not be correlated with the error term. So, intuitively they provide a way to analyze the relationship between the outcome variable and the endogenous covariates. In the second stage, the regression of interest is estimated as usual, except that in this each endogenous covariate is replaced with its approximation estimated in the first stage. 

## Instrumental Variable Estimation: 2nd Stage Regression Coefficients  

|  | Change in Days Intoxicated with Alcohol | Change in Days with Drug Problems |
| --- | --- | --- |
| Inpatient Index Episode | -1.50** | -1.36* |
| Mean Psychiatric ASI Problem Score | -1.84 | -1.67** |
| Mean Days with Alcohol Problems | -0.465** | -0.161** |
| Mean Days with Drug Problems | 0.0003 | N/A |
| Stimulant Use | -0.800** | -8.21** |
| Mean Days in a Controlled Environment | 0.101** | 0.080** |
| Heroin Use | N/A | -5.37** |

-  * significant at 5%; ** significant at 1%

**Notes:**
Coefficients

If more days in a controlled environment chnge score will be more positive

## Instrumental Variable Estimation: 2nd Stage Regression Coefficients 

|  | Index Care Costs | Total Index and Follow-up Care Costs |
| --- | --- | --- |
| Inpatient Index Episode | 8,595** | 10,549** |
| Mean Psychiatric ASI Problem Score | 1,084** | 7,083** |
| Mean Days with Drug Problems | 1.22 | 31.6 |
| Heroin Use | 1,316** | 228 |
| Hepatitis C | 1,316** | 1,102 |
| Mean Days Experienced Medical Problems | -10.2 | 9.39 |
| Mean Days Intoxicated with Alcohol | -3.07 | 63.9* |
| Pre-Study Substance Abuse Care Costs | 0.351** | 0.214** |
| Pre-Study Psychiatric Care Costs | -0.005 | 0.188** |
| Pre-Study Other Care Costs | 0.234** | 0.366** |

-  * significant at 5%; ** significant at 1%

**Notes:**
Coefficients 

## Estimation of Effect of Setting While Controlling for Baseline Case-mix

- Used regression parameters to simulate outcomes and cost of typical patient
- Estimated mean value for setting while controlling for baseline substance use and case-mix variables
- Estimated value of outcomes and cost at the mean of the covariates

## Estimation of Effect of Setting While Controlling for Baseline Case-mix

- Summarized findings from 5 outcome regressions and 6 cost regressions
- Mean change in substance use for the average patient treated in inpatient and outpatient setting
- Mean costs of care for the average patient treated in inpatient and outpatient setting

## Evaluation of Parameter at the Mean: Effect of Treatment Setting on Outcomes

|  | Inpatient Setting | Outpatient Setting |
| --- | --- | --- |
| Outcomes |  |  |
| Mean Change in Days Intoxicated with Alcohol | -5.45** | -3.96 |
| Mean Change in Days with Drug Problems | -5.23* | -3.87 |
| Follow-up Heroin Use (%) | 9.90 | 9.20 |
| Follow-up Cocaine Use (%) | 27.0 | 24.9 |
| Follow-up Amphetamine Use (%) | 10.8 | 11.8 |

-  * significant at 5%; ** significant at 1%

## Evaluation of Parameter at the Mean: Effect of Treatment Setting on Cost

|  | Mean Cost in
Inpatient Setting | Mean Cost in
Outpatient Setting |
| --- | --- | --- |
| Index Care | 12,567** | 3,972 |
| Follow-up Psychiatric | 3,714 | 2,890 |
| Follow-up Substance Abuse Care | 3,535 | 2,216 |
| Sub-total, Follow-up Care Psychiatric and Substance Abuse Care | 7,299 | 5,070 |
| Follow-up Other Care | 2,985 | 3,434 |
| Total, Index and Follow-up Care | 22,977** | 12,428 |

-  * significant at 5%; ** significant at 1%

**Notes:**
“inpatient” group combines both patients treated in both inpatient and residential settings.   So I assume the cost for the “inpatient” group underestimates the cost of inpatient treatment and overestimates the costs of residential treatment – correct?  On the other hand, the patients seen in inpatient unit probably have greater severity (e.g., psychiatric diagnoses). 

## Results 

- Improvement
- Greater for inpatients than outpatients
- Change in days intoxicated with alcohol
- -5.45 vs. -3.96
- Change in days with drug problems greater
- -5.23 vs. -3.87

## Results 

- Abstinence
- No significant difference observed between settings
- Mean Costs
- Significantly greater in inpatient setting
- Index care ($14,592 vs. $1,964)
- Total, index and follow-up care ($25,343 vs. $9,999)

## Conclusion

- Inpatient setting treats patients with severe psychiatric, medical, and substance use problems
- An analysis that controls for selection bias found inpatient setting yields significantly better outcomes
- Setting affects cost

## Implications

- Is it worth spending more on treatment for greater outcomes?
- Inpatient treatment is ~$8,600 greater than outpatient treatment for an extra 1.5 days reduction in substance use

**Notes:**
We assumed that all values on the ASI have the same value.  For example a reduction from .9 to .8 should be treated equivalently as .6 to .5.  If intervals among people with more problems at the outset are given a greater value, then we’ve potentially understated the value of settings that treat individuals with more problems.   

Since ASI is not on a an interval scale are policy makers willing to spend more for changes in those with higher problem scores?

## Summary

- Substance abuse treatment has shifted from inpatient to outpatient setting
- Extra cost of inpatient setting
- Lack of evidence of inpatient superiority

## Summary

- Instrumental variables control for selection bias
- Instrumental variables are associated with treatment
- Instrumental variables are independent of outcomes

## Summary

- Instrumental                         Treatment
- variable(s)		          variable(s)
- 	Other 						Outcome
-  	variables 						measure
- 				(Adapted from Newhouse and McClellan, 1998)

## Summary

- VA Outcomes Monitoring Project
- IV captures policy
- Some sites make greater use of inpatient setting
- After controlling for selection bias
- Change in substance use severity greater in inpatient care
- More costs incurred in inpatient setting

## References

- Alterman A, O’Brien C, McLellan T, August D, Snider E, Droba M, Cornish J, Hall C, Raphaelson A, Schrade F. 1994. Effectiveness and costs of inpatient versus day hospital cocaine rehabilitation. J Nerv Ment Dis 182:157-63
- Finney J, Hahn A, Moos R. 1996. The Effectiveness of inpatient and outpatient treatment for alcohol abuse: the need to focus on mediators and moderators of setting effects. Addiction 91(12):1773-96
- Greenwood G, Woods W, Guydish J, Bein E. 2001. Relapse outcomes in a randomized trial of residential and day drug abuse treatment. J Sub Abuse Treatment 20: 15-23
- McKay J, Alterman A, McLellan T, Boardman C, Mulvaney F, O’Brien C. 1998. Random versus nonrandom assignment in the evaluation of treatment for cocaine abusers. J Consulting and Clinical Psychology 66(4):697-701
- McKay J, Donovan D, McLellan T, Krupski A, Geary K, Cecere J. 2002. Evaluation of full vs. partial continuum of care in the treatment of publicly funded substance abusers in washington state. Am J Drug Alcohol 28(2):307-38

## References

- McLellan T, Alterman A, Metzger D, Grissom G, Woody G, Luborsky L, O’Brien C. 1994. Similarity of outcome predictors across opiate, cocaine, and alcohol treatments: role of treatment services. J Consulting and Clinical Psychology 62(6):1141-58
- Mojtabai R, Graff Zavin J. 2003. Effectiveness and cost-effectiveness of four treatment modalities for substance disorders: a propensity score analysis. Health Services Research 38(1):233-59
- Newhouse J, McClellan M. 1998. Econometrics in outcomes research: the use of instrumental variables. Annu. Rev Public Health 19:17-34
- Rychtarik R, Connors G, Wirtz P, Whitney R, McGillicuddy N, Fitterling J. 2000. Treatment settings for persons with alcoholism: evidence for matching clients to inpatient versus outpatient care. J Consulting and Clinical Psychology 68(2):277-89
- Schneider R, Mittelmeier C, Gadish D. 1996. Day versus inpatient treatment for cocaine dependence: an experimental comparison. J Ment Health Admin 23(2):234-45
- Weithmann G, Hoffman. 2005. A randomised clinical trial of in-patient versus combined day hospital treatment of alcoholism: primary and secondary outcome measures. Eur Addict Res 11:197-203