Comparison of Cost and Outcomes of Inpatient and Outpatient Treatment for Substance Use Disorders

Paul Barnett PhD

Mistry Gage MPH

John Finney PhD

Outline
        - Background on Effect of Inpatient Setting on Substance Use Treatment
        - Instrumental Variables (IV)
        - VA Outcomes Monitoring Project 

Overview
        - Assess effectiveness of outcomes and  cost in inpatient and outpatient substance use treatment while controlling for selection bias

Background
        - Treatment for substance use disorders has shifted from inpatient to outpatient setting
        - Payers have shifted from inpatient treatment

Extra cost of inpatient setting 

number of studies have not found any advantage to inpatient care

number of studies that suggest that inpatient treatment is not anymore effective than outpatient treatment

Clinical Trials
        - No advantage to inpatient setting

(Alterman et al. 1994; McKay et al. 1998; Rychtarik et al. 2000; Weithmann and Hoffman, 2005)
        - Short-term advantages of inpatient treatment not sustained over the long-run

(Greenwood et al. 2001;Schneider et al. 1996)

In the randomized experiment, exogenous variation in treatment is provided by the random assignment of participants to the treatment and control conditions 

Randomized trial where people are randomly placed into the treatment and control groups have strong internal validity, it would be easy to estimate the treatment effect by comparing the mean outcomes of the treated and control groups. 

Clinical Trials
        - Limitations

Exclusion criteria

Substance use disorder severity

Psychiatric co-morbidities

                                   (Finney et al. 1996)

Estimate treatment effect

External validity

Exclusion/Inclusion criteria

	co-morbidites

Effectiveness

	desired results obtained under optimal conditions but not necessarily under standard conditions

External Validity

	Population included in trials nort representative of the population under 	treatment. Results cannot be generalized

	Women and the elderly are often underrepresented

Observational Studies
        - Few studies show better outcomes from inpatient treatment 

	(McKay et al. 2002; Mojtabai and Graff Zivin, 2003)
        - Majority find no significance once patient characteristics are controlled

(Harrison and Asche, 1998;McLellan et al. 1994)

Observational studies allow for the ability to estimate treatment effect in real world conditions. 

In observational studies there is no control over treatment assignment

 How can we adjust for selection bias in observational studies) ? (go to next slide)

Observational Studies
        - Limitations

Selection bias

Non-randomized groups differ in observed and unobserved characteristics 

Sicker patients are assigned to inpatient care 

Differential selection into treatment group 

Observational studies allow for the ability to estimate treatment effect in real world conditions. 

In observational studies there is no control over treatment assignment

However, studies lack randomization of subjects into treatment groups and must address selection bias to estimate the effect of treatment.

Non randomized groups usuaully differ in observed and unobserved characteristics resulting in differential selection into treatment groups causing selection bias when evaluating the effect of treatment 

How does one adjust for selection bias in observational studies? (next slide)

Instrumental Variables (IV)
        - Predictor of treatment that has no independent effect on outcome
        - Uses exogenous variables to control for selection bias

Several ways to control for selection bias 

Econometric technique

IV produce a consistent estimator of a parameter when the explanatory variables (covariates) are correlated with the error terms. Such correlation can be caused by endogeneity, by omitted covariates, or by measurement errors in the covariates. 

In OMP some sites make greater use of residential and inpatient setting

Instrumental Variables (IV)

Y=β0 + β1T + β2X +εi
        - Treatment (T) may be correlated with unobserved case-mix εi
        - Violation of assumptions of OLS
        - β1 may be biased

Parameter may understate or overstate effect of T on Y

Violation of OLS-Errors correlated with independent variables 

Instrumental Variables (IV) Assumptions
        - Relevance: Instrumental variable causes variation in the treatment variable
        - Exogeneity: Instrumental variable has no direct effect on the outcome variable

					(Newhouse and McClellan, 1998)

Instrumental Variables (IV) Examples
        - Randomization to intervention group leads to more likely receipt of intervention
        - Distance from patient residence to nearest cardiac facility predicts likelihood of coronary angioplasty
        - Inclusion of drug in nearest pharmacy formulary predicts greater likelihood that drug will be dispensed to patient

Instrumental Variables (IV)

Instrumental                         Treatment 

variable(s)		          variable(s)

	Other 						Outcome 

 	variables 						measure

				(Adapted from Newhouse and McClellan, 1998)

Instrumental Variables (IV) Estimation

2 Stage Least Square (2LS)

Stage 1: T=α0 + α1Z + α2X + ε

             ^

Stage 2: Y=β0 + β1T + β2X + υ

^

T: predicted value of T from the first

stage 

Z: instrumental variable

X: patient characteristics

IV assumes you want to estimate a single equation from a system of equation 

Under the 2SLS approach, in a first stage, each endogenous covariate (predictor variable) is regressed on all valid instruments, including the full set of exogenous covariates in the main regression. Since the instruments are exogenous, these approximations of the endogenous covariates will not be correlated with the error term. So, intuitively they provide a way to analyze the relationship between the outcome variable and the endogenous covariates. 

In the second stage, the regression of interest is estimated as usual, except that in this each endogenous covariate is replaced with its approximation estimated in the first stage. 

Testing Assumptions
        - Good instrument

statistical significance of the instrument in the first stage

the instrument is not a direct determinant of outcomes

For a good instrument, you are looking for statistical significance

of the instrument in the first stage. Also, intuitively, it should be clear that Z isn’t a direct

determinant of y.

Outcomes Monitoring Project
        - Evaluate performance of VA substance abuse programs
        - Selection of study sites

Enumerated and stratified

141 VA treatment sites
        - Selection of study participants

Up to 50 new patients at each site

Outcomes Monitoring Program
        - Enrollment

3 waves FY01-04

4,905 at baseline

3,286 at follow-up
        - Outcomes Measure 

Improvement in substance use

Abstinence

To select study sites we enumerated all VA treatment programs and stratified by setting.   Sites were then randomly selected from each strata resulting in 136 study sites. At each program, up to 50 new patients were randomly selected and administered a brief, self-report questionnaire that assessed Addiction Severity Index. 

4,905 patients completed a self-report version of the Addiction Severity Index (ASI) at baseline and at follow-up, an average of 6.7 months later 3,286 patients completed the questionnaire.  

Addiction Severity Index (ASI)
        - Domains

Alcohol

Drug

Psychiatric
        - Measures problems/functioning in last 30 days
        - Composite score ranges from 0 to 1

The ASI consists of 7 domains (list domains we used).  ASI composite scores reflect patient functioning/problems over the past 30 days. Composite scores range from zero to one, with zero indicating no problems and one indicating severe problems. 

Statistical Methods Outcomes
        - Multiple Imputations

To address missing ASI follow-up data and attrition bias

(Schafer, 1997; Shafer & Graham, 2002)
        - Estimated regressions for outcomes for each imputation and combined with Rubin’s method.

Methods to control for confounding and measurement error 

Options for controlling selection bias in observational studies include:

(list those on slide)

Fixed effects (person as own control)

Propensity score

	constructing matched pairs or sets or strata that balance observed covariates

	does not balance unobserved covariates

Statistical Methods Costs
        - Cost adjusted to FY03
        - Complete not imputated
        - Estimated regressions with index and follow-up costs

Pre-study cost 

Patient case-mix

Methods to control for confounding and measurement error 

Options for controlling selection bias in observational studies include:

(list those on slide)

Fixed effects (person as own control)

Propensity score

	constructing matched pairs or sets or strata that balance observed covariates

	does not balance unobserved covariates

Patient Characteristics by Setting Where Enrolled

 * significant at 5%; ** significant at 1%
| Inpatient Setting N=2,208          | Outpatient SettingN=2,697 |
|------------------------------------|---------------------------|
| Psychiatric Diagnoses              |
| Schizophrenia (%)                  | 7.73                      | 7.12 |
| Bipolar Disorder (%)               | 12.7**                    | 8.61 |
| Depression (%)                     | 42.5**                    | 35.6 |
| Other Non-Psychotic Disorder (%)   | 68.7                      | 71.6 |
| Post-Traumatic Stress Disorder (%) | 18.4*                     | 15.6 |
| Medical Diagnoses                  |
| AIDS (%)                           | 1.68                      | 1.04 |
| Hepatitis C (%)                    | 18.5**                    | 8.87 |
| Hypertension (%)                   | 28.6**                    | 20.5 |
| Lower Back Pain                    | 14.8**                    | 10.2 |

For this analysis inpatient setting includes inpat, res, and dom

Outpatient setting includes IOP and outpt

Patient Characteristics by Setting Where Enrolled

 * significant at 5%; ** significant at 1%
| Inpatient Setting N=2,208             | Outpatient SettingN=2,697 |
|---------------------------------------|---------------------------|
| Baseline Substance Use Diagnoses      |
| Mean Days Intoxicated with Alcohol    | 7.75**                    | 5.81 |
| Mean Days with Drug Problems          | 8.06**                    | 5.43 |
| Mean Days with Alcohol Problems       | 9.80**                    | 7.69 |
| Mean Days in a Controlled Environment | 9.55**                    | 7.22 |
| Heroin Use (%)                        | 6.11*                     | 3.23 |
| Cocaine Use (%)                       | 34.7*                     | 24.6 |
| Amphetamine Use (%)                   | 6.11                      | 4.30 |
| Stimulant Use (%)                     | 37.4*                     | 26.9 |

Challenge
        - Selection bias 

Influence of unobserved covariates correlated with treatment

Some medical centers make greater use of inpatient setting 
        - Program-level error terms

Patients from same program correlated

Why use Instrumental Variables (IV)?
        - To take advantage of VA administrative data on facility level variation in the use of inpatient treatment

Instrumental Variables (IV)
        - Facility-Level Index 

Percentage of the facility's patients who received their substance abuse treatment in the inpatient setting

Count of patients who had an 

inpatient substance use stay

Total number of patients who received any type of 

substance use service inpatient or outpatient

          - Facility level index captures policy that some sites more likely to use inpatient setting

Instrumental Variables (IV)

Instrumental                         Treatment 

variable(s)		          variable(s)

	Other 						Outcome 

 	variables 						measure

				(Adapted from Newhouse and McClellan, 1998)

Instrumental Variables (IV) Relevance
        - Treatment is correlated with error term

Patients with severe co-morbidities in inpatient programs 

Some sites make greater use of residential and inpatient settings
        - IV captures policy

“natural experiment” caused by policy

Characteristics of Patients by Type of Facility Where They Enrolled

 * significant at 5%; ** significant at 1%
| Centers More Likely to Use Inpatient Setting N=2,083 | Centers Less Likely to Use Inpatient SettingN=2,822 |
|------------------------------------------------------|-----------------------------------------------------|
| Treatment                                            |
| Inpatient Index Episode (%)                          | 67.3**                                              | 26.2 |
| Facility-Level Index (IV)                            | 0.655                                               | 0.345 |
| Baseline Substance Use Diagnoses                     |
| Mean Days Intoxicated with Alcohol                   | 7.30                                                | 6.23 |
| Mean Days with Drug Problems                         | 7.77**                                              | 5.76 |
| Mean Days with Alcohol Problems                      | 9.06                                                | 8.33 |
| Mean Days in a Controlled Environment                | 8.34                                                | 8.23 |
| Heroin Use (%)                                       | 6.58**                                              | 3.01 |
| Cocaine Use (%)                                      | 33.7*                                               | 25.7 |
| Stimulant Use (%)                                    | 36.3*                                               | 28.2 |

IV only works is outpatient treatment at medical centers more likely to use inpatient setting

Two Stage Least Square Regression 

Stage 1: T=α0 + α1Z + α2X + ε

T=Treatment (inpatient treatment 0/1)

Z=Instrumental variable (percent of inpatient substance use treatment)

X=Covariates (baseline substance use and psychiatric co-morbidities)

ε=Error term

IV assumes you want to estimate a single equation from a system of equation 

Under the 2SLS approach, in a first stage, each endogenous covariate (predictor variable) is regressed on all valid instruments, including the full set of exogenous covariates in the main regression. Since the instruments are exogenous, these approximations of the endogenous covariates will not be correlated with the error term. So, intuitively they provide a way to analyze the relationship between the outcome variable and the endogenous covariates. In the second stage, the regression of interest is estimated as usual, except that in this each endogenous covariate is replaced with its approximation estimated in the first stage. 

Instrumental Variable Estimation: 1st Stage Regression Parameters

 * significant at 5%; ** significant at 1%
| Coefficient                                    | Z Statistic |
|------------------------------------------------|-------------|
| Facility-Level Index (IV)                      | 12.5**      | 14.68 |
| Baseline Mean Psychiatric ASI Problem Score    | 0.417       | 0.99 |
| Baseline Mean Days Intoxicated with Alcohol    | 0.012       | 1.08 |
| Baseline Mean Days with Drug Problems          | 0.023*      | 1.99 |
| Baseline Stimulant Use                         | -0.246      | -0.94 |
| Baseline Mean Days in a Controlled Environment | 0.024*      | 2.15 |
| Baseline Heroin Use                            | -.430       | -0.74 |

Testing Assumptions
        - Correlated with treatment

First stage regression treatment significantly predicted by IV
        - No direct effect on outcomes 

Construction/Assumption

Outpatient treatment observed at sites with propensity to treat inpatient

Two Stage Least Square Regression 

       ^

Stage 2: Y=β0 + β1T + β2X + υ

Y=Outcome (improvement in substance use and abstinence)

T=Treatment (inpatient treatment 0/1-instrumented)

X=Covariates (baseline substance use and psychiatric co-morbidity)

υ =Error term

IV assumes you want to estimate a single equation from a system of equation 

Under the 2SLS approach, in a first stage, each endogenous covariate (predictor variable) is regressed on all valid instruments, including the full set of exogenous covariates in the main regression. Since the instruments are exogenous, these approximations of the endogenous covariates will not be correlated with the error term. So, intuitively they provide a way to analyze the relationship between the outcome variable and the endogenous covariates. In the second stage, the regression of interest is estimated as usual, except that in this each endogenous covariate is replaced with its approximation estimated in the first stage. 

Instrumental Variable Estimation: 2nd Stage Regression Coefficients  

 * significant at 5%; ** significant at 1%
| Change in Days Intoxicated with Alcohol | Change in Days with Drug Problems |
|-----------------------------------------|-----------------------------------|
| Inpatient Index Episode                 | -1.50**                           | -1.36* |
| Mean Psychiatric ASI Problem Score      | -1.84                             | -1.67** |
| Mean Days with Alcohol Problems         | -0.465**                          | -0.161** |
| Mean Days with Drug Problems            | 0.0003                            | N/A |
| Stimulant Use                           | -0.800**                          | -8.21** |
| Mean Days in a Controlled Environment   | 0.101**                           | 0.080** |
| Heroin Use                              | N/A                               | -5.37** |

Coefficients

If more days in a controlled environment chnge score will be more positive

Instrumental Variable Estimation: 2nd Stage Regression Coefficients 

 * significant at 5%; ** significant at 1%
| Index Care Costs                       | Total Index and Follow-up Care Costs |
|----------------------------------------|--------------------------------------|
| Inpatient Index Episode                | 8,595**                              | 10,549** |
| Mean Psychiatric ASI Problem Score     | 1,084**                              | 7,083** |
| Mean Days with Drug Problems           | 1.22                                 | 31.6 |
| Heroin Use                             | 1,316**                              | 228 |
| Hepatitis C                            | 1,316**                              | 1,102 |
| Mean Days Experienced Medical Problems | -10.2                                | 9.39 |
| Mean Days Intoxicated with Alcohol     | -3.07                                | 63.9* |
| Pre-Study Substance Abuse Care Costs   | 0.351**                              | 0.214** |
| Pre-Study Psychiatric Care Costs       | -0.005                               | 0.188** |
| Pre-Study Other Care Costs             | 0.234**                              | 0.366** |

Coefficients 

Estimation of Effect of Setting While Controlling for Baseline Case-mix
        - Used regression parameters to simulate outcomes and cost of typical patient

Estimated mean value for setting while controlling for baseline substance use and case-mix variables

Estimated value of outcomes and cost at the mean of the covariates

Estimation of Effect of Setting While Controlling for Baseline Case-mix
        - Summarized findings from 5 outcome regressions and 6 cost regressions

Mean change in substance use for the average patient treated in inpatient and outpatient setting

Mean costs of care for the average patient treated in inpatient and outpatient setting

Evaluation of Parameter at the Mean: Effect of Treatment Setting on Outcomes

 * significant at 5%; ** significant at 1%
| Inpatient Setting                            | Outpatient Setting |
|----------------------------------------------|--------------------|
| Outcomes                                     |
| Mean Change in Days Intoxicated with Alcohol | -5.45**            | -3.96 |
| Mean Change in Days with Drug Problems       | -5.23*             | -3.87 |
| Follow-up Heroin Use (%)                     | 9.90               | 9.20 |
| Follow-up Cocaine Use (%)                    | 27.0               | 24.9 |
| Follow-up Amphetamine Use (%)                | 10.8               | 11.8 |

Evaluation of Parameter at the Mean: Effect of Treatment Setting on Cost

 * significant at 5%; ** significant at 1%
| Mean Cost inInpatient Setting                                  | Mean Cost inOutpatient Setting |
|----------------------------------------------------------------|--------------------------------|
| Index Care                                                     | 12,567**                       | 3,972 |
| Follow-up Psychiatric                                          | 3,714                          | 2,890 |
| Follow-up Substance Abuse Care                                 | 3,535                          | 2,216 |
| Sub-total, Follow-up Care Psychiatric and Substance Abuse Care | 7,299                          | 5,070 |
| Follow-up Other Care                                           | 2,985                          | 3,434 |
| Total, Index and Follow-up Care                                | 22,977**                       | 12,428 |

“inpatient” group combines both patients treated in both inpatient and residential settings.   So I assume the cost for the “inpatient” group underestimates the cost of inpatient treatment and overestimates the costs of residential treatment – correct?  On the other hand, the patients seen in inpatient unit probably have greater severity (e.g., psychiatric diagnoses). 

Results 

Improvement

Greater for inpatients than outpatients

Change in days intoxicated with alcohol

-5.45 vs. -3.96

Change in days with drug problems greater 

-5.23 vs. -3.87

Results 

Abstinence

No significant difference observed between settings

Mean Costs

Significantly greater in inpatient setting

Index care ($14,592 vs. $1,964)

Total, index and follow-up care ($25,343 vs. $9,999)   

Conclusion
        - Inpatient setting treats patients with severe psychiatric, medical, and substance use problems
        - An analysis that controls for selection bias found inpatient setting yields significantly better outcomes
        - Setting affects cost

Implications

Is it worth spending more on treatment for greater outcomes?

Inpatient treatment is ~$8,600 greater than outpatient treatment for an extra 1.5 days reduction in substance use

We assumed that all values on the ASI have the same value.  For example a reduction from .9 to .8 should be treated equivalently as .6 to .5.  If intervals among people with more problems at the outset are given a greater value, then we’ve potentially understated the value of settings that treat individuals with more problems.   

Since ASI is not on a an interval scale are policy makers willing to spend more for changes in those with higher problem scores?

Summary
        - Substance abuse treatment has shifted from inpatient to outpatient setting
        - Extra cost of inpatient setting
        - Lack of evidence of inpatient superiority

Summary
        - Instrumental variables control for selection bias
        - Instrumental variables are associated with treatment
        - Instrumental variables are independent of outcomes 

Summary

Instrumental                         Treatment 

variable(s)		          variable(s)

	Other 						Outcome 

 	variables 						measure

				(Adapted from Newhouse and McClellan, 1998)

Summary
        - VA Outcomes Monitoring Project 

IV captures policy

Some sites make greater use of inpatient setting 

After controlling for selection bias 

Change in substance use severity greater in inpatient care

More costs incurred in inpatient setting 

References

Alterman A, O’Brien C, McLellan T, August D, Snider E, Droba M, Cornish J, Hall C, Raphaelson A, Schrade F. 1994. Effectiveness and costs of inpatient versus day hospital cocaine rehabilitation. J Nerv Ment Dis 182:157-63

Finney J, Hahn A, Moos R. 1996. The Effectiveness of inpatient and outpatient treatment for alcohol abuse: the need to focus on mediators and moderators of setting effects. Addiction 91(12):1773-96

Greenwood G, Woods W, Guydish J, Bein E. 2001. Relapse outcomes in a randomized trial of residential and day drug abuse treatment. J Sub Abuse Treatment 20: 15-23

McKay J, Alterman A, McLellan T, Boardman C, Mulvaney F, O’Brien C. 1998. Random versus nonrandom assignment in the evaluation of treatment for cocaine abusers. J Consulting and Clinical Psychology 66(4):697-701

McKay J, Donovan D, McLellan T, Krupski A, Geary K, Cecere J. 2002. Evaluation of full vs. partial continuum of care in the treatment of publicly funded substance abusers in washington state. Am J Drug Alcohol 28(2):307-38

References

McLellan T, Alterman A, Metzger D, Grissom G, Woody G, Luborsky L, O’Brien C. 1994. Similarity of outcome predictors across opiate, cocaine, and alcohol treatments: role of treatment services. J Consulting and Clinical Psychology 62(6):1141-58

Mojtabai R, Graff Zavin J. 2003. Effectiveness and cost-effectiveness of four treatment modalities for substance disorders: a propensity score analysis. Health Services Research 38(1):233-59

Newhouse J, McClellan M. 1998. Econometrics in outcomes research: the use of instrumental variables. Annu. Rev Public Health 19:17-34

Rychtarik R, Connors G, Wirtz P, Whitney R, McGillicuddy N, Fitterling J. 2000. Treatment settings for persons with alcoholism: evidence for matching clients to inpatient versus outpatient care. J Consulting and Clinical Psychology 68(2):277-89

Schneider R, Mittelmeier C, Gadish D. 1996. Day versus inpatient treatment for cocaine dependence: an experimental comparison. J Ment Health Admin 23(2):234-45

Weithmann G, Hoffman. 2005. A randomised clinical trial of in-patient versus combined day hospital treatment of alcoholism: primary and secondary outcome measures. Eur Addict Res 11:197-203