Is Physician Organizatio

Is Physician Organizatio

Related to Efficienc

y an

Patient Outcomes? 

Todd Wa

gne

r

Joe Doyle (MIT)

Steve Ewer (U Wisconsin) Steve Ewer (U Wisconsin)

June 2011

Thanks to Andrew Siroka for research assistance.
Premise

 The key to improving the effectiveness of h

lies w t c c a s. with clinicians.

 Although they often lack information and re

1 to provide optimal care, and often face perv

incentives to overtreat, they are responsible 

2 production and distribution of care.

 Major gains in effectiveness requires unders

Major gains in effectiveness requires unders

and changing clinicians’ behavior.

1. Committee on Quality Health Care in America IoM. Crossing the quality chasm: a new health system for the 21st cen

National Academics Press; 2001.

2. Fuchs VR. Who shall live? Health, economics, and social choice. New York,: Basic Books; 1974.
Clinician Behavior

 A large literature has focused on using incen

c

h

a

nge

clini

c

i

an be

hav

i

o

r.

1 c a ge c c a be av o .

 Less work has focused on culture. Culture matters because it shapes clinicians

judgment communication and teamwork judgment, communication, and teamwork. 

1. Rosenthal MB, Dudley RA. Pay-for-performance: will the latest payment trend improve care? JAMA. Feb 21 2007
Contribution of this Stu

 We follow a natural experiment in which pa

r

a do y ass g ed to o e o two es de cy 

ndoml

y ass

i

g

ned to

o

n

e

of two res

ide

ncy 

the same VA hospital

 We explore differences in practice patterns 

patient outcomes.

Doyle JJ, Jr., Ewer SM, Wagner TH. Returns to physician human capital: Evidenrandomized to physician teams randomized to physician teams. J Health Econ Aug 24 2010;29(6):866 J Health Econ. Aug 24 2010;29(6):866–882.
Outline

 Background Description of the natural experim Methods Results Discussion
Background

 People in the US receive substantially differ

a ou ts a d types o ed ca ca e depe d 

mounts and types of medical care dependin

where they live

– Wennberg J and A Gittelsohn (19

Wennberg, J. and A. Gittelsohn (19

 Thi k l h d fi ld f h i 

This work launched new fields of research i

health disparities and quality of care
C fS A i 

Causes o

f

Small 

Area Var

i

 Observe

d

– Patient outcomes – Facility resources– Patient characteristics

 Partially observed or unobserved

– Ph

ysician characteristics

– Organization of clinical teams– Patient preferences Patient preferences– Patient risk– Patient Patient s’ selection of provider selection of provider
Physician Training

 A considerable share of variation in practice 

a d pat e t outco es ca be att buted to p 

n

d pat

i

e

n

t outco

mes can be attributed to

p

h

practice styles

 Practice styles are influenced by training an

organization

 If we can understand the role of training and 

organization we can improve system perfororganization, we can improve system perfor

and patient health
Natural Experiment

 A large, urban VA hospital is affiliated with 

res de cy p og a s a d pat e ts a e ass g e

idency programs and patients are assigne

residency team based on the last digit of the( ) odd or even)

 Random assignment only happens at the ma

hospital– not at substations

 Random assignment happens for most speci

Random assignment happens for most speci

not all (e.g., neurology)
Residency Program

 Anecdotally: program A and B do not get al Rounds are conducted at different times so t

minimal overlap minimal overlap

 Different attending physicians Ancillary staff (e.g., nurses) are the same 
Methods

 Linked real SSN to all discharges at VAMC Random assignment: 

– Odd ending SSN = program A Odd ending SSN = program A– Even ending SSN = program B

 At this time, we don’t have information abo

residents
Data

 Utilization data from Patient Treatment File

(1993-2009)

 Cost data from HERC Average Cost Data a

Cost data from HERC Average Cost Data a

Decision Support System 

(1998-1999)

 Mortality data from VA Vital Status File Mortality data from VA Vital Status File
Outcomes

 Length of stay (logged) Cost (logged 2006$): total and DSS subtota 28 day and 1 year hospital readmission 30 day and 1 year mortality 30 day and 1 year mortality Timing p ofrocedures
Subsamples

 Chronic heart failure (CHF) Acute myocardial infarction Acute myocardial infarction Chronic obstructive pulmonary disease (CO Gastro-i i l bl d (V l 2007) intestinal bleed (Volpp, 2007) Stroke patients Patients at substations

No randomizatRobustness ch
Control Variables

 Age Ge de nr Race (white, nonwhite) Marital status (married divorced other) Marital status (married, divorced, other) Disease severity (Deyo-modified Charlson I Time da month and ear of admission Time, day, month, and year of admission Zip code characteristics (education, age, rac

density)
Analytical Questions

 Was randomization applied consistently? Do outcomes differ by residency programs? 

And if so, why? And if so, why?

 Is the quality of care different?
Analytical Models

 Continuous and count data

– Semi

-log models log models

– General linear models (log link and gamma dist

 Logistic models for dichotomous outcomes Model choice has no effect on results
Random Assignment: Demogr

Program A Program

A Program B Program

B

(Odd SSN) (Even SSN) 

age 63.0 62.8 

18-34 0.019 0.022 35-44 0.074 0.075 45-54 0.186 0.186 55-64 0 229 . 0 229 . 65-69 0.134 0.131 70-74 0.149 0.146 75-84 0.179 0.184 84+ 0.030 0.027 

male 0.976 0.978 

white 0 466 . 0 472 . married 0.443 0.446 divorced 0.271 0.269 Observations (discharges) 35932 36434
Random Assignment: Admissi

Assigned to Higher- Assigned to Lowe

Ranking Program Ranking Program

(Odd SSN

)

(Even SSN

)

Program A Program B

( ) ( )

Charlson index = 0 0.294 0.290 Charlson index = 1 0.274 0.278 Charlson index = 2 0.433 0.432 Midnight-6am 0.096 0.098 6am-12 noon 0.237 0.233 12 noon 6pm 12 noon-6pm 0 420 . 0 425 . 6pm - Midnight 0.247 0.245 weekend 0.163 0.162 Observations (discharges) 35932 36434
Random Assignment: Zip Cod

Assigned to Higher Assigned to Higher- Assigned to Low

Assigned to Low

Ranking Program Ranking Progra

(Odd SSN) (Even SSN) 

median HH Income 33714 33945 

Program A Program B

median HH Income 33714 33945fraction HS dropout 0.249 0.247 fraction HS only 0.317 0.318 fraction Some Colle

ge 0.271 0.272 

fraction white 0.628 0.633 fraction black 0.331 0.327 fraction aged 19-34 0.214 0.213 fraction aged 35-64 0.368 0.369 fraction aged 65+ 0.141 0.141 population per 1000 sq meters 1.102 1.072 Ob ti (di h ) Observations (disc

harges

) 35932 36434
Summary of Randomizat

 Patients treated by the two residency progra

not stati i ll diff ist

ically different.

 “Successful” randomization “Successful” randomization
Results: Length of Sta

Dependent Variable:

(1) (2) 

log(length of stay)

Assigned to 0.108 0.114 0Lower Ranking Program [0.0086]** [0.0075]** [0.00

Program B

Diagnosis Fixed Effects No Yes YFull Controls No No YObservations 72366Mean of De

p Var 1.4309

Models estimated using OLS. Robust standard errors in brackets, clustered by pa

Full controls include variables listed in Table 1, as well as month, year, and day-oweek indicators. * si

g ;g nificant at 5%; ** si

gnificant at 1%
Length of Stay

Figure 1: Log(Length of Stay) vs. Last Digit of SSN

1 45

1.5

1.55

1 3

1.35

1.4

1.451.25

1.3

0

1

2

3

4

5

6

7

8

Last Digit
Eff t Ti Effec

t over Time

N Np

Not significant; every other year is sig
Results: Cost

Dependent Variable:

(4) (5) (

Assigned to 0 113 0 123 0 

log(accounting cost)

Assigned to

0.113

0.123

0.

Lower Ranking Program [0. Program B 0136]** [0.0116]** [0.0Diagnosis Fixed Effects No Yes YFull Controls No No YObservations 34098Mean of Dep Var 8.6297

Models estimated using OLS. Robust standard errors in brackets, clustered by patiencontrols include variables listed in Table 1, as well as month, year, and day-of-the-we

* si

g ;g nificant at 5%; ** si

gnificant at 1%
Costs

DSS HERC

Figure 2: Log(Accounting Cost) vs. Last Digit of SSN

8.8

Figure 3: Log(Est. Expenditure) vs

8.9

8.48.58.68.7

8.58.68.78.8

8.3

1 2 3 4 5 6 7 8 9 10

Last Digit

8.4

1 2 3 4 5 6 

Last Digit
Cost Subtotals

DSS C t bt t l

Assigned to 

P B Ob ti

DSS

Cos

t subt

o

t

a

l

Pro

gram 

B Observations

Nursing 292 [88.2776]** 34098 Surgery -123 [30.5502]** 34098 Radiology 40 [12.1013]** 34098 Lab 53 [8.8733]** 34098 Pharmacy 112 [48 6039]

* 34098 

M d l ti t d i OLS R b t t d d i b k t l t d b ti t F 

Pharmacy 112 [48.6039] 34098 All Other 253 [46.0791]** 34098 

M

o

d

els estimate

d using OLS. 

R

o

bust stan

dar

d errors in 

brac

k

ets, clustere

d

by patient. 

F

u

variables listed in Table 1, as well as month, year, and day-of-the-week indicators. * significant at 5%; ** significant at 1%
Readmission

Dependent Variable:

(1) (2) (3) (4) (

Assigned to -0 002 -0 002 -0 002 0 006 0 0

30-day Readmission 1-year Re

Program B Assigned to -0.002 -0.002 -0.002 0.006 0.0

Lower Ranking Program [0.0032] [0.0031] [0.0030] [0.0058] [0.0Diagnosis Fixed Effects No Yes Yes No Y

Program B

Full Controls No No Yes No NObservations 71954 66938Mean of Dep Var Mean of Dep Var 0 1315 . 0 4287 .

Models estimated using OLS. Robust standard errors in brackets, clustered by patient. Fuvari bl li t d i T bl 1 ll th d d iables listed in Table 1, as well as month, year, and day-of-the-week i di t k indicators. 

* significant at 5%; ** significant at 1%
Readmission

Figure 4: 30-Day Readmission vs. Last Digit of SSN

0.15

0.2

0.05

0.1

0

1 2 3 4 5 6 7 8 9 

Last Digit
Mortality

Dependent Variable:

(1) (2) (3) (4) 

Assigned to -0.001 -0.001 -0.001 -0.007 -

30-day Mortality 1-year 

Lower Ranking Program [0.0020] [0.0019] [0.0019] [0.0051] [0Diagnosis Fixed Effects No Yes Yes No 

Program B

Full Controls No No Yes No Observations 71954 66938Mean of Dep Var Mean of Dep Var 0 0642 . 0 2418 .

M d l ti t d i OLS R b t t d d i b k t l t d b ti t F Models estimated using OLS. Robust standard errors in brackets, clustered by patient. Fu

variables listed in Table 1, as well as month, year, and day-of-the-week indicators.* significant at 5%; ** significant at 1%
Mortality

Figure 5: 1-year Mortality vs. Last Digit of SSN

0 30.2

0.25

0.3

0 05

0.1

0.15

0

0.05

1 2 3 4 5 6 7 8 9 

Last Digit
B Di i yagnosis

Dependent Coeff. On 

Variable Program B S.E. 

Heart failure log(length of stay) 0.252 [0.0272]** 

1-year mortality 0.005 [0.0210] 

Acute myocardial infarction log( g y) length of stay) 0.089 [ ] 0.0372]* 

1-year mortality -0.030 [0.0201] 

Chronic Obstructive Pulmonary Disease log(length of stay) 0.191 [0.0343]** 

1-year mortality year mortality 0 001 . [0 0256] [0.0256] 

GI Bleed log(length of stay) 0.163 [0.0370]** 

1-year mortality -0.015 [0.0221] 

Models estimated using OLS. Robust standard errors in brackets, clustered by patient. Fuvariables listed in Table 1, as well as month, year, and day-of-the-week indicators. * sig ;g nificant at 5%; ** significant at 1%
Robustness Checks

Coeff. On 

Dependent Variable Program B S.E. 

Sample: nervous system patients log(length of stay) 0.047 0.048 

30-day readmission -0.011 0.022 1-year mortality -0.040 0.021 

Sample: Outside main facility log(length of stay) -0.012 0.014 

1-year mort lit a

y 0 005 . 0 004 . 

Sample: Drop transferred patients. log(length of stay) 0.114 0.007*

30-day readmission -0.003 0.003 

1 year mortality 0 007 0 004 

M d l ti t d i OLS R b t t d d i b k t l t d b ti t F 

1-year mortality -0.007

0.004

M

o

d

els estimate

d using OLS. 

R

o

bust stan

dar

d errors in 

brac

k

ets, clustere

d

by patient. 

F

u

variables listed in Table 1, as well as month, year, and day-of-the-week indicators. * significant at 5%; ** significant at 1%
Summary of Results

 Patients treated by program A have 8-20% l

LOS h i d i B LOS t

han pat

ients treate

d

in program 

B

 Cost differences also exist and remain after Cost differences also exist and remain after

controlling for LOS

 No differences in readmission and mortality
What’s Causing This Effe

 We ranked diagnosis by mortality rate and b

diagnoses into quartiles diagnoses into quartiles

 Anal

y yq zed LOS b

y

quartiles

Coeff. On Mea

log(length of stay) Program B S.E. Dep

1 il (l lik l di ) 1st quartile (least lik

ely to di

e) 0 023 . [0 0167] [0.0167] 1 

2nd quartile 0.112 [0.0131]** 1.3rd quartile 0.119 [0.0116]** 1.4th 

quartile (most likel

y to die) 0.142 [0.0141]** 1.

 Effect gets bigger as patients get sicker

q ( y) [ ]
Procedures and S i urger 

Dependent Variable: Number of Number o

Procedures Surgerie

(1) (2)

Assigned to 0.250 -0.002

Lower Ranking Program [0.0143]** [0.0036]Observations 72366 72366

Program B

Observations 72366 72366Mean of Dep Var 1.681 0.290

Models estimated using OLS. Robust standard errors in brackets, clustered by patiencontrols include variables listed in Table 1, as well as month, year, and day-of-the-we

* significant at 5%; ** significant at 1%
Use and Timing of Proced

Use and Timing of Proced

Pr (test) # | any days 

AB AB A

All casesobservations 35932 36434

any diagnostic any diagnostic 68 4%. 73 1%. ** 2 99 . 3 25 . ** 1 4

. 

xray 22.4% 25.1% ** 1.77 1.77 3.04 chest xray 6.3% 7.5% ** 1.11 1.13 * 4.39 endoscopy 5.2% 5.7% ** 1.26 1.30 ** 4.90 angiography angiography 8 1%. 8 3%. 2 70 . 2 67 . 3 16 . cardiac stress test 6.4% 7.8% ** 1.02 1.02 3.96 Other cardiac test, including echo 12.7% 15.0% ** 1.12 1.11 1.39 

• Story within AMI, COPD, CHF and GI bleed patients is consistent
Residency Programs Residency Programs

 P A rogram 

A (t t dd (trea

ts odd-endi SSN ) ding SSN

s

)

– MCAT ranking 1/126 (mean 12.2) in 2006– NIH funding ranking 4/126 in 2006 NIH funding ranking 4/126 in 2006– Supposedly greater use of specialists as attendings

 P B rogram 

B (t t rea

ts even-endi SSN ) ding SSN

s

)

– MCAT ranking 48/126 (mean 10.4) in 2006– NIH funding ranking 79/126 in 2006 NIH funding ranking 79/126 in 2006– Supposedly more international medical graduate residen

 UCSF (f i ’ k ) UCSF (for compar

ison’s sa

k

e

)

– MCAT ranking 17/126 (mean 11.1)– NIH ranking 3/126 NIH ranking 3/126
Cause?

 At this point, we don’t know the cause Three hypotheses

1. Differences are attributable to selection of hu

capital: program A attracts smarter people thB

2. Diff tt ib t bl t Differences are attributable to t ii ra n ng of h 

f hu

program A does a better job of training than 

3. Differences in the attending / resident teams 

Differences in the attending / resident teams 

the effect
P dR h ropose

d

Researc

h

 How extensive is the effect?

– OQP quality data OQP quality data– Surgery– randomization also happens in surg

 Can we identify the cause?

– Trainin

g p and resident su

pervision

– Qualitative interviews to assess communicat

teamwork
Li i i &C ii i i

m

itat

ions 

&

C

r

i

t

i

c

ism

 We have heard two criticisms

1. Results may not generalize to non-VA2. These data come from a single VA hospital and may no

other VA’s

 VA is a critical component of provider training.

1

– 17,000 medical students are trained in VA facilities every 

med h l d t i 2007) d sc

hoo

l

gra

dua

tes 

in 2007)

– 30% of all US residents receive training in VA facilities e

 This is a “mouse model”– a very unique opportunit

insights on clinician behavior and teamwork

1. AAMC; Winship D, Ullian E. Advisory Committee on Veterans Health Administration (VEvaluation, Report to the Secretary of Veterans Affairs
Summary

 Cost and length of stay differ, and the differ

biggest with sickest patients

 No differences with regard to readmission o

mortality

 N t bl diff i th d ti i f Notable differences in the use and timing of

procedures

 Would love your feedback on extending thi