Award Acceptance Presen

by Dr. Rodney Hayward

Recipient of the 2005 Under Secretary’s Award for OutstanAchievement in Health ServiceResearch
An Unexpected Care

An Unexpected Care





My Name’s Rod and 

I’m an Empiricist

•

The world is not knowable.

•

Rigorous positivistic approaches s

used to assess evidence for caus

•

Scientific empiricism is the only va

to producing such evidence.

•

Empiricism should not determining 

importance or meaning of evidenc
Important Questions Relat

Patient Safety

•

Number & Nature of Preventable Even

•

Overall Impact of Events (positivism)

•

Effectiveness and Efficiency of Safety 

(positivism)

• Importance of Preventing Adverse Eve

(normative)

• Importance of Iatrogenic vs. Non-Iatro

Adverse Events (normative)
Three Conceptual Contrib

1. Listening to Random Noise2. Using Quality Measurement as 3. Connecting the Dots: 

Clinical Evidence to Health Po
How best to measure qual

profile providers in order to 

quality and efficienc

Larry McMahon walked into my office a

after I moved to Ann Arbor and . . 
Quality Measureme

Tools

Quality Measureme

Tools

1. Explicit Evaluation of Structure 2. Implicit Evaluation of Structure 3. Outcomes Measurement
How Good Is Structured Imp

(Annals Internal Med 1994

How Good Is Structured Impl

(Annals Internal Med 1994

•

Quality problems and “prevent

deaths” are common

• Inter-rater reliability is not very •

HCFA should re-evaluate their 

to evaluating quality for paymedecisions
Resource Use Patte

of Ward Attending

Resource Use Patter

of Ward Attending

Ancillary 

Resources 

MD Groups (RVUs) 

High Users 1344 ± 215 Average Users 878 ± 118 Low Users 545 ± 112 
Observed Profiles of 

Hospital Resource Us

(Hayward et al. Med Care 1996)

min

5th 25th 75th 9

Hospital LOS

 Observed -2.3 -1.57

-.63 .68

1
Observed and Simulated Pro

Hospital Resource Use

(Hayward et al. Med Care 1996)

min

5th 25th 75th 

Hospital LOS Observed -2.3 -1.57

-.63 .68

 Expected -2.2 -1.35

-.62 .58 
Resource Use Patte

of Ward Attending

Resource Use Patter

of Ward Attending

Ancillary 

Resources 

MD Groups (RVUs) 

High Users 1344 ± 91 Average Users 878 ± 86 Low Users 545 ± 88 Patients 968 + 910 
Variance in Risk-Adjusted Reso

Attributable to Attending Phy

(Medical Care 1996)

22

Hospital LOSAncillary RVUs
Variance in Risk-Adjusted Res

Attributable to Attending Ph

(Medical Care 1996)

22

Hospital LOSAncillary RVUs

LaboratoryPharmacyImaging
What About The Variation and 

of Profiles of:

•

Residents

•

Subspecialty medicine attend

•

Surgeons

•

Outpatient care for chronic illn

•

Site!!!
To Err Is Human

•

As many as 98,000 people die eac

US hospitals due to medical errors 

•

Medical errors may be the 5th lead

of death 

(Washington Post, 1999)

“ . . . like 3 jumbo jets fully loaded wit

crashing every other day” (NY Times

“Therefore, doctors are approximatel

times more dangerous than gun ow

(Benton County News Tribune, 2000)
Studies of Preventable D

•

Harvard Medical Practice Stud

•

Utah/Colorado Study

• VA Mortality Study•

RAND Mortality Study

Reliability = 0.1 - 0.3Probably Preventability = 5%-10%Possibly Preventable = 20%-35%
VA Mortality Study Res

(Hayward and Hofer. JAMA July 2001)

23% (13,32) 

Rated as at least possibly preventable

Pr

% of Active-Care Death, 

reported as preventable

(95% CI)
VA Mortality Study Res

(Hayward and Hofer. JAMA July 2001)

1.3% (1.0,1.5) 

Adjusted for probability of leaving the hospital alive and reliability/skew of reviews

23% (13,32) 

Rated as at least possibly preventable

Pr

% of Active-Care Death, 

reported as preventable

(95% CI)
Over-Estimating Vari

O

ne re

viewer p

er c

a

s

e

A

verage of 3 re

view

ers per c

a

s

e

3 re

view

ers r

and

o

m effects variance

e

stim

ate

A

verag

e of 100 re

view

ers p

er c

a

s

e

True r

ating di

strib

utio

n

0 .2 .4 .6 .8 1

Probability of preventing

d

e

ath

V

aria

bility in preventable 

d

eath estim

ate

s

0 .2 .4 .6 .8 1

Probability of preventing death

On

e

 r

eviewer

A

verag

3 r

e

view

S

hrin

k

a

ge a

in individual c

a

s

e 

E

f

f

e

ct of m

ultiple re

view

s on estim

ations of prevent

ab
Three Conceptual Contrib

1. Listening to Random Noise2. Using Quality Measurement as 3. Connecting the Dots: 

Clinical Evidence to Health Po
Conceptual Underpinni

a Good Quality Meas

•

The majority of variation in sco

to quality of care

•

The easiest way to improve yo

by improving efficient high-qua
Sample size estimates for 

vs... process in AM

Hospital B Sample size neeed 

to detct a difference 

in 

Mortality 

(%) 

U

ptake of 

effective 

interventions 

(%) 

# extra 

lives 

saved in 

B vs A

Outco

me Process 

29 6 4.5 32846 15527 18 13.8 3619 4825 31 22.5 1290 2723 43 31.5 651 1821 55 40.5 389 12

Hospital A

Mortality

30%

Mant 
If > 90% of Variation in a 

Resides at the Patient L

•

Unmeasured Casemix

can cre

–

Unfair judgments

– Incentives to deselect patients
1991 1992

-0.4-0.2

0.00.20.4

Deviations of MD profiles from mean Hgb A1c levels

These physicians eliminate from their 1992 panels the patients who in 1991 had HgbA1c levelsin the top 5%. 

Outlier Physicians (1991)

Non-outlier Physicians (1991)

These physicians have the same patient panels in 1992 as in 1991. 

The “Advantages” of De-selectin

(Hofer et al, JAMA 1999)
Tightly-linked Measu

(Kerr et al. 2001)

1) High-risk pt with LDL > 120/mg

not on appropriate statin dose

2) Persistent BP > 135/80 & 

not on 3-4 anti-hypertensive m
Why Not Set Strict 

Performance Measures

•

May put excessive emphasis on b

low-risk cases or care.

•

May encourage devaluing or dese

outliers.

•

Can canonize care that is contrary 

preferences. 
Relationship betwee

Receipt of Care & Qua

Quality Value/Dollar 

0% 

Demand 
Kerr & Asch et al

1. What you measure is what impr2. Perhaps sometimes other aspe

improves

3. Implicit Review might be much 

you think
Reminder:

Impart Some Words 

Wisdom & Inspirati
Mentors

• Will Man• Bob Oye• Roger Grek• Joel Howel• Larry McMa• Nicki Lurie• Dennis Cop• Jack Billi• Sharon Ha

• Martin Shapiro• Larry Lynn• Shelly Greenfield• Bob Brook• Howard Freeman• Al Williams• Bill Kelley• Ken Warner
Research Colleagu

•

Tim Hofer

•

Will Manning

• Joel Howell• Larry McMahon• John Piette• Martin Shapiro• Shelly Greenfield• Richard Kravitz• Haya

Rubin

• Joel Weissfeld• Steve Asch

• •• • • • • • • 

•

Sandeep Vijan

•

Sarah Krein

• David Kent• Susan Goold• Caroline Richardson• Bill Herman• Annette Bernard• Judi Zemencuk• Mary Hogan• Sonya DeMonner
Trainees

TNTC
Ann Arbor HSR&

Center of Excellen


Why Are You Doing Res

Good Reasons•

To help improve the effectivene

efficiency of health care (make the maximum impact on length and 

with the available resources)

•

Try to make the world more hum

just

•

To improve our understanding o
Why Are You Doing Res

Bad Reasons•

To prove or demonstrate your 

theories, beliefs or political be

•

To be a disease, occupation o

disciplinary advocate

•

To stay in an academic or tea

environment

•

To acquire impressive titles