Introduction to 

Medical Decision Making
 

and Decision Analysis
 

Jeremy D. Goldhaber-Fiebert, PhD
 

Presented October 24, 2012
 
Agenda
 

• Decision analysis 

• Cost-effectiveness analysis
 

• Decision trees 

• Sensitivity analysis 

• Markov models 

• Microsimulations 
WHAT IS A DECISION ANALYSIS?
 
What is a decision analysis?
 

•	 A quantitative method for evaluating decisions 

between multiple alternatives in situations of 

uncertainty 
What is a decision analysis?
 

•	 A quantitative method for evaluating decisions 

between multiple alternatives in situations of 

uncertainty 

Decisions between multiple alternatives: 

•	 Allocate resources to one alternative (and not the others) 

•	 There is no decision without alternatives => making a choice 
 

What is a decision analysis?
 

•	 A quantitative method for evaluating decisions 

between multiple alternatives in situations of 

uncertainty 

Quantitative method for evaluating decisions: 

•	 Gather information 

•	 Assess the consequences of each alternative 

•	 Clarify the dynamics and trade-offs involved in selecting each 

•	 Select an action to take that gives us the best expected outcome
 

We employ probabilistic models to do this
 
The steps of a decision analysis
 

1. Enumerate all relevant alternatives 

2. Identify important outcomes 

3. Determine relevant uncertain factors 

4. Encode probabilities for uncertain factors
 

5. Specify the value of each outcome 

6. Combine these elements to analyze the 

decision 

Decision trees and related models important for this
 
What is a decision analysis called when its 

important outcomes include costs?
 

1. Enumerate all relevant alternatives 

2. Identify important outcomes 

3. Determine relevant uncertain factors 

4. Encode probabilities for uncertain factors 

5. Specify the value of each outcome 

6. Combine these elements to analyze the 

decision
 

Cost-effectiveness analysis a type of decision analysis 

that includes costs as one of its outcomes 
WHAT IS A COST-EFFECTIVENESS 

ANALYSIS? 
 

What is a cost-effectiveness analysis?
 

•	 In the context of health and medicine, a costeffectiveness analysis (CEA) is a method for 

evaluating tradeoffs between health benefits 

and costs resulting from alternative courses of 

action 

•	 CEA supports decision makers; it is not a 

complete resource allocation procedure 
 

Cost-Effectiveness Ratio (CER): 

How to compare two strategies in CEA
 

•	 Numerator: Difference between costs of the intervention 

(strategy) and costs of the alternative under study 

•	 Denominator: Difference between health outcomes 

(effectiveness) of the intervention and health outcomes of the 

alternative 

Incremental resources required 

by the intervention 

Incremental health effects 

gained with the intervention 
 

Models for decision analysis and CEAs
 

•	 Decision model: a schematic representation of all 

of the clinically and policy relevant features of the 

decision problem 

–	 Includes the following in its structure: 

• Decision alternatives 

• Clinical and policy-relevant outcomes 

• Sequences of events 

– Enables us to integrate knowledge about the decision 

problem from many sources (i.e., probabilities, values) 

– Computes expected outcomes (i.e., averaging across 

uncertainties) for each decision alternative 
Building decision-analytic model
 

1. Define the model’s structure 

2. Assign probabilities to all chance events in 

the structure 

3. Assign values (i.e., utilities) to all outcomes 

encoded in the structure 

4. Evaluate the expected utility of each decision 

alternative 

5. Perform sensitivity analyses 

Simple enough to be understood; complex enough to 

capture problem’s elements convincingly (assumptions)
	
“!ll models are wrong; 

but some models are 

useful” 

-- George Box and Norman Draper, 1987 
Building decision-analytic model
 

1. Define the model’s structure 

2. Assign probabilities to all chance events in 

the structure 

3. Assign values (i.e., utilities) to all outcomes 

encoded in the structure 

4. Evaluate the expected utility of each decision 

alternative 

5. Perform sensitivity analyses 
WHAT ARE THE ELEMENTS OF A 

DECISION TREE’S STRUCTURE? 
Decision node
 

A place in the decision tree at which there is 

a choice between several alternatives 

Surgery The example shows a choice 

between 2 alternatives, but
 

Other
 a decision node can 

accommodate a choice 

between more alternatives / 

provided alternatives are 

mutually exclusive. Medical tx 
 

 

Chance node
 

A place in the decision tree at which chance 

determines the outcome based on probability 

No complications The example shows only 

2 outcomes, but 

a chance node can 

complications accommodate more 

outcomes / 

provided they are 

mutually exclusive AND Dies collectively exhaustive. 
 

What do mutually exclusive and 

collectively exhaustive mean?
 

• Mutually exclusive 

– Only one alternative can be chosen 

– Only one event can occur 

• Collectively exhaustive 

– At least one event must occur 

– One of the possibilities must happen 

– Taken together, the possibilities make up the 

entire range of outcomes 
 

Terminal node
 

Final outcome associated with each pathway 

of choices and chances 

Final outcomes must be 

valued in relevant terms 

(cases of disease, Life years, 

30 Yrs Quality-adjusted life years, 

costs) so that they can be 

used for comparisons 
 

Summary
 

•	 Decision nodes: enumerate a choice between 

alternatives for the decision maker 

•	 Chance nodes: enumerate possible events 

determined by chance/probability 

•	 Terminal nodes: describe outcomes 

associated with a given pathway (of choices 

and chances) 

The entire structure of the decision tree 

can be described with only these elements 
 

Example: decision tree
 

•	 Patient presents with symptoms 

•	 Likely serious disease; unknown w/o treatment
 

•	 Two treatment alternative: 

–	 Surgery, which is potentially risky 

–	 Medical management, which has a low success rate 

•	 With surgery, one must assess the extent of 

disease and decide between curative and 

palliative surgery 

•	 Goal: maximize life expectancy for the patient
 
The initial decision is between 

surgery and medical management 

Surgery 

Medical
 

Mgmt
 
Treatment is initiated on patients
 

w/ symptoms; some w/o disease
 

Surgery 

Disease 

Present 

Medical 

Mgmt 

Disease
 

Absent
 
 

Those with disease have a chance 

to benefit from treatment 

Surgery 

Cure 

Disease
 

Present
 

Medical No

Mgmt Cure 

Disease 

Absent 
 

Likewise with surgery 
 Disease
 

Present 

Surgery 

Disease
 

Absent
 

Cure 

Disease
 

Present
 

Medical No

Mgmt Cure 

Disease 

Absent 
 

 

Surgery is risky even for those with no disease 
 Disease
 

Surgery 

Disease 

Present 

Cure 

No 

Cure 

Disease 

Absent 

Live 

Surg 

Dth 

Medical
 

Mgmt
 

Present
 

Disease
 

Absent
 
 

 

Surgery 

Cure 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

For disease, try cure vs. palliate? 

Disease
 

Present
 

Medical No

Mgmt Cure 

Disease 

Absent 
 

 

 

 

Surgery 

Cure 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

Live 

Surg 

Dth 

Live 

Surg 

Surgical risks here too Dth

Disease
 

Present
 

Medical No

Mgmt Cure 

Disease 

Absent 
 

 

 

 

 

 

Disease 

Present 

Disease 

Absent 

Surgery 

Medical 

Mgmt 

Cure 

No 

Cure 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

Live 

Surg 

Dth 

Live 

Surg 

Dth 

Cure 

No 

Cure 

Cure 

No 

Cure 

Chance of cure for survivors 
No 

Cure 

g 

Live 

 

 

No 

Cure 

 

Disease 

Present 

Disease 

Absent 

Surgery 

Medical 

Mgmt 

Cure 

Disease 

Present 

Disease 

Absent 

Live 

Sur 

Dth Palliate 

Try 

Cure 

Surg 

Dth 

Live 

Surg 

Dth 

Cure 

Cure 

No 

Cure 

Paths define course of events 

1. Surgery 

2. in a patient with disease 

3. where curative surgery chosen 

4. and patient survives 

5. and is cured 
 

 

 

 

 

 

Surgery 

Disease 

Present 

Disease 

Absent 

Cure 

No 

Cure 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

Live 

Surg 

Dth 

Live 

Surg 

Dth 

Cure 

No 

Cure 

Cure 

No 

Cure 

10% 90% 

10% 90% 

10% 

90% 

99% 

1% 

10% 90% 

10% 

90% 90% 10% 

2% 

98% 

Add probabilities 

(studies, experts) 

Medical
 

Mgmt
 
 

 

 

 

 

 

 

Surgery 

Disease 

Present 

Disease 

Absent 

Cure 

No 

Cure 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

Live 

Surg 

Dth 

Live 

Surg 

Dth 

Cure 

No 

Cure 

Cure 

No 

Cure 

10% 90% 

10% 90% 

10% 

90% 

99% 

1% 

10% 90% 

10% 

90% 90% 10% 

2% 

98% 

Now add outcomes 

Medical
 

Mgmt
 
 

 

 

 

 

 

Surgery 

Disease 

Present 

Disease 

Absent 

Medical 

Cure 

No 

Cure 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

Live 

Surg 

Dth 

Live 

Surg 

Dth 

Cure 

No 

Cure 

Cure 

No 

Cure 

10% 

90% 

10% 

90% 

10% 

90% 

99% 

1% 

10% 

90% 

10% 

90% 

90% 

10% 

2% 

98% 

0 Y 

0 Y 

0 Y 

Death yields 0 years of 

additional life 

Mgmt
 
 

 

 

 

 

 

 

Surgery 

Disease 

Present 

Disease 

Absent 

Medical 

Cure 

No 

Cure 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

Live 

Surg 

Dth 

Live 

Surg 

Dth 

Cure 

No 

Cure 

Cure 

No 

Cure 

10% 90% 

10% 90% 

10% 

90% 

99% 

1% 

10% 90% 

10% 

90% 90% 10% 

2% 

98% 

0 Y 

0 Y 

0 Y 

2 Y 

2 

2 

Uncured disease 

confers 2 years of 

additional life 

Mgmt
 
 

 

 

 

 

 

 

 

 

 

Surgery 

Disease 

Present 

Disease 

Absent 

Medical 

Cure 

No 

Cure 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

Live 

Surg 

Dth 

Live 

Surg 

Dth 

Cure 

No 

Cure 

Cure 

No 

Cure 

10% 90% 

10% 90% 

10% 

90% 

99% 

1% 

10% 90% 

10% 

90% 90% 10% 

2% 

98% 

20 Y 

20 Y 

20 

20 

20 Y 

0 Y 

0 Y 

0 Y 

2 Y 

2 

2 

Cure yields 20 years 

of additional life 

Mgmt
 
 

 

 

 

 

 

 

 

 

 

 

Disease 

Present 

Disease 

Absent 

Surgery 

Medical 

Mgmt 

Cure 

No 

Cure 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

Live 

Surg 

Dth 

Live 

Surg 

Dth 

Cure 

No 

Cure 

Cure 

No 

Cure 

10% 

90% 

10% 

90% 

10% 

90% 

99% 

1% 

10% 

90% 

10% 

90% 

90% 

10% 

2% 

98% 

20 Y 

20 Y 

20 

20 

20 Y 

0 Y 

0 Y 

0 Y 

2 Y 

2 

2 

Now average out & fold back 
 

 

 

 

 

 

 

 

 

Disease 

Present 

Disease 

Absent 

Surgery 

Medical 

Mgmt 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

Live 

Surg 

Dth 

Live 

Surg 

Dth 

Cure 

No 

Cure 

Cure 

No 

Cure 

10% 

90% 

10% 

90% 

99% 

1% 

10% 

90% 

10% 

90% 

90% 

10% 

2% 

98% 

20 Y 

20 

20 

20 Y 

0 Y 

0 Y 

0 Y 

2 

2 

3.8 Y 

10%*20 + 90%*2 = 

3.8 years (expected) 

Now average out & fold back 
 

 

 

 

 

 

 

 

 

 

Disease 

Present 

Disease 

Absent 

Surgery 

Medical 

Mgmt 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

Live 

Surg 

Dth 

Live 

Surg 

Dth 

Cure 

No 

Cure 

Cure 

No 

Cure 

10% 

90% 

10% 

90% 

99% 

1% 

10% 

90% 

10% 

90% 

90% 

10% 

2% 

98% 

20 Y 

20 

20 

20 Y 

0 Y 

0 Y 

0 Y 

2 

2 

3.8 Y 

Same calculation here 

Now average out & fold back 
 

 

 

 

 

 

 

 

 

Disease 

Present 

Disease 

Absent 

Surgery 

Medical 

Mgmt 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

Live 

Surg 

Dth 

Live 

Surg 

Dth 

Cure 

No 

Cure 

10% 

90% 

10% 

90% 

99% 

1% 

10% 

90% 90% 

10% 

2% 

98% 

20 Y 

20 

20 Y 

0 Y 

0 Y 

0 Y 

2 

3.8 Y 

3.8 Y 

10%*20 + 90%*2 = 

3.8 years (expected) 

Now average out & fold back 
 

 

 

 

 

 

 

 

 

Disease 

Present 

Disease 

Absent 

Surgery 

Medical 

Mgmt 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

Live 

Surg 

Dth 

Live 

Surg 

Dth 

Cure 

No 

Cure 

10% 

90% 

10% 

90% 

99% 

1% 

10% 

90% 90% 

10% 

2% 

98% 

20 Y 

20 

20 Y 

0 Y 

0 Y 

0 Y 

2 

3.8 Y 

3.8 Y 

Since disease presence 

unknown, we do this 

again 

Now average out & fold back 
 

 

 

 

 

 

 

Surgery 

Medical 

Mgmt 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

Live 

Surg 

Dth 

Live 

Surg 

Dth 

Cure 

No 

Cure 

10% 

90% 99% 

1% 

10% 

90% 90% 

10% 

2% 

98% 

20 

20 Y 

0 Y 

0 Y 

0 Y 

2 

18.38 Y 

3.8 Y 

10%*3.8 + 90%*20 = 

Now average out & fold back 

18.38 years
 
 

 

 

 

 

 

 Surg
 

Surgery 

Medical 

Mgmt 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

Live 

Surg 

Dth 

Live 

Dth 

Cure 

No 

Cure 

10% 

90% 99% 

1% 

10% 

90% 90% 

10% 

2% 

98% 

20 

20 Y 

0 Y 

0 Y 

0 Y 

2 

18.38 Y 

3.8 Y 

Now average out & fold back 
 

 

 

 

 Surg
 

Surgery 

Medical 

Mgmt 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

Live 

Dth 

Cure 

No 

Cure 

10% 

90% 99% 

1% 

10% 

90% 90% 

10% 

20 

20 Y 

0 Y 

0 Y 

2 

18.38 Y 

3.72 Y 

Now average out & fold back 
 

 

 

 

 Surg
 

Surgery 

Medical 

Mgmt 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

Live 

Dth 

Cure 

No 

Cure 

10% 

90% 99% 

1% 

10% 

90% 90% 

10% 

20 

20 Y 

0 Y 

0 Y 

2 

18.38 Y 

3.72 Y 

Now average out & fold back 
 

 

 

 

 

Surgery 

Medical 

Mgmt 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

Live 

Surg 

Dth 

Cure 

10% 

90% 99% 

1% 

10% 

90% 

20 Y 

0 Y 

0 Y 

18.38 Y 

3.72 Y 

18.2 Y 

Now average out & fold back 
 

 

 

 

 

Surgery 

Medical 

Mgmt 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

Live 

Surg 

Dth 

Cure 

10% 

90% 99% 

1% 

10% 

90% 

20 Y 

0 Y 

0 Y 

18.38 Y 

3.72 Y 

18.2 Y 

Now average out & fold back 
 

 

 

 

 

Surgery 

Medical 

Mgmt 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

10% 

90% 99% 

1% 

20 Y 

0 Y 

18.38 Y 

3.72 Y 

16.38 Y 

Now average out & fold back 
 

 

 

 

 

Surgery 

Medical 

Mgmt 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

10% 

90% 99% 

1% 

20 Y 

0 Y 

18.38 Y 

3.72 Y 

16.38 Y 

This one is different: 

Decision node: 

Surgeon picks option with 

greatest expected benefit: 

Try Cure (16.38 years) preferred 

(called “folding back”) 
 

 

 

 

 

Dth 

Surgery 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

10% 

Try 

Cure 

90% 99% 

1% 

20 Y 

0 Y 

16.38 Y 

Now average out & fold back 

Medical 

Mgmt 18.38 Y 
 

 

 

 

 

Dth 

Surgery 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

10% 

Try 

Cure 

90% 99% 

1% 

20 Y 

0 Y 

16.38 Y 

Now average out & fold back 

Medical 

Mgmt 18.38 Y 
 

 

Now average out & fold back Try 

Disease Cure 

Present 

Surgery 10%
 

90%
 Disease
 

Absent
 

Medical
 

Mgmt
 18.38 Y 

16.38 Y 

19.8 Y 
 

 Surgery 

Medical 

Mgmt 18.38 Y 

19.46 Y 
 

 Surgery 

Medical 

Mgmt 18.38 Y 

19.46 Y Decision node again (overall) 

Surgery is preferred to Medical 

Management because the 

incremental benefit of surgery is: 

19.46 – 18.38 = 1.08 years
 

Recommendation: Choose surgery 

(with “try cure” surgical option) 
 

 

 

Surgery 

Medical 

Mgmt 18.38 Y 

$100 

19.46 Y 

$10,000 

Use same approach for CEA but 

now with second set of outcomes 

19.46 – 18.38 = 1.08 years 

$10,000 – $100 = $9,900 

$9,900 / 1.08 = 

$9,167 per life year gained 

Surgery if willing to pay at least 

$9,167 per life year gained, 

otherwise medical management 
SENSITIVITY ANALYSIS
 
 

 

 

 

 

 

 

 

 

 

Surgery 

Disease 

Present 

Disease 

Absent 

Medical 

Mgmt 

Cure 

No 

Cure 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

Live 

Surg 

Dth 

Live 

Surg 

Dth 

Cure 

No 

Cure 

Cure 

No 

Cure 

10% 90% 

10% 90% 

10% 

90% 

99% 

1% 

10% 90% 

10% 

90% 90% 10% 

2% 

98% 

20 Y 

20 Y 

20 

20 

20 Y 

0 Y 

0 Y 

0 Y 

2 Y 

2 

2 

But probabilities and 

Outcome values 

uncertain/ 
Sensitivity Analysis
 

•	 Systematically asking “what if” questions to 

see how the decision result changes 

•	 Determines how “robust” the decision is 

–	 Threshold analysis: one parameter varied
 

– Multi-way analysis: multiple parameters 

systematically varied 
 

 

 

 

 

 

 

 

 

 

Surgery 

Disease 

Present 

Disease 

Absent 

Medical 

Mgmt 

Cure 

No 

Cure 

Disease 

Present 

Disease 

Absent 

Live 

Surg 

Dth Palliate 

Try 

Cure 

Live 

Surg 

Dth 

Live 

Surg 

Dth 

Cure 

No 

Cure 

Cure 

No 

Cure 

10% 

90% 

10% 

90% 

10% 

90% 

99% 

1% 

10% 

90% 

10% 

90% 

90% 

10% 

2% 

98% 

20 Y 

20 Y 

20 

20 

20 Y 

0 Y 

0 Y 

0 Y 

2 Y 

2 

2 

If probability of surgical 

death with curative 

surgery uncertain 
Base Case Threshold 
 

 

Probability of Curative Surgical Death 

Medical Management Preferred 

Surgery Preferred 

Base Case 

Prevalence of Disease
 
 

Advanced: Probabilistic Sensitivity 

Analysis (2nd order Monte Carlo)
 

•	 Decision tree estimates of probabilities and 

utilities are replaced with probability 

distributions (e.g. logistic-normal) 

•	 The tree is evaluated many times with random 

values selected from each distribution 

•	 Results include means and standard 

deviations of the expected values of each 

strategy 
MARKOV MODELS VS. DECISION 

TREES 
WHAT TO DO WHEN THERE IS A 

POSSIBILITY OF REPEATED EVENTS 

AND/OR DECISIONS? 
Decision about one-time, immediate 

action
 
Decision about one-time, immediate 

action
 

Intervention
 
Decisions: repeated actions and/or 

with time-dependent events
 
Repeated in what sense?
 
Disease process involves events 

occurring at multiple time points
 

/
	

/
	

/
	

/
	
/

/

Intervention (can) be delivered 

repeatedly too
 

/ 

/ 

• Repeated events can occur throughout an 

individual’s life. 

• Interventions delivered at multiple time 

points. Subsequent transitions depend on 

prior intervention outcomes. 
 

What is a Markov Model?
 

•	 Markov Model: Mathematical modeling 

technique, derived from matrix algebra, that 

describes the transitions a cohort of patients 

make among a number of mutually exclusive 

and exhaustive health states during a series of 

short intervals or cycles 
Properties of a Markov Model
 

•	 Individuals are always in one of a finite number of 

health states 

•	 Events are modeled as transitions from one state 

to another 

•	 Time spent in each health state determines 

overall expected outcome 

– Living longer without disease yields higher life 

expectancy and quality adjusted life expectancy
 

•	 During each cycle of the model, individuals may 

make a transition from one state to another 
Constructing a Markov Model
 

•	 Define mutually exclusive health states 

•	 Determine possible transitions between these 

health states 

–	 State transitions 

–	 Transition probabilities 

•	 Determine clinically valid cycle length 
 

 

Cycle Length
 

•	 Short enough that for a given disease being 

modeled the chance of two events/transitions 

occurring in one cycle is essentially 0 

–	 Many applications: weekly or monthly 

–	 Some (e.g., ICU) may hourly or daily 
Natural history disease model: health 

states
 

HEALTHY SICK DEAD 

•	 Mutually exclusive and collectively exhaustive health states 

•	 Best defined by actual biology/pathophysiology 

•	 Markovian assumptions: 

–	 Homogeneity: All individuals in the same state have the 

same costs, quality of life, risks of transition 

–	 Memorilessness: The current state determines future risks 

– Note: Stratification and tunnel states used to ensure 

Markov assumptions hold (advanced topic) 
 

Natural history disease model: 

transitions
 

HEALTHY SICK DEAD 

•	 Transitions between health states (arrows) 

•	 The proportion that do not transition stay in 

current state 

•	 Risk of death at all times and from all states! 

•	 If no transition out of a state = absorbing state 

(i.e., death) 
 

 

 

 

Natural history disease model: time 

and matrix representation
 

HEALTHY SICK DEAD 

For example pSH is the 

Probability of going from 

Sick to Healthy 

pHH pSH 0 

pHS pSS 0 

pHD pSD 1 
 

 

 

 

Natural history disease model: time 

and matrix representation
 

HEALTHY SICK DEAD 

At time t, cohort 

has proportions 

in various states 

(Sum to 1!) 

propH 

propS 

propD 

pHH pSH 0 

pHS pSS 0 

pHD pSD 1 

time=t
 
 

 

 

Natural history disease model: time 

and matrix representation
 

HEALTHY SICK DEAD 

pHH pSH 0 propH propH
 

pHS pSS 0 propS = propS
 

pHD pSD 1 propD propD
 

time=t time=t+1
 

NOTE: transition probabilities can be time dependent as well
 
 

 

 

Natural history disease model: time 

and matrix representation
 

HEALTHY SICK DEAD 

propH 

propS 

propD 

pHH pSH 0 

pHS pSS 0 

pHD pSD 1 

=
 

propH 

propS 

propD 

time=t time=t+1
 
 

 

 

Natural history disease model: time 

and matrix representation
 

HEALTHY SICK DEAD 

propH 

propS 

propD 

pHH pSH 0 

pHS pSS 0 

pHD pSD 1 

=
 

propH 

propS 

propD 

time=t time=t+1
 
Model trace
 

Proportion 

Model time
 
Model trace
 

Proportion 

• Is proportion the prevalence? 

• Is model time the age? 

Model time
 
Underlying the trace
 

Stage propH_t propS_t propD_t NotD 

0 1.00 0.00 0.00 1.00 

1 0.90 0.09 0.01 0.99 

2 0.75 0.10 0.15 0.85 

3 0.50 0.25 0.25 0.75 

4 0.20 0.40 0.40 0.60 

5 0.10 0.30 0.60 0.40 

6 0.05 0.15 0.80 0.20 

7 0.00 0.00 1.00 0.00 
Quality Adjusted Life Years (QALYS) & 

quality-of-life weights
 

HEALTHY SICK DEAD 

1.0 0.6 0.0 
Valuing outcomes
 HEALTHY SICK DEAD 

1 

. 

0 

0 

. 

6 

0 

. 

0 

Stage propH_t propS_t propD_t NotD 

0 1.00 0.00 0.00 1.00 

1 0.90 0.09 0.01 0.99 

2 0.75 0.10 0.15 0.85 

3 0.50 0.25 0.25 0.75 

4 0.20 0.40 0.40 0.60 

5 0.10 0.30 0.60 0.40 

6 0.05 0.15 0.80 0.20 

7 0.00 0.00 1.00 0.00 

T

t

QALYs propHtq H propStq S propDt

0

* * 0*

T

t

t t

propDtCOSTs propH cH propS cS

0

* * 0*
 

 

 

Interventions?
 

HEALTHY SICK DEAD 

pHH pSH 0 propH propH
 

pHS pSS 0 propS = propS
 

pHD pSD 1 propD propD
 

time=t time=t+1
 
 

 

Screening before treatment
 

•	 Screening 70% sensitivity, 100% specific 

•	 Treatment 90% effective 

•	 Intervention occurs after natural hx transitions every 

cycle 

•	 Calculations 

–	 pHS_i = pHS*(0.3) + pHS*(0.7*0.1) 

–	 pSS_i = pSS*(0.3) + pSS*(0.7*0.1) 

–	 pSH_i = pSH + pSS*(0.7*0.9) 

–	 pHH_i = pHH + pHS*(0.7*0.9) 
 

 

 

Natural History
 

0.5 0.2 0
 

0.4 0.6 0
 

0.1 0.2 1
 
 

 

 

Screening before treatment
 

pHH_i pSH_i 0
 

pHS_i pSS_i 0
 

pHD pSD 1
 
 

 

 

Screening before treatment
 

0.752 0.222 0
 

0.148 0.578 0
 

0.100 0.200 1
 
With and w/o intervention
 

Proportion

1 

0.8
 

0.6
 

0.4
 

0.2
 

1 11 21 31 41 51 61 71 81 91
 

0
 
Proportion

With and w/o intervention 

1 

0.8 

0.6 

0.4 

0.2 

0 

The additional area 

represents the gain 

in life expectancy 

and/or QALYs from 

the intervention 

1 11 21 31 41 51 61 71 81 91
 
Natural History To Healthy 

To Sick Healthy 

To Dead 

To Healthy 

To Sick Sick 

M 

To Dead 

To Dead 

Dead 
 

 

 

Intervention
 

M 

To Healthy T- => No Tx => To Healthy 

Tx Effective -To Healthy 

T+ 

To Sick Healthy Tx Ineffective - To Sick 

T- => No Tx => To Sick 

To Dead 
 

 

 

Intervention
 

M 

To Healthy T- => No Tx => To Healthy 

Tx Effective -To Healthy 

T+ 

To Sick Sick Tx Ineffective - To Sick 

T- => No Tx => To Sick 

To Dead 
 

Cohorts vs. individuals
 

Deterministic vs. stochastic
 

•	 Markov cohort model (i.e., the matrix version) is 

smooth model (infinite population size) of the 

proportion of a cohort in each state at each time 

• Can use same structure to simulate many individuals 

(first-order Monte Carlo) (simple microsimulation)
 

•	 The matrix becomes the probability of an individual 

transition from one state to another instead of the % 

of those in a given state who deterministically flow 

into another state 
Microsimulation 

Healthy Dead Sick 

0
 

1
 

2
 

3
 

4
 

5 
Microsimulation 

Healthy Dead Sick 

0
 

1
 

2
 

3
 

4
 

pHS 

pSS 

pSH 

pHS 

pSD 

5 
Microsimulation 

Healthy Dead Sick 

0
 

1
 

2
 

3
 

4
 

5 
Microsimulation 

Healthy Dead Sick 

0
 

1
 

2
 

3
 

4
 

5 
Recall the trace and calculation of 

outcomes from it
 

Stage propH_t propS_t propD_t NotD 

0 1.00 0.00 0.00 1.00 

1 0.90 0.09 0.01 0.99 

2 0.75 0.10 0.15 0.85 

3 0.50 0.25 0.25 0.75 

4 0.20 0.40 0.40 0.60 

5 0.10 0.30 0.60 0.40 

6 0.05 0.15 0.80 0.20 

7 0.00 0.00 1.00 0.00 

T

t

QALYs propHtq H propStq S propDt

0

* * 0*

T

t

t t

propDtCOSTs propH cH propS cS

0

* * 0*
Microsimulation
 

•	 Run with many individuals 

•	 Calculate proportions in each state at each time 

(just like in our Markov cohort table) 

–	 Stage 2: 5100 sick / 100,000 people = 5.1% 

•	 !pproximates the “smooth” cohort version 

–	 5.1% [CI\ is ~= 5.0% in “smooth” cohort 

–	 Advanced 

•	 Larger the number of individuals the closer to the smooth 

cohort (tighter the CI) 

•	 See Kuntz/Weinstein chapter of Michael Drummond’s book 

on Economic Evaluation for more on this for more on this 
Why consider microsimulation? 

•	 It requires longer simulation times 

•	 It is more complex 

•	 Fewer people are familiar with it 

•	 There is “Monte Carlo” noise (random error) 

even with simulating fairly large groups of 

individuals (at least for rare events) 
 

State explosion!
 

•	 Suppose you want to use a Markov model of a 

disease with 2 states and death (H,S,D) 

•	 Suppose you need it stratified by sex and smoking 

status (3 levels), BMI (4 levels), hypertension (4 

levels) 

•	 Now you need 2x3x4x4x2 states (death is not 

stratified = 192 states 

•	 What if you need to stratify states by past history? 

(previous high hypertension, used to be obese) or Tx 

history (has a stent)? 
Microsimulation as alternative
 

•	 Simulate 1 individual at a time 

•	 Assign a set of attributes to the individual 

–	 Sex=M, Smoking=Y, BMI=Overweight, HT=Y 

•	 Define a function for the probability of transitioning from 

H to S 

–	 P(H to S | Sex, Smoking, BMI, HT) 

•	 Have functions for changing attributes 

–	 P(BMI=Obese|Sex, BMI) 

•	 Track previous health states 

–	 P(H to S| Sex, Smoking, BMI, HT, S in the past) 

•	 Note: Could estimate these functions from 

logistic regressions 
Sage advice I have heard
 

•	 Know what information your consumers need
 

•	 Pick a model that is as simple as possible / 

but no simpler 

•	 Know the limits of what your model does and 

make statements within those limits – All 

research studies have limitations 
Summary: 

Medical Decision Analysis
 

•	 Clearly defines alternatives, events, and 

outcomes 

• Formal method to combine evidence 

• Can prioritize information acquisition 

• Can help healthcare providers to make 

medical decisions under uncertainty
 
 

Classic sources on about decision 

analysis and modeling
 

•	 Sox HC, Blatt MA, Higgins MC, Marton KI (1988) Medical 

Decision Making. Boston MA: Butterworth-Heinemann 

Publisher. 

•	 Detsky AS, Naglie G, Krahn MD, Naimark D, Redelmeier DA. 

Primer on medical decision analysis: Parts 1-5. Med Decis 

Making. 1997;17(2):123-159. 

•	 Sonnenberg FA, Beck JR. Markov models in medical decision 

making: a practical guide. Med Decis Making. 1993;13(4):322-

38. 

•	 Beck JR, Pauker SG. The Markov process in medical prognosis. 

Med Decis Making. 1983;3(4):419-458. 

•	 Society for Medical Decision Making (http://www.smdm.org)