Modeling in 

Medical Decision Analysis

Jeremy D. Goldhaber-Fiebert, PhD

Presented May 28, 2014
Agenda 

• Decision analysis

• Cost-effectiveness analysis

• Decision trees

• Sensitivity analysis

• Markov models

• Microsimulations
WHAT IS A DECISION ANALYSIS?
What is a decision analysis?

• A quantitative method for evaluating decisions 

between multiple alternatives in situations of 

uncertainty
What is a decision analysis?

• A quantitative method for evaluating decisions 

between multiple alternatives in situations of 

uncertainty

Decisions between multiple alternatives:

• Allocate resources to one alternative (and not the others)

• There is no decision without alternatives => making a choice
What is a decision analysis?

• A quantitative method for evaluating decisions 

between multiple alternatives in situations of 

uncertainty

Quantitative method for evaluating decisions:

• Gather information

• Assess the consequences of each alternative

• Clarify the dynamics and trade-offs involved in selecting each

• Select an action to take that gives us the best expected outcome

We employ probabilistic models to do this
The steps of a decision analysis

1. Enumerate all relevant alternatives

2. Identify important outcomes

3. Determine relevant uncertain factors

4. Encode probabilities for uncertain factors

5. Specify the value of each outcome

6. Combine these elements to analyze the 

decision

Decision trees and related models important for this
What is a decision analysis called when its 

important outcomes include costs?

1. Enumerate all relevant alternatives

2. Identify important outcomes

3. Determine relevant uncertain factors

4. Encode probabilities for uncertain factors

5. Specify the value of each outcome

6. Combine these elements to analyze the 

decision

Cost-effectiveness analysis a type of decision analysis 

that includes costs as one of its outcomes
WHAT IS A COST-EFFECTIVENESS 

ANALYSIS?
What is a cost-effectiveness analysis?

• In the context of health and medicine, a costeffectiveness analysis (CEA) is a method for 

evaluating tradeoffs between health benefits 

and costs resulting from alternative courses of 

action 

• CEA supports decision makers; it is not a 

complete resource allocation procedure
Cost-Effectiveness Ratio (CER): 

How to compare two strategies in CEA

• Numerator: Difference between costs of the intervention (strategy) 

and costs of the alternative under study

• Denominator: Difference between health outcomes (effectiveness) 

of the intervention and health outcomes of the alternative

CCCCCC =

CCii − CCaaaaaa

ii − aaaaaa

Incremental resources 

required by the 

intervention

Incremental health effects 

gained with the 

intervention
Models for decision analysis and CEAs

• Decision model: a schematic representation of all 

of the clinically and policy relevant features of the 

decision problem

– Includes the following in its structure: 

• Decision alternatives

• Clinical and policy-relevant outcomes

• Sequences of events

– Enables us to integrate knowledge about the decision 

problem from many sources (i.e., probabilities, values)

– Computes expected outcomes (i.e., averaging across 

uncertainties) for each decision alternative
Building decision-analytic model

1. Define the model’s structure

2. Assign probabilities to all chance events in 

the structure

3. Assign values (i.e., utilities) to all outcomes 

encoded in the structure

4. Evaluate the expected utility of each decision 

alternative

5. Perform sensitivity analyses

Simple enough to be understood; complex enough to

capture problem’s elements convincingly (assumptions)
“All models are wrong; 

but some models are 

useful”

-- George Box and Norman Draper, 1987
Building decision-analytic model

1. Define the model’s structure

2. Assign probabilities to all chance events in 

the structure

3. Assign values (i.e., utilities) to all outcomes 

encoded in the structure

4. Evaluate the expected utility of each decision 

alternative

5. Perform sensitivity analyses
WHAT ARE THE ELEMENTS OF A 

DECISION TREE’S STRUCTURE?
Decision node

A place in the decision tree at which there is 

a choice between several alternatives

Medical tx 

Surgery The example shows a choice 

between 2 alternatives, but

a decision node can 

accommodate a choice 

between more alternatives ...

provided alternatives are 

mutually exclusive. 

Other 
Chance node

A place in the decision tree at which chance 

determines the outcome based on probability

Dies

No complications The example shows only

2 outcomes, but

a chance node can 

accommodate more 

outcomes ...

provided they are 

mutually exclusive AND

collectively exhaustive. 

complications
What do mutually exclusive and 

collectively exhaustive mean?

• Mutually exclusive

– Only one alternative can be chosen

– Only one event can occur

Collectively exhaustive

– At least one event must occur

– One of the possibilities must happen

– Taken together, the possibilities make up the 

entire range of outcomes

•
Terminal node

Final outcome associated with each pathway

of choices and chances

Final outcomes must be

valued in relevant terms

(cases of disease, Life years,

Quality-adjusted life years,

costs) so that they can be

used for comparisons

30 Yrs
Summary 

• Decision nodes: enumerate a choice between 

alternatives for the decision maker

• Chance nodes: enumerate possible events 

determined by chance/probability

• Terminal nodes: describe outcomes 

associated with a given pathway (of choices 

and chances)

The entire structure of the decision tree

can be described with only these elements
Example: decision tree

• Patient presents with symptoms

• Likely serious disease; unknown w/o treatment

• Two treatment alternative:

– Surgery, which is potentially risky

– Medical management, which has a low success rate

• With surgery, one must assess the extent of 

disease and decide between curative and 

palliative surgery

• Goal: maximize life expectancy for the patient
Surgery

Medical 

Mgmt

The initial decision is between 

surgery and medical management
Disease

Present

Disease

Absent

Surgery

Medical 

Mgmt

Treatment is initiated on patients

w/ symptoms; some w/o disease
Disease

Present

Disease

Absent

Surgery

Medical 

Mgmt

Cure

No 

Cure

Those with disease have a chance

to benefit from treatment
Disease

Present

Disease

Absent

Surgery

Medical 

Mgmt

Cure

No 

Cure

Disease

Present

Disease

Absent

Likewise with surgery
Disease

Present

Disease

Absent

Surgery

Medical 

Mgmt

Cure

No 

Cure

Disease

Present

Disease

Absent

Live

Surg 

Dth 

Surgery is risky even for those with no disease
Disease

Present

Disease

Absent

Surgery

Medical 

Mgmt

Cure

No 

Cure

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

For disease, try cure vs. palliate?
Disease

Present

Disease

Absent

Surgery

Medical 

Mgmt

Cure

No 

Cure

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg 

Dth

Live

Surg

Surgical risks here too Dth
Disease

Present

Disease

Absent

Surgery

Medical 

Mgmt

Cure

No 

Cure

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg 

Dth

Live

Surg

Dth

Cure

No 

Cure

Cure

No 

Cure

Chance of cure for survivors
Disease

Present

Disease

Absent

Surgery

Medical 

Mgmt

Cure

No 

Cure

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg 

Dth

Live

Surg

Dth

Cure

No 

Cure

Cure

No 

Cure

Paths define course of events

1. Surgery 

2. in a patient with disease

3. where curative surgery chosen

4. and patient survives

5. and is cured
Disease

Present

Disease

Absent

Surgery

Medical 

Mgmt

Cure

No 

Cure

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg 

Dth

Live

Surg

Dth

Cure

No 

Cure

Cure

No 

Cure

10% 90%

10% 90%

10%

90%

99%

1% 

10% 90%

10%

90%90% 10%

2% 

98% 

Add probabilities 

(studies, experts)
Disease

Present

Disease

Absent

Surgery

Medical 

Mgmt

Cure

No 

Cure

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg 

Dth

Live

Surg

Dth

Cure

No 

Cure

Cure

No 

Cure

10% 90%

10% 90%

10%

90%

99%

1% 

10% 90%

10%

90%90% 10%

2% 

98% 

Now add outcomes
Disease

Present

Disease

Absent

Surgery

Medical 

Mgmt

Cure

No 

Cure

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg 

Dth

Live

Surg

Dth

Cure

No 

Cure

Cure

No 

Cure

10%

90%

10%

90%

10%

90%

99%

1% 

10%

90%

10%

90%

90%

10%

2% 

98% 

0 Y

0 Y

0 Y

Death yields 0 years of

additional life
Disease

Present

Disease

Absent

Surgery

Medical 

Mgmt

Cure

No 

Cure

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg 

Dth

Live

Surg

Dth

Cure

No 

Cure

Cure

No 

Cure

10% 90%

10% 90%

10%

90%

99%

1% 

10% 90%

10%

90%90% 10%

2% 

98% 

0 Y

0 Y

0 Y

2 Y 

2 

2 

Uncured disease 

confers 2 years of 

additional life
Disease

Present

Disease

Absent

Surgery

Medical 

Mgmt

Cure

No 

Cure

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg 

Dth

Live

Surg

Dth

Cure

No 

Cure

Cure

No 

Cure

10% 90%

10% 90%

10%

90%

99%

1% 

10% 90%

10%

90%90% 10%

2% 

98% 

20 Y

20 Y

20

20

20 Y

0 Y

0 Y

0 Y

2 Y 

2 

2 

Cure yields 20 years

of additional life
Disease

Present

Disease

Absent

Surgery

Medical 

Mgmt

Cure

No 

Cure

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg 

Dth

Live

Surg

Dth

Cure

No 

Cure

Cure

No 

Cure

10%

90%

10%

90%

10%

90%

99%

1% 

10%

90%

10%

90%

90%

10%

2% 

98% 

20 Y

20 Y

20

20

20 Y

0 Y

0 Y

0 Y

2 Y 

2 

2 

Now average out & fold back
Disease

Present

Disease

Absent

Surgery

Medical 

Mgmt

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg 

Dth

Live

Surg

Dth

Cure

No 

Cure

Cure

No 

Cure

10%

90%

10%

90%

99%

1% 

10%

90%

10%

90%

90%

10%

2% 

98% 

20 Y

20

20

20 Y

0 Y

0 Y

0 Y

2 

2 

3.8 Y

10%*20 + 90%*2 =

3.8 years (expected)

Now average out & fold back
Disease

Present

Disease

Absent

Surgery

Medical 

Mgmt

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg 

Dth

Live

Surg

Dth

Cure

No 

Cure

Cure

No 

Cure

10%

90%

10%

90%

99%

1% 

10%

90%

10%

90%

90%

10%

2% 

98% 

20 Y

20

20

20 Y

0 Y

0 Y

0 Y

2 

2 

3.8 Y

Same calculation here

Now average out & fold back
Disease

Present

Disease

Absent

Surgery

Medical 

Mgmt

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg 

Dth

Live

Surg

Dth

Cure

No 

Cure

10%

90%

10%

90%

99%

1% 

10%

90% 90%

10%

2% 

98% 

20 Y

20

20 Y

0 Y

0 Y

0 Y

2 

3.8 Y

3.8 Y

10%*20 + 90%*2 =

3.8 years (expected)

Now average out & fold back
Disease

Present

Disease

Absent

Surgery

Medical 

Mgmt

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg 

Dth

Live

Surg

Dth

Cure

No 

Cure

10%

90%

10%

90%

99%

1% 

10%

90% 90%

10%

2% 

98% 

20 Y

20

20 Y

0 Y

0 Y

0 Y

2 

3.8 Y

3.8 Y

Since disease presence

unknown, we do this

again

Now average out & fold back
Surgery

Medical 

Mgmt

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg 

Dth

Live

Surg

Dth

Cure

No 

Cure

10%

90% 99%

1% 

10%

90% 90%

10%

2% 

98% 

20

20 Y

0 Y

0 Y

0 Y

2 

18.38 Y

3.8 Y

10%*3.8 + 90%*20 =

18.38 years

Now average out & fold back
Surgery

Medical 

Mgmt

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg 

Dth

Live

Surg

Dth

Cure

No 

Cure

10%

90% 99%

1% 

10%

90% 90%

10%

2% 

98% 

20

20 Y

0 Y

0 Y

0 Y

2 

18.38 Y

3.8 Y

Now average out & fold back
Surgery

Medical 

Mgmt

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg

Dth

Cure

No 

Cure

10%

90% 99%

1% 

10%

90% 90%

10%

20

20 Y

0 Y

0 Y

2 

18.38 Y

3.72 Y

Now average out & fold back
Surgery

Medical 

Mgmt

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg

Dth

Cure

No 

Cure

10%

90% 99%

1% 

10%

90% 90%

10%

20

20 Y

0 Y

0 Y

2 

18.38 Y

3.72 Y

Now average out & fold back
Surgery

Medical 

Mgmt

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg

Dth

Cure

10%

90% 99%

1% 

10%

90%

20 Y

0 Y

0 Y

18.38 Y

3.72 Y

18.2 Y

Now average out & fold back
Surgery

Medical 

Mgmt

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg

Dth

Cure

10%

90% 99%

1% 

10%

90%

20 Y

0 Y

0 Y

18.38 Y

3.72 Y

18.2 Y

Now average out & fold back
Surgery

Medical 

Mgmt

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

10%

90% 99%

1% 

20 Y

0 Y

18.38 Y

3.72 Y

16.38 Y

Now average out & fold back
Surgery

Medical 

Mgmt

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

10%

90% 99%

1% 

20 Y

0 Y

18.38 Y

3.72 Y

16.38 Y

This one is different:

Decision node:

Surgeon picks option with 

greatest expected benefit:

Try Cure (16.38 years) preferred

(called “folding back”)
Surgery

Medical 

Mgmt

Disease

Present

Disease

Absent

Live

Surg 

Dth 

Try

Cure

10%

90% 99%

1% 

20 Y

0 Y

18.38 Y

16.38 Y

Now average out & fold back
Surgery

Medical 

Mgmt

Disease

Present

Disease

Absent

Live

Surg 

Dth 

Try

Cure

10%

90% 99%

1% 

20 Y

0 Y

18.38 Y

16.38 Y

Now average out & fold back
Surgery

Medical 

Mgmt

Disease

Present

Disease

Absent

Try

Cure

10%

90%

18.38 Y

16.38 Y

19.8 Y

Now average out & fold back
Surgery

Medical 

Mgmt 18.38 Y

19.46 Y
Surgery

Medical 

Mgmt 18.38 Y

19.46 Y Decision node again (overall)

Surgery is preferred to Medical 

Management because the 

incremental benefit of surgery is:

19.46 – 18.38 = 1.08 years

Recommendation: Choose surgery 

(with “try cure” surgical option)
Surgery

Medical 

Mgmt 18.38 Y

$100

19.46 Y

$10,000

Use same approach for CEA but 

now with second set of outcomes

19.46 – 18.38 = 1.08 years

$10,000 – $100 = $9,900

$9,900 / 1.08 = 

$9,167 per life year gained

Surgery if willing to pay at least 

$9,167 per life year gained, 

otherwise medical management
SENSITIVITY ANALYSIS
Disease

Present

Disease

Absent

Surgery

Medical 

Mgmt

Cure

No 

Cure

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg 

Dth

Live

Surg

Dth

Cure

No 

Cure

Cure No Cure

10% 90%

10% 90%

10%

90%

99%1% 

10% 90%

10%

90%90% 10%

2% 

98% 

20 Y

20 Y

20

20

20 Y

0 Y

0 Y

0 Y

2 Y 

2 

2 

But probabilities and 

Outcome values 

uncertain...
Sensitivity Analysis

• Systematically asking “what if”questions to 

see how the decision result changes

• Determines how “robust” the decision is

– Threshold analysis: one parameter varied

– Multi-way analysis: multiple parameters 

systematically varied
Disease

Present

Disease

Absent

Surgery

Medical 

Mgmt

Cure

No 

Cure

Disease

Present

Disease

Absent

Live

Surg 

Dth Palliate

Try

Cure

Live

Surg 

Dth

Live

Surg

Dth

Cure

No 

Cure

Cure

No 

Cure

10%

90%

10%

90%

10%

90%

99%

1% 

10%

90%

10%

90%

90%

10%

2% 

98% 

20 Y

20 Y

20

20

20 Y

0 Y

0 Y

0 Y

2 Y 

2 

2 

If probability of surgical 

death with curative 

surgery uncertain
Base Case Threshold
Prevalence of Disease

Probability of Curative Surgical Death

Medical Management Preferred

Surgery Preferred

Base Case
Advanced: Probabilistic Sensitivity 

Analysis (2nd order Monte Carlo)

• Decision tree estimates of probabilities and 

utilities are replaced with probability 

distributions (e.g. logistic-normal)

• The tree is evaluated many times with random 

values selected from each distribution

• Results include means and standard 

deviations of the expected values of each 

strategy
MARKOV MODELS VS. DECISION 

TREES
WHAT TO DO WHEN THERE IS A 

POSSIBILITY OF REPEATED EVENTS 

AND/OR DECISIONS?
Decision about one-time, immediate 

action
Decision about one-time, immediate 

action

Intervention
Decisions: repeated actions and/or 

with time-dependent events
Repeated in what sense?
Disease process involves events 

occurring at multiple time points

... 

... 

... 

... 
Intervention (can) be delivered 

repeatedly too

... 

... 

... 

... 

• Repeated events can occur throughout an 

individual’s life.

• Interventions delivered at multiple time 

points. Subsequent transitions depend on 

prior intervention outcomes.
What is a Markov Model? 

• Markov Model: Mathematical modeling 

technique, derived from matrix algebra, that 

describes the transitions a cohort of patients 

make among a number of mutually exclusive 

and exhaustive health states during a series of 

short intervals or cycles 
Properties of a Markov Model

• Individuals are always in one of a finite number of 

health states

• Events are modeled as transitions from one state 

to another

• Time spent in each health state determines 

overall expected outcome

– Living longer without disease yields higher life 

expectancy and quality adjusted life expectancy

• During each cycle of the model, individuals may 

make a transition from one state to another
Constructing a Markov Model

• Define mutually exclusive health states

• Determine possible transitions between these 

health states

– State transitions

– Transition probabilities

• Determine clinically valid cycle length
Cycle Length

• Short enough that for a given disease being 

modeled the chance of two events/transitions 

occurring in one cycle is essentially 0

– Many applications: weekly or monthly

– Some (e.g., ICU) may hourly or daily
Natural history disease model: health 

states

• Mutually exclusive and collectively exhaustive health states

• Best defined by actual biology/pathophysiology

• Markovian assumptions:

– Homogeneity: All individuals in the same state have the 

same costs, quality of life, risks of transition

– Memorilessness: The current state determines future risks

– Note: Stratification and tunnel states used to ensure 

Markov assumptions hold (advanced topic)

HEALTHY SICK DEAD
Natural history disease model: 

transitions

• Transitions between health states (arrows)

• The proportion that do not transition stay in 

current state

• Risk of death at all times and from all states!

• If no transition out of a state = absorbing state 

(i.e., death) 

HEALTHY SICK DEAD
Natural history disease model: time 

and matrix representation

HEALTHY SICK DEAD

pHH pSH 0 

pHS pSS 0 

pHD pSD 1 

For example pSH is the 

Probability of going from 

Sick to Healthy
Natural history disease model: time 

and matrix representation

propH 

propS 

propD 

time=t

HEALTHY SICK DEAD

pHH pSH 0 

pHS pSS 0 

pHD pSD 1 

At time t, cohort

has proportions

in various states

(Sum to 1!)
Natural history disease model: time 

and matrix representation

propH 

propS 

propD 

time=t

propH 

propS 

propD 

time=t+1

= 

NOTE: transition probabilities can be time dependent as well

HEALTHY SICK DEAD

pHH pSH 0 

pHS pSS 0 

pHD pSD 1 
Natural history disease model: time 

and matrix representation

propH 

propS 

propD 

time=t

propH 

propS 

propD 

time=t+1

= 

HEALTHY SICK DEAD

pHH pSH 0 

pHS pSS 0 

pHD pSD 1 
Natural history disease model: time 

and matrix representation

propH 

propS 

propD 

time=t

propH 

propS 

propD 

time=t+1

= 

HEALTHY SICK DEAD

pHH pSH 0 

pHS pSS 0 

pHD pSD 1 
Model trace

Proportion

Model time
Model trace

Proportion

Model time

• Is proportion the prevalence?

• Is model time the age?
Underlying the trace

Stage propH_t propS_t propD_t NotD 

0 1.00 0.00 0.00 1.00 

1 0.90 0.09 0.01 0.99 

2 0.75 0.10 0.15 0.85 

3 0.50 0.25 0.25 0.75 

4 0.20 0.40 0.40 0.60 

5 0.10 0.30 0.60 0.40 

6 0.05 0.15 0.80 0.20 

7 0.00 0.00 1.00 0.00 
Quality Adjusted Life Years (QALYS) & 

quality-of-life weights

HEALTHY SICK DEAD

1.0 0.6 0.0 
Valuing outcomes

Stage propH_t propS_t propD_t NotD 

0 1.00 0.00 0.00 1.00 

1 0.90 0.09 0.01 0.99 

2 0.75 0.10 0.15 0.85 

3 0.50 0.25 0.25 0.75 

4 0.20 0.40 0.40 0.60 

5 0.10 0.30 0.60 0.40 

6 0.05 0.15 0.80 0.20 

7 0.00 0.00 1.00 0.00 

∑[( ) ( ) ( )] =

= + +

T

t

QALYs t qHpropH t qSpropS propDt

0

* * 0*

∑[( ) ( ) ( )] =

= + +

T

t

t t propDt COSTs propH cH propS cS

0

* * 0*

HEALTHY SICK DEAD

1

.

0 

0

.

6 

0

.

0 
Interventions? 

propH 

propS 

propD 

time=t

propH 

propS 

propD 

time=t+1

= 

HEALTHY SICK DEAD

pHH pSH 0 

pHS pSS 0 

pHD pSD 1 
Screening before treatment

• Screening 70% sensitivity, 100% specific

• Treatment 90% effective

• Intervention occurs after natural hx transitions every 

cycle

• Calculations

– pHS_i = pHS*(0.3) + pHS*(0.7*0.1)

– pSS_i = pSS*(0.3) + pSS*(0.7*0.1)

– pSH_i = pSH + pSS*(0.7*0.9)

– pHH_i = pHH + pHS*(0.7*0.9)
Natural History 

0.5 0.2 0 

0.4 0.6 0 

0.1 0.2 1 
Screening before treatment

pHH_i pSH_i 0 

pHS_i pSS_i 0 

pHD pSD 1 
Screening before treatment

0.752 0.222 0 

0.148 0.578 0 

0.100 0.200 1 
0

0.2

0.4

0.6

0.8

1

1 11 21 31 41 51 61 71 81 91

Proportion

With and w/o intervention
0

0.2

0.4

0.6

0.8

1

1 11 21 31 41 51 61 71 81 91

Proportion

With and w/o intervention

The additional area 

represents the gain 

in life expectancy 

and/or QALYs from 

the intervention
M 

To Healthy

To Sick

To Dead

Healthy

To Healthy

To Sick

To Dead

To Dead

Sick

Dead

Natural History
M 

To Healthy

To Sick

To Dead

Healthy Tx Ineffective - To Sick

To Sick

Tx Effective -To Healthy

T+

T- => No Tx =>

T- => No Tx => To Healthy

Intervention 
M 

To Healthy

To Sick

To Dead

Sick Tx Ineffective - To Sick

To Sick

Tx Effective -To Healthy

T+

T- => No Tx =>

T- => No Tx => To Healthy

Intervention 
Cohorts vs. individuals

Deterministic vs. stochastic

• Markov cohort model (i.e., the matrix version) is 

smooth model (infinite population size) of the 

proportion of a cohort in each state at each time

• Can use same structure to simulate many individuals 

(first-order Monte Carlo) (simple microsimulation) 

• The matrix becomes the probability of an individual 

transition from one state to another instead of the % 

of those in a given state who deterministically flow 

into another state
Microsimulation

Healthy Sick Dead

0 

1 

2 

3 

4 

5 
Microsimulation

Healthy Sick Dead

0 

1 

2 

3 

4 

5 

pHS

pSS

pSH

pHS

pSD
Microsimulation

Healthy Sick Dead

0 

1 

2 

3 

4 

5 
Microsimulation

Healthy Sick Dead

0 

1 

2 

3 

4 

5 
Recall the trace and calculation of 

outcomes from it

Stage propH_t propS_t propD_t NotD 

0 1.00 0.00 0.00 1.00 

1 0.90 0.09 0.01 0.99 

2 0.75 0.10 0.15 0.85 

3 0.50 0.25 0.25 0.75 

4 0.20 0.40 0.40 0.60 

5 0.10 0.30 0.60 0.40 

6 0.05 0.15 0.80 0.20 

7 0.00 0.00 1.00 0.00 

∑[( ) ( ) ( )] =

= + +

T

t

QALYs t qHpropH t qSpropS propDt

0

* * 0*

∑[( ) ( ) ( )] =

= + +

T

t

t t propDt COSTs propH cH propS cS

0

* * 0*
Microsimulation

• Run with many individuals

• Calculate proportions in each state at each time 

(just like in our Markov cohort table)

– Stage 2: 5100 sick / 100,000 people = 5.1% 

• Approximates the “smooth” cohort version

– 5.1% [CI] is ~= 5.0% in “smooth” cohort

– Advanced

• Larger the number of individuals the closer to the smooth 

cohort (tighter the CI)

• See Kuntz/Weinstein chapter of Michael Drummond’s book 

on Economic Evaluation for more on this for more on this
Why consider microsimulation? 

• It requires longer simulation times

• It is more complex

• Fewer people are familiar with it

• There is “Monte Carlo” noise (random error) 

even with simulating fairly large groups of 

individuals (at least for rare events)
State explosion!

• Suppose you want to use a Markov model of a 

disease with 2 states and death (H,S,D)

• Suppose you need it stratified by sex and smoking 

status (3 levels), BMI (4 levels), hypertension (4 

levels)

• Now you need 2x3x4x4x2 states (death is not 

stratified = 192 states

• What if you need to stratify states by past history? 

(previous high hypertension, used to be obese) or Tx

history (has a stent)?
Microsimulation as alternative 

• Simulate 1 individual at a time 

• Assign a set of attributes to the individual

– Sex=M, Smoking=Y, BMI=Overweight, HT=Y

• Define a function for the probability of transitioning from 

H to S

– P(H to S | Sex, Smoking, BMI, HT)

• Have functions for changing attributes 

– P(BMI=Obese|Sex, BMI)

• Track previous health states

– P(H to S| Sex, Smoking, BMI, HT, S in the past)

• Note: Could estimate these functions from 

logistic regressions
Sage advice I have heard

• Know what information your consumers need

• Pick a model that is as simple as possible ... 

but no simpler

• Know the limits of what your model does and 

make statements within those limits – All 

research studies have limitations
Summary: 

Medical Decision Analysis

• Clearly defines alternatives, events, and 

outcomes

• Formal method to combine evidence

• Can prioritize information acquisition

• Can help healthcare providers to make 

medical decisions under uncertainty
Classic sources on about decision 

analysis and modeling

• Sox HC, Blatt MA, Higgins MC, Marton KI (1988) Medical 

Decision Making. Boston MA: Butterworth-Heinemann 

Publisher. 

• Detsky AS, Naglie G, Krahn MD, Naimark D, Redelmeier DA. 

Primer on medical decision analysis: Parts 1-5. Med Decis 

Making. 1997;17(2):123-159.

• Sonnenberg FA, Beck JR. Markov models in medical decision 

making: a practical guide. Med Decis Making. 1993;13(4):322-

38.

• Beck JR, Pauker SG. The Markov process in medical prognosis. 

Med Decis Making. 1983;3(4):419-458.

• Society for Medical Decision Making (http://www.smdm.org)