1/25/2011

1

Using System Dynamics Tools to

Integrate Evidence into VA Stroke Care:

Part 1

Kristen Hassmiller Lich, PhD, MHSA (klich@unc.edu)

Project team: Hayden B. Bosworth PhD, Linda Williams MD,

Dawn Bravata MD, Chris Beadles MD, Amy Kaufman MSW,

Jack Homer PhD, Eric Cheng MD, David Matchar MD

January 25, 2011

Our approach

• This project uses System Dynamics (SD)

mod li e ng to h l e p key st k h ld akeholders of the

Stroke QUERI achieve a comprehensive

understanding of the complex systems

involved in stroke prevention and treatment

and provides a tool to support effective

stakeholder communication and the

establishment of strategic actionable

priorities.
1/25/2011

2

Our goals today:

• To review why we chose system dynamics modeling

• To (briefly) (briefly) summarize summarize the model structure structure

• Present the “base case” and get familiar with key inputs and

model output

• Present simulated intervention results

• To share what it took to implement these methods, lessons

learned, and next steps

Why System Dynamics modeling?
1/25/2011

3

Difficulties with standard approaches

• Challenges to effective, sustainable translation of research into

action in the real world (our QUERI mission!):

– Limited resources. f d un ing does not cover d l eve opment and

evaluation of policies and clinical interventions. Furthermore, mistakes

in strategic direction are costly.

– Numerous policy options. It is difficult to develop a single strategic

plan from the large and diverse evidence on stroke.  

– Multiple stakeholders, multiple visions. When dealing with complex

problems, stakeholders often operate from conventional and often

narrowly focused ‘wisdom’ about how to improve systems of care that

all limit their ability to see new ways of operating.

– Absence of a forum for integration. Multiple stakeholders are key to

successful and sustainable implementation. There is a lack of existing

linking structures in which key participants can come together to make

change happen.

Steps in Standard Problem Solving

Identify

problem

Collect

data

Evaluate

alternatives

Select

solutions Implement
1/25/2011

4

System Dynamics Modeling

Dynamic Modeling for Complex Policy Environments

Origins

• Jay Forrester, MIT, Industrial Dynamics, 1961 (“One of the seminal books 

of the last 20 years.”-- NY Times)

• Public policy applications starting late 1960s Public policy applications starting late 1960s

• Population health applications starting mid-1970s 

Forrester JW. Industrial Dynamics. Cambridge, MA: MIT Press; 1961.

Sterman JD. Business Dynamics: Systems Thinking and Modeling for a Complex World. Boston, 

MA: Irwin/McGraw-Hill; 2000.

System Dynamics Health Applications

1970s to the Present

• Disease epidemiology 

– Cardiovascular, diabetes, obesity, HIV/AIDS, 

cervical cancer, chlamydia, dengue fever, 

drug-resistant infections

• Substance abuse epidemiology 

– Heroin, cocaine, tobacco

• Health care patient flows 

– Acute care, long-term care 

• Health care capacity and delivery

– Managed care, dental care, mental health 

care, disaster preparedness, community 

health programs health programs

• Health system economics

– Interactions of providers, payers, patients, 

and investors 

Homer J, Hirsch G. System dynamics modeling for public health: Background and opportunities. 

American Journal of Public Health 2006;96(3):452-458.
1/25/2011

5

System Dynamics Modeling

Dynamic Modeling for Complex Policy Environments

Origins

• Jay Forrester, MIT, Industrial Dynamics, 1961 

(“One of the seminal books of the last 20 

years.”-- NY Times)

Good at Capturing

• Differences between short- and long-term consequences of an action

• Time delays (e.g., incubation period, time to detect, time to respond)

• Accumulations (e.g., prevalences, resources, attitudes)

y )

• Public policy applications starting late 1960s

• Population health applications starting mid1970s 

( g,p , , )

• Behavioral feedback (reactions by various actors)

• Nonlinear causal relationships (e.g., threshold effects, saturation effects)

• Differences or inconsistencies in goals/values among stakeholders

Forrester JW. Industrial Dynamics. Cambridge, MA: MIT Press; 1961.

Sterman JD. Business Dynamics: Systems Thinking and Modeling for a Complex World. Boston, 

MA: Irwin/McGraw-Hill; 2000.

Time Series Models

Describe trends

Events

Increasing

Mathematical models to inform strategic planning

Describe trends

Multivariate Statistical Models

Identify historical trend drivers 

and correlates Patterns

Increasing:

• Depth of causal theory

• Robustness for longerterm projection

• Value for developing 

policy insights

• Degrees of uncertainty

L fh Dynamic Simulation Models

Structure

• Leverage for change y a c S u at o ode s

Anticipate new trends, 

learn about policy consequences, 

and set justifiable goals

Homer, 2010
1/25/2011

6

Modeling for Learning

Multi-stakeholder 

Dialogue

Dynamic Hypothesis (Causal Structure)

High cholesterol

Hypertension

Smoking

Obesity

Not

obese

Obese

Not high

cholest

High

cholest

Non

diab

Non

smoker

Non

hypt

Hypert

Smoking and equivalent

secondhand smoke and air

pollution exposure

Secondhand

Smoker smoke

Healthiness

of diet

Extent of

physical activity

Chronic

stress

First-time CV

events and deaths

One-year costs from

first-time CV and other

risk factor complications

Access to quit

services

Access to mental

health services

Sources of chronic

stress (unalleviated by

social supports)

Access to healthy

food options

Air pollution

exposure

Marketing of

healthy behaviors

Anti tobacco

policy

Clean indoor

air policies

Diabetes

Marketing of health

& social services

Plausible Futures (Policy Experiments)

Obese fraction of Adults (Ages 20-74)

10%

20%

30%

40%

50%

Fraction of popn 20-74

Morecroft JDW, Sterman J. Modeling for learning organizations. Portland, OR: Productivity Press, 2000.

Sterman JD. Business dynamics: systems thinking and modeling for a complex world. Boston, MA: Irwin McGraw-Hill, 2000.

SYSTEM DYNAMICS

Diabetic Diagnosis and

control Access to physical

activity options Access to weight

loss/maintenance

services

Access to

primary care

Effectiveness of

chronic care

system

0%

1970 1980 1990 2000 2010 2020 2030 2040 2050

Why Use System Dynamics Methods...

• Help us develop a shared understanding of the system

• Teach us to think differently about how systems behave (that

is, in terms dynamics, circular causal feedbacks,

accumulations, etc)

• Allow stakeholders to view the larger system they are

embedded within

• Provide a framework for integrating what we know, and

determining importance of what we don’t know

• Support identification of high impact leverage points

• Offer a virtual world in which to “try out” and compare

policies
1/25/2011

7

“Effective” models of complex systems

• Causal (not correlational)

• D i ynam c ( t no equilib i ) r um

• Grounded in empirical tests (econometrics,

ethnography...)

• Broad boundaries (not limited to one

disciplinary domain)

Engage stakeholders

who develop “ownership”

THE MODEL: WHAT IT LOOKS LIKE

AND HOW IT HAS BEEN DEVELOPED
1/25/2011

8

Project time line

 May 2008‐October 2008: 6‐month RRP to develop preliminary

model

 May 2010: 12‐month RRP start‐up!  Initial day‐long meeting

with Core Modeling Team to walk through structure and

assumptions in the current model, and ask for feedback

 May‐August 2010: Model refinement (structural and

numerical), recalibration and analysis

 August 23 2010 W kh or s op: Mod l e ‐if d n ormed di i scuss on of

Stroke QUERI strategic plan

 August 31 2010:

Project time line

 May 2008‐October 2008: 6‐month RRP to develop preliminary

model

 May 2010: 12‐month RRP start‐up!  Initial day‐long meeting

with Core Modeling Team to walk through structure and

assumptions in the current model, and ask for feedback

 May‐August 2010: Model refinement (structural and

numerical), recalibration and analysis

 August 23 2010 W kh or s op: Mod l e ‐if d n ormed di i scuss on of

Stroke QUERI strategic plan

 August 31 2010: Pause!
1/25/2011

9

Project time line

 May 2008‐October 2008: 6‐month RRP to develop preliminary model

 May 2010: 12‐month RRP start‐up!  Initial Initial day‐long meeting meeting with Core

Modeling Team to walk through structure and assumptions in the

current model, and ask for feedback

 May‐August 2010: Model refinement (structural and numerical),

recalibration and analysis

 August 23 2010 Workshop: Model‐informed discussion of Stroke

QUERI strategic plan

 August 31 2010: Pause!

 August‐May 2010:  Designing and executing sensitivity analysis,

preparing manuscripts and toolkit, presenting work, brainstorming

next steps

A Framework for Stroke Strategic Planning

NON‐EVENT

VA USER POPN POST‐STROKE

by age group & subsets of VA USER POPN

smokers, hypertensives,

1st Strokes in

New VA Users

Rankin 4 & 5

1st stroke

Fatal

Survived

Recurrent

Rankin 2 & 3

Rankin 0 & 1 atrial fib, diabetes, CVD;

and interventions targeting

particularsubsets Post‐1st stroke

Rankin score

Post‐recurrent stroke

change in Rankin

POST‐TIA VA USER POPN

1st TIA 1st TIAs in

New VA Users

Undx Dx

stroke

Fatal

Survived Recurrent TIA
1/25/2011

10

A Framework for Stroke Strategic Planning

NON‐EVENT

VA USER POPN POST‐STROKE

VA USER POPN

Primary

prevention efforts

(HTN & AF mgmt)

by age group & subsets of

smokers, hypertensives,

1st Strokes in

New VA Users

Rankin 4 & 5

1st stroke

Fatal

Survived

Recurrent

Rankin 2 & 3

Rankin 0 & 1 atrial fib, diabetes, CVD;

and interventions targeting

particularsubsets Post‐1st stroke

Rankin score

Post‐recurrent stroke

change in Rankin

Post TIA CEA

& 2o

POST‐TIA VA USER POPN

1st TIA 1st TIAs in

New VA Users

Undx Dx

stroke

Fatal

Stroke rehab Survived

effort

Stroke acute care efforts:

‐Timely to hospital

‐ Effective tPA use

‐DVT prophylaxis

‐Dysphagia screening Post stroke

CEA & 2o prevention

efforts

& 2o

prevention

efforts

Recurrent TIA

TIA

diagnosis

effort

A Framework for Stroke Strategic Planning

NON‐EVENT

VA USER POPN POST‐STROKE

VA USER POPN

Primary

prevention efforts

(HTN & AF mgmt)

by age group & subsets of

smokers, hypertensives,

1st Strokes in

New VA Users

Rankin 4 & 5

1st stroke

Fatal

Survived

Recurrent

Rankin 2 & 3

Rankin 0 & 1 atrial fib, diabetes, CVD;

and interventions targeting

particularsubsets Post‐1st stroke

Rankin score

Post‐recurrent stroke

change in Rankin

Post TIA CEA

& 2o

POST‐TIA VA USER POPN

1st TIA 1st TIAs in

New VA Users

Undx Dx

‐ Total strokes & TIAs

‐ Fatal strokes

‐ DALYs

‐ Medical costs

OUTCOMES IN

VA USER POPN

stroke

Fatal

Stroke rehab Survived

effort

Stroke acute care efforts:

‐Timely to hospital

‐ Effective tPA use

‐DVT prophylaxis

‐Dysphagia screening Post stroke

CEA & 2o prevention

efforts

& 2o

prevention

efforts

Recurrent TIA

TIA

diagnosis

effort
1/25/2011

11

Model overview: Outcomes

• Total number of diagnosed TIAs and strokes: the number of 

diagnosed TIAs and strokes that occur per year, the incidence rate per 

1 000 enrollees per year and the total number of strokes occurring in 1,000 enrollees per year, and the total number of strokes occurring in 

the next 5 years, 10 years, and 20 years.

• Total number of fatal strokes: the number of stroke fatalities per 

year (enrollees who died as a result of their stroke within 3 months 

after the event), the rate per 1,000 enrollees per year, and the total 

number of fatal strokes occurring in the next 5, 10, and 20 years.

• Average Rankin score for enrollees post stroke: the average 

modified Rankin score 3 months post stroke per year.

• Di bilit Disability-adj t d lif (DALY ) djusted life years (DALYs): th diff b t the difference between

scenarios, over 5 years, 10 years, and 20 years.

• Medical costs: the costs related to stroke for which the VA would be 

responsible each year and cumulatively for the next 5, 10, and 20 

years.

Model overview

• Stock and flow model

• Programmed using Vensim software (www.vensim.com)

• F i d fi d VA ll h h Focus is on veteran users, defined as VA enrollees that have

had at least one primary care clinic visit in the past 12 months

• Simulates population of veteran users from 2010-2030

• Used VA data whenever possible

– Worked closely with Dr. Bruce Vogel to identify relevant VA data 

sources, including published reports, published journal articles, 

and VA databases (including VA Enrollment File, VA Decision 

Support System National Data Extracts, and VA Functional Status 

Outcomes Datab ) ase

• When VA data was unavailable, estimates were based on 

scientific literature and/or expert opinion

• Cost data is most imperfect
1/25/2011

12

Model overview: Inputs

• Stock-related parameter values: parameters that initialize the 

number of users in each stock.

• Flow-related values: parameters that shape the flows between

stocks over time (for example, controlling the incidence rate of stroke 

among pre-event users, functional loss post stroke, and death rates).

• Current quality of care: parameters that establish the current 

quality of care in the VA along the key dimensions delineated.

• Cost parameters: parameters that specify the per-event costs and 

the per-person per year costs included in the model.

• Level of functioning per stock: estimates of the level of 

functioning per stock that are used to calculate DALYs.

• Ability parameters: parameters that specify the extent to which 

interventions can alter model flows.

Interventions – What is possible?
1/25/2011

13

Illustrative key input parameters

Variable Value Primary source

% of the population that is post-stroke 4.1% Cornell, 2007

% of the population that is post % of the population that is post-TIA and TIA and diagnosed diagnosed 0 9%. Cornell 2007 Cornell, 2007

% of all strokes preceded by a TIA 7% OQP 2007

% of all strokes that are recurrent 18% OQP 2007, AHA 2010

Annual incidence rate of stroke in pre-event population 3-6 per 1,000 per year AHA 2010

Average life expectancy post stroke 6-7 years AHA 2010

% of strokes that are fatal (30 days) 8-12% OQP 2007, AHA 2010

% of survived first strokes that are R23 31% VA FSOD

% of survived first strokes that are R45 46% VA FSOD

% TIAs have a recurrent TIA w/in 90 days 16.5% OQP 2007

% TIAs have a stroke w/in 90 days 13.8% OQP 2007

% TIAs have a stroke after 90 days 0.8% per year AHA 2010

% strokes have a recurrent stroke w/in 90 days 5% Johnston, 2007

% strokes have a recurrent stroke after 90 days 3.6% per year Hankey, 1998

Base Run Assumed VAU Population by Age Group

VAU popn by age group

2 M

VAU popn by age group (stack)

6 M

1.5 M

1 M

500,000

0

2010 2015 2020 2025 2030

Year

4 M

2 M

0

2010 2015 2020 2025 2030

Year

VAU popn by age group[ageunder45] : Base2010

VAU popn by age group[age45to64] : Base2010

VAU popn by age group[age65to74] : Base2010

VAU popn by age group[age75plus] : Base2010

VAU popn by age group[ageunder45] : Base2010

VAU popn by age group[age45to64] : Base2010

VAU popn by age group[age65to74] : Base2010

VAU popn by age group[age75plus] : Base2010
1/25/2011

14

THE MODEL: RESULTS

Base Run Stroke Mortality & Morbidity

Fatal frac of strokes

0.2

0.15

0.1

Fatal strokes per year

6,000

5,500

5,000

0.05

0

2010 2015 2020 2025 2030

Year

Base2010

4,500

4,000

2010 2015 2020 2025 2030

Year

Base2010

DALYs lost per year

200,000

150,000

Avg loss of function post-stroke

0.5

0.45

100,000

50,000

0

2010 2015 2020 2025 2030

Year

Base2010

0.45

0.4

0.35

0.3

2010 2015 2020 2025 2030

Year

Base2010
1/25/2011

15
1/25/2011

16

THE MODEL: NEXT STEPS

Sensitivity analysis

GOAL: QUESTION:

Assess rob t us ness of H iti

interventions

How sensitive are

intervention rankings to

uncertainties in the

model?

Prioritize uncertainties Which uncertain

parameters are most

capable of changing the

magnitude of the impact

of each intervention on

DALYs lost the most?
1/25/2011

17

Implementation science

• To date, our work has really helped us learn

ab t ou how powerf l u pot ti l en a i t ti n erventions

could be

• But, models such as this can be incredibly

useful for helping a group identify new

leverage p , oints conceptualize interventions,

and think about the trade‐off between

feasibility and impact

 next workshop (May 2011?)

LESSONS LEARNED
1/25/2011

18

Lessons learned

• Start broad; add detail iteratively

• Keep it simple  

• (It won’t take much “reality” to make things quite

complex!)

• Data issues will arise

• Work with a diverse diverse group of stakeholders stakeholders throughout throughout

• It’s hard to know what data, etc you need until you are

going (again, iterate)

Lessons learned

• Start broad; add detail iteratively

• Keep it simple  

• (It won’t take much “reality” to make things quite

complex!)

• Data issues will arise

• Work with a diverse group of stakeholders throughout

ITERATE • Work with a diverse group of stakeholders throughout

• It’s hard to know what data, etc you need until you are

going (again, iterate)
1/25/2011

19

Lessons learned

• Having the right project team matters

– Engaged stakeholders are critical

– Core modeling team teleconferences were

incredibly useful

– RAs need to understand modeling, and need to be

able to immerse themselves in the evidence

– Communication between the modeling team and

system stakeholders is important, and difficult!

– Steep learning curve for programming these

models

This is very much a work in

progress, so stay tuned!

( ee F l free to co tact contact us for mo er

information: klich@unc.edu)