HSR&DCenter for Clinical Management Research

VAAnn Arbor Healthcare System

Laura J. Damschroder, MS, MPH DiabetesQUERIInitially presented in collaboration with:

Cheryl Stetler, PhDCIPRS Consultant

Teresa Damush, PhD Stroke QUER

Christian Helfrich, PhD IHD QUERI
 Survey:What exposure have you had to th

presentation for today?

 Attended the EIS training lastJuly in Denver Listened to the presentation online

 Reviewed only the slides online

 This will be my first time

 Use whatever wording you’ve used in the p

thisis to get the gist of it.

2
 Short introduction

 Using theory in hybrid & implementation implementation 

with concrete examples

3
 Replicate successful implementation

 Core components Rationale

 Generalize knowledge about how to impl

and sustain interventions

 Navigate complex implementations Improve Improve prospects prospects for sustainability sustainability

Theory‐driven implementation enable

4

Theory driven implementation enable

accomplishment of these objectives
 Multiple theories often needed

 Process theories – (aka prescriptive, planned action)

 How implementation should be planned, organ

scheduled

 Explanatory theories (aka descriptive, impact)

 Hypotheses and assumptions about how

implementation activities will facilitate a desire

chang , e as well as the facilitators and barriers f

 Mixed theories

 Elements of both

5

Adapted from: Grol RP, Bosch MC, Hulscher ME, Eccles MP, Wensing M. Planning and studying improvement in p

theoretical perspectives. Milbank Q. 2007;85(1):93‐138.
Phase 1

Phase 2

Phase 3Phase 4

Pronovost, P. J et al. BMJ 2008;337:a1714
 Quantitative measures of predictors of

implementation effectiveness

 Strength of relationships emp y irically estimated

Klein KJ, Conn AB, Sorra JS. Implementing computerized technology: An organizational analysis. J Appl Psychol. 200
 Consider nature of the theory

 Process v. explanatory

 Context (e.g., policy, organization)

 Discipline ( g, e. . social science, p y gy) psychology)

 Consider level at which it will be applied

 Individuals Teams Organization System

 P i rev ous fi di n ngs, experience Greatest potential for adding to the know

base

8
 Survey: How confident do you feel about

conducting a theory‐based implementatiostudy?

 Not at all confident

 Somewhat confident

 Moderately Confident

 Very confident

 I’m ready to dive in!

9
Intervention Focus Implementation Approach

YES N

YES

HybridType II:

Test clinical

intervention, test

Hybri

Test 

interv

observ

ClinicalEff ti

,

implementation

intervention

observ

inform

implem

HybridType III: Effectiveness

NO

HybridType III:

Test implementation

intervention,

observe/gather

f l l NO Observati

information on clinical

intervention and

outcomes

Observati

Implementation Study

From:Combining Elements of Clinical Effectiveness and Implementation ResearchTrials: Hybr

Designs. CurranG., Bauer M., Mittman B., Stetler, C. Enhancing Implementation Science Serie

CyberSeminar.January 2011.
Implementation 

Research

Effectiveness 

Studies

Efficacy 

Studies

Spatially speaking, hybrid designs 

here

Spatially speaking, formative/proc

evaluations go here evaluations ...go here...

From:Combining Elements of Clinical Effectiveness and Implementation ResearchTrials: Hybr

Designs. CurranG., Bauer M., Mittman B., Stetler, C. Enhancing Implementation Science Serie

CyberSeminar.January 2011. 11
Proactive Diabetes Case

Management Management x 2 studies studiesNPs / PharmD

Medication changes & phonefollow‐upVA primary care/managedcareRCTPrimary Clinical OutcomeA1c
Proactive Diabetes Case

Management x 2 studies

AIM

Management x 2 studiesNPs / PharmD PharmD

Medication changes & phonefollow‐up

PharmDMedication changes phone follow‐up

VA primary care/managedcareRCT

VA primary care/ma

care

RCT RCTPrimary Clinical OutcomeA1c

RCTPrimary Clinical Out
Pre-I l t ti Implementation During Implementation During Implementation Post-Implemen

Implemen

Formative Evaluation

Formative

Evaluation

Summ

Evalu

Interp

Improve & Tailor

Evalu

Improve & Tailor

Intervention Improve & Adjust Implementation Explain s

evaluatio

Evaluate 

attainm

sustain

*Stetler CB, Legro MW, Wallace CM, Bowman C, Guihan M, Hagedorn H, Kimmel B, Sharp ND, Smith JL: The role of formative evaluation in implementation research

and the QUERI experience. J Gen Intern Med 2006, 21 Suppl 2: S1‐S8
Assess targeted Assess targeted Select targeted Develo

EBP change and 

context

Select targeted

theory(s) & provide 

rationale

Develo

imple

st

Assess fit of 

findings with 

•Observational

•HybridType I

initial theory

Execu

imple

st

Evaluate effectiveness

of implementation

strategy

Adapted from: Sales A, Smith J, Curran G, Kochevar L: Models, strategies, and tools. Theory in ievidence-based findings into health care practice. J Gen Intern Med 2006, 21 Suppl 2:S43-49.
Assess targeted Assess targeted Select targeted Develo

EBP change and 

context

Select targeted

theory(s) & provide 

rationale

Develo

imple

st

Assess fit of 

findings with 

Developmental Formative Evaluation:

Assess EBP intervention and

initial theory barriers, facilitators, and setting

Execu

imple

st

Evaluate effectiveness

of implementation

strategy

Adapted from: Sales A, Smith J, Curran G, Kochevar L: Models, strategies, and tools. Theory in ievidence-based findings into health care practice. J Gen Intern Med 2006, 21 Suppl 2:S43-49.
Assess targeted Assess targeted Select targeted Develo

EBP change and 

context

Select targeted

theory(s) & provide 

rationale

Develo

imple

st

Assess fit of 

findings with 

•Identify potential factors or theories from prior research related

to this type of implementation

initial theory

yp

•Identify applicable types of published theories (SeeA SAMPLE

CLASSIFICATION OFTHEORIES handout)

•Select specific theory(s) Execu

imple

st

Evaluate effectiveness

of implementation

strategy

•Select specific theory(s)

Adapted from: Sales A, Smith J, Curran G, Kochevar L: Models, strategies, and tools. Timplementing evidence-based findings into health care practice. J Gen Intern Med 202:S43-49.
 StudyAims

 Describe adoption and implementation of a C

associated Urinary Tract Infection Prevention 

(aka Bladder Bundle)

 ID and compare barriers & facilitators of imple

the Bl dd a er B dl un e

 Develop and evaluate implementation strat

the Bladder Bundle

 Facilitate implementation in small sample of 

18
CFIR F

Pronovost, P. J et al. BMJ 2008;337:a1714
 Intervention Characteristics

 8 Constructs (e.g., evidence strength & quality, complexity)

 Outer Setting

 4 Constructs (e.g., patient needs & resources)

 Inner Setting

 14 constructs (e.g., leadership engagement, available resourc

 Individuals Involved

 5 Constructs (e.g., knowledge, self‐efficacy)

 Process

 8 Constructs (e.g., plan, engage, champions)

20
 Selected High Priority Constructs from CF

 Narrow the focus

 Data collection

 Semi‐structured phone interviews

 On‐s tei vis ts i

 Validated and customized quantitative survey 

 Selected Organizational Readiness for Change

Assessment (ORCA) items

 Other measures

21
Assess targeted Assess targeted Select targeted Develo

EBP change and 

context

Select targeted

theory(s) & provide 

rationale

Develo

imple

st

Assess fit of 

findings with •ID general implementation approach(es); consider theory and

initial theory pragmatic findings

•Operationalize specific implementation interventions consistent

with theoretical and pragmatic intent Execu

imple

st

Evaluate effectiveness

of implementation,

strategy

with theoretical and pragmatic intent•ID implementation tools to support implementation activites

Adapted from: Sales A, Smith J, Curran G, Kochevar L: Models, strategies, and tools. Theory evidence-based findings into health care practice. J Gen Intern Med 2006, 21 Suppl 2:S43-49
Types of

Theories

Potential

Theories Rationale Compone

ExplanatoryTheory CFIR

• Provides systematic

approach for evaluating

context

B i & f ilit t

Online toolkit to address sk

about suppor

•Barriers & facilitators evidence
Types of

Theories

Potential

Theories Rationale Compone

ExplanatoryTheory CFIR

• Provides systematic

approach for evaluating

context

B i & f ilit t

Online toolkit to address sk

about suppor

•Barriers & facilitators evidence

Process theory Pronovost’s 4E’s Model

• Used in similar setting

(hospital) and practice

(preventing catheter‐related

Data collectio

for catheter d

4E s Model (preventing catheter related indications

infections)

indications
Types of

Theories

Potential

Theories Rationale Compone

ExplanatoryTheory CFIR

• Provides systematic

approach for evaluating

context

B i & f ilit t

Online toolkit to address sk

about suppor

•Barriers & facilitators evidence

Process theory Pronovost’s 4E’s Model

• Used in similar setting

(hospital) and practice

(preventing catheter‐related

Data collectio

for catheter d

4E s Model (preventing catheter related indications

infections)

indications

Individual‐level Adult Guide education approach –

•Select appro

materials from 

Individual levelTheory Learning Theory minimize didactic and maximize hands‐on toolkit •Plan multi‐le

educational s
Types of

Theories

Potential

Theories Rationale Compone

ExplanatoryTheory CFIR

• Provides systematic

approach for evaluating

context

B i & f ilit t

Online toolkit to address sk

about suppor

•Barriers & facilitators evidence

Process theory Pronovost’s 4E’s Model

• Used in similar setting

(hospital) and practice

(preventing catheter‐related

Data collectio

for catheter d

4E s Model (preventing catheter related indications

infections)

indications

Individual‐level Adult Guide education approach –

•Select appro

materials from 

Individual levelTheory Learning Theory minimize didactic and maximize hands‐on toolkit •Plan multi‐le

educational s

Nurse cha

Pragmatic Findings What worked for others may work here Nurse cha

facilitate ed

exerci
Assess targeted Assess targeted Select targeted Develo

EBP change and 

context

Select targeted

theory(s) & provide 

rationale

Develo

imple

st

Assess fit of 

findings with 

initial theory Progress‐focused Formative Evaluation

Execu

imple

st

Evaluate effectiveness

of implementation

strategy

Adapted from: Sales A, Smith J, Curran G, Kochevar L: Models, strategies, and tools. Theory in evidence-based findings into health care practice. J Gen Intern Med 2006, 21 Suppl 2:S43-49. 
Assess targeted Assess targeted Select targeted Develo

EBP change and 

context

Select targeted

theory(s) & provide 

rationale

Develo

imple

st

Assess fit of 

findings with 

initial theory Implementation Progress‐focused‐focused Formative Formative Evaluation Evaluation

Execu

imple

st

Evaluate effectiveness

of implementation

strategy

Adapted from: Sales A, Smith J, Curran G, Kochevar L: Models, strategies, and tools. Theory in evidence-based findings into health care practice. J Gen Intern Med 2006, 21 Suppl 2:S43-49.
Pre-I l t ti Implementation During Implementation During Implementation Post-Implemen

Implemen

Formative Evaluation

Formative

Evaluation

Summ

Evalu

Interp

Improve & Tailor

Evalu

Improve & Tailor

Intervention Improve & Adjust Implementation Explain s

evaluatio

Evaluate 

attainm

sustain

*Stetler CB, Legro MW, Wallace CM, Bowman C, Guihan M, Hagedorn H, Kimmel B, Sharp ND, Smith JL: The role of formative evaluation in implementation research

and the QUERI experience. J Gen Intern Med 2006, 21 Suppl 2: S1‐S8
CFIR Construct Potential Barrier Æ Tailor AIM

Case managers

concerned about overlap

between patients being Æ

• Pre‐implementat

with case manager

concerns

Networks &C i ti

between patients beingcase managed and those

targeted byAIM

Æ

• Promote close on

communications bepharmacist and cas

Communications h d 

Providers not used to

working

i h li i l h i

Æ

• Pharmacist introd

provider• Pharmacists partimonthly staff meet

with clinical pharmacists monthly staff meet

• Co‐sign providers 

changes within the 

30
CFIR Construct Barrier

Æ Adjustment

AvailableResources

Inadequate spacerequired pharmacist tonegotiate for space on

Æ

Unable to resolve. 

explain process or Resources negotiate for space on

a daily basis summative outcom

Access to

Information

&

Motivationalinterviewin

g

Æ

Continued webinarpharmacists shared 

Information

&

Knowledge

g

techniques were

challenging to apply

Æ

pexperiences and pr

solved with an MI e

Pressure to treat Negotiated for

a ne

External Policy& Incentives

patients who did not

meet performancemeasure goals

Æ Negotiated for

a ne

initiative to start af

end of AIM
Conducting ConductingTheory‐based Implement

Implement

Assess targeted Assess targeted Select targeted Develo

EBP change and 

context

Select targeted

theory(s) & provide 

rationale

Develo

imple

st

Assess fit of 

findings with 

Don’t skipthis step initial theory

Execu

imple

st

Evaluate effectiveness

of implementation

strategy
 Does theory still app yl ?

 Modifications/refinements needed? Building validity

 Quantitative theory testing

 Test hypotheses Path analyses

 Qualitative theory testing

 Is terminology/language terminology/language coherent? coherent? Does it promote comparison of results across s

and studies over time?

 Does it stimulate new theoretical development

33
Conducting ConductingTheory‐based Implement

Implement

Assess targeted Assess targeted Select targeted Develo

EBP change and 

context

Select targeted

theory(s) & provide 

rationale

Develo

imple

st

Assess fit of 

findings with Add to the knowledge‐base initial theory Add to the knowledge base

Execu

imple

st

Evaluate effectiveness

of implementation

strategy
 CFIR embraces, consolidates, and standar

key constructs constructs from other theories theories

 Agnostic to specific models and theories

 Can be used to b ld ui , affirm, or evaluate them

 Provides a pragmatic structure for evaluat

complex il i mp ementations

 Helps to organize findings across disparat

implementations

 Paves the way for cross‐study synthesis

35
 Come to our workshop at HSR&D to explo

more about using the CFIR to synthesizefindings:

Implementation Research:An

approach to promote more timely and

informative synthesis for real‐world

implementation findings

Presented by:

Laura Damschroder

Carmen Hall

Cheryl Stetler

Thursday 4:15‐5:45pm February 17th

36

y 4 5 5 45p y 7
 Survey:Think back to the level of confiden

indicated at the beginning of this presentNow, how confident do you feel aboutconducting a theory‐based implementatiostudy?

 Less confident than before hearing this present

 I’m feeling about the same

 Slightly more confident Much more confident

37
38
 Identify barriers and facilitators of use or

implementation

 Study designs

 Ethnographic and other qualitative approache

 Quantitative measures

 Mixed methods

39
Aim 2:

 Identify factors that facilitate & impede ad

and implementation of evidence‐based prep act ces ractices

 Sequential mixed methods

 Quantitative survey Qualitative: semi-structured interviews, on-site 

observations

 Theory-guided evaluation guided evaluation

40
Characteristics of 

the intervention

O i ti l Organizational Eff ti

characteristics

Adoption decision

Effective 

implementation

O

Environmental context

41

Application to infection prevention practices

Krein SL, Olmsted RN, Hofer TP, Kowalski C, Forman J, Banaszak‐Holl J, et al. Translating infection preventio

practice using quantitative and qualitative research. Am. J. Infect. Control 2006;34(8):507‐12.
Practice 1 Practice 2

Category II evidence Category II evidenceEvidence debated, dismissed Evidence accepted

Mechanical issue Challenging to change behav

Selective use in

patients Inconsistent im

plementation

across units (e.g., MICU v. SIC

42
 Identify gaps in use and implementation

 Adds to knowledge about barriers & facil

use and implementation

 Tend to be single or small sample case st Use theoretical framework

 Guide evaluation

 Organize findings

 Evaluate the theory

 Theory allows us to build on knowledge a

studies & settings

43