Enhancing Implementation Science in VA 2012, Introductory 

1: Introduction to Implementation Science i

Brian S. Mittman, PhD

Director, VA Center for Implementation

Practice and Research Support

VA Greater Los Angeles Healthcare System

May 3, 2012
O tli u ne

 P t ar 1: I t n roduction: overview of implementa

science

Part 2: Policy/practice foundations

Part 3: Implementation science frameworks

Part 4: Overview of EIS 2012 Introductory Progr
Increased investment and activity in

i l mp ement ti a on and i l mp ement ti a on re

are critical to achievement of key societ

in health and health care: Reducing gaps in quality and outcom

 quality of life; excess mortality, morbidit

 gender/racial equity, use of effective car

prevention

 Improving efficiency, reducing cost

 Addressing obesity, substance abuse, 
Poll question

Please indicate your primary affiliation and role; 

one

• VA clinician researcher• VA social scientist researcher

• Non‐VA clinician researcher

• Non‐VA social scientist researcher

• Non‐researcher, VA• Non‐researcher , non‐VA
Poll question

If you are a researcher, please indicate your prim

research focus:

• Basic science / lab research

• Cli i l n ca research (stud d y rugs, d i ev ces, h l ea th

promotion programs)

• Health services services researcher researcher (study health servic

servic

costs, quality, accessibility, delivery, organizatiofinancing, outcomes)

• Implementation researcher
Poll question

Please indicate your highest role to date in

implementation research:

• Principal investigator• Co‐investigator / consultant

• I have watched colleagues conduct implement

research

• None
What is implementation researc

1. Development of new evidence, innovation

2. Initial efforts to promote implementation

3. Measurement Measurement of rates of adoption adoption and

implementation (quality) gaps

4. Research Research to develop develop and evaluate evaluate implemen

implemen

(or QI) programs to increase implementatio

The next few slides illustrate illustrate this sequence sequence for a 

innovation in heart failure treatment
Phase 1. New evidence
Phase 2. Advocacy for adoption
Phase 2. Guidance for adoption
Phase 2. Incentives for adoption
Phase 2. Medical society, healthca

system support for adoption

• American Heart Association “Get with the

Guidelines”

• VA/DoD guideline development, implemen• Kaiser Permanente Permanente, HealthPartners HealthPartners, other p

p

systems
Phase 3. Measurement of adoption 

(US 2002‐03; UK 2005)
Phase 4. Trials of implementation prog
Phase 4. Trials of implementation progr
What is implementation researc

1. Development of new evidence, innovation

2. Initial efforts to promote implementation

3. Measurement Measurement of rates of adoption adoption and

implementation (quality) gaps

4. Research Research to develop develop and evaluate evaluate implemen

implemen

(or QI) programs to increase implementatio

Effectiveness Effectiveness of implementation implementation and QI program

program

varies, but is generally low
•

Barriers to implementation research pro

Lack of rigor; limited limited internal internal validity; validity; too few 

• Limited external validity; too many RCTs (or t

many flawed RCTs); use of “black box” evalua

evalua

approaches

• Lack of theory; lack of appropriate theory

• Too many theories; lack of guidance in using t• Implementation Implementation phenomena phenomena are extraordina

extraordina

complex (simple vs. complex vs. wicked probl
O tli u ne

Part 1: Introduction: overview of implementati

science

 Part 2: Policy/practice foundations

Part 3: Implementation science frameworksPart 4: Overview of EIS 2012 Introductory Prog
Policy/practice foundations of

implementation, implementation res

1. The implementation gap

2. The quality chasm
The e tat o e imp el m

e

ntatio

n gap

(second translational roadbloc

Basic

/

Lab

Clinical

Lab Research

Research

Research

Basic

/

LabResearch

ClinicalResearch

Imd

Research

Improved

Health

He
Refined research‐implementation pipelin

Implementation Implementation research research and clinical clinical resea

resea

“Type 1”Translation

Innovation/EvidenceDevelopment

Pre‐Clinical Implemen‐ Pre ClinicalTranslationalResearch

BasicScience

tationResearch

ClinicalResearch

“Type 2”Translation
The Implementation Gap:

A component of the Clinical Research C 

r 

• AAMC Clinical Research Summit: Clinical Researc

A National Call to Action (Nov 1999)

• IoM Clinical Research Roundtable (2000‐2004)
The “Quality Chasm” in US healthcare d

• Institute of Medicine (1999, 2001)• US and international quality measurement studies
Quality comparisons: VA vs. oth
Outline

Part 1: Introduction: overview of implementati

science

Part 2: Policy/practice foundations

 Part 3: Implementation science frameworks

Part 4: Overview of EIS 2012 Introductory Progr
Implementation research definitio

Implementation research is the scientific study ofmethods to promote the systematic uptake of res

findings and other evidence‐based practices into 

practice practice, and, hence, to improve improve the quality quality and

effectiveness of health services.

It includes includes the study of influences influences on healthcare healthc e ar

professional and organizational behavior.

Eccles and Mittman, 2
Implementation research goa

1. Develop reliable strategies for improving heal

related processes and outcomes; facilitate

widespread adoption of these strategies

2. Produce insights and generalizable knowledge

regarding implementation processes, barriers,

facilitators, strategies

3. Develop, test and refine implementation theo

and hypotheses; methods and measures
Implementation research framework

Planning Planning, designing designing, conducting conducting, reporti

reporti

1. Pipeline diagrams (and gaps)2. QUERI six‐step process (pre‐implementation,

implementation)

3. QUERI 4‐phase framework (phased trials)

4. SDP template template (not covered covered today)
Refined research‐implementation pipelin

Implementation Implementation research research and clinical clinical resea

resea

Innovation/Evidence

TranslationalResearch

Development Implementation

Research

Pre‐Clinical Implemen‐ Pre ClinicalTranslationalResearch

BasicScience

tationResearch

ClinicalResearch
Gaps in the pipeline:

Effectiveness and Hybrid E‐I studies

Efficacy studies

Clinical

Hybrid effectiveness‐implementation studies

ImpPro

Health Behavior Effective‐ Implemen‐ Pro

Out

tationResearch

Health Services 

nessStudies

Guidelines, Evidence Syntheses Syntheses 
Gaps in the pipeline:

Pre‐implementation studies

Efficacy studies

Clinical

Health Behavior Implem

Effective‐ness

Docum ent and diagnos 

en‐tationResearc

Health Services h

Studies diagnos 

e quality gaps Health Services h
The Classic Six‐Step QUERI Proc

1. Identify high risk/high

burden conditions

2. Identify b t es practices3. Define existing practice

patterns patterns in VA andvariations from best

practices

4. Identify (or develop

implement program

promote promote best pract

pract

5. Document outcome 

system improvemen

6. Document improve

health related quali
Pre‐QUERI Steps

Step M: Develop Develop measures measures, methods methods and da

resources

Step C: Develop Develop clinical clinical evidence evidence, effective effective

practices

Step E: Effectiveness studies
Annotated QUERI Six‐Step Proc

Step 1: Select Diseases/Conditions/Patient Populations

1A. Identify and prioritize (via a formal ranking procedu

risk/high burden clinical conditions

1B. Identify high priority clinical practices/outcomes wit

condition

Step 2: Identify Evidence‐Based Guidelines/Recommend

2A. Identify evidence‐based clinical practice guidelines a

recommendations

2B. Identify Identify evidence evidence‐based clinical clinical practices practicesPrioritize recommendations for implementation (based on gap

for outcomes, feasibility of improvement)
Expanded QUERI Six‐Step Proce

Step 3: Document and Diagnose Quality/Performanc

3A. Measure existing practice patterns and outcomes 

and identify variations from evidence‐based pract

benchmark benchmark outcomes outcomes (quality quality, outcome outcome and perfo

perfo

gaps)

3B. Identify determinants of current practices

3C. Diagnose quality gaps

3D. Identify barriers and facilitators to improvement
Gaps in the pipeline:

Observational implementation studies

Efficacy studies

I l

Clinical

Implem

en‐tation

Health Behavior Practice

Effective‐ness

Docum ent and diagnos 

Health Services 

Studies diagnos 

e quality gaps 

Implemen‐

Health Services tation

Trials
Observational studies of implementat

processes (QUERI Step O)

• Internal vs. external validity• Artificial vs. real‐world conditions
Annotated QUERI Six‐Step Proce

Step 4: Implement Improvement Programs

4A. Identify quality improvement strategies, programs and pro

components or tools to address quality gaps

(e.g., via literature reviews)

4B. Develop or adapt quality improvement strategies, program

components or tools (e.g., educational resources, decision 

4C. Implement quality improvement strategies and programs

Step 5/6: Evaluate Improvement Programs

5. Assess improvement program feasibility, implementation a

patient patient, family and system outcomes outcomes

6. Assess improvement program impacts on health related qu

(HRQOL)
Gaps in the pipeline:

Phased implementation trials

Efficacy studies

I l

Clinical

Implem

en‐tation

Health Behavior Practice

Effective‐ness

Docum ent and diagnos 

Health Services 

Studies diagnos 

e quality gaps 

Implemen‐

Health Services tation

Trials

Phase 1PilotProjects

Phase 2Small‐ScaleEfficacyTrials

Phase 3Large‐ScaleEffectiveness Trials
QUERI Four‐Phase Implementation Res

Framework

Phase Study Type Form of EvaluationPre‐trial Conceptual design of implementation program an

design (logic) model from theory, prior empir

Phase 1 Pilot / Pilot test, assess feasibility, formative evaluation a

Formative develop intervention/evaluation protocols

Phase 2 Efficacy Small‐scale rigorous trial in controlled settings wit

intervention support; internal validity

Phase 3 Effectiveness Large‐scale rigorous trial under routine conditions 

settings; external validity

Phase 4 Monitoring Ongoing monitoring and feedback
‐

Gaps in the pipeline:

Challenges in implementation trials

Most implementation trials fail to account for:• Heterogeneity Heterogeneity of settings settings, interventions interventions, effects effects• Context dependence, weak main effects• Process and mechanism (vs. outcome/impact) focus

• Protocol‐driven intervention adaptation

• External validity vs. internal validity, artificiality of tria

• Sustainability Sustainability and scale‐up/spread up/spread potential potential, economi

economi

evaluation

• Implementation phenomena are different

(e.g., representative representative sampling sampling paradox paradox)
Poll question

Which of the following types of research have you

conducted? (select (select all that apply)

• QUERI Step 3 pre‐implementation implementation research• Small‐scale / pilot implementation trials (Phase 1)

• Larger efficacy‐oriented implementation trials (Pha

• “Regional roll‐out” or “spread” trials (Phase 3)

• Observational studies of implementation processes 

O)

• Formative or process evaluation in implementation

science
Outline

Part 1: Introduction: overview of implementati

science

Part 2: Policy/practice foundations

Part 3: Implementation science frameworks

 Part 4: Overview of EIS 2012 Introductory Pr
EIS 2012 Lectures

1. Overview, QUERI Steps/Phases

2. Hybrid Effectiveness‐Implementation Studies; Q

3

3. Designing Implementation Strategies, Part 1

4. Designing Implementation Strategies, Part 2

5. Evaluating Implementation Strategies and Study

Implementation Processes, Part 1

6. Evaluating Implementation Strategies and Study

Implementation Processes, Part 2