QUERI 

Implementation Guide

HSR&D Cyber Seminar

March 6, 2014

http://www.queri.research.va.gov/implementation/
Why is implementation important?

• Pace of innovation in health care is rapid and 

increasing

• Innovation and evidence are piling up

– Managing the flow of new knowledge and best 

practices is very complex

– Routines of practice and care delivery are complex and 

difficult to change
Why is implementation research 

important?

• Understanding reasons for 

– Gaps in performance (suboptimal care delivery)

– Practice not based on best current evidence

• Developing reliable, effective, and efficient approaches to 

implementing evidence based care

– Reducing performance gaps

– Decreasing time to use of best evidence

– Improving the capacity of individuals and systems of care to 

take up new information
Roadmap to the presentation
APPLYING FRAMEWORKS, 

THEORIES, AND MODELS

Julie Lowery, PhD

Diabetes QUERI Center

Ann Arbor, MI
Implementation research uses frameworks, 

theories, and models for both prescriptive and 

explanatory purposes

6

Prescriptive Domain Explanatory Domain

Individual 

Characteristics

Organizational 

Characteristics

Clinical 

Outcomes

Implementation 

Strategy

Implementation 

Effectiveness

Targeted 

Intervention 

Change
Explanatory

Frameworks, Theories, and Models
Definitions for the 

Explanatory Domain

• Framework

• Broad set of propositions that organize diagnostic and/or prescriptive 

inquiry

• “Meta-theoretical” language; promotes comparison across theories

• Middle-range theory

• Small “t” theory: a set of context-independent propositions that 

specifies a denser and more logically coherent set of relationships

• (Big “T” Theory: a well-substantiated explanation of some aspect of 

the natural world)

• Model

• A simplified representation of a complex reality; precise assumption 

about a limited set of variables; context-dependent 
Explanatory Domain (continued)

• Frameworks: Consolidated Framework for 

Implementation Research (CFIR), Theoretical 

Domains Framework (TDF)

• Theory: Normalization Process Theory (NPT)

• Model: Klein, Conn, & Sorra
Consolidated Framework for Implementation 

Research: 5 Domains

(focus on the organization)

• Intervention

– 8 Constructs (e.g., evidence strength & quality, complexity)

• Outer Setting

– 4 Constructs (e.g., patient needs & resources)

• Inner Setting

– 14 constructs (e.g., leadership engagement, available resources)

• Individuals Involved

– 5 Constructs (e.g., knowledge, self-efficacy)

• Process

– 8 Constructs (e.g., plan, engage, champions)

10
Theoretical Domains Framework: 

12 Domains

(focus on the individual)

• Knowledge

• Skills

• Social/professional role and identity

• Beliefs about capabilities (self-efficacy)

• Beliefs about consequences

• Motivation and goals

• Memory, attention, and decision processes

• Environmental constraints

• Social influences

• Emotion 

• Behavioral regulation

• Nature of the behaviors
Model of Implementation Effectiveness:

Klein, Conn, & Sorra

12

Leadership 

Engagement Relative Priority Networks & 

Communications

Available 

Resources Champion(s) Compatibility

Implementation 

Effectiveness

Klein, K.J., A.B. Conn, and J.S. Sorra, Implementing computerized technology: An 

organizational analysis. Journal of Applied Psychology, 2001. 86(5): p. 811-824.

Helfrich, C. D., Weiner, B. J., McKinney, M. M., & Minasian, L. (2007). Determinants of 

implementation effectiveness: adapting a framework for complex innovations. Med 

Care Res Rev, 64(3), 279-303. 
Prescriptive Framework:

Intervention Mapping

• Step 0: Needs assessment

• Step 1: Creation of a matrix of proximal objectives

• Step 2: Selection of theory-based intervention 

methodologies, practical strategies, and suggestions 

from targeted users

• Step 3: Design and organization of the program

• Step 4: Adoption and implementation of the 

program

• Step 5: Monitoring and program evaluation
If you have questions on the previous section, please contact:

Julie Lowery, PhD

Julie.Lowery@va.gov

or 

Laura Damschroder, MS, MPH

Laura.Damschroder@va.gov
DIAGNOSING A GAP AND 

DESIGNING AN INTERVENTION

Teresa Damush, PhD

Stroke QUERI Center

Indianapolis, IN
Diagnosing Gaps and Designing 

Interventions

• Teresa M. Damush, Ph.D. & Arlene A Schmid, OTR, Ph.D.

 VA Stroke QUERI Center

a. An Introductions to Systems Thinking 

(http://www.queri.research.va.gov/implementation/section2/partA)

b. What Does Systems Thinking Contribute to Diagnosis and 

Intervention Design 

(http://www.queri.research.va.gov/implementation/section2/partB)

c. Conducting Diagnosis and Intervention Design 

(http://www.queri.research.va.gov/implementation/section2/partC)

d. Tools for Implementation Strategy Design 

(http://www.queri.research.va.gov/implementation/section2/partD)

e. Web Resources 

(http://www.queri.research.va.gov/implementation/section2/partE)
Introduction to Systems Thinking

FORMAL INFORMAL 

Objective Subjective

Documented processes, 

inputs, sub systems

(e.g., VISN)

Exist as observer constructs

(e.g., informal support on/off 

job)

Prescribed, mandated 

organization and processes 

(e.g., Acute Stroke Care 

Centers in VHA-Primary, 

Limited Hours, Supporting 

Centers)

Interactions among entities 

and associated behaviors, 

observed goals (e.g., informal 

support from academic 

affiliates and VA neurology 

services)
Systems Thinking Contributions to 

Diagnosis and Intervention Design

Allows us to recognize when a system is not 

functioning as designed

 Process mapping – map out a task model or 

performance model to determine the system 

effectiveness at each point in the system

 Identify System Gaps

 Work Flow

 Users of the system – process owner

 Process transfers across users

 Design Interventions to Fix Gaps
RNs not

Screening

Correctly

Lack of

Education

Other

Patient

Needs

Clinical

Status

Change

Unconscious

Assessment

Using 

Previous

Assessment Admit

RN 

Capacity

RN

Distractions

No

Sip Test

Questions

Not

Appropriate

MD Cancels

Order

H&P

Impact

Assessment RN Not

Checking Consult

Box

No RN 

Policy

Stroke

History

Clinical

Judgment

RN Choice

Per Policy

No Consult

To Surgery

Positive

Screen?

Dr. 

Approves

Consult?

NPO

Order 

Signed?

RN Contacts Dr for 

Order

No Policy

To Call

Lack of 

Communication

Patient

Not Labeled

NPO

Current State Process Map

Assessment  Order Consult

Doctor signs order

Yes

No

Yes Yes

Dysphagia

Assessment

Conducted

RN 

makes 

NPO

No Consult 

Ordered

No

Consult 

ordered

No
Intervention Mapping

 Intervention Mapping is a planning framework that 

utilizes theory, evidence, practical strategies to 

design implementation interventions and may target 

multi-level changes including a system and its users. 

 Developed for Health Promotion Programs

 Applied across fields including healthcare

Bartholomew LK, Parcel GS, Kok G, Gottlieb NH. Planning Health Promotion Programs: 

An Intervention Mapping Approach. 2nd ed. 2006, Jossey-Bass; San Francisco, CA.

20
Intervention Mapping 6 Steps

• 1. Needs Assessment –

• 2. Create Matrices of 

Expected Change 

Objectives and Specify 

Determinants

• 3. Identify Theory-Based 

Methods and Practical 

Strategies to Design 

intervention strategies

• 4. Program plan –develop 

and pretest materials

• 5. Specify Adoption and 

Implementation Plan 

• 6. Generate an 

Evaluation Plan

21
Intervention Mapping: Example

• Used intervention mapping to develop and implement 

a locally tailored, evidence-based secondary stroke 

prevention program in two VA medical centers

• Practice Gap = VA/DoD Guidelines and American Stroke 

Association recommend the provision of secondary 

stroke prevention at time of acute stroke event prior to 

discharge and across continuum of stroke care

Schmid A, Anderson J, Kent T, Williams LS, Damush TM. Using intervention mapping to develop and 

adapt a secondary stroke prevention program in VHA medical centers. Implementation Science, 

2010, 5:97

22
Step 2: Matrices of Change 

Provider Performance 

Objectives

Community Resources for 

Stroke Risk Management

Delivery System Design

Assesses pt stroke risk 

factors during 

hospitalization for acute

stroke

Access to local resources 

available to assess stroke 

risk factors

Work flow of discharge 

planning includes stroke

risk factor assess/educ

Orders lab tests as needed Access to lab tests and 

interpretation of results

System alerts lab results; 

prescribes based on results

Prescribes appropriate 

medications

Access and provides pt eds

materials on meds

Med reconciliation prior to 

discharge

Motivates pt to modify 

lifestyle

Write orders for home 

equipment 

Motivational interviewing is 

built into pt education

Refers pt to local programs Recommends and refers pt 

to local support programs

Access to local programs is 

available and up to date

23

Secondary Stroke prevention program matrix of program objectives (Provider)
Step 3: Theory Based Methods & Practical Strategies

Provider Performance 

Objectives

Theoretical Strategies of 

(Theory of Planned 

Behavior)

Practical Strategies (From 

provider interviews)

Assess patient stroke risk 

factors during 

hospitalization for stroke

Perceived Social Norms –

clinical champion 

promotes; added into 

annual competency

evaluation

Attitudes, Beliefs, Values –

training 

Self-efficacy – role playing 

to improve skills, 

vicarious/peer modeling

Behavioral Intentions – ask 

commitment to perform

Stroke risk factor 

assessment template is 

included in electronic

medical record; 

Checklist available at 

neurology workstation 

where discharge planning 

for stroke patients occurs

24
Informed, Empowered Patients

• Who recognize symptoms of stroke

• Who adhere to prescribed meds

• Who practice lifestyle strategies in 

collaboration with provider

Self-management

Patient Readiness to 

Change

Promote self-efficacy to 

activate lifestyle and 

meds changes

Individual/Group meetings

Telephone support

Delivery System

Local champion of stroke 

risk reduction in 

inpatient/outpat services

Discharge planning

Open access gym/rehab 

facilities at VA

Decision Support

VA/Dept of Defense 

Guidelines for stroke 

reduction

JCAHO guidelines to 

educate stroke 

patients

Clinical Information 

System

Clinical reminders for 

meds and lifestyle 

counseling

Chart forms

Community Resources

American Heart Association

American Stroke Association

Health System

VA FIM Performance Measure

Prepared, Proactive Practice Teams

• Who initiate meds treatment

• Who counsel on lifestyle changes

• Who collaborate & educate patients

• Who conduct timely follow-up of

stroke survivors Goals:

↑ Medication Adherence

↑ Patient activation for lifestyle change

↑ Physical functioning and Quality of Life

Using the Chronic Care Model to improve Stroke Risk Factors

25
Diagnosing Gaps and Designing 

Interventions

d. Tools for Implementation Strategy Design 

(http://www.queri.research.va.gov/imple

mentation/ section2/partD)

e. Web Resources 

(http://www.queri.research.va.gov/imple

mentation/ section2/partE)
If you have questions on the previous section, please contact:

Teresa Damush, PhD

Teresa.Damush@va.gov
METHODS USED IN IMPLEMENTING 

RESEARCH INTO PRACTICE

Hildi Hagedorn, PhD

Substance Use Disorder QUERI Center

Minneapolis, MN
QUERI Six Step Process

1. Select high prevalence, high burden 

conditions within Veteran population.

2. Identify evidence-based guidelines, 

recommendations, or best practices

3. Measure and diagnose quality/performance 

gaps.

4. Implement improvement programs.

5/6. Evaluate improvement programs.
QUERI Four Phase Framework

• Phase 1: Pilot project to develop/refine an 

improvement program and assess basic 

feasibility.

• Phase 2: Small clinical trial to further refine 

and evaluate improvement program. (4-8 

facilities)

• Phase 3: Regional roll-out projects (10-20 

facilities)

• Phase 4: “National roll-out” effort
An Example: 

The Liver Health Initiative

Step 1: Select

Condition

Step 2: 

Identify bestpractice

Step 3: 

Measure 

quality 

gap

Step 4:

Implement

Step 5/6: 

Evaluate

Hepatitis 

infections 

common and 

frequently 

unaddressed

in SUD 

patients.

Literature 

review and 

stakeholder 

meetings to 

identify/refine 

strategy

SUD

Clinic 

Director 

Survey

One Clinic 

Pilot

10-Site 

Training 

Program

Evaluate 

feasibility,

implementation, 

change in clinic 

processes

Phase 1

Phase 2
Hybrid Designs

Hybrid Type 1 Hybrid Type 2 Hybrid Type 3

Primary Question:

Will the clinical 

treatment work in this 

setting or with these 

patients?

Secondary Question: 

What are the potential 

barriers/facilitators to a 

treatment’s 

implementation?

Primary Questions:

Will the clinical 

treatment work in this 

setting or with these 

patients?

Does the 

implementation strategy 

show promise?

Primary Question:

Which method works 

better for implementing 

the clinical treatment?

Secondary Question:

Is the clinical treatment 

effective in this setting 

or with these patients?
If you have questions on the previous section, please contact:

Hildi Hagedorn, PhD

Hildi.Hagedorn@va.gov

or 

Jeff Smith, PhDc

Jeffrey.Smith6@va.gov
FORMATIVE EVALUATION

Tim Hogan, PhD

eHealth QUERI Center

Bedford, MA
Purpose and Need for FE in 

Implementation

* Formative evaluation offers, as Stetler 2006 suggests, a “rigorous 

assessment process designed to identify both potential influences 

and actual influences on the progress and effectiveness of 

implementation efforts.”

* FE is focused on the process of implementation, rather than the 

outcomes. 

* It helps to determine how factors amplify/facilitate, mitigate, or 

impede implementation efforts in a given context.

* Allows for more precise replication and further dissemination of 

evidence-based practices.
Data Collection Strategies Used in FE

* A variety of data collection strategies can have a role in FE

* Quantitative assessments can be used to collect data about 

stakeholder groups and implementation processes.

• Surveys

• Analysis of data from existing databases

* Qualitative assessments offer a “deeper-dive,” usually with 

smaller numbers of individuals to understand more fully the 

factors that influence implementation, implementation 

strategies, and best practices.

• Semi-structured interviews, focus groups, and other strategies may be 

used depending on the perspective sought. 
FE Research Processes

* There are a number of essential steps to conducting FE research

* The first and most important step is to identify the aims of your FE and how it 

fits into broader study goals.

• It is also important to include the theoretical or conceptual framework 

that is guiding the broader study in the development of your FE.

* Subsequently, the evaluation team must also

• Develop instruments and procedures to collect the data

• Collect the data 

• Analyze and report that data (in a rapid, continuous way)
FE Research Processes/Challenges

* There are a variety of potential challenges to conducting FE

* Data collection issues

• Selecting and effectively applying the appropriate approaches to data 

collection and analysis

• Identifying the “right” informants/participants

* Regulatory issues 

• Describing FE to organizational entities like research and development 

committees, institutional review boards, and other bodies that may 

not be familiar with it
Writing about FE

* Knowing the needs and perspectives of 

audience members is key

* A thorough description of FE will typically 

address 

• Settings and samples of participants

• Data collection techniques to be used

• Data collection procedures

• Managing and analyzing the data that results from 

the FE
If you have questions on the previous section, please contact:

Tim Hogan, PhD

Timothy.Hogan@va.gov

Jennifer Hill, MA

Jennifer.Hill3@va.gov

Henry Anaya, PhD

Henry.Anaya@va.gov
TOOLS AND TOOLKITS

Rani Elwy, PhD

HIV/Hepatitis QUERI Center

Bedford, MA

Anju Sahay, PhD

Chronic Heart Failure (CHF) QUERI Center

Palo Alto, CA
Section A: VA HSR&D Cyberseminars

• http://www.hsrd.research.va.gov/cybersemin

ars/default.cfm

• Traumatic Brain Injury Toolbox, May 5, 2014

o VIReC Clinical Informatics Series

o Register at URL or email cyberseminar@va.gov
VA HSR&D Cyberseminar website:

http://www.hsrd.research.va.gov/cyberseminars/default.cfm
Tool/Toolkit Example

Hepatitis C 

Self-Management 

Program 

∗ Timely Topics of 

Interest (TTOI) 

∗ Erik J. Groessl, PhD, 

Core Investigator, 

HHQUERI 

∗ http://www.hsrd.research.va.

gov/for_researchers/cyber_se

minars/archives/video_archiv

e.cfm?SessionID=607
Section B: Implementation Toolkits, 

Conferences, Consortiums and Trainings

Implementation Toolkits: 

• Quality tools posted on Agency for Healthcare Research 

and Quality (AHRQ) website:

http://www.innovations.ahrq.gov/innovations_qualitytool

s.aspx

o Example - Heart Failure Toolkit for Providers. This toolkit developed by the CHF QUERI offers a 

comprehensive set of downloadable resources to assist 

providers in managing care for heart failure patients. 
Heart Failure Toolkit website: 

http://www.queri.research.va.gov/chf/products/hf_toolkit
Conferences

∗ Dissemination and Implementation (D&I) Conference by the 

National Institutes of Health (NIH): 

http://obssr.od.nih.gov/scientific_areas/translation/dissemination_and_implementation/index.aspx

∗ Global Implementation Conference (GIC): http://globalimplementation.org/gic

∗ Seattle Implementation Research Collaborative (SIRC): 

http://www.seattleimplementation.org/
Training Opportunities

Within the VA

∗ Center for Implementation Practice and Research Support (CIPRS):

http://www.queri.research.va.gov/ciprs/

∗ Enhancing Implementation Science (EIS): http://www.queri.research.va.gov/meetings/eis/

Outside the VA

∗ Training Institute for Dissemination and Implementation Research in 

Health (TIDIRH). Sponsored by the Office of Behavioral and Social Sciences Research 

(OBSSR), National Institutes of Health (NIH), in collaboration with the National Cancer 

Institute (NCI), the National Institute of Mental Health (NIMH), and the U.S. Department of 

Veterans Affairs (VA): http://conferences.thehillgroup.com/OBSSRinstitutes/TIDIRH2012/index.html

∗ GEM (Grid Enabled Measures) Database - Dissemination and 

Implementation Initiative (GEM-D&I). Initiated and co-developed by the Cancer 

Research Network Cancer Communication Research Center at Kaiser Permanente Colorado 

and the National Cancer Institute’s (NCI) Division of Cancer Control & Population Sciences : 

https://www.gem-beta.org/public/wsoverview.aspx?wid=11&cat=8
If you have questions on the previous section, please contact:

Rani Elwy, PhD

Rani.Elwy@va.gov

or 

Anju Sahay, PhD

Anju.Sahay@va.gov
ORGANIZATIONAL UNITS OF THE VA 

RELEVANT TO IMPLEMENTATION 

RESEARCH

Christian Helfrich, PhD

Ischemic Heart Disease QUERI Center

Seattle, WA
Organizational Units Relevant to 

Implementation Research

• Veterans Health Administration is large & 

complex, with many org units relevant to 

implementation 

http://vaww.ush.va.gov/docs/VHAOrgChart103113.pdf

– Institutional policy and planning

– Operations and management

– Monitoring and oversight

– Training and development

• Develop understanding of mission & functions
Creating Effective Partnerships with VA 

Organizational Units

• What makes an effective partner?

• How do you meet the needs of your org 

partner?
Creating Effective Partnerships with VA 

Organizational Units

• Effective partnerships built on long-term 

relationships

• Effective partnered projects have:

– A real implementation research question 

• E.g., comparing E-Consults vs. group tele-video consultations 

(Specialty Care Access Network program-Extension for 

Community Healthcare Outcome (SCAN-ECHO)) 

– Frequent communication & expectation management

• E.g., baseline interviews of mini-residencies becomes priority; 

SCAN-ECHO follow-up delayed

– Early products/wins planned

• Papers & presentations actually a win/win
If you have questions on the previous section, please contact:

Christian Helfrich, PhD

Christian.Helfrich@va.gov

or 

Blake Wood, MSc

Gordon.Wood@va.gov
RESOURCES FOR IMPLEMENTING 

RESEARCH INTO PRACTICE

Amanda Midboe, PhD

HIV/Hepatitis QUERI Center

Palo Alto, CA
Resources

• Gain familiarity with key program offices:

– See Org Chart - http://vaww.ush.va.gov/docs/VHAOrgChart103113.pdf

– More detail about VA program offices – http://vaww1.va.gov/health/programs.asp

• For more detail, see http://www.valu.va.gov/Home/GetPDF/57

• Learn more about specific QUERI centers:

– http://www.queri.research.va.gov/
Resources

• Implementation research and practice - http://www.queri.research.va.gov/ciprs/

• Database and information systems - http://www.virec.research.va.gov/

• Health economics and cost-effectiveness - http://www.herc.research.va.gov/home/def

ault.asp
Example of Hierarchy

Primary Care 

Program Office

Office of Specialty 

Care Services

Office of Mental 

Health Services
HIV/Hepatitis QUERI Project Example

Clinical Public Health 

Group – Multi-VISN, 

HIV Testing Project
HIV/Hepatitis QUERI Project Example

Office of Homelessness –

HIV & HCV Homeless 

Project

Office of Public 

Health’s Clinical 

Public Health Group
If you have questions on the previous section, please contact:

Amanda Midboe, PhD

Amanda.Midboe@va.gov
• For more information, please visit:

http://www.queri.research.va.gov/implementation/