# Presentation: 223688

## Applying Multiple Frameworks and Theories in Implementation Research

- Jeffrey Smith
- Implementation Research Coordinator
- Mental Health QUERI

## “In theory there is no difference between theory and practice... in practice there is.”
Yogi Berra

**Yogi Berra**

## Definitions

- Formative evaluation (FE)
- Rigorous assessment process designed to identify potential and actual influences on the progress and effectiveness of implementation efforts (Stetler et al, *JGIM* 2006; 21:(Suppl 2):S1-8.)
- Summative evaluation
- Systematic process of collecting data on impacts, outputs, products, outcomes for a study
- Used to assess success, effectiveness, or goal achievement

## MH QUERI Approach to Implementation

- Design interventions based on theory and/or results from formative evaluation
- Conduct formative evaluation
- identify determinants of care
- tailor intervention design and implementation to local context
- assess barriers to implementation
- Use external facilitation techniques
- Identify and test new approaches and methods for overcoming barriers
- Conduct summative or impact evaluation

**Notes:**
there is a need, after a translation study, to have gained an understanding of the implementation efforts, potential causal events leading to change, and suggestive evidence re: the specific translation components and processes that appeared to most influence success. (adaptation of Altman)
Note:  this note is from the slide presentation by Cheryl Stetler to QUERI Translation Coordinators, February 2003

## An Approach to Using Theory for Implementation Planning

- Select theory of planned behaviorchange

- Assess fit with initial theory

- Identify potential strategies for achieving change

- Select interventionsthat fit with plannedstrategies (based on theory)

- Identify interventiontools that fit bothstrategy and theory

- Launch interventionusing identified toolsand strategies

- Evaluate effectivenessof intervention,strategies, tools

- From: Sales A, Smith JL, Curran G, Kochevar L. Models, strategies and tools: The role of theory in implementing evidence-based findings into health care practice.  *Journal of General Internal Medicine* 2006; 21:S43-49.

## Antipsychotic Treatment Improvement Program (ATIP)

- Goals
- Improve adherence to CPGs for psychosis
- Reduce use of very high doses of antipsychotic medications
- Encourage use of newer “atypical” antipsychotic medications for non-responders to conventional antipsychotic medications

## ATIP Application of Frameworks, Theory and Planning Models (cont)

- Social Influence Theory, Diffusion of Innovation, Social Cognitive Theory
- Utilize influential local clinician leaders (**opinion leaders**) to inform other clinical staff about evidence-based antipsychotic medication management, model-targeted prescribing behaviors, and motivate practice change
- Complexity Theory
- Initial conditions in HCOs are important in intervention planning, but HCOs are adaptive and change over time
- Need mechanism for assisting clinical partners in modifying or adapting initial intervention strategy based on changing circumstances, initial success

## ATIP Application of Frameworks, Theory and Planning Models (cont)

- Promoting Action Research Implementation in Health Services (PARiHS)
- Use **external facilitation** techniques to maintain contact with clinical opinion leader to assist with problem-solving and addressing challenges to intervention implementation
- PRECEDE Planning Model (*Predisposing, Reinforcing, and Enabling Constructs **in Education Diagnosis and Evaluation*)
- Address **predisposing** factors (eg, knowledge, attitudes) that influence EBP adoption
**Enable** providers to follow guideline recommendations at the point of care
- Apply social incentives through performance audit/feedback to **reinforce** providers’ implementation of EBPs

## Facilitation is a process of interactive problem-solving and support to meet specific implementation goals, which occurs in the context of a recognized need for improvement and a supportive interpersonal relationship.

- “Refined” Description of Facilitation Based on VA QUERI Experience*

- * Stetler CB, et al. *Implementation Science* 2006; 1:23.

## Operationalization of External Facilitation in MH QUERI 

- Facilitator
- Maintains regular contact w/ local QI leader / team
- Email communication
- Telephone
- Participate in QI Team meetings (as feasible)
- Assist in goal-setting; monitor implementation of project tools/strategies; performance on clinical QI goals
- Identify and problem-solve barriers to implementation
- Assist in adapting tools/strategies as needed or suggested by site partners to meet project goals

## ATIP Intervention Components

| Intervention Component | Supporting Theory or Planning Model |
| --- | --- |
| Clinical Opinion Leader | Diffusion of Innovation, Social Cognitive Theory, Social Influence Theory |
| External Facilitation | PARiHS, Complexity Theory |
| Psychosis Guidelines Help File | PRECEDE (predisposing) |
| Pocket Card on Antipsychotic Treatment | PRECEDE (enabling) |
| Pharmacy Order-Entry Reminder on Antipsychotic Dose Recommendations | PRECEDE (enabling) |
| Clinical Reminder on Metabolic Side Effects Associated with Olanzapine  | PRECEDE (enabling) |
| Audit/Feedback Performance Reports on Antipsychotic Dosing / Side Effect Monitoring | PRECEDE (reinforcing) |

## Formative Evaluation: Barriers and Lessons Learned

- Physicians do not always agree on who is an opinion leader; some sites may have no opinion leader
- Some sites had poorly developed formal and informal social networks among physicians
- A focus on physicians only as agents of change
- How much directive should be given to the opinion leaders concerning how to influence attitudes and behaviors?

- From: Curran GM, Thrush CR, Smith JL, et al. Jt Commission J Qual Safety 2005; 31(12):700-707.

## Summative Evaluation

- ATIP intervention improved antipsychotic medication management in concordance with CPGs
- Reduced pharmacy costs for antipsychotics
- Participating clinicians reported positive experiences with ATIP educational materials and clinical support tools

## Summary 

-  Application of multiple frameworks/theories in guiding intervention design and implementation can be successful in implementing EBPs
-  accommodates tailoring to setting when combined with formative evaluation
-  acknowledges there are generally multi-level determinants to complex, clinical QI issues
-  organizational-level
-  team-level
-  interpersonal-level
-  individual-level

## Multiple theory approach 

**Strengths**
-  useful in designing multifaceted interventions to influence multi-level determinants of care (flexible)
-  allows integration of theory, knowledge, methods from multiple disciplines (multidisciplinary)
**Limitation**
-  Can be unwieldy... need to provide rationale for applying multiple theory approach, and rationale for selecting the specific frameworks / theories applied
**Key Guidance on Evaluation**
** **combine with rigorous formative evaluation
-  conduct summative (impact) evaluation to assess intervention effectiveness on key study outcomes
-  confirm, refute or propose refinements to selected theory(ies) based on study findings

## Stages of FE

- Developmental
-  Identify determinants of current practice
-  Identify potential barriers / facilitators
-  Assess feasibility of proposed intervention
-  Integrate findings into intervention design and refinement prior to implementation

- Implementation-Focused
-  Assess discrepancies between implementation plan and execution, exploring issues of fidelity, intensity, exposure
-  Understand and document nature and implications of local adaptation

- Progress-Focused
-  Monitor impacts and indicators of progress toward project goals
-  Use data to inform need for modifying or refining original strategy
-  Provide positive reinforcement to high performers; negative reinforcement to low performers

- Interpretive
-  Assess intervention usefulness/value from stakeholder perspective
-  Elicit stakeholder recommendations for further intervention refinements
- Assess satisfaction with intervention and implementation process
-  Identify additional barriers / facilitators

*Post-Implementation*

*Pre-Implementation*

*Implementation*