Funded by the VA HSR&D (Project# SDR 10-012)

Spotlight on Women Cyberseminar Series

Purpose of This Talk

 Understand what QI and EBQI are Understand what QI and EBQI are Explore development of an EBQI 

approach to improving healthcare foapproach to improving healthcare fo

women veterans
Framework for EBQI
Efficacy Studies

QUESTION: Can an intervention improve h lth d ti l health un

der optima

l

conditions?

Intervention Effect
Efficacy

 A backbone of clinical science

 Aim to eliminate contextual factors, 

Aim to eliminate contextual factors, 

causal link to intervention

 Because efficacy studies optimize 

Because efficacy studies optimize 

neutralize context, their link to re

improvement is weak improvement is weak

 Someone else is responsible for usin

information
Efficacy

Routine Care

Effectiveness

Quality 

Improvement
Classical Effectiveness 

Studies

Question: Is the intervention effective real

-world contexts? world contexts?

Intervention Effect

Ctt on

tex

t
Healthcare Interventions

Traditional: 

E.g., pills, 

proce

dures

Multicompon

Changes in 

Healthcare

Intervent

Context: E.g., 

reminders
Classical Effectiveness

 M i l i t fd i May involve a varie

ty o

f

designs

 Randomized, cluster randomized, q

experimental or advanced observaexperimental, or advanced observa

 Involves partnership with care s O t t b j t li i 

O

u

tcomes may no

t

be 

jus

t cli

nic

cost)

 I ff t d b t t Is affecte

d

by con

tex

t

 Goal is not, ,however, to change th
Efficacy

Routine Care

Effectiveness

CER

Quality 

Improvement

Clinical ProblemSolving
Quality Improvement Effectiven

Studies

Question: Can the intervention changcontext to improve routine care? context to improve routine care?

Intervention Impact

Ctt on

tex

t
QI Study Goal Is To Change the 

Setting

 Develop a chan

ge to context that w

 Im

p qy rove 

quality of care

 Spread across sites Endure if successful 

past the research

 QII studies are usually excluded from 

systematic effectiveness reviews on systematic effectiveness reviews on 

implementations, e.g. pills, technolo

collaborative care for depression collaborative care for depression
Context

Organization

& Practice Provider

Environment P

E

Provider

Behavior

H E A L T H O U T C O M 

VA/RAND/UCLA Center for the Study of Healthcare Prov

Behavior (Rubenstein, Mittman, Yano, Mulrow Medical Care
Defining QI Method

and Change Strateg

QI Project/Research QI Project/ResearchRe-engineering strategy
Context & QIIs Are Intertwined

 QII s’ manipulate aspects of context manipulate aspects of context QII’s are immersed in particular 

contexts

 Classical efficacy methods emphasize 

internal validity and eliminating contexinternal validity and eliminating contex

 QII’s make no sense without context QII’ b t b h i QII’s are a

bou

t

b

e

havior 
Complexity of the CQI Literature

 Two views of CQI innovation:

naturally emergent

rationascienti
Evidence-Based Quality 

Improvement

 Aims to systematically insert evidence Aims to systematically insert evidence

knowledge, data at all points in 

development of a QI intervention development of

a QI intervention

 Builds on a research/clinical partnersh

 QI leadership development QI leadership development If needed, aims for policy change

 Most useful when sufficient evidence Most useful when sufficient evidence

about the problems, their origins, and their potential solutions exists their potential solutions exists
Top Down and Bottom UP

 Centralized Centralized strategic strategic decision decision

-making

and local tactical decision-making

 Multilevel organizational approach Multilevel organizational approach

 National strategic plans, guidelines, models Regional priority Regional priority

-setting process setting process

 Local care model development, adaptation, 

im

plementation

 Regional then national spread (if 

successful)
EBQI Requirements

 An evidence-based problem, e.g.

 Needs assessment Needs assessment Variations

 Guidelines governing clinical care goal

Guidelines governing clinical care goal

 Sufficient relevant evidence regarding 

potential methods for improvement potential methods for improvement

 Barriers/facilitators Care models

 A research/clinical partnership 

 including subject matter & QI methods 

experts 
EBQI For Women’s Health: 

Evidence on Problems 
VA problems With/For Women Veterans VA problems With/For Women Veterans C

’

 Problematic outcomes

 Women vet satisfaction with VA care Women vet satisfaction with VA care Quality of prevention, chronic disease care

mental health care

 Homelessness, esp. with MST, PTSD, disa

 Contributors

 Limited workforce training in female-spec

care modalities

 Ph i l en i onment Physical en

vironment

 Limited in-house technology (meds, 

equ p e t) ip

m

e

nt)

 Substantial contract care
Variations in Care/Needs

Examples: Deficits in care in the ED (Washington 

2006)

 Local implementation of gender-sensitive 

mental health care in VA settin

g g

s is hi

ghl

variable (Oishi 2011)
Problem Analysis

 Two overarching problems:

1. Women vet satisfaction with VA care2. Quality of care for women vets

○ Chronic illness, prevention, mental 

health, social health
How Serious are These Problem

 Only 30% of women veterans attendin

Only 30% of women veterans attendin

VA one year return the next (FY 2010 data)

○ Continued reports of gender 

insensitivity in physical and staff components of care

○ Continued disparities in e.g. LDL care 

andl l eve

s
EBQI Requirements

 An evidence-based problem, e.g.

 Nd t ee

ds assessmen

t

√

 Variations 

√

 G id li i li i l 

G

uid

elines 

governin

g cli

nical care 

go

 Sufficient relevant evidence regardi

pot ti l th d f i t tenti

al meth

o

ds 

for improvemen

t

 Barriers/facilitators C dl are mo

d

els

 A research/clinical partnership 

including subject matter & QI experincluding subject matter

& QI exper
EBQI For Women’s Health: Guidelin
Guidelines for Clinical Care Goals

 Several national strategic planning e

 Research Clinical

 N ti l f f N

ational performance measures 

for 

gender-specific prevention e.g. Pap 

smears, mammo

grams

 National male/female performance 

measures for many chronic conditio
Maybe No VA Guidelines....

 ? for menstrual disorders, 

contraceptive management contraceptive management, 

pregnancy, infertility, parenting issu

 ? For gender ? For gender

-sensitive management sensitive management

 Across primary care sites

Across the continuum (ED, LTC, 

telehealth, inpatient)
EBQI Requirements

 An evidence-based problem, e.g.

 Nee

ds assessment 

√

 Variations 

√

 G dl l l l Gui

d

elines 

governin

g clinical care 

goal

 Sufficient relevant evidence regarding 

potential methods for improvement

 Barriers/facilitators Care models

 A research/clinical partnership includin

subject matter & QI experts
EBQI For Women’s Health: Evidence 

on Methods for Improvement
Evidence on Methods for 

Improvement

 Systematic reviews show few Systematic reviews show few

intervention studies (Goldzweig 2006, BeanMayberry 2011)

 Mostly descriptive Little information about what works or 

doesn’t work, especially re. quality of care

 Rich information on 

barriers/facilitators
Barriers/Facilitators: Examples

 Knowledge deficits about VA eligibility an

services and perceived worse VA care quservices and perceived worse VA care qu

predict outside health care use (Washington 

 Women veterans have less social suppor

Women veterans have less social suppor

male veterans (Frayne 2006)

 African American women, older women v

African American women, older women v

receive less depression care (Sambamoorthi 2

 Hi

g / h level of contract care for ob

/gy

n

 Women veterans are dispersed across ru

urban locations
Evidence Regarding Improvement Met

 Specialized women veteran clinics 

associated with: associated with:

(Washington 2011)

 I d ti f ti f tt d Increase

d sati

sfaction 

for atten

dees

 No impact on quality of primary-type  No information about impact across t

continuum
EBQI Requirements

 An evidence-based problem, e.g.

 Nee

ds assessment 

√

 Variations 

√

 G dl l l l Gui

d

elines 

governin

g clinical care 

goal

 Sufficient relevant evidence regarding 

potential methods for improvement 

√

 Barriers/facilitators 

√

 Care models 

√ --

 A research/clinical partnership includin

subject matter & QI methods experts
EBQI For Women’s Health: A 

Research/Clinical Partnership
Research/Clinical Partnership

 Women s’ health practice health practice

-based

research network established

 Strong participation of research and Strong participation of research and

clinical experts

 National clinical/research leadership National clinical/research leadership ? Regional leadership involvement ? Methods resources for collaborating 

? Methods, resources for collaborating 

quality improvement, access to QI 

facilitation in development facilitation in development
EBQI Requirements

 An evidence-based problem, e.g.

 Nee

ds assessment 

√

 Variations 

√

 G dl l l l Gui

d

elines 

governin

g clinical care 

goal

 Sufficient relevant evidence regarding 

potential methods for improvement 

√

 Barriers/facilitators 

√

 Care models - -

 A research/clinical partnership includin

subject matter and QI methods exper
EBQI For Women’s Health: Is it Time
Readiness Assessment: Key 

Issues 

 Paucity of information on effective Paucity of information on effective

care models

 Incomplete guidelines/measures for Incomplete guidelines/measures for

clinical care goals (esp. gender 

sensitivity)

 Need for regional engagement Need for QI facilitation/expertise
Ways Forward Ways Forward....
Additional Evidence Development C

Begin in Parallel with 

QI 

 What makes an environment 

gender-sensi

 What kinds, formats of messaging best pro

the information women vets seek? 

 How can gender-sensitive care be provide

across the continuum, outside of current women s’ health clinics? health clinics?

 How can gender-specific technologies best 

made available?

 What are the origins of quality of chronic 

disease care deficits?

 Are there applicable lessons learned outsid

Are there applicable lessons learned outsid

Champions Pick Problem(s) to Address & 

E M li Engage M

ulti-L l S k h ld Level Sta

k

e

h

olders

 Problems that

 Fall within national priorities

H

a o a a po ( ) ve l

o

cal ch

a

m

pio

n

(

s)

 Conduct a regional expert panel-typ

priority

-setting process setting process

 Prioritizes problems & potential solutio (Rubenstein 1995) (Rubenstein 1995)
Rationales for Picking Problems

 The one on which all others depend The most frequently experienced The largest impact The largest impact

 E.g., on health, economics, QOL

 The easiest to address The easiest to address Synergy with other local efforts 
Champions Identify QI Team(s)

 Clinicians/researchers who will carry Clinicians/researchers who will carry

out QI projects

 Should be interdisciplinary (e g MD R

Should be interdisciplinary (e.g., MD, R

admin & include any other key disciplinfo t e p oject/p ob e ) r 

t

h

e

p

roject/p

roble

m

)

 Local clinic, medical center, and VIS

partners
Champions Identify Relevant 

Evidence & Tools

 Relevant

 Clinical guidelines Local data Local data Behavioral theory/research Evaluation methods & measures Evaluation methods

& measures

 Documented policies, procedures, 

manuals

Begin PDSA

...

 Rapid cycle tests of change Rapid cycle tests of change
PDSA Cycles

Plan Act

Plan the•Improvement

D t ll ti

•H

•Data collection Plan Act

D Do the Study o Do the •Improvement •Data collection 

•Analysis 
Start Small

N = 10

N = 20 N = 10

N = 1 N = 10

Implementation

 Intervention is in place including Intervention is in place including

policies, procedures, tools, etc.
Assess Outcomes

 RE

-AIM

 Reach Effectiveness/Impact/Success Adoption Implementation (Maintenance)Glasgow, et al. Ann Behav Med 2004;27(1):3-12

Maintenance and Spread

 QI intervention is in steady state & QI intervention is in steady state

&

assessed as successful

 Regional testing of developed Regional testing of developed

approach

 Additi l d t ti d t ti Additional a

dap

tation an

d

testi

n

g

 Revision/completion of tools, policie

perf dh ormance measures, adherence 

measures
“Knowing is not 

enou

gh; we must apply.

Willin

g is not enou

gh; 

we must do.”

---Goethe