Funded by the VA HSR&D (Project# SDR 10-012)

Spotlight on Women Cyberseminar Series
Improving Care for Women Vetera

Moving from Observational to 

Interventional and Implementation Res

Susan Frayne MD, MPH

Linda R. Lipson MA

L i B ti MD MPH Lori Bastian MD, MPH

Ciaran Phibbs PhD

Elizabeth Yano PhD Elizabeth Yano PhD

VA HSR&D CyberSeminar

November 2, 2010
Why This Workshop?

z Rapid growth of observational studies 

women’ h lth ’s 

health

– These studies have documented concerns 

women veterans women veterans’ health and their health c

health and their health c

– Intervention/Implementation studies (I/I) ar

sparse
Literature on Women Veterans Accele

506030401020

0

3 Bean‐Mayber
Literature on Women Veterans Accele

5060 ...but only

3 (out of 191)

women

’

s health studies

3040

women s health studies

in 2004

‐2008 were

experimental

1020

0

4 Bean‐Mayber
Objectives of Workshop

z

Opp y ortunit

y to consider how YOU mi

gh

your own work from observational to I/I 

z Provide tools to help 

y g ou 

get started

z Caveats

– Workshop is a basic introduction– Additional resources available through

Women’s Health Research ConsortiumPractice-Based Research Network
Audience Poll – QUESTION 1

z Do you have prior experience with 

Intervention/Implementation resea

A: YesB: No
Audience Poll - QUESTION 2

z Do 

y pp ou have 

prior ex

perience 

Women’s Health research? 

A. YesB No . 
Audience Poll - QUESTION 3

z Do 

y p ou 

plan to conduct Wome

Health Intervention/Implementresearch in the future? research in the future?

A Yes . B. MaybeC N. 

N

o
Workshop Overview

z Charge to the Field (Linda Lipson MA)z Intervention Research (Lori Bastian Mz Im

plementation Research

(y 

Beck

y Yan

z Multi-Site Research (Ciaran Phibbs, Pz Women’s Health Practice-Based Res

Network (Susan Frayne MD, MPH)
Charge to the Field Charge to the Field

Linda R. Lipson Di t VA HSRDirec

tor, VA HSR

VA Central Offic
Intervention Research

Intervention Research

Getting Started

Lori Bastian MDCenter for Health ServiCenter for Health Servi

Durham VA;Duke University Medica
Outline

z Use a case-based presentation to desc

th t d d t f b the steps needed to move from observ

results to intervention research.

– Hypotheses based on observational data Hypotheses based on observational data– Systematic review of prior interventions– Develop a preliminary intervention and get 

Develop a preliminary intervention and get 

from your target audience

– Pilot the Intervention and make adjustmen– Test the intervention in a randomized cont

trial
Observational Studies

z Several observational studies have info

th fi ld b t t the fi

eld a

bou

t women ve

terans.

z For example, my research focuses on 

hi h t f ki hi

g

h ra

tes o

f smoking among women 

veterans and the need to develop gend

specific smoking cessation interventionspecific smoking cessation intervention

the VA. 
Observational Data

z As part of a recently completed HSR&D 

As part of

a recently completed HSR&D 

trial, we identified 471 chronically ill vetsmokers.

z Compared to men, women (N=40) were

– Reported higher depressive symptoms on – Experienced higher temptations to smoke i

social and stressful situations
Identify Research Question

z Based on known observational data, w

veterans may benefit from an interventveterans may benefit from an intervent

that addresses self-efficacy to abstain smokin

g y in certain ke

y situations.

z Preliminary Question:

Would women veterans benefit from an intervention that focuses on psychologsupport to resist temptations to smoke?
How to get started

z Intervention research is an intelligence 

g gp p atherin

g

process that identifies 

possib

strategy options to address the issue in question. 

z It is important to kno ho others ha e It is important to know how others ha

v

e

addressed similar issues, the strategy used, the sequence of strategies, key lessons from others lessons from others’ efforts and how th

efforts, and how th

be used to chose strategies for your situation. 
Systematic Review

z Focuses on taking an evidence-based 

approach to developing new interventioapproach to developing new interventio

using prior research to inform strategy selection.

z R hi i d t Researching previous an

d curren

t

interventions serves to help identify whworks, when and in what circumstance

z It is useful to start by searching for 

systematic reviews on your particular tinterest I find it useful to start with the interest. I find it useful to start with the

Cochrane Collection at www.cochrane
Cochrane Reviews

z Reviews are organized by topics.z Tobacco Addiction (62)

– Cessation (50)

z P h l i l t f b h i l h (9

Psyc

h

ological suppor

t

for 

b

e

havioral c

hange (9

– Self help (1)– Individual counseling (1)– Motivational interviewing (1)
Developing Intervention

z Based on results from Cochrane Revie

M ti ti l I t i i M

otivational In

terviewing i ff ti is effective

– MI is more effective if delivered by PCPs– One session is as effective as several ses

One session is as effective as several ses

– 20 minute MI session is more effective tha

shorter session
Input from your target audien

z Develop written and other intervention 

mat i l d ti tf terials an

d ge

t inpu

t

from women 

smokers.

– Focus groups Focus groups– One-on-one structured interviews
Pilot Study 

z If possible, apply for pilot fundingz Goals of pilot study:

– Test logistics of study– Feasibility of recruitment

z Small numbers needed (n=10-20) and 

can be done at one site can be done at one site
Applying for VA Funding

z Contact the PBRN Clinical Trials Divisi

help with: help with:

– Determining sample size estimates and 

approximate number of sites needed to merecruitment goals (Special populations of wveterans may need more sites).

– Identifying site coordinators in preparation 

Identifying site coordinators in preparation 

writing an HSR&D IIR or a Cooperative Stproposal.
Advancing Implementation Res

Moving Evidence into Practi

Elizabeth YanoVA HSR&D Center of ExcStudy of Healthcare ProvUCLA School of Public He
What is implementation researc

z Majority of research evidence 

about effective care does not

Researc

Evidenc

about effective care does notmake it into routine practice

– Estimates span 15-30 yrs

z Implementation research 

focuses on methods to promote uptake of evidence promote uptake of evidence

– No longer studying the original 

intervention (we know it works

)

N t d i i l t ti

Routin

–

Now s

t

u

d

ying an implemen

t

ation 

strategy as the “intervention” Care
Research-to-Practice “Pipelin

Translation to 

Basic 

Biomedical

Research

Clinical

Science &

Health 

human studies

Efficacy to effectiveness Adoption, implementation

spread, sustainability

Knowledge

ea

Services

Research

Implementation

Research

“T1” “T2”

• Interventions, practice guidelines• Document and diagnose gaps

D i /t t i l t ti t t i 

Adapted from: VA Quality Enhancement Research Initiative (QUERI) pipeline model and Sung NS, et al., Central challenges facing the national clinical research enterprise, JAMA 2003.

•

Design/test implementation strategies

• Organizational, provider, individual theorie
S h i i l t ti f So w

hy 

is 

imp

lemen

t

ation o

f

evidence into practice so hard

z Under what conditions was “evidence” 

– Patients selected to reduce complexity– Tested in favorable environments– F t d t ti t id d t t 

Fac

tore

d ou

t patien

t, provider an

d con

tex

t

variables (organizational and area)

– Researchers work to ensure 

protocol adhe

z How different are “routine care” setting

those in which evidence was derived?
E li R h Evo

l

v

ing 

Researc

h

-Cli i l Cli

n

ica

l

Partnerships

z From researcher control to control of o

– Need for tools to support their control and 

“ownership” in ways that support fidelity

– Need for processes/procedures for ongoin

Need for processes/procedures for ongoin

adaptation (but in view of evidence of wha

– Need for tools/materials to orient new team 

members, new leaders

z Need for ongoing consultation as evide

ch h ll anges, new c

h

allenges emerge
Example:

TIDES Depression Collaborati

Provider/patient educatio

Evidence base: >35 RCTs

Depression care manage

35 RCTs

EBQI* QI Informatics support

Performance feedback

“adaptation”

“priority setting” Leadership support - “practice tailoring”

*EBQI = evidence-based quality improvement
Context Matters

z Design for it

– 2 1 i t ti 2:1 in

terventionto-cont l it 3 t k (6 i t 

trol sites x 

3 ne

twor

ks (6 in

te

3 control sites total) 

Æ unit of analysis typically the 

– VA network leaders chose sites, we randomized w

net k (bl k d t k h t i ti ) twor

k (bloc

k

e

d on ne

twor

k c

harac

teristics

)

z Importance of input from sites

– Attitudes/beliefs/ex

periences (gp e.

g., 

perceived need for in

competing demands, staff open to innovation, PC-MH relations

– Resources (e.g., perceived time to use program and particip

implementation, organizational structure, staffing, prior QI expe

access to informatics support) access to informatics support)

z Variation in people/places 

Æ variable implem
Need Multiple Data Sources

Data Sources Types of MeasuresSemi-structured interviews (leaders structured interviews (leaders, Level of participation

managers, teams, providers)

Level of participation,

implementation, experien

Organizational surveys (VA net

w

ork, facility,

p

ractice, pa

rtn

e

rs)

Clinic structure, process c

et o , ac ty, p act ce, pa t e s) fac tato s, ba e s ilitato

rs, barri

e

r

s

Administrative data (includingelectronic medical record data)

Diagnoses, visits, prescricosts

Patient surveys Patient surveys Process and outcome cha

Process and outcome cha

(e.g., PHQ9), comorbiditie

Provider surveys Knowledge, attitudes, beh

Practice checklists Practice checklists Implementation of new ca

Implementation of new ca

components (including fid
G ti th E id Generating the 

E

vidence:

Depression

z Identify target problem

– Epidemiologic studies on prevalence, impa

depression and location in primary care

z Define best practices (guidelines) Define best practices (guidelines)

– Systematic reviews (and meta-analysis) sh

CBT and antide

p qy ressants e

quall

y effective

– Expert panel methods to develop guideline

Adapted from Rubenstein LV et al. Application of QUERI evaluation me
G ti th E id Generating the 

E

vidence:

Depression

z Assess care variations

– Q lit b d di l d 

Quality measures 

base

d on medical recor

d 

survey

– Worse care for minorities, mana

ged care 

identified in large studies

z Develop intervention models and evalu

effectiveness

– Provider behavior and QI theory for design– Randomized trials of collaborative care

Adapted from Rubenstein LV et al. Application of QUERI evaluation me
G ti th E id Generating the 

E

vidence: 

Depression Collaborative Car

z Identify successful model characteristic

d l i l t ti d l develop implemen

t

ation mo

d

els

– Qualitative research on models, organizatio

predictors of success and barriers predictors of success and barriers

– Quasi-experiments based on diffusion, pro

behavior change, and QI theory

– Literature synthesis and meta-analysis

Adapted from Rubenstein LV et al. Application of QUERI evaluation me
I l ti mp ementing th E id the Evidence: 

Depression Collaborative Car

z Identify successful implementation mod

implement as routine policies/procedurimplement as routine policies/procedur

– Quality improvement theories used to eng

organizations and their leaders organizations and their leaders

– Quality improvement type measurement– Qualitative research on organizations/mod– Policy analysis and theory to understand/f

policy uptake, incentive changes for sprea

z Elt t f Evaluate system performance measureAdapted from Rubenstein LV et al. Application of QUERI evaluation me
M lti u -Sit R h Site Research

Ciaran Phibbs PHealth Economics ResHealth Economics Res

VA Palo AltoStanford University
Multi-Site Research: Benefits

z Meeting recruitment targetsz Preventing burn-out in available pool of 

subjects/clinicians

z Enhancing generalizabilityz Building professional connections
Multi-Site Research: Barriers

z Identifying sites and site leadsz Addressing local logistics

– IRB/R&D– Engaging clinicians– Hires

z U d t di th l l ti t l t

U

n

ders

tanding the local patien

t populat

and clinical environment

z Coordinating efforts across sites Coordinating efforts across sites
Lessons from Cooperative Studies Prog

z Recruitment is hard

– Even with extensive experience most CSP 

Even with extensive experience, most CSP 

have recruitment problems

– Need to allow for this in planning– Have contingency plans for when, not if, rec

lags

z N t k/P ll h l N

e

twork/Program can really 

h

elp

– Provides expertise that is hard for any indiv

investigator to assemble investigator to assemble

– Help with planning study so that you plan 

appropriately and realistically
VA Women’s Health

P ti rac ce-B dR hN 

Base

d

Researc

h

N

e

Susan Frayne MCenter for Health Care VA Palo Alto;Stanford University

Funding: VA HSR&D S
W ’ H lth PBRN Women

’s 

Health PBRN: 

Overview of Design

z Network of partnered VA facilities Network of partnered VA facilitiesz Provides infrastructurez Targets

– researcher-initiated, funded research proje–

c ca lini

cian-ini

t ated qua ty p o e e t p oje

iated quality im

p

r

o

v

e

m

e

n

t

p

roje
B ildi th N t k B

uilding the 

N

e

twor

k: I iti l S

Initia

l S

Frayne

Sadler

Bean-Mayberry

B
Rationale for a VA Women’s Healt

z Facilitate conduct of multi-site women’s 

h lth h health researc

h

– Address complexities of multi-site researc– Overcome problem of small N at any one f

Overcome problem of small

N at any one f

z Promote the study of questions informe

clinical practice clinical practice

z Foster disseminationz Encourage sense of Encourage sense of community community among

women’s health researchers and clinic
Women’s Health PBRN: Women’s Health PBRN:

Bi-Directional Collaboration

Practice

‐based research question

CLINICIANS RESEAR

Dissemination of research finding
Infrastructure Components

z Facilitate conduct of studies in loca

– Build relationships w/ local facility leaders/– Id tif li i Identify cli

nic-specifi b i / l ti t 

ific 

barriers

/solutions 

to 

recruitment and minimize disruptions in cli

z Facilitate local research administrat

Facilitate local research administrat

– Determine site-specific research requirem

(R&D, IRB etc.) and contacts

– M i t i i iti d i ti t 

M

ain

t

ain generic position 

descriptions a

t s

expedite hiring
Infrastructure Components

z Solicit research using PBRN

– Outreach to investigators

z Women’s health researchersz Other researchers wanting to increase enrollme

Other researchers wanting to increase enrollme

women

– Provide information about known priority a– Detect emerging practice-based priority ar
Infrastructure Components Infrastructure Components

z Support the grant application proce

– Discuss evolving study aims 

z Alignment with priority areas Alignment with priority areasz Emphasis on intervention/implementation researchz Other studies welcome, e.g., qualitative methods, o

studies

– Gauge interest of site clinicians re particip– Connect researchers with resources (CSP

other collaborators) other collaborators)

– Provide information relevant to grant prop

preparation 

– P id i li d th d ti Provide specialize

d meth

o

ds expertise

– Review protocols to provide feedback
Infrastructure Components Infrastructure Components

z Promote success of funded PBRN st

– Upon notification of funding, assist PI with 

issues (e g IRB logistic issues related to issues (e.g., IRB, logistic issues related to 

settings)

z Note: PI will negotiate with participating site leads reg

services to be provided locally budgetary needs of thservices to be provided locally, budgetary needs of th

whether the site lead will be an investigator

– Monitor progress of ongoing studies (with u

responsibility resting with the project responsibility resting with the project s’ PI)

– Ongoing contact with sites

z local problem-solving and team-buildingz training in relevant research methods (to enhance sta

training in relevant research methods (to enhance sta

of measurement and quality of data collection across 
Infrastructure Components

z Promote dissemination

– Women’s Health Research Consortium– PBRN Site clinicians– W ’ H lth St t i H lth C G 

Women’s 

Health Stra

tegic 

Health

Care 

Gr

Women Veteran Program Managers

– Steerin

g Committee

Patricia Hayes PhD; David Atkins MD; Joseph Francis MD, G

PhD; Susan McCutcheon EdD, RN; Paula Schnurr PhD; G

McGlynn MEd; Margaret Mikelonis ANP; Michael Parchma

Salganicoff PhD; Amy Street PhD; Donna Washington MD Salganicoff PhD; Amy Street PhD; Donna Washington MD
Accepting the Charge to Increase Accepting the Charge to Increase

Intervention/Implementation Women’s Health Research

z What can YOU do?z How can the Consortium/PBRN help
Extra Slides
What is a PBRN? What is

a PBRN?

Definition (AHRQ):“...a group of ambulatory practices .... Typicall

PBRNs d th i d i i ht 

draw on the experience an

d

ins

ight 

practicing clinicians to identify and frame res

questions whose answers can improve ... p

.... By linking these questions with rigorous methods, the PBRN can produce research fthat are immediately relevant to the clinician that are immediately relevant to the clinician 

theory, more easily assimilated into everydapractice.”

Clinician-centric Uses Clinical Practices as Laboratories
Research-to-Practice Implementation: 

Trajectory Toward National Rollout

Depression

Collaborati e

20+ year evidence base (efficacy/effectivenes

Collaborati

v

e Single and m lti component inter entions

Care Model

TIDES WAVES COVES ReTIDES

Single and

m

ulti-component inter

ventions

Multiple settings

Process 

Evaluation

Outcomes 

Evaluation

Cost Assessment

Stakeholder Anal

ysis

Impact evaluation

y Cost-effectiveness

Formative Evaluation 

DESIGN PROGRESSION

National “bridge” interview

Process tools

Building Research-Clinical Partnerships