www.Research-Practice.org

Applying and Building the Science of 

Dissemination and Implementation: 

Kaiser Permanente’s Center for Health 

Dissemination and Implementation 

Research

Russ Glasgow & Jim Dearing

Institute for Health Research

Kaiser Permanente Colorado

VA/QUERI Cyber Seminar

September 1, 2009
www.Research-Practice.org

Vision

 to advance the field of dissemination and 

implementation research and to offer concrete 

assistance to narrow the gap between health 

research, practice and policy, especially as they 

apply to reducing health disparities

 We pursue this vision by partnering with 

organizations and on projects targeting disparities 

in health and health care among racial, ethnic, 

socioeconomic, health literacy, and geopolitical 

populations and offering tools and technical 

assistance at the planning, implementation, 

evaluation, and reporting stages for interventions. 
www.Research-Practice.org

Kaiser Permanente

 Nation’s largest private nonprofit, 

integrated healthcare system

 Serving 9 states and the District of 

Columbia

 As of 12/07, 8.7 million members

 32 medical centers, 421 medical offices

 159,700 employees, 13,000 physicians

 $37.8 billion operating revenues 
www.Research-Practice.org

Research at KP

 8 research centers nationwide

 45 years experience with EMRs

 In 2006, 2,800 open research projects

 In 2009, founding a national CER center

 Partnerships with extra-organizational networks 

(such as the HMO Research Network)

 Research about translational processes is partly 

vested in the KP Colorado Region’s Institute for 

Health Research, home of the Center for Health 

Dissemination and Implementation Research 

(CHDIR) 
www.Research-Practice.org

CHDIR description

 We are new & small

 Our target audiences are both inside & outside 

the Kaiser system

 We emphasize a practice-based type of research

 We pursue generalization of theories, 

frameworks, tools, and approaches

 Our expertise is in external validity and diffusion 

research

 Our key challenge is aligning with funding 

streams and applied settings that are both 

typically organized in disease or condition-specific 

silos 
www.Research-Practice.org

www.research-practice.org
www.Research-Practice.org

Our focus

 The Center, while focused at multiple levels, 

including regional, national and international, also 

has a research and technical assistance role with 

the national Kaiser Permanente system. This role 

helps to improve the quality, equity and efficiency 

of healthcare delivery through the provision and 

study of:

 Formative evaluation, including social network 

analysis and health literacy issues;

 Research-based translation strategies;

 Tools to enhance implementation and dissemination 

success; and

 Organizational processes.
www.Research-Practice.org

Research tracks

 One research track concerns tests of the purposive 

spread of evidence-based practices among diverse 

settings and populations. Another track concerns tests 

of the factors responsible for the generalizability and 

replication of program effectiveness. In both cases, a 

third CHDIR research emphasis of economic and costeffectiveness analyses is being integrated with other 

measures, such as:

 Rate and feasibility of adoption;

 Program reach, especially among hard to reach 

populations;

 Organizational adaptation;

 Theoretic fidelity (implementation of theory-based 

principles); and

 Program re-invention or modification, and maintenance.
www.Research-Practice.org

Example 2009 achievements

 Continued application of the RE-AIM model across a 

number of nonprofit, healthcare system, and universitybased organizations and projects. Noteworthy among 

these activities in 2009 were

 a subcontract from Project LAUNCH, a national multi-site 

study to enhance child and family health; and 

 a subcontract with the University of Pittsburgh to evaluate a 

diabetes quality improvement program throughout the state 

that is being considered as a national model program.

 Social network data-collection, analysis, and technical 

assistance to the KP Colorado Senior Leadership Group in 

the establishment of a Regionwide Sociometric Database. 

Use of this database will support future improvement 

efforts.

 A new dissemination channel;

 CHDIR as process intermediary.
www.Research-Practice.org

Objectives for 2010

 1. To establish relationships with selected, key 

organizational partners interested in or conducting 

research with underserved populations.

 2. To investigate and publish results on the reach and 

impact of programs on different population subgroups 

including especially Latino and low health literacy 

groups.

 3. To secure core funding for Center infrastructure (or 

switch to categorical funding approach)

 4. To continue as a resource for social network 

analyses and for organizational performance 

improvement more generally.
www.Research-Practice.org

Any questions at this time about...

 KP or KP Colorado?

 our new dissemination & 

implementation research center 

(the CHDIR)?

 ...or observations on similarities or 

differences with the VA or other 

systems?
www.Research-Practice.org

What do we do in terms of 

diffusion and dissemination?

Dissemination is a purposive attempt to spread 

evidence-based innovations (practices, programs, 

policies, processes, technologies) through a 

population of potential adopters.

Dissemination is applied diffusion.

In the dissemination of EBPs, then, much of our 

approach is to “trip” existing levers in the social 

system in question so that dissemination occurs like 

diffusion.

More pull than push.
www.Research-Practice.org 13
www.Research-Practice.org

Dissemination is premised in

 Validated concepts from the diffusion of 

innovation paradigm

 The achievement of efficiency more so 

than efficacy or effectiveness

 Centralized coordination and 

decentralized wisdom

 A combined understanding of micro, meso 

and macro level factors that affect unit 

and system change

Beck A, Bergman DA, Rahm AK, Dearing JW & Glasgow RE (2009), 

Permanente Journal 13(3):10-17. 
www.Research-Practice.org

In terms of dissemination strategy, we 

conduct research and help 

stakeholders with... 

 Timing and framing of the 

introduction of innovations

 Attributes of innovations

 Understanding of social structure
www.Research-Practice.org

What’s the status quo in KP?1

 Bottom up

 Intra-region via grassroots innovation, local 

demonstrations, word-of-mouth, committees

 Inter-region via formal recognition such as Vohs, 

Lawrence, and via conferences and committees

 Top down

 Inter-region via CMI, CEC, KPAN, DSLs, nationallyfunded pilots and demonstrations

 Authority adoption decisions by leaders, chiefs

1Wallace P. IOM presentation, May 25, 2007
www.Research-Practice.org

Dissemination Factor 1: 

Timing & framing

 Carrying capacity and the 

absorption of innovations

 Battling against context is a losing 

fight

 Encouraging stakeholders to time

the introductions of innovations

 Encouraging stakeholders to frame

innovations so that adoption 

promises the achievement of other, 

complementary goals 
www.Research-Practice.org

Dissemination Factor 2: Innovation 

attributes as perceived pros & cons

 Key attributes

 Cost ****

 Simplicity  

 Compatibility ** **

 Evidence **

 Trialability *

 Observability *
www.Research-Practice.org

Dissemination Factor 3: Identification 

and use of advice networks

 When conceptualized for dissemination 

intervention, potential adopters comprise 

a social system

 They know one another

 They can comprise a patterned network 

structure

 They attend to mediated information sources

 They attend to interpersonal influence 

sources
www.Research-Practice.org

Advice Network among Teachers in a Public Middle School

“At your school, whose advice do you most value for new ideas or better ways

of doing things in the curriculum related to health education?”

School nurse: 

centrality = 0.596;

28 links

Principal: centrality = 

0.085; 4 links

Response rate 90%; N=54 

individuals; thicker lines mean 

more frequent general 

communication
www.Research-Practice.org

Advice Network among Healthcare Providers in a Pediatric Clinic

“Whose advice do you most value for new ideas or better ways of 

doing things concerning pediatric care?”
www.Research-Practice.org

Any questions at this time about...

 diffusion theory concepts that we 

test and apply?

 projects outside or inside KP?

 QI efforts using some of these same 

concepts?

 ...or?
www.Research-Practice.org

What do we do in terms of 

implementation research and helping 

stakeholders?

 Making research relevant

 Help make KP a leading rapid 

learning healthcare organization 

(ala Etheridge)

 Provide leadership and coordination 

around practical research

Etheridge LM. Health Affairs, Web Exclusive Collection, 2007;w107-w118.
www.Research-Practice.org

Definition

 IMPLEMENTATION = Use of 

strategies to adopt and integrate 

evidence-based interventions and 

changes in practice patterns within 

specific settings (NIH).

(Success in Particular Settings)
www.Research-Practice.org

What can the CHDIR do for KP?

 Conceptual models and frameworks

 Practical trials expertise

 Tools

 Measures

 TBD (You tell us)
www.Research-Practice.org

As Tested

Critical

Elements

Program or Policy

Simplified Systems Model for Translational Research

Organization

Clinic

Program 

Delivery 

Staff

Delivery Site(s)

Fit

Research

Design

Appropriate

For Question

Partnership Broader Health

Policy and

Cultural

Context

Estabrooks PA, Glasgow RE. Am J Prev Med 2006;31(4S):S45-S56
www.Research-Practice.org

RE-AIM framework for focusing on 

key issues in integrating research 

into practice
www.Research-Practice.org

History/Background for RE-AIM

Late 1990s: Increasingly clear that 

major problems moving research into 

practice.

Helpful models for understanding (e.g., 

Rogers) and planning (e.g., Green & 

Kreuter) health care programs but no 

systematic models for translation.

Almost total focus on efficacy. Assumed 

that linear “automatic” process of 

efficacy  effectiveness 

dissemination.
www.Research-Practice.org

RE-AIM can help in planning, 

evaluating, and reporting studies

R Increase Reach

E Increase Effectiveness

A Increase Adoption

I Increase Implementation

M Increase Maintenance

Glasgow, et al. Ann Behav Med 2004;27(1):3-12
www.Research-Practice.org

Purposes of RE-AIM

 To broaden the criteria used to evaluate programs to 

include external validity and context.

 To evaluate setting issues relevant to program adoption, 

implementation, and sustainability.

 To help close the gap between research studies and 

practice by:

 Informing design of interventions

 Providing guides for adoptees

 Suggesting standard reporting criteria to 

increase transparent reporting
www.Research-Practice.org

Building Programs

and Policies 

with a Large Public 

Health Impact

Adoption

RE-AIM BUILDING BLOCKS THAT TOGETHER 

PRODUCE PUBLIC HEALTH IMPACT

Efficacy

Effectiveness

Reach

Implementation

Maintenance
www.Research-Practice.org

0

20

40

60

80

100

Reach Effectiveness Adoption Implementation Maintenance

SCORES: 36 70 50 40 30

RE-AIM PROFILE
www.Research-Practice.org

Proposed translational research 

measurement package

1) Contextual factors

2) Implementation

3) Generalization (Settings, Staff, Patients)

4) Behavior change (multiple levels?)

5) Economic outcomes

6) Quality of life

Glasgow, et al. Diabetes Care 2003;26(8):2451-2456
www.Research-Practice.org

Behavior change measures

 Brief, practical measures.*

 Often triangulate when no “gold 

standard”.

 Focus on sensitivity to change.

 Measures of patient, staff, change 

agents (e.g., family), system and 

policy changes.

* Glasgow, et al. Ann Fam Med 2005;3:73-81
www.Research-Practice.org

Economic outcomes 

using standardized methods

 Assess cost of intervention delivered.*

 Estimate replication costs.**

 Optional, more sophisticated analyses of 

cost-effectiveness, cost-utility, costbenefit, return on investment.

 “Costs are not costs are not costs”.

* Gold, et al. Cost-effectiveness in health and medicine. New York: Oxford Univ. Press, 2003

** Meenan, et al. Med Care 1998;36:670-678

Ritzwoller D et al. Annals Behav Med 2009 37: 218-227
www.Research-Practice.org

Any questions at this time about...

 RE-AIM?

 the ways that RE-AIM components 

are most used?

 actual or potential practice-based 

research applications of the RE-AIM 

framework?

 ...or? 
www.Research-Practice.org 37

www.research-practice.org

russg@re-aim.net

james.w.dearing@kp.org