Scale-up and S pre

1928 

Quality David C. Aron, MD, MS Q y Louis DVAMC Stokes Cleveland Formerly 

Enhancement 

Research 

Initiative 

Louis DVAMC Stokes Cleveland 

10701 East Blvd., Cleveland, OH 

Case Western Reserve University School of 

Medicine, Cleveland, OH 

David aron@va gov 

y 

David.aron@va.gov 
The following presentation is ra

(Opining and sarcasm may

“inadvertently” seep in.) Viewer

discretion is

a

d i v se

d. However, f 

free to challenge everything I sa
Definition

" Scaling up: Deliberate efforts to incre

impact of health service innovations suc

tested in pilot or experimental projects 

benefit more people ".

Source: WHO, Reproductive Health Research & ExpandNe

projects to policies & programmes: Practical guidance for 

health service innovations (WHO, 2006). 
The “what” of what is scale

• A “practice”, such as a way of carrying out a work t

example a health worker using a checklist to ensur

done all the parts of the task,

• A combination of practices, such as a number of in

for expectant mothers

• A way of organizing a service, such as creating a clo

coordinated team of practitioners,

• Other types of intervention, such as a new way of 

providers.

John Ovretveit. Scale Up and Spread – The International Health Experience, Conto Advance the State of the Science and Practice on Scale-up and Spread of Effective 

Programs, Washington, DC, July 6-8. , 2010).
The “how” of scaling up (mo

Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion o

innovations in service organizations: systematic review and 

recommendations. The Milbank Quarterly 2004; 82(4):581-629. 
The “direction direction ” of scaling of scaling 

Vertical 

Diff l l 

H 

Different levels 

of the 

organization 

H Horizontal 

# units at a given le

i th i ti 

W 

in the organizatio

D 

Depth 

The scope of the 

services/ practice 
Industrial (Mechanic

up‐Make it Hap

Early penicillin culture 

facility at the Sir William 

Dunn School of 

Pathology, Oxford, Pathology, Oxford, 

England. 

Museum of the History of 

Science, Oxford

Upper part of fermentors (tanks) used to 

produce penicillin and vitamin B12. 

Merck Archives 

Fermentati

used in pu

penicillin i

Merck Ar
Diffusion of Innovation Let it Ha

Usi h f d i f h b id b f i h l 

ing the rate of adoption of hybrid corn by farmers in the early 1930s, 

were to derive some very important insights. These two researchers interv

in Iowa about their use of hybrid corn, when the farmers first heard abou

st t d it tarted us

i

ing it. 

Diffusion of Innovation Diffusion of Innovation 

Everett Rogers

Plsek Original

ThinkingParadigm‐

busting

thinking

Creativeconnectionthinking

Cleverthinking

Potentialbetterpracticethinking

Usual thi

Fraser Enthusiast Visionary Pragmatist Conservative Skeptic 
Structural 

context 

Nature of the 

intervention 

CO

sc

ta

The 

The 

Joe McCannon 

11 Centers for Medicare and Medicaid Services 
B Druce. Aig ns, MD MPH D MD MPH, Di t rec or, 

HEALTHQUAL International 

•Massoud MR, Donohue KL, and McCannon CJ. 2010. Options for Large‐scale Spread of Simple, Highimpact Interventions. Technical Report. Publishe
•Model for Evaluating Scaling Up a

•Complex Adaptive System

Da

JohJoh
Conceptualizing Conceptualizing Scale‐Up & Sp

Adding multiBuilding the level context 

Evidence Base 

Int’n

Int’n Int’n

Elizabeth M. Yano, PhD, MSPH, VA GLA HSR&D Center of Excellence, UCLA Scho
Building the 

Adding multilevel context 

Evidence Base 

Int’n

Int’n Int’n

Int’n Int’n

Int’n Int’n

Int’n Int’n

Int’n Int’n

• What ties connect each new target for spread? 

• Stakeholder communication  who, what levels, how? 

• Can we map to different contexts or treat each uniquely? Can we map to different contexts or treat each uniquely? 

Elizabeth M. Yano, PhD, MSPH, VA GLA HSR&D Center of Excellence, UCLA Scho
Building the 

Adding multilevel context 

Evidence Base 

Int’n

Int’n Int’n

Int’n Int’n

Int’n Int’n

Int’n Int’n

Int’n Int’n

• S i il hil h ld Some contexts are similar, while others couldn’ t b diff e more differ

• Not all partnerships “created equal” (competition, resources

• Requires teams  single “champion” unlikely to be right fit fo

• Bi- to multi-directional opportunities  active commun

i

lti di iti icatio

Elizabeth M. Yano, PhD, MSPH, VA GLA HSR&D Center of Excellence, UCLA Scho
All of these issues and more w

observed/dealt with in the

WAVES/TIDES/COVES/RIPPLE/

TIDES series of projects to sca

collaborative care for depress
What is TIDES?

(Translating Initiatives in Depression into Effective S

• 1990’s: Researchers tested depression c

improvement models

– Collaborative care with care management n

and sufficient (over 36 high quality random

• 2000: TIDES = Can VA implement collab

care as part of routine care?

– VISN leadership were decision‐makers

• 2006: TIDES part of national VA rollout

Lisa Rubenstein QUERI MEETING 2008. 

• NOTE THAT THE PROCESS IS STILL GOING 
Lessons from WAVES: Trials May Not R

Real World Implementation

• Real world interventions are implemente

some type of quality quality improvement improvement metho

– Interventions only sustain if integrated into o

real world activities

– Difficult to use trials to study QI implementat

distortion

• Trial can’t capture some strong determin

world program functioning

– E.g., primary care provider preferences and e

determine determine use of the treatment treatment model

Lisa Rubenstein QUERI 

MEETING 2008. 
Rycroft‐Malone et al. A pragmatic cluster rand

evaluation evaluation three implementation implementation interven

interven

Implementation Science. 2012;7:80

Rycroft-Malone et al. Implementation Science 2012 7:80 
Table 4

Intervention group across pre‐and post‐intervention timepoinIntervention Pre‐intervention Post‐interve

Food Food Fluid ANOVA Flui

ANOVA ANOVA

Standard

p = 0.981 p = 0.951 p = 0.872 p 

dissemination

SD + web‐

resource/opinio p = 0.410 p = 0.716 p = 0.536 p 

n leaderSD + PDSA p = 0.958 p = 0.981 p = 0.748 p 
Rycroft-Malone et al. A pragmatic clus

three implementation interventions. Im
Rycroft-Malone et al. A pragmatic cluster 

randomised trial evaluation three 

im

plementation interventions. 

Implementation Science. 2012;7:80 

Rycroft-Malone et al. Implementation Science 2012 

7:80 
Conclusions 

This was a large, complex study and one first national randomised controlled trials conducted within acute care in implemenresearch. The evidence base for fasting prwas accepted by those participating in thiand the messages from it simple; howeveimplementation and practical challenges influenced the interventions’ impact. A seconditions for implementation emerges frfindings of this study, which are presentetheoretically transferable propositions thainternational relevance. (my underlining

Rycroft-Malone et al. Implementation Science 2012 7:80 
My interpretation:

Although it would have been interesting 

true control group, no strategy works bet

anything else... INDEPENDENT OF CONTE
There are a set of conditions and antecedents for implem

emerge from the findings of this study, which we have dev

number of propositions:

• 1. Implementation Implementation is more likely to be successful successful in case

topic/issue is a strategic and organisational priority...

• 2. A historical lack of clear leadership, structure, and pro

guideline dissemination and implementation, in which sabout their responsibilities, will negatively impact on an 

ability to routinely use guideline recommendations.

• 3. Robust and believable evidence is not always sufficie

decision making and practice, therefore implementation 

and eff t or s need to ext d en beyond i di n vidual d i ec sion m

for certain clinical topics) and take account of the system

connections between individuals, teams and organisatio

Rycroft-Malone et al. Implementation Science 2012 7:80 
• 4. In areas where there is more effective teamwork with 

communication, practice change will be easier to achiev

• 5. New improvement and implementation projects have 

chance of success success if they are embedded embedded into existing existing pr

pr

and structures.

• 6. Change agent effectiveness is a function of the prote

and dedicated dedicated time to fulfil the role, i e. ., it has to be pa

job.’

• 7. Change agents will be more effective if they have p 

eo 

management skills, work collaboratively, handle difficul

and people with diplomacy, understand where people ‘

from,’ develop and motivate a team, including effective considerate d l e egation of work, using team members sk

Rycroft-Malone et al. Implementation Science 2012 7:80 
• 8. E l va uations of implementation i t n erventions that 

different types of impacts over the course of the

study/programme programme are more likely to provide provide a reali

reali

knowledge use, and intended and unintended cons

• 9. Complex Complex interventions interventions ...need to be deconstructe

deconstructe

greater understanding of the linkages between the components/mechanisms of action and the impact 

process and summative outcomes.

• It is likely that these p p ro ositions will be theoretica

to other implementation studies, particularly when alongside the growing empirical and theoretical eviabout the successful ingredients for successful imp

Rycroft-Malone et al. Implementation Science 2012 7:80 
Time for researchers to get re

RCT Realistic Evaluation Realistic Evaluation Intervention is independent from other external c

hanges 

External changes in or outare part of interventions, a

reported h i h h 

d toget

her wit

h t

he 

Isolating of confounding 

factors enables the factors enables the 

researcher to infer a direct link between intervention 

and outcome and outcome 

In real world interventionsconfounding factors is not confounding factors is not 

Context and mechanisms factors which initiate or tri

causal relationship Hence causal relationship. Hence

cannot be seen isolated frand mechanisms 

Internal validity ensures the ability to generalize. Differences in contextual Differences in contextual 

factors are eliminated given large samples 

Generalization is based on of relevant context and me

Pedersena LM, Nielsena KJ, Kinesb P. Realistic evaluation as a new wa

evaluate occupational safety interventions. Safety Science 50 (201
Take home points/

Issues to think about

• There is no easy solution. If there were, 

would have gotten around by now. (par

J. Derrida)

• Is scaling‐up a “wicked problem” (Rittel

If so, there is no Solution, only solution

• Is every instance of scaling‐up unique?

• Are our “researcher researcher” mental models no

wrong (as all models are to one degree another), b t u dangerously wrong?
Supplementary Materia
What we don’t know

• Direct and indirect costs of scale up.

• Requirements for initial versus sustained sc

• Comparative effectiveness of approaches th

to different types of innovations and contex

• Intersection Intersection of scaling scaling‐up and scaling scaling‐down

processes.

• Unintended consequences

Adapted Adapted from: Scaling Scaling‐up Health Innovations Innovations and Interventions Interventions in Public Health: Health: A Brief Review of the 

Science Nancy Edwards, RN, PhD, School of Nursing and Department of Epidemiology and Communi

Ottawa and Institute of Population and Public Health, Canadian Institutes of Health Research. O
Recommendations for Resea

•

A typology of innovations to facilitate comparisons

•

A framework for efficiently assessing context.

• Include equity outcomes.

• Temporal dimensions of scaling

‐up require further 

• Systems integration – how can/do innovations sca

become

just the wa

y we do business

• De

‐scaling

• Systems approaches
Definitions of scaling (adapted from: IIRR (International Institute for Rura

2000. Going to scale: Can we bring more benefits to more people more 

W k or

k

s

hop, Silang, PHPH . 114 p.

H. Menter, S.Kaaria, N.Johnson & J. Ashby 

http://ciat-library.ciat.cgiar.org/Articulos_Ciat/scaling_up_chapter_1.pdfHar
MODELS

S

ser

J

Capabi

•8 key themes which highlight the conditions 

8 key themes which highlight the conditions 

to successfully scale up innovation across 

sector:

Culture: build a culture that rewards and encourages scalin

E id v ence: mak th b i d d t t th 

ke the business case and demonstrate the 

Skills: embed skills needed for scaling up and understand tha

and to scale up are different 

Networks: develop and use networks to make connections, pro

knowledge and create dialogue 

Processes: embed processes and mechanisms that facili

Ownership: recognise that a feeling of ownership acts as an 

learning about what works learning about what works 

Resources: manage resources, funding, expertise and sup

encourage scaling up. 

Credibility: credibility, endorsement and reputation provide the 

scaling up 
‐

Additional References

Harris JR, Cheadle Cheadle A, Hannon PA, Forehand Forehand M, Lichiello Lichiello P, Mahoney Mahoney 

Yarrow J. A framework for disseminating evidence‐based health 

practices. Prev Chronic Dis 2012;9:110081.

Tabak RG, Khoong EC, Chambers Chambers DA, Brownson Brownson RC.• Bridging Research and Practice Models for Dissemination and Im

Research. Am J Prev Med 2012;43(3):337–350)

Jowett A, Dyer C. Scaling Scaling‐up successfully: successfully: Pathways Pathways to replication replication fo

NGOs. International Journal of Educational Development 32 (20

Schillinger, D. (2010). An Introduction to Effectiveness, Disseminatio

Implementation Implementation Research Research. P. Fleisher Fleisher and E. Goldstein Goldstein, eds. From 

UCSF Clinical and Translational Science Institute (CTSI) Resource 

Guides to Community‐Engaged Research, P. Fleisher, ed. Publish

Translational Science Institute Community Eng ga ement Program

California San Francisco. http://ctsi.ucsf.edu/files/CE/edi_introg
References

Mangham, L J. ., & Hanson, K. (2010). Health Economics Economics and financing financing program program, L

Hygiene and Tropical Medicine. Scaling Up in International Health: what a

McCannon, C.J. (2010) Conference to Advance the state of the Science and Prac

Spread of effective Programs. Framing a discussion on Scale Up and Sprea

Gilson, L., & R h ap aely, N. (2008).The terrain of h lth ea policy anal i ys s in low and 

countries: A review of published literature 1994‐2007. Health Policy & Pla

Greenhalgh, T., Robert, G., MacFarlane, F., Bate, P., & Kyriakidou, O. (2004). Diff

in service organizations: System review and recommendations. Milbank Q

629.

Reich, M. (1995). The politics of health sector reform in developing countries: t

pharmaceutical policy. Health Policy, 32: 47‐77.

Walt, G., Lush, L., & Ogden, J. (2004). International organizations transfer of infe

iterative loops of ad ti op on, ad t ti ap a on and mark ti e ng. Governance, 17 18

: 

Shiffman, J. (2007). Generating political priority for maternal mortality reductio

countries. American Journal of Public Health, 97: 796‐803.
Schneider, H., Gilson, L., Ogden, J., Lush. L.

& Walt, G. (2006). Health systems and 

of disease programs: case studies from South Africa. Global Public Health, 1

Smith J.,

& Colvin

C. (2000). Getting Getting to scale in young adult reproductive reproductive health pro

pro

Service series 3. Washington, DC: Focus on Young Adults.

Pedersen, L., Nielsen, K. J.,

& Kines P. (2012) Realistic evaluation as

a new way to d

occupational safety interventions. Safety Science 50 (2012) 48–54

Senderowitz Senderowitz, J. (2000)

A Review of program program approaches approaches to Adolescent Adolescent Reproductiv

Reproductiv

Assignment Number 2000.176. Population Technical Assistance Project. Arlin

Ovretveit, J. (2010). Scale Up and Spread – The International Health Experience, Co

the State of the Science and Practice on Scale‐up and Spread of Effective He

Washington Washington, DC, July

6‐8

Simmons, R., Fajans, P.,

& Ghiron, L. (2006). Scaling Up Health Service Delivery: Fro

to Policies and Programs. Geneva: Expandnet

& World Health Organization (

World Health Organization (WHO, 2006).Reproductive Health Research

& ExpandN

projects projects to policies policies

& programmes: programmes: Practical Practical guidance guidance for scaling scaling up health s

s
• Wolff N. Using Randomized Controlled Trials to Evaluate Socia

Services: Problems, Challenges and Recommendations. J Men

Econ.

3, 97–109 (2000)