Shared Medical Appointments for 

Chronic Medical Conditions: 

A (Traditional) Systematic Review

David Edelman MD, Jennifer R. McDuffie PhD, Eugene Oddone MD,

Jennifer Gierisch PhD, John W. Williams Jr. MD.
Shared Medical Appointments (SMAs)

• Subset of Group Visits

• Groups of patients with a defining chronic 

condition or other health care state meet over 

time for comprehensive care

• Involve both self-management training and 

medication management
SMAs-- Structure 

• 60-120 minutes, usually 1-3 months apart

• Usually both a prescribing provider and a 

trained educator/facilitator

• Interactive education, often with techniques 

such as motivational interviewing; goal is 

patient activation

• Prescribing provider does med changes, often 

in one-on-one breakouts
Studies of SMAs

• Some studies in frail elderly

• Most in a single disease, most commonly 

diabetes

• Wide variability in:

– Setting and patients

– Intervention approaches, including staffing

– Chosen outcomes to measure (ie, behavioral, 

clinical, cost/utiliization) 
Objectives

• Summarize the effects of SMA on:

– Patient outcomes

– Staff outcomes

– Economic outcomes

• Evaluate whether these effects vary by clinical 

condition or specific intervention 

components.
Outline of Methods

• Topic development

– Key questions

– Protocol

• Systematic searches of the literature

• Study selection via eligibility criteria

– Screening 

– Full text review

• Data abstraction and quality assessment

• Data synthesis and report generation

• Peer review
Key Question 1

• For adults with chronic medical conditions, do 

shared medical appointments (SMAs), compared 

with usual care, improve the following:

– Patient and staff experience?

– Treatment adherence?

– Quality process measures?

– Biophysical markers (laboratory or physiological 

markers of health status such as HbA1c and blood 

pressure)? 

– Symptom severity and functional status?

– Utilization of medical resources or health care costs?
Key Questions 2-3 

• Key Question 2. For adults with chronic 

medical conditions, do the effects of SMAs 

vary by patient characteristics such as specific 

chronic medical conditions and severity of 

disease?

• Key Question 3. Is the intensity of the 

intervention or the components used by SMAs 

associated with intervention effects?
Protocol

• The protocol provides the analytic framework 

for the report and outlines a priori how each 

remaining step of the procedure is to be 

conducted

• The protocol is reviewed and vetted by both 

internal and external experts as well as the 

major stakeholders for the report
Literature Search Strategy

• Databases

– MEDLINE® (via PubMed®), Embase®, CINAHL®, 

PsychINFO® and Web of Science® 

• Search terms

– Consult master librarian

– Key words and MeSH Analyzer

• Supplemental searches

– Bibliographies of exemplary articles

– ClinicalTrials.gov
Modifiers

Patient 

Social support

Health care 

system

Adults with

- Asthma

- CAD 

- CHF

- COPD

- Diabetes

- High lipids

- HTN

SMA model

Group size

Components 

Team composition

Rxing professional

Visit frequency

Usual care 

Traditional office 

visit

Other systems 

improvements Adverse 

effects

Intermediate 

outcomes

Adherence

Satisfaction

QI Measures

(A1c, BP, LDL)

Final outcomes

Symptoms 

Functional status

HR QOL

Health utilization
Study Selection Criteria

• Exclusion

– Publication is NOT English language or not peerreviewed

– Population selected for substance abuse or is from 

inpatient setting

• Inclusion:

– Based on model in previous slide
Inclusion (1)– Study Quality

• Quality of study

– Study designs recommended by Cochrane EPOC 

Group

– Trials, or observational studies with 

contemporaneous comparator
Inclusion (2)– Patients

• Adult with one or more of 7 chronic medical 

conditions of a priori interest

– Asthma

– CAD

– CHF

– COPD

– Diabetes

– High lipids

– HTN

• Also reviewed– extant literature on older adults 

without a single unifying disease
Inclusion (3)– 

Intervention and its context

• Setting

– Outpatient primary care or specialty 

clinic/practice

• SMA Model

– Intervention defined as ≥2 medical visits where ≥1 

healthcare professional (includes prescribing 

clinician) cares for a patient group

• Comparator

– Defined as usual care or other quality 

improvement strategy
Inclusion (4)– Outcomes 

• One of following outcomes reported at ≥ 3 

months:

– Patient or staff experience 

– Adherence (treatment, medication or selfmanagement) 

– Biophysical marker, (e.g., HbA1c, LDL, BP)

– Symptom severity or functional status

– Utilization of medical resources
Data Abstraction 

• Extraction of pertinent information from each 

eligible article into a customized, uniform 

database in DistillerSR® 

• Performed by 1st reviewer and independently 

over-read by a 2nd reviewer

• Disagreements are resolved by discussion and 

consensus or referral to a 3rd reviewer
Robustness Score

• Devised to attempt to describe the more potent elements 

of an SMA intervention

• Seven variables

– Education session

• Qualifications of leader 

• Based on theoretical framework?

– Group composition

• Closed membership? 

• Stable healthcare team?

– Intervention Process

• Individual breakout sessions?

• Medication changes within visit?

• Number and length of visits 

• Potential score range = 0-9 
Quality Assessment

• Elements rated for RCTs

– Adequacy of randomization

– Adequacy of allocation concealment

– Comparability of groups at baseline

– Blinding of subjects and/or investigators

– Completeness of and differential loss to followup

– Management of incomplete data

– Validity of outcome measures

– Potential conflicts of interest

• Elements rated for observational studies

– Selection bias

– Performance bias

– Detection bias

– Reporting bias

• Reference: Agency for Healthcare Research and Quality’s (AHRQ) Methods 

Guide for Effectiveness and Comparatie Effectiveness Reviews
Data Synthesis

• Summary table of key outcomes

• Quantitative meta-analysis, if feasible

– Dichotomous outcomes combined using RR; OR

– Continuous outcomes combined using standardized 

mean difference and a random effects model

– Tests for statistical heterogeneity (Q and i2) 

• Qualitative synthesis otherwise (e.g., too few 

studies or subgroup and sensitivity analyses)

• Assessment of publication bias
Strength of Evidence

• Assessment of four domains

– Risk of bias

– Consistency

– Directness 

– Precision

• The strength of the evidence for the proposed answer 

to each key question is graded – high, moderate, low or 

insufficient

• Reference: AHRQ’s Methods Guide for Effectiveness 

and Comparative Effectiveness Reviews
Literature Flow

Excluded = 1009 references 

Excluded at screening level 

Search results 

= 1104 references

Excluded = 71 references 

Not peer-reviewed, or primary data = 34

Not study population of interest = 6

Not eligible study design = 17

Comparator not of interest = 1

Intervention doesn’t meet definition = 7

No outcome of interest = 6

Retrieved for full-text review 

= 95 references

Included =18 unique studies 

and 6 companion articles

KQ 1: 19 unique 

studies + 1 companion 

KQ 2: 16 unique 

studies + 4 companions 

KQ 3: 13 unique 

studies + 4 companions 
Study and subject characteristics

Study Characteristic Adults With Diabetes Older Adults

N studies (participants) 16 (3221) 3 (1851)

Mean age of sample: median 

(range)

60.8 (27 to 69.8) 74.1 (73.5 to 78.2)

Randomized controlled trial 13 (2921) 2 (615)

Observational 3 (300) 1 (1236)

Study quality: N (%)

 Good

 Fair

 Poor

6 (46%)

6 (46%)

1 (8%)

0

2 (67%)

1 (33%)

Single-site 14 (2106) 1 (321)

Multisite 2 (1115) 2 (1530)

Duration 6 to 12 months 4 (410) 0

Duration >12 months 12 (2811) 3 (1851)
Effect on A1c: mean = - 0.55%
Effect on SBP: mean = - 5.2 mmHg
Effect on LDL-C: mean = - 6.6 mg%
Other outcomes—patient-level

• Patient experience?

– No effect– but only 2 studies measured 

• Staff experience?

– Completely unmeasured

• Treatment Adherence?

– No effect– but only 3 studies measured, and no single 

behavior was assessed in more than 2 studies

• HRQoL? 

– 3 studies used a disease-specific measure, found a 

positive effect, not highest quality trials

– 2 used a general measure, no effect
Other Outcomes– 

costs and utilization

• Utilization?

– 5 studies, 4 of 5 with reduced admissions

– Same 5 studies, variable results on ER visits

• Costs

– 4 studies, mixed results on overall costs
Older Adults?

• 3 studies, 2 trials, one observational

• Lower study quality than diabetes studies

• All studies measured patient satisfaction but with 

different, non-validated measures

– All showed satisfaction improvement

• No change in global health or function

• Both trials showed lower ER use and admissions, 

ER statistically significant in both

• Costs lower, but not significantly so, in both trials
Key Questions 2 & 3

• No study reported specific patient 

characteristics that led to better response to 

SMAs (Key Question 2)

– We evaluated whether baseline A1c was 

associated with response; it was not

• No study reports specific intervention 

components, or intensity, associated with 

effects of SMAs (Key Question 3)

– We evaluated whether robustness was associated 

with effect size; it was not
Evidence Synthesis found no data to 

assess:

• Cost-effectiveness

• Non-patient benefits, such as improved access 

or staff satisfaction

• Key elements to successful implementation, 

especially outside academic or vertically 

integrated systems
Lessons learned

• Precise, highly scientifically valid estimates of 

SMA efficacy

– SMAs are pretty efficacious; effects on A1c and SBP 

close to those seen in drug trials

– Effect sizes 0.5, 0.33, and 0.25 for SBP, A1c, and LDL-c 

respectively 

– Taken together, a marked improvement in risk of 

complication; would still be important risk reduction 

if half the efficacy were lost in translation
Are these findings generalizable?

• Use PICOTS framework

• Population— likely generalizable, populations were well 

demographically balanced

• Intervention—components VERY heterogeneous, remains 

quite possible that not all group strategies are effective.

• Comparator– Also heterogeneous, “usual care” poorly 

described in most studies

• Outcome—likely generalizable, biophysical outcomes are 

the generally agreed upon set

• Timing—likely generalizable, general agreement that 6 

months improvement is important

– But no studies of maintenance of improvement

• Setting—All studies in highly academic settings, not any 

“real-world” studies.
Where do we go from here?

• Other chronic illnesses

• Study designs that allow evaluation of particular 

components

• Multi-site implementation studies with good 

measurement of patient and staff impacts

– Strongly consider mixed-methods studies

– Carefully measure unintended consequences on the 

system

• Cost and cost-effectiveness analyses
Theory Driven, Context Dependent Studies 

of Shared Medical Appointments: 

A Realist Work in Progress

 

Susan Kirsh, MD, MPH and David Aron, MD, MS

with Kim Johnson, RN PhD, Katherine Jones, PhD, 

Brian Mittman, PhD, John Øvretveit, PhD, Laura Santurri, PhD, 

MPH, CPH, Lauren Stevenson, PhD

Louis Stokes Cleveland DVAMC and Case Western Reserve University

Cleveland, OH 44106

The views are those of the presenters and do not reflect those of VHA or any 

other agency/institution.
Objectives

• Discuss theory-driven context-dependent 

review (realist) and why needed

• Describe our experience

• Q&A 
Burke RE et al. The effectiveness of group medical visits on diabetes mellitus type 

2 (dm2) specific outcomes in adults: a systematic review. JBI Library of Systematic 

Reviews JBL000433 2011;9(23):833-885

• Main results: benefits 

HbA1c; some evidence for 

SBP; none for LDL

• Conclusions: should be 

considered by clinicians as an 

effective, non-pharmacologic 

intervention that can have a 

positive impact on biologic 

markers such as HbA1c and 

SBP.

• Implications for Practice: 

most powerful model includes 

clinician prescriber

• Implications for Research: 

RCTs needed

What managers want to 

know is what works when 

and for whom.

In other words, context 

matters.
45 CFR 46 .102(d) Research means 

a systematic investigation, 

including research development, 

testing and evaluation, designed to 

develop or contribute to 

generalizable knowledge. 
Nancy Cartwright

Scientists, including social scientists, are often dismissive of 

philosophy. Philosophy, it is said, is too abstract, too fussy 

and too taken up with its own problems to matter to real 

practice. With the issues discussed here (efficacy, evidence, 

RCTs and policy) I think just the opposite is the case. Bad 

practice, I maintain, is being recommended without 

intention and without sufficient notice in part because 

prissy issues that philosophy fusses about are being 

ignored, issues like what counts as a proper definition and 

whether an argument has been laid out with all the 

necessary premises.

What is This Thing Called ‘Efficacy’? 

http://personal.lse.ac.uk/cartwrig/PapersOnEvidence/What%20is%20that

%20thing%20called%20efficacy.%2018%20June%20edited%20for%20web

%20page.pdf
Philosophical differences: Positivism, 

Realism and Constructivism

Realist Evaluation: an overview Report from an Expert Seminar with Dr. Gill Westhorp. Edited by Dr. G. 

Westhorp, Prins, E., Kusters, C.S.L., Hultink, M., Guijt, I., Brouwers, J.H.A.M. Centre for Development 

Innovation, Wageningen University & Research centre, May 2011
RCT Theory Driven/Context Dependent

High Internal validity-independent from 

other external changes.

External validity-external changes part of 

interventions and must be reported 

together with the results.

Intervention group is compared with 

similar control or an internal control is 

used. 

Based on identification of relevant context 

and mechanisms, the intervention group 

can be compared with similar external or 

internal control. Differences between 

controls and intervention are reported. 

Isolation of confounding factors Isolation of confounding factors is not 

possible, outcome cannot be seen 

isolated from context and mechanisms

Nielsena & Kinesb. Realistic evaluation as a new way to design and evaluate occupational safety 

interventions. Safety Science. 2012;50:48-54
Realist Evaluation and Realist Synthesis

• Real – deals with the real world

• Realist – grounded in ‘scientific realism’

• Realistic – “The whole point is that it is a form of 

applied research, ... pursued to inform the thinking 

of policy makers, practitioners, program 

participants and public.” 

Patricia Rogers, RMIT University

8 

1997 2006

“What causes something to happen has nothing to do with the number of 

times we observe it happening” (Sayer, 2000 p. 14).

Not: “does it work or not?”

But rather, “what works, for whom, and in what circumstances?”

 Justin Jagosh, Ph.D , Participatory Research at McGill (PRAM)

 Department of Family Medicine, McGill University, Montréal, Canada.
Search for 

evidence

Appraise 

studies & 

extract 

data 

Organization 

of data; 

Iterative 

theming

process

Revision of 

conceptual 

model

Description of 

evidence; 

Generation of 

hypotheses

Initial 

Conceptual 

model and 

management 

questions

1. identifying the review question

2. searching for primary studies (A search to track 

program theories and a search for primary studies)

3. quality appraisal (Assessment of relevance and 

rigour) 

4. extracting the data (Annotation, Collation, 

Reportage)

5. synthesis

Methods, 

but...
•Albright F: A page out of the history of hyperparathyroidism. J Clin Endocrinol 8: 637–657, 1948

•Biographical Memoirs V.48 (1976) National Academy of Sciences (NAS)

There is more than one way to conduct a “realist synthesis.”

Thus, on the continent the disease was pulled to pieces, bit by bit without 

the benefit of chemical determinations: in America it was put together

by a group of men, almost no one of whom had ever looked a parathyroid 

cell face to face in a microscope.

All of which proves that there are two ways of killing a cat (see Fig. 8).
What Realist Review Does

• Identifying mechanisms, the contexts in which they are (or are 

not) activated, and the outcomes to which they lead

• Categorizing and building these Context-MechanismOutcome clusters into Demi-Regularities

– Not laws, but things that tend to happen

• Bringing to bear mid-range theories to help understand the 

patterns of these demi-regularities

• Ultimately building or testing a theoretical model of how a 

program works

AC Macaulay, J Jagosh, R Seller, J Henderson, M Cargo, T Greenhalgh, G Wong, J Salsberg, LW Green, C 

Herbert, P Pluye. Benefits of Participatory Research: A Rationale For a Realist Review (in press). Global Health 

Promotion. 18(2) June. 2011  Justin Jagosh, Ph.D , Participatory Research at McGill (PRAM)

 Department of Family Medicine, McGill University, Montréal, Canada.
Understanding Mechanisms:

• Mechanism may be defined as:

– “...underlying entities, processes, or structures which 

operate in particular contexts to generate outcomes of 

interest.”*

• Mechanism: 

– Are usually hidden

– Sensitive to variations in context

– Generate outcomes

• For social interventions, mechanism typically refer to a cognitive 

process or what ‘turns on’ in the mind of program participants 

to make them want to participate in the program

*Astbury B, Leeuw F. Unpacking Black Boxes: Mechanisms and Theory Building in Evaluation American Journal 

of Evaluation 2010 31(3):363-381

 Justin Jagosh, Ph.D , Participatory Research at McGill (PRAM)

 Department of Family Medicine, McGill University, Montréal, 

Canada.
How do you do that?

• By identifying the basic logic (theory) behind programs under review;

• By configuring the contextual features and mechanisms which determine 

outcomes (C-M-O configuring) and comparing cases;

• By refining the theory that was originally identified, based on the CMO 

synthesis. 

 Modified from Justin Jagosh, Ph.D , Participatory Research at McGill (PRAM), Department of Family Medicine, McGill 

University, Montréal, Canada.

Context 

(C1) Mechanism (M1) 

Outcome (O1) 

Pawson R, & Tilley N. 1997 [2003]. Realistic Evaluation. Thousand Oaks, CA: Sage Publications.

Basic components of realist causal explanation

Context 

(C2) Mechanism (M2) 

Outcome (O2) 
Identifying the theory:

For a realist synthesis of a single case, the underlying logic is understood as 

‘program theory.’ Every program has a theory, whether it is obvious or not 

• For a realist review synthesizing many cases, the underlying theory is 

considered “Middle-range” 

• Middle-Range Theory: not abstract to the point of being disconnected from 

the actual on-the-ground realities of program planning and implementation, 

yet, not specific to the point of being relevant to only one type of program.

• Middle-Range Theory According to Merton*: 

 “theory involves abstraction, of course, but it is close enough to 

observed data to be incorporated in propositions that permit empirical 

testing.”

*

Merton R. On Theoretical Sociology. Five Essays, Old and New. New York: The Free Press, 1967.

 Justin Jagosh, Ph.D , Participatory Research at McGill (PRAM)

 Department of Family Medicine, McGill University, Montréal, 

Canada.
Initial Conceptual Model

(from prior work on SMAs)

Informed, 

Activated 

Patient

Patient at 

3 Months

Patient 

at 12 

Months

Medication 

Adjustments

A1c

Patient 

Factors 

(demographics, 

adherence, 

other 

factors

Shared Medical Appointment

Peer support

Multidisciplinary Expertise

Patient 

with 

Diabetes

A1c

*Improving SelfManagement Activites

*Improving Problem 

Solving Skills

*Overcoming Barriers

to Treatment

03/08

Series of 

management-related 

questions provided by 

J. Øvetvreit 
• Number of articles by year

– 2008 or earlier: 38

– 2009: 10

– 2010: 9

– 2011: 14

• Number of articles by U.S. or 

international

– U.S.: 56 (78.9%)

– International: 15 (21.1%) 
Characteristics of SMAs

• Educational component – 88.7%

• Multidisciplinary members – 64.8%

• Included a behavioral intervention – 50%

• Included medication adjustment – 55.7%

• Included peer-to-peer support – 87.3%

• Included clinician training – 42.9%
Characteristics of SMAs

Visit duration

– <60 minutes: 1.7%

– 60-89 minutes: 9.9%

– 90-120 minutes: 59.2%

– >120 minutes: 12.7%

– Missing: 16.9%

– Missing: 23.9%

Visit frequency – Once: 8.5%

– Weekly: 16.9%

– Every 2 weeks: 2.8%

– Monthly: 29.6%

– Every 2 months or more: 

18.3%
Informed, 

Activated 

Patient

Outcomes 

Clinical

Cost

Satisfaction

Medication 

Adjustments

Patient 

Factors 

(demographics, 

adherence, 

other factors

Peer support

Patient with 

Chronic 

Disease

•Education about:

•Self-Management

•Problem-Solving 

Skills

SMA

Multidisciplinary 

Expertise

Q1, 2 

Q3, 4 and 

5 

Q6,7, 

10

Q7

Q8: 

 

Q9

Choose to 

participate 

in SMA

Improving SelfManagement 

Activites

 Insulin 

Inertia

Version 

2_11-8-11

10/11b
Patient Context

Social/Behavioral 

Context 

Educational 

attainment/ Health 

literacy/ numeracy

Social/ Psychological 

Factors 

Physiological Context

Disease and severity

Co-morbid conditions 

Psychological Factors

Desire to 

participate in 

SMA

Ability to 

participate 

in SMA

External Context

Context (high level) Mechanisms Outcomes

Education

Behavioral 

Intervention, e.g., 

motivational 

interviewing

SelfManagement 

Peer to Peer 

Support

Multi-professional 

participation

Medication 

Adjustment

Self-efficacy 

Intervention Dose, 

e.g., length, 

frequency, size

Quality of life

Clinical 

outcomes 

SMA 

Q3,4,5

Q7A,8

Q7B, 

10

Q6

Q9

04/12

Geography/location 

Healthcare system

Functional status

cost/health 

services 

utilization 

Depression/ 

anxiety 

Process 

measures of care 

Treatment 

adherence 

Multi-professional 

leadership

(Unpacked) Mechanisms
Some preliminary (hypothetical) demiregularities

Context Mechanism Outcome

C1

Across geographies (urban 

vssuburban vsrural)

M1 

Travel distances affect 

participation rate

O1 

Effects similar

C2

Across patient factors

(Socioeconomic status)

M2 

SES associated with 

educational attainment

O2 

Effects similar

C3

Across chronic diseases

M3 

Self management 

principles similar across 

disease

O3

Effects similar
Some preliminary hypothetical demiregularities

Context Mechanism Outcome

C1 M1 

Behavioral component

O1 

better outcomes if present

C1 M2 

Multiprofessional

O1 

better outcomes if present

C1 M3 

Medication Adjustment

O1 

better outcomes if present

C1 M4 

Duration of SMA

O1 

better outcomes if longer 

duration (90-120 minutes)

C1 M5 

Peer support

O1 

better outcomes if present

Also found that SMAs that used more of these had better outcomes.
Further unpacking a mechanism seeking further 

support from established theory

• Peer Support is a strategy – a high level mechanism that becomes the context for 

the next level.

Context Mechanisms

Outcome (O)

Beliefs about 

/attitudes towards 

self-management

+++ Beliefs about 

/attitudes towards 

self-management

More effective 

self-management

M1 motivation 

to comply with 

others

M4 learning 

in context

M3 evaluation 

of outcomes

M2 trust in 

peers
Further unpacking a mechanism seeking further 

support from established theory

• Multiprofessional/interprofessional team is a strategy – a high level mechanism 

that becomes the context for the next level.

Context Mechanisms

Outcome (O)

Beliefs about 

/attitudes towards 

self-management

+++ Beliefs about 

/attitudes towards 

self-management

More effective 

self-management

M1 motivation to comply with 

important others reinforced by 

multiplicity of professionals

M2 trust in professionals 

others reinforced by 

multiplicity of 

professionals

M3 interprofessional

practice is more 

patient centered.
Model Revised Model Re-Revised Model

•Iterative development of conceptual model for contexts, 

mechanisms, outcomes

•Based on principles and not a series of sequential steps

•No particular preference for quantitative or qualitative methods

•Multidisciplinary stakeholders and participants

•Stakeholders are regarded as key sources for eliciting program theory 

and providing data on how the program works

How is this theory driven, context 

dependent evaluation different?
Lessons Learned

• Generalizability is not “just” a philosophical question. 

It is core to practice in the real world. Not necessarily 

a need for further “research” but a need for 

researchers to report on different information. 

• Cannot manufacture new data - can only look at 

existing data in a different way.

• Current literature has very little information on 

patient perspective. Need for qualitative data.
Traditional ‘Cochrane’ Review Realist Review

1. Identify the review question 1. Clarify scope of review (identify review ; question; 

refine purpose of review; Articulate key theories to 

be explored

2. Search for primary studies, using clear 

predefined inclusion and exclusion criteria

2. Search for relevant evidence, refining inclusion

criteria in the light of emerging data

3. Appraise quality of studies using a predefined 

and validated critical appraisal checklist, 

considering relevance to research question and 

methodological rigour

3. Appraise quality of studies using judgement to

supplement formal checklists, and considering

relevance and rigour from a ‘fitness for purpose’

perspective

4. Extract standard items of data from all primary 

studies using template or matrix

4. Extract different data from different studies using an 

eclectic and iterative approach

5. Synthesise data to obtain effect size and 

confidence interval and/or transferable themes 

from qualitative studies

5. Synthesise data to achieve refinement of programme

theory – that is, to determine what works for whom, 

how and under what circumstances

6. Make recommendations, especially with 

reference to whether findings are definitive or 

whether further research is needed

6. Make recommendations, especially with reference to 

contextual issues for particular policymakers at 

particular times

7. Disseminate findings and evaluate extent to 

which practitioners’ behaviour changes in a 

particular direction

7. Disseminate findings and evaluate extent to which 

existing programmes are adjusted to take account of 

elements of programme theory revealed by the review

Pawson et al. ESRC Research Methods Programme, Working Paper Series, August ‘04