1

Practitioners

GUIDE

PRINCETON, NJ - ANN ARBOR, MI - CAMBRIDGE, MA - CHICAGO, IL - OAKLAND, CA - SEATTLE, WA 

TUCSON, AZ - WASHINGTON, DC - WOODLAWN, MD

EDI: HIGH WYCOMBE, UK - BUKOBA, TANZANIA - DAR ES SALAAM, TANZANIA

SEPTEMBER 2019

Organizations use a variety of strategies to help themselves consistently deliver an 

intervention as it was designed—in other words, to implement an intervention with 

fidelity. These strategies, or “implementation supports,” include processes such as workforce training, plans for continuous quality improvement (CQI), and supportive data 

systems. Having well-structured, formal implementation supports in place can be useful 

for ensuring that personnel and partners implement interventions consistently over time 

and across a variety of situations. Such measures increase the likelihood that all participants receive the intervention as it was intended to be delivered and can benefit from it. 

Having such supports in place can be especially beneficial when organizations are scaling 

an intervention, such as by serving more participants at a site or offering the intervention 

at new sites, because the supports can provide a framework for assessing implementation 

fidelity and considering changes to the intervention or implementation support processes. 

This guide will help practitioners set up supports in a way that facilitates implementing 

an intervention with fidelity. Well-structured implementation supports can be set up 

in three phases: (1) describe them in complete and operational terms and consider 

including details on how implementation might be modified if the intervention is 

scaled; (2) identify the personnel needed to conduct them and define their roles and 

responsibilities; and (3) specify the best way to document, communicate, review, and 

revise implementation support plans for CQI. 

How to Structure Implementation 

Supports

Katie Eddins and Karen Needels

MATHEMATICA-MPR.COM

Three phases of structuring an implementation support

Describe the implementation 

support in a way that is complete 

and operational, and may include 

considerations for possible scaling.

Identify the personnel 

needed to conduct the 

support, and define their 

roles and responsibilities.

Specify how implementation support activities will be 

documented, communicated, 

reviewed, and revised.

1

3 2
2

DESCRIBE IMPLEMENTATION 

SUPPORTS

Implementation science has identified six key 

supports that help organizations implement and 

scale an intervention with fidelity: (1) implementation monitoring, (2) performance monitoring, 

(3) CQI processes, (4) workforce training, (5) 

communication systems, and (6) data systems.1,2

For more information about some of these 

implementation supports, see the Further Reading 

section below. Each support plays a critical role in 

delivering an intervention as it was designed. For 

example, implementation monitoring focuses on 

whether personnel are adhering to the details of 

an intervention, such as delivering content in the 

specified time frame, and performance monitoring 

focuses on whether personnel are achieving certain 

benchmarks, such as serving a specified number 

of participants.3,4 While each support is important 

on its own, the combination of supports can also 

promote implementation fidelity. For example, 

implementation monitoring can generate ideas for 

additional training to help personnel better adhere 

to intervention guidelines. Through conversations with practitioners, we found that having a 

complete and operational description for each support can help personnel conduct the activities in a 

consistent way across different sites and situations.

Complete. To ensure that personnel understand 

and meet the various goals of implementation support activities, a complete description 

includes specifying the goals and purposes of 

the support and details each key activity or task. 

For example, implementation monitoring may 

consist of documenting and observing service 

delivery and reviewing case files to examine 

whether the intervention is being implemented 

with fidelity. To reduce burden on personnel, 

organizations can also specify how support 

activities can be integrated with each other in 

mutually reinforcing ways. For example, observations may be used primarily to monitor implementation fidelity, but they may also be used as 

workforce training opportunities or a data source 

for CQI discussions.5

Operational. The description should include all 

the procedures, tools and materials, and policies 

or guidelines that enable personnel to implement the support consistently. For example, a 

description that implementation monitoring 

will occur “regularly” is not specific enough to 

provide consistent monitoring across locations 

where an intervention is being provided. By 

contrast, implementation monitoring that is 

described as in-person observations by specific 

personnel of specific activities that happen once 

a month, using a fidelity checklist, is operational. 

Organizations may wish to specify how policies 

and procedures vary for different personnel or 

partners, such as by personnel role, experience level, or length of operation. For example, 

supervisors, program managers, and administrators likely require training different from that of 

frontline personnel, and experienced personnel 

may require training less frequently than new 

personnel. An operational description includes:

• Procedures, or how the implementation 

supports have to be carried out, such as the 

frequency (how often); duration (how long); 

and mode (whether in-person versus virtual, 

one-on-one versus group, and paper versus 

electronic). For example, in-service workforce 

training might occur virtually every month, 

for one hour, for all workforce members with a 

particular position description. 

• Tools and materials, such as standard 

templates, guides, checklists, and other materials. For example, pre-service training might 

include a standard presentation or interactive 

exercises for program managers to deliver 

consistent information across sites. 

• Policies or guidelines, such as which tasks 

are critical versus optional and which tasks 

must be carried out as described versus those 

that can be adapted. For example, a basic set 

of pre-service training information might 

be required, but sites might add training 

components specific to their population or 

location. Organizations might also wish to 

describe backup plans or required actions if a 

task cannot be completed as planned. 

Considerations for scaling. When scaling 

an intervention, organizations might describe 

specifically how supports should be carried out 

across various situations and, based on their 

capacity, prioritize certain implementation support activities. For example, when replicating an 

intervention at a new site, they might conduct 

observations to ensure that it starts up correctly, 

and during expansion of services at an existing 

site, they might find that less personnel-intensive documentation and case file reviews provide 

equal quality of monitoring. When adapting an 

intervention to a new location, organizations 

may wish to pay particular attention to specifyIMPLEMENTATION 

SUPPORTS

• Implementation 

monitoring: ensures 

implementation of 

an intervention the 

way it was designed. 

Focuses on improving adherence to the 

specific details of an 

intervention using 

tools, such as fidelity 

checklists or observing service delivery.

• Performance 

monitoring: specifies performance 

measures or goals 

for personnel and 

partners, such as the 

number of weekly 

participant contacts 

per workforce member or the number of 

participants served 

annually per partner 

site, and tracks progress toward achieving them. 

• CQI processes:

identify and communicate challenges 

with intervention 

implementation and 

develop strategies to 

address them. Test 

the intervention and 

make adjustments as 

needed, for example, 

by pilot testing a new 

service before delivering it to all intervention participants. 

• Workforce training: ensures that 

personnel receive 

necessary training 

prior to intervention 

implementation (preservice training) and 

on an ongoing basis 

(in-service training). 
3

Review and

revise

• Information gathering 

activities

• Revisions to activities

• Documentation and 

communication of 

revisions

Communicate

• Training

• Personnel meetings

• One-on-one supervision

• Electronic communication

• One- and two-way

communication

ing which implementation support activities are 

critical and which can establish a baseline that 

can apply in all situations. 

DETERMINE THE 

PERSONNEL NEEDED

From conversations with practitioners, we learned 

that having clear descriptions of which personnel 

need to be involved with each implementation 

support and of their respective responsibilities 

can help ensure both that the implementation 

supports are carried out by the most appropriate 

workforce member and that roles and responsibilities are clearly assigned. When determining 

the personnel needed for an implementation 

support, organizations may wish to consider: 

Specifying the required personnel or team 

for each implementation support activity.

This can help organizations be specific about 

involving personnel and reduce personnel burden 

in carrying out each support. Implementation 

support activities will require the varied skills of 

different types of workforce members, who may 

be internal or external to the organization. For 

example, data collection activities might be best 

completed by frontline personnel skilled in interacting with participants, and data analysis might 

best be performed by someone with specialized 

data skills. The number or types of personnel 

may also differ when an organization is scaling 

an intervention. For example, if an organization 

expands the number of personnel, participants, 

or sites, then more or different personnel may 

be required to deal with the additional volume 

or increased complexity in monitoring program 

implementation and performance. 

Defining personnel roles and responsibilities. Various personnel might play different roles 

and interact with each other differently within and 

across implementation support activities. For example, it might be valuable to specify that a data specialist analyzes data on participants’ service receipt 

but then shares information with both (1) program 

administrators, who monitor the performance of 

personnel and the fidelity of program implementation; and (2) frontline personnel who, along with 

administrators, contribute to CQI efforts. 

DOCUMENT, COMMUNICATE, 

REVIEW, AND REVISE

Effective documentation of implementation support procedures and policies can help organizations 

make sure the supports are carried out completely 

and consistently and can be particularly important 

in instances of personnel turnover. When documenting implementation support goals, activities, 

policies, and procedures, organizations may wish to 

consider using easily accessible formats; describing complex policies in plain, understandable 

language; and providing documentation in the 

languages spoken by personnel, if English is not 

their primary language. Different types of documentation can reinforce each other: for example, 

implementation manuals may lay out expectations 

for personnel, and checklists help personnel deliver 

on those expectations on a daily basis. 

Establishing communication procedures can 

help organizations share implementation support policies with the personnel responsible for 

conducting the support. Organizations may wish 

to consider aligning the format and frequency 

of communication about supports with ongoing 

communication about service delivery, and they 

may also wish to consider additional communication about implementation support topics that 

fall outside regular service delivery. For example, 

frontline personnel may not be regularly 

involved in discussions about changes to data 

systems, but they might benefit from information about changes that could affect their work. 

Different types of communication can also be 

Document

• Implementation manuals

• Guidebooks

• Checklists/logs

implementation

supports (cont'd)

• Communication 

systems: set up 

and monitor formal 

communication procedures among personnel and partners 

to address implementation challenges 

and continually 

improve the intervention. For example, 

plans for one-on-one 

mentoring meetings 

or group personnel 

meetings. 

• Data systems: track, 

measure, and store 

information about 

intervention implementation; participants' characteristics, 

service receipt, and 

outcomes; and 

program impacts. For 

example, management information 

systems or standardized spreadsheets. 

EXAMPLE OF 

WORKFORCE 

CHANGES DURING 

SCALING

An organization that 

runs a reading program 

for 7th- and 8th-graders 

recently expanded the 

number of students 

served in a current 

location and replicated 

its services in another 

location. Previously, 

one program manager had conducted 

monthly observations 

of all reading tutors to 

ensure fidelity to the 

tutoring model. 
4

used to reinforce the same information, from 

initial pre-service training through ongoing 

meetings. Presenting information in different 

modes (in person, over the phone, via webinar, 

electronically) can also help reinforce it, as can 

having channels of communication be both oneway (such as electronic newsletters) and two-way 

(such as open chat forums).

Reviewing and revising implementation support 

policies and procedures can help (1) ensure 

that the supports are being implemented as 

intended, and (2) identify needed changes. 

This can be especially important during scaling 

because unanticipated challenges often arise; for 

example, original performance measures may not 

work when an intervention is adapted to a new 

population. For reviewing and revising supports, 

organizations may wish to establish a schedule 

that aligns with other implementation dates; for 

example, updates could be aligned with yearly 

personnel training or reporting deadlines. To 

review implementation support procedures, organizations can gather and analyze information 

from program and implementation data, CQI 

activities, personnel feedback, and new or current 

funder requirements, such as those for updated 

data collection. Organizations can then make 

any needed revisions to their implementation 

support activities, procedures, or policies; update 

the implementation support documentation; and 

communicate the updates with personnel.

Follow us on:

Mathematica® is a registered trademark of Mathematica Policy Research, Inc.

Scan this QR code 

to visit our website.

FURTHER READING

1 National Implementation Research Network (NIRN)

"Implementation Drivers" (https://nirn.fpg.unc.edu/module-2/implementation-drivers)

"Active Implementation Hub" (https://nirn.fpg.unc.edu/ai-hub)

2Mathematica. "Planned Scaling Activities of CNCS-Funded Organizations: Benchmark 

Findings," 2018

3 Breitenstein et al. "Implementation Fidelity in Community-Based Interventions," 2010 (http://

doi.org/10.1002/nur.20373)

4 U.S. Department of Education. "Performance Management: Collecting and Using Data to Measure Progress, Improve Results," 2014 (https://www2.ed.gov/about/inits/ed/implementationsupport-unit/tech-assist/performance-management-collecting-data.pdf)

5 U.S. Office of Adolescent Health. "Continuous Quality Improvement," 2016 (https://www.hhs.

gov/ash/oah/sites/default/files/cqi-intro.pdf)

6 Mathematica. "Practitioners Guide: Build Organizational Capacity to Implement an Intervention," 

2019 (https://nationalservice.gov/impact-our-nation/evidence-exchange/build-organizationalcapacity-implement-intervention)

FOR MORE INFORMATION

For more information, contact Scott Richman, director of the Scaling Evidence-Based 

Models project, at srichman@mathematica-mpr.com.

ABOUT THE SERIES

The Corporation for National and Community Service (CNCS) supports the scaling of effective 

interventions that it funds. CNCS engaged Mathematica Policy Research to conduct the Scaling 

Evidence-Based Models project (contract GS10F0050L/CNSHQ16F0049). As part of that project, 

Mathematica developed a series of guides to help practitioners assess their scaling efforts critically, 

collect evidence on the effectiveness of their interventions, and increase the likelihood of effective 

scaling of successful interventions.

Each guide provides a succinct, but non-exhaustive, overview of a topic relevant to practitioners. The 

guides are based on implementation science research, as well as information collected during site visits 

conducted by Mathematica staff with three CNCS-funded grantees during fall 2018. As part of the effort, 

Mathematica staff reviewed program documents (such as manuals and grant applications) and conducted 

in-depth interviews with organizational leaders and frontline personnel, partner personnel, AmeriCorps 

members, and other stakeholders. More in-depth information on the topic of this guide can be found 

through references in the Further Reading section.

For more information 

about building organizational capacity, which 

can help to ensure that 

the personnel and infrastructure are in place to 

provide implementation supports, refer to 

Practitioners Guide: 

Build Organizational 

Capacity to Implement 

an Intervention.6

example of workforce

changes during scaling

(cont'd)

To accommodate the 

expansion and replication, three team leaders 

now observe all tutors 

monthly in both sites 

and have incorporated 

observation reviews 

into their supervision 

time with tutors. The 

team leaders received 

training to conduct 

the observations, 

and they review the 

results monthly with 

the program manager. 

Personnel will review 

the new process and 

revise it, if necessary, 

at their annual training 

institute meeting.