August 2020

aspe.hhs.gov 

Office of the Assistant Secretary for Planning and Evaluation

U.S. Department of Health and Human Services

OFFICE OF HUMAN SERVICES POLICY

Advancing the Use of Core 

Components of Effective Programs: 

Suggestions for Researchers 

Publishing Evaluation Results Project Officers: Sarah Oberlander

Authors: Allison Dymnicki, Lisa Trivits, Cheri Hoffman, and David Osher

_________________________________________________________

Abstract

Core components are the parts, features, attributes, or characteristics of a program that 

a range of research techniques show influence its success when implemented 

effectively.1 These core components can serve as the unit of analysis that researchers 

use to determine “what works,” and they become the areas practitioners and 

policymakers seek to replicate within and across a range of related programs and 

systems in order to improve outcomes. Research techniques such as meta-analysis can 

shed light on which components make programs successful across a range of programs 

and contexts, and help researchers identify with greater precision what works, in which 

contexts, and for which populations. This brief explains why it is important to collect and 

report on a wide range of program characteristics and the kinds of characteristics that 

should routinely be collected and reported in order to facilitate future meta-analyses that 

can help the field identify core components of effective programs.

_________________________________________________________

Background

This issue brief is part of a series of working papers designed to increase awareness, 

understanding, and use of the “core components” approach, designed to identify parts of a 

program that research suggests influence its success when implemented effectively. The first 

issue brief developed by the Forum for Youth Investment described what core components of 

effective programs are. In that paper, Ferber, Sileo, and Wiggins (2019) also described the benefits 

of using core components of effective programs, five steps researchers and policymakers can take 

to identify and implement these core components, and examples of this approach being used in 

afterschool, juvenile justice, and mental health settings. 

1 Ferber, T., Sileo, A., & Wiggins, M. E. (2019). Advancing the use of core components of effective programs. 

Washington, DC: Forum for Youth Investment. Retrieved from https://forumfyi.org/knowledge-center/advancingcore-components/
2

This second issue brief, developed by ASPE in its role as Chair of the Interagency Working Group 

on Youth Programs2 and the American Institutes for Research, builds upon a multi-year effort. 

Given the focus of the Interagency Working Group on Youth Programs on youth, the brief 

frequently describes youth-based work but the concept of core components applies broadly to 

program evaluation. It dives deeper into Steps 1 and 2 of the “core components” approach—to 

identify effective components and determine whether these components predict improvements in 

target outcomes across multiple contexts and subpopulations (see Appendix A for more 

information). It is important for researchers to be able to empirically test and identify core 

components within the context of a systematic review or meta-analysis because this offers 

practitioners and others more flexibility when choosing, adapting, and evaluating programs and 

services for a local community (Melendez-Torres, Bonell, & Thomas, 2015). However, 

researchers conducting systematic reviews or meta-analyses are often limited in their ability to test 

core components across effective programs because of inadequate information about the content, 

approach, and/or training and supports provided.

There are existing resources and checklists 

that encourage researchers to report more 

detailed information to describe key 

characteristics of interventions and evaluation 

studies. For example, the CONSORT-SPI, 

TIDieR, BCT Taxonomy, and Oxford 

Implementation Index provide suggestions for 

people who evaluate social and behavioral 

programs and intend to publish their results, 

whether in peer-reviewed literature, grantee 

annual reports, or other mechanisms that will 

be made available for future meta-analyses or 

systematic reviews. This issue brief adds to 

existing resources by translating some of the 

information in these resources and lifting up 

details that may be challenging to understand 

from the technical explanatory documents. It 

also provides more information on how to use 

the existing checklists and helps to identify 

which characteristics or domains appear 

across checklists to indicate those 

characteristics people agree are important to report on. There is no checklist that is right for every 

study, so we recommend reviewing all of these resources to determine which checklist(s) might 

best fit your specific needs given the research aims/goals of your study and local context.

We begin by describing what core components of effective programs are, some benefits of using 

this approach, how and why we developed these suggestions, and then provide suggestions for 

reporting on setting, participant, program, and implementation characteristics. 

What Are “Core Components” of Effective Programs?

As described by Ferber, Sileo, and Wiggins (2019), core components are the parts, features, 

attributes, or characteristics of a program that research suggests influence its success when 

implemented effectively.3 For example, a component might be a particular way staff and youth 

interact, a key feature of the relationships or environment a program creates for the target 

2 The Interagency Working Group on Youth Programs (IWGYP) (https://youth.gov/about-us), representing 21

federal departments and agencies, was formed in 2008 to improve outcomes for young people by promoting 

collaboration among federal, state, tribal, and local organizations, and through the identification and 

dissemination of promising strategies and practices that have been proven effective through rigorous evaluation. 3 The field has not yet adopted a common nomenclature for this work. What we call core in this brief has also 

been called evidence-based, evidence-informed, essential, common, and active. What we call components in this 

brief have also been called practices, features, ingredients, elements, characteristics, and kernels.

Why use meta-analysis to identify core 

components? It offers several benefits over 

standard evaluation of programs and 

services including (a) improving accuracy of 

the size of an effect, (b), resolving 

uncertainty when reports disagree, and (c) 

allowing results to be generalized to a larger 

population. 

What kinds of information do metaanalyses rely on? They require having 

information about outcomes (for people 

participating in the program and, ideally, for a 

comparison group) and characteristics such 

as those setting, participant, program, and 

implementation characteristics that can be 

coded across studies. 
3

population (e.g., a youth-driven environment), one of many activities within a program (e.g., 

coaching), the way the program is delivered (e.g., a combination of in-person and virtual events), 

or the amount (“dosage”) of a particular activity. Components can serve as the unit of analysis 

that researchers use to determine “what works,” and they become the areas practitioners and 

policymakers seek to replicate within and across a range of related programs and systems. 

Through using research techniques that attempt to figure out what makes programs successful 

across a range of programs and contexts, researchers can identify with greater precision what 

works, in which contexts, and for which populations.

Why Use Core Components of Effective Programs?

The use of core components of effective programs complements existing federal efforts 

encouraging the use of evidence-based programs in the following ways:

1. More flexibility to apply and adapt evaluation findings in policy and practice. This 

approach can enable practitioners and policymakers to integrate evidence into their work in a 

range of ways, including:

– Identifying strategies or activities that may drive change across a range of 

programs, settings, and populations. For example, if certain staff behaviors or 

activities prove especially effective with youth, policy and program leaders can 

better articulate and lift them up for adoption across youth-serving systems.

– Being able to adapt a new or existing program. This approach allows people to 

adapt programs in ways that fit the particular context and constraints of a 

community, organization, or population while still implementing with fidelity those 

strategies and activities that research shows matter the most. 

– Supporting continuous improvement over time. By identifying specific strategies 

and activities that are effective, focusing on core components may help further 

strengthen programs that are already showing some positive effects. 

2. Ability to target funding and services to improve impact. Using this approach can help 

agencies, provider organizations, and even individual practitioners allocate resources and 

time to specific strategies or activities likely to succeed for a population, setting, or individual.

How Did We Develop These Suggestions?

The resources and information used to develop these suggestions were gathered from input from 

federal and nonfederal experts and informant interviews over a two-year time period with both 

federal and nonfederal staff who have expertise in implementing and evaluating social, 

emotional, and behavioral programs (see Appendix B for additional detail).

The following suggestions build upon the four documents referenced earlier (CONSORT-SPI, 

TIDieR, BCT Taxonomy, Oxford Implementation Index) and focus on “setting, participant, 

program, and implementation characteristics.” These particular characteristics matter because 

they have been shown to influence the extent to which programs improve outcomes for youth 

and their families in real-world conditions. For example, even if an evidence-based program is 

being delivered with fidelity, if it is being delivered in ways that do not align with the setting or 

participant needs (e.g., there is no child care provided for parents while attending behavioral 

management classes), participants will not attend the program regardless of how high-quality it 

is, and outcomes will not be achieved. 
4

While it would benefit the field to have 

researchers and evaluators report on all of 

these aspects, this is often not feasible. The 

research aims/goals of a study should guide 

people in determining which characteristics 

are of highest priority to collect and report. For 

example, if the research aims/goals of a study 

are to understand to what extent a program 

can be adapted to address the needs of older 

Hispanic youth (aged 18-25) because the 

original program was developed and effective 

for White youth aged 10-17, then it would be 

important to prioritize collecting information 

regarding participants, the setting of the 

program, and how the program was adapted. 

The following suggestions should be 

considered with specific research aims/goals 

of a study and related program goals in mind. 

Again, it might not be feasible to report on all 

of these factors, but the more information 

available, the better the analyses of core 

components will be. 

Setting Characteristics 

All four checklists recommend describing 

these aspects related to setting characteristics:

• Contextual factors such as setting(s), geographic location(s), and date/time; 

characteristics of the delivering organization and wider service environment; unique 

ethical considerations; and significant events occurring at the time of the intervention

• The type(s) of location(s) where the intervention occurred, including any necessary 

infrastructure or relevant features

• The expertise, background, and any specific training given for each intervention provider 

(for example, psychologist, nursing assistant)

Examples of information researchers should consider including in the setting characteristics 

section of a checklist include: 

• Describe the research site(s) in terms of country, region, state, and locale 

(urban/rural/suburban) as well as characteristics of the community, site, and staff where 

the program occurred—and of the organization that operated the program. For example, 

it would be helpful to report on: 

o community characteristics such as economic opportunities, violence, and crime 

rates;

o site characteristics such as whether it was a residential institution/outpatient 

clinic/school/or other setting, age of organization, agency finances, other services 

provided, organizational climate and culture, structure, role of leadership, need 

and degree of support for the program (both in terms of local support and/or 

implementation resources), and micro- and macro-level influences; and 

CONSORT-SPI was developed for reporting 

results of evaluations for social and 

psychological interventions. The CONSORTSPI is an extension of the CONSORT, which 

was designed to help biomedical researchers 

report randomized controlled trials 

transparently. 

TIDieR is a guide and reporting checklist that 

was developed to improve the completeness 

of reporting, and ultimately the replicability, of 

interventions done in clinical settings.

BCT Taxomony is a consensually agreed, 

reliable taxonomy that could be used across 

behaviors, disciplines and areas of interest 

(e.g. health, the environment) to understand 

“active ingredients” of behavior change 

interventions.

Oxford Implementation Index is a 

methodology designed to develop and test 

methods for assessing implementation 

fidelity that can be applied to many domains.
5

o staff characteristics such as demographics (and match between demographic 

characteristics of staff and participants being served), staff mobility rates, 

educational credentials of staff, attitudes and beliefs of staff who will be 

implementing the program, and skills, knowledge, and training relevant to the 

program being implemented.

Participant Characteristics 

All four checklists recommend reporting on participant characteristics.

Examples of information researchers should consider including in the participant characteristic 

section of a checklist include: 

• Describe the people who participated in the study in terms of gender, race/ethnicity, 

socioeconomic status, age/grade levels, distinctive characteristics of participants, risk 

and protective factors of participants, and measures of responsiveness, engagement, 

and motivation to participate in the program with as much granularity as possible over 

time to address issues of intersectionality, ecological fit, and cultural competence. For 

example, it would be helpful to report on:

o distinctive characteristics of participants such as percentage of participants with 

disabilities, percentage who are uninsured, or percentage who are 

homeless/runaway youth; 

o risk and protective factors such as internalizing/externalizing symptoms; racism, 

violence/bullying, or social isolation experienced; involvement in the justice system; 

family and peer relationships; academic performance; social-emotional 

competencies; interest/skills in the arts, sports, or other activities; leadership, 

advocacy, or community service experience; and

o measures of responsiveness, engagement, and motivation to participate in the 

program, including the extent to which participants want to be part of the program, 

the program is something they are interested in and passionate about, they feel 

ready to make a change, and they want to persevere in their goals.

Program Characteristics 

All four checklists recommend describing these aspects related to program characteristics:

• Any rationale, theory, or goal of the elements essential to the intervention 

• Any physical or informational materials used in the intervention, including those provided 

to participants or used in intervention delivery or training of intervention providers 

• Each of the procedures, activities, and/or processes used in the intervention, including 

any enabling or support activities

• Interventions for each group with sufficient details to allow replication, including how and 

when they were actually administered 

• The core program components including (among other things) whether the program 

includes goals and planning (e.g., goal setting, problem solving, action planning), 

feedback & monitoring (e.g., feedback on behavior, feedback on outcome of behavior), 

social support (e.g., either practical or emotional social support, and antecedents (e.g., 

restructuring the physical and social environment) 
6

Examples of information researchers should consider including in program characteristics 

section of a checklist include: 

• Describe the program and comparison program(s) participants received It is critical to 

report similarly detailed information on the comparison program(s) participants received, 

with as much detail as is provided for the program of focus. For example, it would be 

helpful to report on:

o the program(s)’ name and whether it was manualized or branded,

o if it was intended for everyone, individuals who are at risk of or presenting 

particular problems, 

o integration of program practices that have been shown to be effective from other 

research, and

o the program’s theory of change, which articulates the core program components, 

mechanisms of action/change, and the primary and secondary short- and longterm outcomes the program aims to change.

Implementation Characteristics 

All four checklists recommend describing these aspects related to implementation characteristics 

for the people receiving the program and people in the comparison group (if they received a 

program):

• Sequence of intervention components; technology, materials, and technical 

requirements/support staff; staff characteristics, training and support; steps to promote 

staff and participant compliance.

• The modes of delivery (such as face to face or by some other mechanism, such as 

internet or telephone) of the intervention and whether it was provided individually or in a 

group.

• The number of times the intervention was delivered and over what period of time 

including the number of sessions, their schedule, and their duration, intensity or dose.

• If the intervention was modified during the course of the study and describe the changes 

(what, why, when, and how). 

• If intervention adherence or fidelity was assessed, describe how and by whom, and if any 

strategies were used to maintain or improve fidelity, describe them.

• If there was not a program provided for people in the comparison group, a description of 

what people in the comparison group received (e.g., access to services). 

Examples of information researchers should consider including in the program characteristics 

section of a checklist include: 

• Describe how the program was implemented, when, by whom, how often and for how 

long, in what format, and if implementation strategies were adapted or tailored to the 

local context. For example, it would be helpful to report on:

o how site leadership was trained and included in program implementation and 

whether a team oversaw program implementation;
7

o supports provided to those implementing the program such as training (prior to 

the program being delivered, advanced training, and follow-up training); 

resources, manuals, toolkits or other supporting materials designed to help

people learn about the program and how to deliver it; ongoing coaching or 

consultation; peer learning communities in which groups of people implementing 

the program learn from one another to foster better implementation; and financial 

incentives that allow people implementing the program ways to access new or 

existing money to facilitate implementation; 

o whether the program was delivered online/in person/both and whether the 

program was delivered to individuals, a group of participants, participants and 

family members, groups of families, or mixed;

o whether there were quality standards for program delivery and information 

collected to monitor implementation quality, including collecting information 

related to program appropriateness, fidelity, and reach of program (defined in 

bullet below) through peer reviews, data from people implementing/supervising 

the programs, and using administrative and electronic record data; and

o results of implementation monitoring that indicate perceptions of key stakeholders 

regarding whether the program was appropriate, relevant, and compatible for the 

setting and participants (appropriateness); the degree to which the program was 

implemented as prescribed in the original protocol or as intended by program 

developers (fidelity); and the degree to which the program was integrated within a 

setting (reach).

Cost

None of the existing resources recommend reporting on cost but this type of information would 

be very beneficial for return-on-investment analyses designed to understand the short- or longterm benefits of a specific program relative to the program’s cost. We suggest including a 

description of the average cost of the program per participant, any return-on-investment analyses 

that were done, and the results of those return-on-investment analyses. 

Next Steps and Future Directions

This brief is designed to increase people’s awareness about the existing resources in the field 

that should be used so that evaluators and researchers can pool information across research 

studies to identify and test core components. There are several ways in which these suggestions 

can be used by different audiences, some of which are described below:

• Individual researchers could use these resources and suggestions by following them 

in their own work.

• Professional organizations could consider how to use these resources and 

suggestions to promote consistency in how members of their organization are 

documenting and reporting the implementation of programs and evaluation findings.

• Funders (both private and federal) could adapt these resources and suggestions into 

their foundation or agency grantee reporting. For example, the suggestions could be 

converted into a checklist for researchers evaluating grantee programs. This checklist 

might be completed before funding is received, as part of annual progress monitoring 

reports, and/or at the study end.
8

Conclusion

Overall, this series of issue briefs is intended to increase the field’s awareness of the resources 

available that can advance work related to a core components approach and to highlight additional 

steps the field needs to take to fully leverage a core components approach as a critical tool for 

advancing evidence-based policy and practice.
9

References

Begg, C., Cho, M., Eastwood, S., Horton, R., Moher, D., Olkin, I., ... Stroup, D.F. (1996). 

Improving the quality of reporting of randomized controlled trials: The CONSORT 

statement. Journal of the American Medical Association, 276(8), 637–39.

Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A. W., Cronin, E., ... Williamson, P. R. 

(2008). Systematic review of the empirical evidence of study publication bias and 

outcome reporting bias. PLoS One, 3(8), e3081.

Ferber, T., Sileo, A., & Wiggins, M. E. (2019). Advancing the use of core components of effective 

programs. Washington, DC: Forum for Youth Investment. Retrieved from 

https://forumfyi.org/knowledge-center/advancing-core-components/

Grant, S., Mayo-Wilson, E., Montgomery, P, Macdonald, G., Michie, S., Hopewell, S., & Moher, 

D. (2018). CONSORT-SPI 2018 Explanation and Elaboration: Guidance for reporting 

social and psychological intervention trials. Trials, 19: 406. Doi: 10.1186/s13063-018-

2735-z

Glasziou, P., Meats, E., Heneghan, C., & Shepperd, S. (2008). What is missing from descriptions 

of treatment in trials and reviews? The BMJ, 336(7659), 1472–1474.

Hoffmann, T. C., Glasziou, P. P., Boutron, I., Milne, R., Perera, R., Moher, D., ... & Lamb, S. E. 

(2014). Better reporting of interventions: Template for intervention description and 

replication (TIDieR) checklist and guide. BMJ, 348, g1687. doi:10.1136/bmj.g1687

Melendez-Torres, G. J., Bonell, C., & Thomas, J. (2015). Emergent approaches to the metaanalysis of multiple heterogeneous complex interventions. BMC Medical Research 

Methodology, 15(1), 47. doi 10.1186/s12874-015-0040-z

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & PRISMA Group. (2009). Preferred reporting 

items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med 

6(7), e1000097. doi:10.1371/journal.pmed1000097

Moher, D., Schulz, K. F., & Altman, D, G. (2001). The CONSORT statement: Revised 

recommendations for improving the quality of reports of parallel-group randomised trials. 

Lancet, 357(9263), 1191–1194. 

Montgomery, P., Underhill, K., Gardner, F., Operario, D., & Mayo-Wilson, E. (2013). The Oxford 

Implementation Index: A new tool for incorporating implementation data into systematic 

reviews and meta-analyses. Journal of Clinical Epidemiology, 66(8), 874-882. doi: 

10.1016/j.jclinepi.2013.03.006
10

Appendix A. Five Steps for Advancing the Use of Core Components 

of Effective Programs

There are a number of ways policymakers, practitioners, and researchers are identifying and 

advancing the use of core components of effective programs. Although each approach has 

unique features, most include many or all of the following steps. Please see Advancing the Use 

of Core Components of Effective Programs (2019) for more information and examples of these 

steps in action.

5. Scaling: “Can we implement these supports on a broader scale?”

Implementing a strategy to scale up the use of the tools and methods that were proven to 

increase practitioners’ use of the core components 

4. Validating: “Did the supports work?”

Testing the tools and methods to see if they increased the use of the core components and if 

this led to better participant outcomes

3. Empowering: “Can we create supports to help people use these components?”

Creating guides, tools, assessments, protocols, techniques, and processes that facilitate the 

translation and dissemination of core components for use by practitioners

2. Testing: “Do the data say these are the core components?”

Winnowing the identified components based on which ones empirically predict the targeted 

population’s improvement in desired outcomes across multiple contexts and subpopulations

1. Identifying: “We think these are core components ...”

Developing theories about which identified components of programs might be instrumental 

in helping targeted populations achieve desired outcomes
11

Appendix B. Additional Information Related to Developing These 

Suggestions

To develop these suggestions, we completed a multi-phased process. First, we convened a twoday in-person meeting with experts doing this type of work in different disciplines and 

approximately 25 federal staff doing relevant work. The meeting focused on sharing current 

efforts taking place and determining appropriate next steps to advance work in this area. A 

smaller working group of federal staff, led by AIR and several experts in the field, determined that 

an important next step was developing suggestions for researchers and evaluators. With this 

goal in mind, a checklist was developed and shared with experts in the field and federal staff. 

Both groups raised important questions related to the audience, purpose, motivation for, and 

timing of completing the checklist. In response to this feedback, the current document was 

developed and shared with the working group of federal staff and two experts for a final round of 

input.

Several resources that are related to reporting core components, have been widely endorsed in 

the field, and we recommend readers review. These include: 

• The CONSORT SPI checklist and CONSORT SPI- Explanation & Elaboration which 

describes reporting aspects such as: 

o Introduction: background and objectives of the study, scientific background and 

explanation of rationale, and specific objectives or hypotheses.

o Methods: the interventions for each group with sufficient details to allow 

replication, including how and when they were actually administered.

o Discussion: generalizability (external validity, applicability) of the trial findings.

• The TIDieR provides guidance for describing an intervention via a checklist of items with 

examples including: 

o The why behind an intervention: Describe any rationale theory, or goal of the 

elements essential to the intervention (Item 2).

o The what (procedures) of an intervention: Describe each of the procedures, 

activities, and/or processes used in intervention, including any enabling or 

support activities (Item 4).

o The how well the intervention was implemented: If intervention adherence or 

fidelity was assessed, describe how and by whom, and if any strategies were 

used to maintain or improve fidelity, describe them (Item 11).

• The BCT Taxonomy provides a way of describing core program components by including 

a taxonomy of: 

o 93 behavior change techniques (e.g., negative reinforcement, self-monitoring of 

behavior, review of outcome goals, emotional social support practices) and 

o 16 hierarchical clusters organizing these behavioral change techniques (e.g., 

scheduled consequences, feedback & monitoring, goals & planning, social 

support), as well as definitions for each of these.

• The Oxford Implementation Index highlights characteristics of implementations broadly 

organized by four domains: 
12

o intervention design (e.g., intended dosage, setting, materials, training), 

o the actual delivery by clinicians (e.g., staff qualifications, the quality and use of 

materials, dosage administered and efforts to monitor adherence or drift 

overtime), 

o the uptake of the intervention by participants (e.g., number of sessions attended 

and whether participants sought out support outside of the trial period), and 

o contextual factors (e.g., socioeconomic characteristics, culture, geography, legal 

environment, and service structures).