Analysis of Evidence-Based Health Promotion and 

Disease Prevention Program Review Processes
Contract Number: HHSP233201500039I / 75P00120F37027

Submitted on: September 6, 2023

Recommended citation: Administration for Community Living. (2023). Analysis of EvidenceBased Health Promotion and Disease Prevention Review Process. U.S. Department of Health and 

Human Services.
Analysis of Evidence-Based Health Promotion and Disease Prevention Program Review Processes

Final Report ii

ACKNOWLEDGMENTS

We would like to thank the RTI International team for preparing this report. We would also like 

to thank the interviewees who contributed information reflected in this analysis. We gratefully 

acknowledge the many contributions of ACL staff. 

This report was funded by the U.S. Department of Health and Human Services’ (HHS’s) 

Administration for Community Living (ACL). The views expressed are those of the authors and 

should not be attributed to HHS or ACL. Funders do not determine research findings or the 

insights and opportunities of the authors.
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report iii

Contents

Section Page

Acknowledgments............................................................................................................................ii

Acronyms.........................................................................................................................................v

Executive Summary..........................................................................................................................1

Background ......................................................................................................................................1

Methodology....................................................................................................................................2

Literature Review....................................................................................................................2

Stakeholder Interviews...........................................................................................................2

Findings............................................................................................................................................4

Literature Review Findings .....................................................................................................5

EBP Application Process.................................................................................................5

EBP Review Processes....................................................................................................6

Stakeholder Interview Findings ..............................................................................................9

Meaning of EBP..............................................................................................................9

Program Identification and Application Experiences ................................................. 10

EBP Applicant Data Tracking....................................................................................... 11

Reported EBP Review Successes and Challenges....................................................... 12

Discussion...................................................................................................................................... 13

References .................................................................................................................................... 16

Appendices

APPENDIX A Literature Review: Publication Identification Guidelines ........................................ 19

APPENDIX B Literature Review: Publication Selection Matrix...................................................... 20

APPENDIX C Interviews: NVIVO Codebook................................................................................... 21

APPENDIX D Summary of Federal and Nonfederal EBP Review Processes.................................. 22
Analysis of Evidence-Based Health Promotion and Disease Prevention Program Review Processes

Final Report iv

Tables 

Table 1. Total Interviewee Counts........................................................................................ 3

Table 2. EBP Definitions and Criteria.................................................................................... 8

Table A. Summary of Federal and Nonfederal EBP Review Processes............................... 22

Figures

Figure 1. Federal EBP Review Process Flexibility ................................................................. 14
Analysis of Evidence-Based Health Promotion and Disease Prevention Program Review Processes

Final Report v

ACRONYMS

ACL Administration for Community Living

CDC Centers for Disease Control and Prevention

EBP Evidence-Based Program

EBPRC Evidence-Based Practice Resource Center

FY Fiscal Year

HHS U.S. Department of Health and Human Services

HomVEE Home Visiting Evidence of Effectiveness

HRSA Health Resources and Services Administration

IES Institute of Education Sciences

NCOA National Council on Aging

NIJ National Institute of Justice

OAA Older Americans Act

OASH Office of the Assistant Secretary for Health

QED Quasi-experimental design

RCT Randomized control trial

SAMHSA Substance Abuse and Mental Health Services Administration

SPRC Suicide Prevention Resource Center

TA Technical Assistance
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report ES-1

EXECUTIVE SUMMARY

Background

Since 1987, Title III-D of the Older Americans Act (OAA) has authorized grants to states and 

territories to support programs that promote health and well-being among older adults (60+). A 

Fiscal Year (FY) 2012 congressional appropriations law added a requirement that programs 

funded via OAA Title III-D must be evidence-based, meaning the programming has been tested 

and undergone thorough review to confirm that it can improve older adults’ health and wellbeing and reduce disease and injury (OAA, 2020). Consequently, the Administration for 

Community Living (ACL) and key stakeholders worked together to establish detailed criteria and 

evidence-based definitions to support Title III-D programming and create a formal ACL 

Evidence-Based Program (EBP) Review Process to identify programs that meet the criteria.

In 2022, some ten years after the initial development of the evidence-based criteria and EBP 

Review Process, ACL sought to examine the existing Review Process, facilitated by the National 

Council on Aging (NCOA), to explore whether any updates may be needed and to learn more 

about current EBP review efforts employed across other federal offices and agencies. ACL 

contracted with RTI International to conduct this exploratory study.

Methodology

Between February and June 2023, RTI conducted a rapid literature review and key informant 

interviews to address five research questions. 

Research Questions

What processes do other federal agencies use to determine evidence-based approval?

• What does the EBP application and review process include?

What is the history of the Title III-D EBP Review Process? 

• What was NCOA’s process? 

• What types of entities are developing programs? What are their protocols? What are their 

standards?

For the rapid literature review, the team identified 27 relevant peer-reviewed and gray 

literature publications. For the interviews, the team completed 33 interview conversations, 

across five groups of interviewees: (1) ACL staff; (2) Federal staff who work on EBP review 

processes with AmeriCorps, the Substance Abuse and Mental Health Services Administration 

(SAMHSA), the Centers for Disease Control and Prevention (CDC), the Office of the Assistant 

Secretary of Health (OASH), the Health Resources and Services Administration (HRSA), and the 

Department of Education; (3) nonfederal EBP stakeholder organizations; (4) ACL EBP Review 

Process applicants with approved programs; and (5) ACL EBP Review Process applicants whose 

programs were not approved.
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report ES-2

Findings

From the literature review and interviews, the team identified key themes concerning EBP 

Review Processes: 

• The existing ACL-supported approach to identifying EBPs is similar to approaches

used by other federal offices and agencies.

• Each agency or office has its own specific requirements, but all indicated that the 

gold standard for EBPs are those that can show positive outcomes in one or more 

randomized controlled trials (RCTs) or well-designed quasi-experimental studies.

• All EBP application and review processes use a systematic approach to measure 

program effectiveness.

• These processes include multiple steps: (1) evaluation of the research methodology 

to ensure rigor, (2) assessment of publications in peer-reviewed journals and other 

publicly available reports, and (3) review of the extent to which the program 

outcomes aligned to their initial program design and goals.

• Applicants find EBP review processes to be time-consuming and sometimes 

challenging, even with federal agencies and offices offering technical assistance.

• Applicants appreciate that some federal agencies offer a tiered EBP review (i.e., the 

most rigorous programs earn full EBP designation; less rigorous program designs, 

such as non-RCTs, earn designation as promising programs, rather than being 

eliminated from consideration).

• The NCOA-facilitated and ACL-supported EBP Review Process employs a systematic 

approach to determining whether a program is evidence-based according to ACL’s 

criteria.

• The process focuses specifically on (1) program emphasis on the research-based 

evidence of effectiveness, and (2) program readiness for implementation and 

dissemination.

• Most successful EBP Review Process applicants have both staff with research 

experience and sufficient resources (e.g., time, funding) to design and implement 

multistage, large-scale programs with rigorous designs, publications, and 

measurable outcomes.

• Many of the successful EBP Review Process applicants are based within academic 

institutions. 

• Some non-applicant program designers shared that they feel they cannot apply for 

EBP designation because they lack resources and staff to implement robust 

research designs (e.g., RCTs).

• Many unsuccessful applicants and non-applicant program designers are housed 

within smaller community-based organizations.

Conclusion

This study provided a useful snapshot of the criteria and processes employed by ACL and other 

entities to identify EBPs and some suggestions for enhancing these criteria and processes. 
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 1

BACKGROUND

Since 1987, Title III-D of the Older Americans Act (OAA) has authorized grants to states and 

territories to support programs that promote health and well-being among older adults (60+). 

An FY2012 congressional appropriations law added a requirement that programs funded via 

OAA Title III-D must be evidence-based, meaning the programming has been tested and 

undergone thorough review to confirm that it can improve older adults’ health and well-being 

and reduce disease and injury (OAA, 2020). In 2012, the Administration for Community Living 

(ACL) and key stakeholders worked together to establish a detailed Evidence-Based Program 

(EBP) Review Process, identifying programs that effectively support older adults. However, ACL 

does not require that all programs delivered using Title III-D funds undergo the formal EBP 

Review Process and be listed in the Evidence-Based Program Registry hosted by the National 

Council on Aging (NCOA). Title III-D grantees are able to deliver programs listed on the NCOA

Evidence-Based Program Registry, programs deemed evidence-based for older adults by other 

agencies within the U.S. Department of Health and Human Services (HHS), or other programs 

that have been determined to meet the Title III-D criteria. Title III-D prioritizes providing EBPs to

historically marginalized older adults, including those with the greatest economic need and 

those residing in medically underserved communities (e.g., rural areas).

This project explored the extent to which the ACL-supported EBP Review Process has 

experienced changes, while also analyzing and summarizing the existing process (facilitated by 

the NCOA). In addition, this effort identified opportunities for moving forward based on 

feedback from EBP applicants, other federal offices with their own EBP processes, and key 

stakeholders. The research questions that guided this project were as follows:

1. What processes are other federal agencies using to determine EBP designation, and 

what are their suggestions for modifying or improving these processes? 

2. What does the EBP application and review process include?

a. What does the rigor behind an EBP look like?

3. What is the history of the Title III-D EBP Review Process?

a. What are the requirements or criteria that must be met for a program to be an 

EBP?

b. Who submitted programs for EBP review? 

4. What is NCOA’s process in determining EBP designation for ACL? 

a. What is it currently, and how has it evolved over time?

b. What steps were taken by NCOA and partners to attract applicants?

c. What benefit does a developer/administrator receive for their program meeting 

ACL’s requirements of an EBP?

5. What types of entities are developing EBPs? What are their protocols? What are their 

standards?

a. What type(s) of technical assistance (TA) is provided to unsuccessful applicants? 

Did these applicants reapply? Are they then successful?
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 2

b. What feedback has been received from developers who did not apply or applied 

but did not receive approval?

c. Are there other programs that may be appropriate, but are not applying? What 

are other gap areas, if any?

To address these questions, the team conducted a rapid literature review from February 

through March 2023 and a series of key informant interviews from May through June 2023. 

This report outlines the process by which the literature review and interviews were conducted, 

coded, and analyzed and highlights key themes. 

METHODOLOGY

Literature Review

The rapid literature review was designed to investigate the published research on the history of 

the ACL EBP Review Process, EBP reviews more broadly, and current processes used to review 

and evaluate EBPs. The team employed a rapid approach, meaning that the research team 

considered a range of peer-reviewed and gray literature within a limited time frame. Rapid 

reviews are recommended as a method of synthesizing available research on what is already 

known about a policy or practice using systematic review techniques in an expeditious manner

(Tricco, Langlois, & Straus, 2017). Appendix A presents the search terms, parameters 

developed, and sources used during the identification process. 

In total, the team identified 13 peer-reviewed publications and 32 gray literature publications, 

screening each for relevance and potential inclusion. A team member imported the publications 

into a data charting tool, created to capture and organize information collected from the 

literature review. The tool allowed information from the publications to be extracted and 

categorized based on applicability to the research question(s) and associated themes.

Upon an initial abstract review, 6 publications were excluded based on publication type and 

population of focus (e.g., conference abstracts with no corresponding journal article; literature 

aimed toward communities, clinicians, and policymakers with minimal focus on resources for 

older adults). The titles and abstracts of the remaining 39 publications were further screened

for relevancy, and 4 were excluded. The remaining 35 peer-reviewed and gray publications 

underwent a full-text review by two team members. Of these, 8 were excluded based on year, 

publication type, or relevance (e.g., an abstract indicating content related to meta-analysis and 

EBP, but the actual article focused almost exclusively on meta-analysis with minimal EBP 

Review Process content). The team completed a qualitative thematic analysis on the final 27 

publications selected for inclusion. The publication selection process is summarized in 

Appendix B.

Stakeholder Interviews

In addition to the rapid literature review, RTI conducted a series of 33 virtual interviews with 

stakeholders including ACL staff, non-ACL federal staff, nonfederal partners, and program 
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 3

developers who either successfully obtained or did not receive EBP designation through the 

ACL-supported EBP Review Process. Through these interview conversations, the team sought a 

better understanding of the EBP Review Process, including benefits, challenges, gap areas, 

program examples, feedback, and suggestions for improvement. Table 1 illustrates the 

completed interviews by interviewee group. 

Table 1. Total Interviewee Counts

Interviewee Group Interviews Conducted

ACL Staff N=4

Other Federal Partners N=6

Nonfederal Partners N=7 

Successful ACL EBP Developers N=8

*

Unsuccessful ACL EBP Developers N=8

Total N=33

*Participants classified as unsuccessful, even if non-EBP applicant (N=4 applied unsuccessfully; N=4 did not apply)

The team worked with ACL to identify potential interviewees, including representatives from 

federal and nonfederal entities, who support or conduct EBP review processes. These 

interviewees represented seven distinct review processes (see Appendix D), with hundreds of 

programs having gone through these processes seeking evidence-based designation. The team 

also interviewed contacts from ACL’s nonfederal partner organization, NCOA, in February 2023, 

and they subsequently provided contact information for program developers who had applied 

or expressed interest in applying for EBP designation. With this information, the team compiled 

a list of prospective interviewees that included both successful and unsuccessful program 

developers. Unsuccessful program developers were defined as those who either (1) applied to

the ACL-supported EBP Review Process and did not receive EBP designation, or (2) inquired but 

did not apply because they perceived their applications would be unsuccessful. Upon finalizing 

the selection of potential interviewees, the team initiated recruitment efforts via email. 

Interviewees who did not respond to the first email received up to four follow-up emails and

telephone calls to encourage participation. 

With guidance from ACL, the team developed five interview guides, each tailored to one of the

following interviewee types: 

1. ACL staff from the National Institute of Disability, Independent Living, and Rehabilitation 

Research and the Administration on Aging. 

2. Non-ACL federal staff (e.g., AmeriCorps, CDC, SAMHSA, HRSA, Office of the Assistant 

Secretary of Health [OASH], and Department of Education). 

3. Nonfederal partners (i.e., organizations that help federal agencies implement EBP 

review processes). 
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 4

4. Staff from programs with evidence-based designation through the ACL-supported EBP 

Review Process. 

5. Staff from programs without evidence-based designation through the ACL-supported 

EBP Review Process. To better understand the unique barriers Tribal developers face in 

achieving evidence-based designation, as well as opportunities to address them, a 

version of the applicant without evidence-based designation interview guide was 

adapted slightly for use with Tribal non-applicant interviewees reflecting reasonings for 

not submitting programs for review. 

Having five types of interviewees allowed the team to capture diverse perspectives regarding

both the ACL-supported EBP Review Process and other federal EBP review processes. Each 

interview guide prioritized a different aspect of EBP review. For interviews with ACL staff, nonACL federal staff, and stakeholders, the priority topics included identification of EBP programs, 

including scoring processes and ongoing follow-up and support. Conversations with program

developers focused on their experiences with the ACL-supported EBP Review Process and other 

EBP review processes. The team also asked all interviewees to define “evidence-based” in their 

own words and to discuss benefits, challenges, and, where appropriate, desired changes to the 

ACL-supported EBP Review Process. 

The team conducted all interviews virtually (Zoom) between May and June 2023, with each 

lasting no more than 1 hour. The Zoom transcription feature was enabled for all interviews, and 

conversations were conducted by both a senior interviewer and a notetaker to document 

interviewer reactions that could not be captured via transcript (e.g., facial expressions). The 

team cleaned the Zoom transcripts and used NVivo qualitative analysis software to code and 

organize interview findings. Using a structural narrative analysis approach (Riessman, 2008), 

transcripts were analyzed by identifying key topics described in both individual sentences 

(micro level) and across broader paragraphs in response to each interview question (macro 

level). By analyzing both micro and macro levels, the team was able to identify not only “what is 

said” but also “how content is organized by the speaker” (pp. 100–101). This organizational 

component is an especially important methodology to apply when interviewees are describing 

step-by-step processes, as was the case with the EBP interviews. Using the structural narrative 

approach, the team developed both a priori NVivo codes based on the original interview 

questions (i.e., fitting responses by question asked) and ad hoc codes developed as 

interviewees shared specific details and nuances related to their EBP experiences. These ad hoc 

themes emerged through the coding process and are a central focus of key findings. All coders 

were trained qualitative research specialists who cross-coded to ensure interrater reliability 

(confirmed interrater reliability using Cohen’s Kappa, Κ > 0.65). A summary of the NVivo 

codebook is provided in Appendix C. 

FINDINGS

Findings from the rapid literature review and stakeholder interviews are presented below, 

organized by method of data collection and key themes. 
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 5

Literature Review Findings

The rapid literature review focused specifically on documentation of the ACL-supported EBP 

application and Review Process, and the review processes and applications employed by other 

federal agencies and offices. The following subsections highlight findings specific to EBP review 

and applications. 

EBP Application Process

To support EBP reviews, ACL funded NCOA to conduct a 

systematic application review and identify programs that 

meet ACL’s OAA Title III-D criteria.1 Based on gray 

literature findings, ACL’s EBP application consists of two 

stages: an initial step to provide a program overview, 

followed by an invitation-only second stage that requires 

more details about program implementation and 

outcomes. This two-stage structure is intended to 

provide a sufficiently rigorous review to ensure that 

designated EBPs will benefit older adults with quality-oflife improvements, increased self-efficacy and personal 

health management, and sustained independence 

(NCOA, 2022b). Blank applications for both Stage 1 and 2 and associated scoring criteria are 

posted to NCOA’s website, enabling prospective applicants to review the requirements.

• Stage 1 requires program developers to detail their research-based evidence, program 

effectiveness, and associated evaluation results. This stage is further divided by program 

focus, with one application for falls prevention programs and a separate application for 

all other health promotion and disease prevention programs. NCOA’s Review Council, 

composed of experts in research and evaluation, reviews all Stage 1 applications. Only 

successful Stage 1 applicants are invited to complete Stage 2.

• Stage 2 requires successful Stage 1 applicants to provide additional information on 

program implementation, training, and dissemination. Based on material submitted in 

Stage 2, NCOA’s Review Council makes final determinations as to which applicant 

programs achieve EBP designation.

Like the two-stage ACL application, many federal offices and agencies include multiple EBP 

application steps. For example, the AmeriCorps’ Office of Research and Evaluation employs a 

five-step review process for EBP applicants to the Social Innovation Fund program, which 

invests in economic opportunity, healthy futures, and youth development in low-income 

communities. This process involves (1) reviewing a self-assessment provided by the program 

developer, (2) determining methods used and number of existing implementation sites and 

populations influenced, (3) assessing program effectiveness via literature review, (4) reviewing 

other programs that have utilized the program’s model using RCT, and (5) determining program 

1 As of June 2022, the EBP Review Process was paused, pending results from this exploratory study (NCOA, 2022a).

The ACL-supported EBP application 

consists of two separate stages, with 

only successful Stage 1 applicants 

receiving invitations to complete Stage 

2. ACL has funded NCOA to review 

applications at both stages and 

determine which Stage 2 applicant 

programs should receive EBP

designation.
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 6

effectiveness based on prior results (Spera et al., 2015). These types of multistep structures 

allow applicants to highlight all the details of their programs, with particular importance placed 

on methodological design and implementation. Use of experimental (e.g., RCT) or quasiexperimental models is a hallmark of robust research design in EBP applications, as are 

publications in peer-reviewed journals and validated evaluation results that demonstrate 

intended outcomes. Multistep applications allow program developers and EBP application 

reviewers to focus on one or more of these key EBP domains at each stage.

EBP Review Processes

The ACL-supported EBP Review Process, administered by 

NCOA, awards EBP designation to falls prevention and 

health promotion programs that demonstrate rigor 

through experimental or quasi-experimental research 

designs. Additionally, successful programs have 

demonstrated dissemination and have published 

program outcomes in peer-reviewed journals.

Based on peer-reviewed and gray literature sources, 

ACL’s EBP Review Process is similar to that used by other federal offices and agencies, such as

the Centers for Disease Control and Prevention (CDC), the Substance Abuse and Mental Health 

Services Administration (SAMHSA), and the Health Resources and Services Administration 

(HRSA). The team found that all of the reviewed offices and agencies have systematic processes 

for defining and designating EBPs that promote health and well-being. Examples include the 

CDC Standards for the Development of Evidence-based Guidelines, the SAMHSA EvidenceBased Practices Resource Center (EBPRC), and the HRSA Home Visiting Evidence of 

Effectiveness (HomVEE). These agency review processes consider EBP applications based on a 

host of criteria, including the number and quality of published, peer-reviewed manuscripts 

related to the EBP; the rigor of the research methodology (i.e., dependent variables or 

outcomes measured, composition of the study sample, and overall quality of the study design 

and methods); and fidelity of the program outcomes to the original program design and goals. 

All of the EBP gray literature indicated that these rigorous criteria are designed to minimize bias

and enhance consistency in the review process while also

ensuring quality across all designated EBPs.

Although the ACL-sponsored EBP Review Process 

identifies programs that support older adults, other 

federal offices and agencies that review EBP applications 

for child- or family-centric programming apply EBP review 

criteria that are similar to ACL’s (Baron, 2018; Fixsen et 

al., 2013). For instance, the Department of Education’s Education Innovation and Research

program, HRSA’s Maternal, Infant, and Early Childhood Home Visiting Program, and OASH’s 

Teen Pregnancy Prevention program (Baron, 2018) all follow similar protocols and standards, 

including evaluating the target demographic and intended outcomes, identifying evidence of 

ACL’s EBP Review Process mirrors the 

processes used by other federal offices 

and agencies to review programs and 

assign EBP designation.

The purpose of the EBP Review

Process is to identify programs that (a) 

have proven success in designing and 

implementing a model that meets 

their stated goals to support their 

target population, and (b) can be 

replicated in other communities with 

the same successful results. 
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 7

prior successful implementation, and assessing the program’s potential for public 

dissemination. 

Nonfederal EBP registries and clearinghouses support a slightly modified EBP review process

that is based on organizational outreach, rather than structured applications (Mayo-Wilson et 

al., 2021). Registries such as the National Institute of Health’s Evidence-Based Cancer Control 

Programs, the Institute of Education Sciences’ (IES’s) What Works Clearinghouse, the National 

Institute of Justice’s (NIJ’s) CrimeSolutions.gov, and some state registries employ a multistep 

process that begins with program identification through staff-led literature reviews or 

nominations from program developers (Best Practice Caregiving, 2022; CEBC, 2023; IES, 2023; 

National Cancer Institute, n.d.; NIJ, n.d.). Programs identified through these literature review or 

nomination processes are then assessed in depth (e.g., identifying findings published in peerreviewed journals, reviewing program dissemination readiness). Registry staff partner with 

internal or external reviewers to assess the quality of these program findings, assigning an 

overall rating or score to each program. These nonfederal registries vary in the nuances of their 

individual scoring guidelines, but to achieve the highest rating, all registries require programs to 

show evidence of positive outcomes in one or more RCTs or well-designed quasi-experimental 

studies. Programs without these rigorous research designs may still be included in the 

registries, but they are listed as “promising” and placed in a lower tier than those with robust 

research methods and associated evidence of program effectiveness. For example, the Social 

Science Research Institute at Penn State provides a five-category, color-coded legend that 

allows users to “quickly see where each program falls on 

a spectrum from negative impact to positive impact” 

(Penn State University, 2023). 

A tiered scoring approach to EBP review also supports 

diversity, equity, and inclusion goals by extending the 

potential for a wider array of applicant programs to earn 

some level of EBP designation. For example, the Suicide 

Prevention Resource Center (SPRC) a national 

nonprofit supported by federal funding from SAMHSA, 

has recently revised its program review process to 

“increase health equity through expanded access and 

representation.” SPRC’s Best Practices Registry explicitly invites submissions for interventions 

that are culturally relevant (SPRC, 2020). Whereas most EBPs have been developed by

institutions with experienced research staff and resources to design rigorous studies (e.g., RCTs) 

and publish findings, the tiered approach may enable smaller entities with alternative 

approaches (e.g., programs with qualitative studies) to apply and receive a lower tier 

designation.

A brief summary of key EBP definitions and criteria is shown in Table 2, and a broader summary 

of EBP review processes is presented in Appendix D.

Concerning process improvements, 

some offices and agencies are moving 

to a tiered system, wherein the most 

rigorous studies are awarded full EBP 

designation, and approaches that do 

not meet all standard EBP criteria may 

earn a lower tier designation that 

acknowledges their potential as 

promising programs.
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 8

Table 2. EBP Definitions and Criteria Entity Name and Type

Entity Entity Type Definition of “Evidence-Based” EBP Eligibility 

ACL (Health 

Promotion) Federal (HHS)

“The program meets the requirements for ACL's Evidence-Based 

Definition; and the program is considered to be an "evidence-based 

program" by any operating division of the U.S. Department of Health and 

Human Services (HHS) and is shown to be effective and appropriate for 

older adults.”1

“Demonstrated through evaluation to be effective for improving the health and 

well-being... among older adults; Proven effective... using experimental or 

quasi-experimental design; ...Published in peer-review journal; Includes 

developed dissemination products available to the public." 

CDC Federal (HHS)

“Evidence-based medicine is the conscientious, explicit, and judicious use 

of current best evidence in making decisions about the care of individual 

patients...[it] means integrating individual clinical expertise with the best 

available external clinical evidence.”2

“Us[es] body of evidence as the scientific backbone behind the 

recommendation...consider well-established clinical or public health principles 

and anticipate adverse outcomes...clear, practical.” 

SAMHSA Federal (HHS)

“The Resource Center contains a collection of scientifically based 

resources...including Treatment Improvement Protocols, toolkits, resource 

guides, clinical practice guidelines, and other science-based resources.”3

(1) Must be clearly defined and replicable; (2) Are currently in use;

(3) Provide evidence of effectiveness; (4) Have accessible resources for 

implementation and fidelity.9

HRSA Federal (HHS) “Unbiased, significant to public health, and rooted in science, experience, 

and policy.”4 

An EBP “conforms to a clear consistent home visitation model that has been in 

existence for at least 3 years and is research-based, grounded in relevant 

empirically-based knowledge, linked to program determined outcomes, 

associated with a national organization or institution of higher education that 

has comprehensive home visitation program standards that ensure high quality 

service delivery and continuous program quality improvement,” among other 

requirements.12

OASH Federal (HHS)

“Published reviews of intervention evaluations and studies to improve 

health that have evidence of effectiveness, feasibility, reach, sustainability, 

and transferability.”5

EBPs are either systematic reviews (i.e., critical assessment of all research 

studies that address a particular issue) or non-systematic reviews (i.e., federal 

guidelines that are based on current science).

IES Federal (Ed)

“Evidence of effectiveness (i.e., randomized controlled trials or quasiexperimental designs) of programs, policies, or practices, using a 

consistent and transparent set of standards.”6

“...Are groups randomly assigned; is sample attrition high or low; are groups 

similar before the intervention began; are there confounding factors or 

concerns with outcomes; ...” 10

NCOA Nonfederal 

Partner

“EBPs offer proven ways to promote health and prevent disease among 

older adults. They are based on research and provide documented health 

benefits.”7

Two-stage application focused on the effectiveness and research-based 

evidence of the program and implementation and dissemination readiness. 

SPRC Nonfederal 

Partner

“Strategic planning, keys to success, and a comprehensive 

approach...these elements work together to make suicide prevention 

efforts successful in achieving desired outcomes and using limited 

resources most efficiently.”8

“The...use of current best evidence in making decisions about the care of 

communities and populations in the domain of health protection, disease 

prevention, health maintenance and improvement.”11

Notes: 1 https://acl.gov/programs/health-wellness/disease-prevention; 2 https://www.cdc.gov/genomics/about/file/print/Evidence-Based_Practice_508.pdf; 3 https://store.samhsa.gov/

sites/default/files/pep20-06-01-001.pdf; 4 https://www.mchevidence.org/documents/ESM-National-Report-2022-2020.pdf ; 5 https://health.gov/healthypeople/tools-action/browseevidence-based-resources/types-evidence-based-resources; 6 https://ies.ed.gov/ncee/wwc/Docs/referenceresources/wwc_info_what_061015.pdf; 7 https://www.ncoa.org/article/aboutevidence-based-programs ; 8 https://sprc.org/effective-prevention/ ; 9 https://www.samhsa.gov/ebp-resource-center/about; 10 https://ies.ed.gov/ncee/wwc/Docs/referenceresources/

wwc_info_rates_061015.pdf; 11 https://sprc.org/keys-to-success/evidence-based-prevention/ ; 12 https://homvee.acf.hhs.gov/HRSA-Models-Eligible-MIECHV-Grantees 
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 9

Stakeholder Interview Findings

Meaning of EBP

At the start of every interview, respondents were asked to define the term “evidence-based” in 

their own words. The results underscore how perceptions differ between interviewee types, 

with federal staff and nonfederal partners focusing more on process rigor and measurable 

outcomes, and program developers highlighting the need for contextual factors and participant 

voice. 

• ACL and non-ACL federal staff first used terms like systematic assessment, rigorous 

methods, and replicability in defining EBPs, with several also mentioning the “gold

standard” for EBPs (i.e., use of RCTs or well-designed quasi-experimental studies). 

Interviewees highlighted the importance of research designs that focus on robust data, 

measurable effects, and analyses that ensure data validity. For many of these 

interviewees, the EBP definition related specifically to the research design and processbased outcomes (e.g., published findings in a peer-reviewed journal, analytic models 

showing improvement over baseline for the intervention group relative to the 

comparison group). 

• Nonfederal partners, who represent organizations that collaborate with federal entities 

to coordinate EBP review processes, provided similar definitions to the federal staff 

interviewees in terms of the need for robust 

research and replication. However, these 

interviewees also focused on the importance of 

having tangible program outcomes that 

demonstrate benefits to the populations they are 

designed to support. Some nonfederal partners 

also acknowledged the challenges their 

organizations have had in defining EBPs, 

specifically because of this perceived disconnect between the need for validated

scientific methods and the benefit of having community or program participant 

perspectives. 

• Program developers were less consistent in their responses, with some highlighting 

robust research and others focusing more on participant perspectives about program 

outcomes. Notably, several developer interviewees, including Tribal community 

members, referenced the triangle of evidence, which includes research, clinical 

expertise, and patient context. These interviewees added that ACL and other federal 

offices focus more heavily on the research pillar of the triangle, without giving equal 

weight to clinicians or patient perspectives. Program developers also described how 

they believe ACL and other entities focus solely on evidence or efficacy in considering 

EBPs, whereas program developers consider multiple dimensions for defining program 

successes (e.g., primary data collection among program participants). 

Definitions of “evidence-based” 

prioritized rigorous research methods, 

with some interviewees also noting 

contextual factors and community 

perspectives. 
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 10

Across all interviewee types, respondents noted that federal offices and nonfederal 

organizations are working to expand their evidence-based definitions to include more realworld context in program outcomes. For example, a nonfederal partner explained that 

nonrandomized control trials and pre-/post-test designs with no control group could be 

included in an expanded EBP definition, provided the programs still had statistically significant 

findings and at least one peer-reviewed journal publication.

An expanded definition of EBPs also was said to encourage greater diversity in the pool of 

program applicants and populations served. Currently programs with small sample sizes, no 

comparison group, and mostly qualitative data are unlikely to be considered for EBP 

designation, but an expanded definition could extend an EBP designation opportunity to these 

programs. Likewise, a narrow EBP definition means 

programs designed specifically to reach historically 

marginalized and underrepresented communities often 

cannot earn EBP designation because they have smaller 

sample sizes and narrower program parameters. A 

broader definition of what it means to be evidencebased could increase the number of EBP-designated, 

population-specific programs designed to support 

diverse communities.

Program Identification and Application Experiences

All interviewees were asked how prospective EBP applicants are identified. Although some 

programs may be recruited (e.g., NCOA identifies active experts from recently published 

literature or presentations at professional conferences, such as American Society on Aging and 

the Gerontological Society of America), most program developers initiate contact directly

because EBP designation is highly coveted as a mark of program success.

Following program identification, all interviewed federal 

staff and nonfederal partners described an application 

process that requires prospective EBPs to submit written 

descriptions of their program purpose, target 

population(s), and implementation and dissemination 

designs. Application materials also highlight replicability, 

detailing how programs can be disseminated effectively 

and implemented across communities nationwide.

Interviewed ACL program developers recalled including

“the outcome measures, the research methodology” and “[program] costs, materials required, 

[and] training required” in their applications. As described in the literature review findings

above, most federal EBP reviews also require that prospective EBPs employ an experimental or 

quasi-experimental design (QED) with comparison groups to measure program effects.

Broadening the definition of EBP to 

include more diversity of research 

designs could increase the number 

and variety of applicant programs, in 

turn expanding the potential reach of 

EBPs to underrepresented 

populations. 

Across federal offices and agencies, 

most EBP review applicants initiate 

contact; applicant recruitment is not 

often necessary because potentially 

eligible programs are already aware of 

and actively pursuing EBP designation.
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 11

Applications for all federal EBP designations, including the ACL-supported EBP Review Process, 

undergo multiple rounds of review and risk analysis by 

panels of subject matter experts to identify any 

weaknesses or programmatic areas of concern. 

Interviewees across federal offices and agencies also 

shared details about specific aspects of their EBP review 

scoring processes. For example, some described sample 

size as an exclusion criterion (e.g., a minimum 

population of 75 people per group) or sample type 

exclusions (i.e., at least half of the study sample being 

composed of the target population).

Because some concerns may be attributable to application errors, all federal EBP reviews 

include an outreach process for applicants to submit corrections, if needed. Some program 

developer interviewees also described the availability of TA during ACL’s EBP Review Process, to 

ensure that applications are complete and correct. Following final review of all application 

materials and corrections, both ACL and non-ACL federal 

interviewees indicated that earning EBP designation 

requires a majority vote from the panel of subject 

matter expert reviewers.

Because these systematic review processes require such 

rigor, very few applications are successful in earning EBP 

designation. One ACL interviewee estimated that only 

about 17% of ACL EBP applicants make it through 

Stage 1 of their EBP Review Process and are invited to 

participate in Stage 2. Between the multistep 

application evaluation and the volume of applications

received (estimated at 150–300 for any given review cycle), all federal EBP reviews were said to 

take several months. For ACL’s EBP Review Process, interviewees noted that reviews can take 

as little as 5 months or as long as 15 months.

EBP Applicant Data Tracking

Given the volume of EBP applications submitted across federal EBP reviews, all ACL, non-ACL 

federal, and nonfederal stakeholder interviewees were asked about the types of data they 

collect from applicants and how those data might be used to track applicant characteristics. 

However, most interviewees noted that their applicant data collection is limited to the details 

needed to consider program applications. Rather, many of these conversations pivoted back to 

the types of data the applications collect to document specific program features (e.g., sample 

sizes, research designs, program implementation costs), with multiple interviewees noting use 

of customized spreadsheets and data tracking tools to document these program details.

When further questioned about applicant demographics, most federal and nonfederal 

stakeholder interviewees provided general answers about the most successful types of 

Successful EBP developers tend to 

represent academic institutions or

large community-based organizations 

that have the research expertise and 

resources to implement rigorous 

experimental or quasi-experimental 

research designs. 

Because EBP designation indicates that 

a given program has not only met its 

own objectives, but also can be 

successfully implemented elsewhere, 

all federal review processes are very 

detailed and lengthy. For ACL, an 

estimated <20% of applicant programs 

are sufficiently rigorous and replicable 

to earn EBP designation.
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 12

applicant organizations. Typically, programs that earn EBP designation were said to have been 

designed by either (a) academic institutions with research staff leading the programs, or (b)

larger community-based organizations that have partnered with experts to evaluate their 

programs and publish associated findings in peer-reviewed journals. Interviewees also said that 

successful program developers tend to have a sizable resource base in terms of funding, staff, 

and population access, compared to unsuccessful program developers, which tend to be 

smaller and less resourced and serve smaller 

subpopulations. 

Reported EBP Review Successes and Challenges

All interviewees were asked about successes of the EBP 

review process. Most ACL and non-ACL federal 

interviewees referred to the ongoing evolution of their 

review processes as their greatest success, citing gains in 

efficiency, increased process transparency, and efforts to 

make the results more relevant to practitioners. These 

interviewees were confident that their review processes 

were positioned to continue evolving to meet changing community needs and greater diversity 

of EBP applicant organizations. Some federal interviewees also added that their EBP application 

and review processes have helped to expand general awareness of the need for concrete 

evidence as a component of program implementation, which has, in turn, expanded the 

number of applicants and successful EBP designations. 

Nonfederal partner interviewees also cited successes in 

streamlining their review processes. Multiple 

interviewees described more standardized review 

processes that have increased equity, such as prioritizing

applications that address gaps in specific health topics or 

populations served. These interviewees noted the 

importance of inclusivity throughout the EBP review.

However, these issues of equity and inclusivity were also cited by program developers as 

growing challenges with EBP reviews. Several developers noted barriers to meeting the strict 

requirements for EBP designation, reiterating the difficulty that many organizations have in 

recruiting research staff or partners and allocating sufficient time and funding to support all of 

the program implementation steps and analyses required to apply for EBP designation. 

Likewise, unsuccessful applicant interviewees described limited capacity to implement rigorous 

experimental or quasi-experimental designs and narrower target populations with samples too 

small to generate both intervention and control groups. 

Some program developers highlighted the fact that although they have evidence of program 

success, they felt their program designs (i.e., non-RCT designs) could not meet EBP 

requirements, meaning they did not even attempt to submit applications. This perception of 

likely failure prevents many small organizations from applying for EBP designation; yet many of 

EBP review successes include growing 

awareness of the importance of 

evidence in program design and 

greater priority given to applications 

that may increase support for diverse 

populations.

Although ACL and other federal EBP 

review processes include multiple 

steps to ensure program rigor, these 

complex applications and program 

requirements may disincentivize 

smaller programs, including Tribal 

entities and others who serve diverse 

subpopulations, from applying for EBP 

designation.
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 13

these smaller organizations serve historically marginalized or underrepresented communities

that federal offices and agencies are increasingly trying to support. As one program developer 

shared, “I would much rather continue to deliver the program that’s so impactful to rural 

programs... than say, ‘Oh, no, let’s take a hiatus on the program and spend a year or two doing 

a randomized control trial.’” From a cost–benefit perspective, smaller organizations choose to 

prioritize continued program delivery rather than diverting resources to pursue EBP 

designation.

Some nonfederal partners also reported that the combination of limited program developer 

time and capacity plus rigid EBP application requirements precludes potentially beneficial 

programs from being accepted, explaining, “They don’t have the access to the research 

enterprise to build the evidence base.” Federal interviewees reiterated a similar concern that 

smaller and more diverse applicants often cannot achieve EBP designation. These interviewees 

explained that EBP reviews are approached academically, including subject matter and research 

experts and focusing on traditional quantitative analyses, without considering more diverse 

outcome measures, like participant feedback. Notably, program developer interviewees who 

represented Tribal communities also underscored that the existing EBP application 

requirements for experimental and quasi-experimental research with quantitative outcomes do 

not align with their more community-driven methods for implementing and assessing

programs.

A broader challenge relates to application of EBPs after achieving designation. The purpose of 

designating EBPs is to make them accessible to replicate for a wider audience; but to achieve 

EBP designation, many programs include so many implementation steps and evaluation 

components that community organizations downstream lack the capacity to implement these 

programs on the same scale. These downstream organizations may need additional support for 

program implementation, but program developers are not incentivized or necessarily equipped 

to get their programs into the hands of communities once they are designated as evidencebased. This lack of support for downstream program implementers results in information gaps 

about who is using EBPs, how they are being used, and whether their use maintains fidelity to 

the original program design.

DISCUSSION

Through both the rapid literature review and key 

informant interviews, the team sought to explore the 

current landscape of EBP review processes and gather 

feedback regarding future considerations. These data 

collection efforts found general alignment between the 

ACL-supported EBP Review Process that NCOA facilitates 

and the EBP reviews conducted by other federal offices 

and agencies. Most federal entities, including ACL,

prioritize systematic, multistage EBP reviews that require

experimental or quasi-experimental research designs 

After achieving EBP designation, 

programs are meant to be applied 

broadly across communities, but 

community-based organizations may 

lack the needed supports to 

implement an EBP fully and maintain

fidelity to the original design.
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 14

with validated and replicable results. This structure ensures that designated EBPs are rigorous 

and well designed to support the needs of their intended audiences.

However, all interviewees recognized the need to begin rethinking these existing EBP 

definitions and review processes. In particular, the requirement for experimental or quasiexperimental evidence results in a level of rigor that may enhance program credibility, but also 

serves as a barrier for many prospective program developers. Current EBP designation tends to 

be limited to academic institutions or larger community organizations that have the capacity to 

meet stringent EBP review requirements (e.g., RCT or QED, peer-reviewed publications); 

smaller organizations or those for whom control- or comparison-group designs may not be 

appropriate have had limited success in achieving EBP designation. By relaxing some of the 

standards typically applied by federal agencies, EBP reviews may reduce barriers for program 

developers with limited research capacity or those for whom RCTs or QEDs are incompatible 

with the communities they serve (e.g., Tribal organizations). Likewise, populating the review 

panels with both subject matter experts and members of the communities served by the 

programs could enhance equity, while increasing accessibility for downstream community 

program implementation.

Some federal offices and agencies are already making these transitions, most notably in the 

breadth of studies that are included in their review processes, as detailed in Figure 1. 

Figure 1. Federal EBP Review Process Flexibility

Note: Centers for Disease Control and Prevention (CDC); Substance Abuse and Mental Health Services 

Administration (SAMHSA); Office of the Assistant Secretary for Health (OASH); Health Resources and Services 

Administration (HRSA); Institute of Education Sciences (IES); Administration for Community Living (ACL)

Applying a tiered or leveled approach, some federal offices and agencies have not only a

formally designated EBP status for rigorous and quantitative study designs, but also a separate, 

lower designation for study designs that may be promising, emerging, or up-and-coming. This 

secondary review status considers other study designs and data collection types (e.g., 

pre-/post-test without control group, qualitative participant feedback) and expands the pool of 

prospective applicants to include more diversity of program developer organizations.
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 15

Alternatively, some entities are also shifting to a review of “study approaches,” rather than 

specific programs, as a means of balancing breadth and depth while producing guidance that 

provides prospective implementers with more flexibility to suit their community needs. These 

review processes do not systematically engage program developers, which precludes the need 

for an application process and TA. Rather, agencies take a proactive approach to researching 

available models, reviewing, and providing publicly shared feedback (e.g., pros/cons, promising 

components, potential concerns). However, this method also requires substantial staff time, 

and since it is agency-based, there is a chance of overlooking programs that have not been 

published or publicized.

Beyond individual office or agency changes, several interviewees noted a desire for increased

collaboration or consolidation across review processes within the federal government. Federal 

EBP review teams could communicate regularly to support program developments and 

evolutions that respond to community-level changes, such as offering a tiered review process 

that extends a degree of recognition to smaller programs or facilitates programming that 

supports equity goals in reaching diverse populations.

In considering potential opportunities to advance the ACL-supported EBP Review Process, ACL 

may consider aligning with other federal agencies that are in the process of reviewing or 

updating their definitions of evidence. Likewise, ACL may consider modifying their processes for

identifying and designating EBPs to include a tiered approach that might extend a lower level of 

EBP designation to a broader range of program developers. 
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 16

REFERENCES

Administration for Children & Families (ACF). (2022). Early childhood home visiting models: 

Reviewing evidence of effectiveness. Available at 

https://homvee.acf.hhs.gov/sites/default/files/2022-

11/homvee_summary_brief_nov2022.pdf

Administration for Children & Families (ACF). (2023). Models eligible for Maternal, Infant, and 

Early Childhood Home Visiting (MIECHV) funding. Available at 

https://homvee.acf.hhs.gov/HRSA-Models-Eligible-MIECHV-Grantees

Administration for Community Living (ACL). (2023). Advancing Aging Network Capacity to 

Recognize and Support Family, Kinship, and Tribal Caregivers. Available at

https://acl.gov/grants/advancing-aging-network-capacity-recognize-and-support-familykinship-and-tribal-caregivers

Administration for Community Living (ACL). (2015). Aging and Disability Evidence-Based 

Programs and Practices (DEPP) guide to reviewing evidence-based programs. Available 

at https://acl.gov/sites/default/files/programs/2017-03/ADEPP-Guide-15.pdf

Administration for Community Living (ACL). (2022a). Health promotion. Available at 

https://acl.gov/programs/health-wellness/disease-prevention

Administration for Community Living (ACL). (2022b). Older Americans Act. Available at 

https://acl.gov/about-acl/authorizing-statutes/older-americans-act

Baron, J. (2018). A brief history of evidence-based policy. AAPSS, 678, 40-50. 

doi:10.1177/0002716218763128

Best Practice Caregiving. (2022). Find evidence-based programs. Available at 

https://bpc.caregiver.org/#methodology

Carande-Kulis, V., Elder, R. W., & Matson-Koffman, D. (2022). Standards required for the 

development of CDC evidence-based guidelines. US Department of Health and Human 

Services/Centers for Disease Control and Prevention. MMWR, 71(1), 1-12.

CEBC. (2023). Welcome to the CEBC: California evidence-based clearinghouse for child welfare. 

Available at https://www.cebc4cw.org/

Derzon, J., & Menne, H. (2022). Implementation science: An approach for evaluating aging 

policies and services. Public Policy & Aging Report, 32(1), 31-35.

Fixsen, D., Blase, K., Metz, A., & Van Dyke, M. (2013). Statewide implementation of evidencebased programs. Exceptional Children, 79(2), 213-230.
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 17

Haby, M., Chapman, E., Clark, R., Barreto, J., Reveiz, L., & Lavis, J. (2016). What are the best 

methodologies for rapid reviews of the research evidence for evidence-informed 

decision making in health policy and practice: A rapid review. Health Res Policy Syst, 

14(1):83.

Institute of Education Sciences (IES). (2023). What Works Clearinghouse. Available at 

https://ies.ed.gov/ncee/wwc/FWW

Mayo-Wilson, E., Grant, S., & Supplee, L. H. (2021). Clearinghouse standards of evidence on the 

transparency, openness, and reproducibility of intervention evaluations. Prevention 

Science, 23, 774-786. doi:10.1007/s11121-021-01284-x 

National Cancer Institute. (n.d.). EBCCP submission and review process: A guide for program 

developers. Available at https://ebccp.cancercontrol.cancer.gov/reviewProcess.do

National Council on Aging (NCOA). (2022a). Apply to become an evidence-based program.

Available at https://www.ncoa.org/article/apply-to-become-an-evidence-basedprogram

National Council on Aging (NCOA). (2022b). Guidelines for operationalizing ACL criteria for 

evidence-based programs. Available at https://www.ncoa.org/article/guidelines-foroperationalizing-acl-criteria-for-evidence-based-programs

National Council on Aging (NCOA). (2022c). About evidence-based programs. Available at

https://www.ncoa.org/article/about-evidence-based-programs

National Institute of Justice (NIJ). (n.d.). CrimeSolutions. Available at 

https://crimesolutions.ojp.gov/

Office of Elementary & Secondary Education. (2023). Education innovation and research: 

Awards. Available at https://oese.ed.gov/offices/office-of-discretionary-grants-supportservices/innovation-early-learning/education-innovation-and-research-eir/awards/

Older Americans Act (OAA), III-D U.S.C. § 361 (amended 2020). Available at 

https://acl.gov/sites/default/files/about-acl/2020-

04/Older%20Americans%20Act%20Of%201965%20as%20amended%20by%20Public%20

Law%20116-131%20on%203-25-2020.pdf

Penn State University. (2023). Results FirstTM clearinghouse database. Available at 

https://evidence2impact.psu.edu/what-we-do/research-translation-platform/resultsfirst-resources/clearing-house-database/

Riessman, C.K., (2008). Narrative Methods for the Human Sciences. Sage Publications.
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 18

Schneider, E., DiCocco, C., & Haynes, M. (2019). Understanding How Programs Are Added to the 

Approved “Evidence-Based” List. [PDF] Presented on June 19, 2019 at Age + Action, 

NCOA, Washington, D.C. 

https://na.eventscloud.com/file_uploads/ccdb03463d8c70ce53aa9769248fcb16_ESchn

eideretal_Evidence-BasedHealthPromotionandDiseaseReviewProcessCombined.pdf

Self-Management Resource Center (SMRC). (2023). Chronic Disease and Caregiver SelfManagement Workshops. Available at https://selfmanagementresource.com/

Spera, C., DiTommaso, A., Morris-Hyde, M., & Zandniapour, L. (2015). The Social Innovation 

Fund: Pioneering an evidence-based investment model. Corporation for National and 

Community Service, Office of Research and Evaluation, Washington, DC.

Substance Abuse and Mental Health Services Administration (SAMHSA). (2023). About the 

Evidence-Based Practices Resource Center. Available at https://www.samhsa.gov/ebpresource-center/about

Suicide Prevention Resource Center (SPRC). (2020). Applying to the SPRC Best Practices 

Registry. Available at https://sprc.org/bpr-submission

Tomioka, M., & Braun, K. L. (2013). Implementing evidence-based programs: A four-step 

protocol for assuring replication with fidelity. Health Promotion Practice, 14(6), 850-858. 

doi:10.1177/1524839912469205

Tricco, A., Langlois, E., & Straus, S. (2017). Rapid reviews to strengthen health policy and 

systems: A practical guide. World Health Organization. ISBN: 978-92-4-151276-3 

U.S. Department of Education. (n.d.). Funding 101. Available at 

https://www2.ed.gov/documents/funding-101/funding-101.pdf
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 19

APPENDIX A

LITERATURE REVIEW: PUBLICATION IDENTIFICATION GUIDELINES

The following table describes the search terms and parameters used to identify peer-reviewed 

and gray literature.

Source Search Terms

Peer-Reviewed Literature

PubMed ((“federal” AND “evidence-based” OR “evidence 

based” OR “EBP”) (OR “title III-D” OR “NCOA” OR 

“national council on aging” OR “ACL” OR 

“administration for community living”) AND 

“process” OR “program” OR “review” OR 

“protocol”)) ((“title III-D” AND “history”)) 

NOT(“Comment”[Publication Type] OR 

“Letter”[Publication Type]) Filters: English

Grey Literature

Administration for Community Living

William T. Grant Foundation

Best Practice Caregiving

AmeriCorps

National Council on Aging

Older Americans Act

Center for Disease Control and Prevention

Health Resources and Services 

Administration

Substance Abuse and Mental Health 

Services Administration

Corporation for National and Community 

Service

Education’s Evidence-Based Clearinghouse

Supplemental Nutrition Assistance 

Program Education

Office of Population Affairs

(“evidence-based” OR “evidence based” OR 

“EBP” AND “federal”) AND (“review” OR 

“process”) OR (“history”)

Parameters: Published within the last 10 years; Focus on EBP Review Process and best 

practices; Based in the United States; English language
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 20

APPENDIX B

LITERATURE REVIEW: PUBLICATION SELECTION MATRIX

The following figure depicts the processes used to identify, screen, and include peer-reviewed 

articles and gray literature sources.
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 21

APPENDIX C

INTERVIEWS: NVIVO CODEBOOK

The following figure shows the structure of the NVivo codebook created to analyze findings 

from across all five types of interviews.

Federal and Nonfederal Partner Codebook Program Developer Codebook

1. Participant Role

a. Description of role

1. Participant Role

a. Description of role

2. Meaning of Evidence-Based

a. Definition of evidence-based

b. Evolution of evidence-based 

definition

c. Reflections on evidence-based 

definition

2. Meaning of Evidence-Based

a. Definition of evidence-based

b. Evolution of evidence-based 

definition

c. Reflections on evidence-based 

definition

3. EBP Review Process Steps

a. Attracting or identifying programs

b. Materials reviewed

c. Eligibility and scoring

d. Review and TA

e. Feedback

f. Data Collected

g. Tracking

3. Applying for ACL EBP Designation

a. Awareness of process

b. Decision to apply

c. Preparing an application

d. Review and TA

e. Follow-up after approval

f. Multiple applications

g. Non-ACL applications

h. Additional information on applying 

for ACL

4. EBP Review Process Reflections

a. Evolution of review process

b. Policy influence on process

c. Successes

d. Challenges

e. Future changes desired or planned

f. Additional thoughts on EBP review 

process

4. Feedback on EBP Designation and Process

a. Advice to program developers

b. Benefits of EBP designation

c. Barriers to applying

d. Future changes desired

e. Additional thoughts

5. Great Quote 5. Great Quote

6. ACL Interviewee 6. Developer with Approved Program

7. Federal Interviewee 7. Developer without Approved Program

8. Nonfederal Partner 8. Tribal Program Developer
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 22

APPENDIX D 

SUMMARY OF FEDERAL AND NONFEDERAL EBP REVIEW PROCESSES

Table A provides a breakdown of EBP review processes across the federal and nonfederal partners discussed in this report.

Table A. Summary of Federal and Nonfederal EBP Review Processes

What is 

being 

designated 

as effective

Rating scale

Selecting 

programs/approach

es for review

Review Process, including 

reviewer (and Timeline)

Accepted study 

types/Criteria

Required 

materials/rubric Grant maker

CDC Community Guide1

Approaches/

categories 

of programs 

and policies

Approaches are 

assigned the 

following rating: 

recommended 

(strong or 

sufficient), 

recommend 

against, or 

insufficient 

evidence. 

Review topics are

based on

Community 

Preventive Services 

Task Force (CPSTF) 

prioritization; the 

Guide does not 

accept applications 

from program 

developers. 

CPSTF members review and 

discuss the evidence 

generated by a targeted 

literature search, consider 

input from partners, and issue 

an opportunity or finding 

based on the strength and 

consistency of the 

effectiveness evidence.

The intervention approaches 

selected for CPSTF reviews 

aim to improve population 

health under the following 

categories:

• Services

• Behavioral or social 

programs

• Environmental or policy 

The evidence 

decision table 

displays evidence of 

an intervention’s 

effectiveness based 

on the suitability of 

study design and 

quality of execution 

of the body of 

evidence and 

consistency of the 

results and 

meaningfulness of 

the effect. 

No; 

organizations 

that wish to 

fund EBPs or 

help fill 

evidence gaps 

can use The 

Community 

Guide to inform 

decisions and 

justify 

interventions.

EBPRC (SAMHSA)2

Approaches/

categories 

of programs 

and policies

Approaches or 

strategies with 

rigorous evidence 

are selected for 

evidence-based 

resource guides; 

some strategies 

with promising 

evidence

Review topics are

based on SAMHSA 

prioritization; the 

EBPRC does not 

accept applications 

from program 

developers, nor do 

they review 

individual programs. 

A targeted literature search 

generates evidence for 

review; expert panels of 

federal, state, and nongovernmental participants 

provide input for each guide. 

Focus on RCT and QED, 

though other study types 

(e.g., retrospective chart 

review, mixed methods) are 

reviewed. 

No; 

organizations 

that wish to 

fund EBPs or 

help fill 

evidence gaps 

can use the 

EBPRC to select 

approaches. 
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 23

What is 

being 

designated 

as effective

Rating scale

Selecting 

programs/approach

es for review

Review Process, including 

reviewer (and Timeline)

Accepted study 

types/Criteria

Required 

materials/rubric Grant maker

Home Visiting Evidence of Effectiveness (HomVEE)3

"Models" 

which may 

be specific 

packaged 

programs

Found to meet 

HHS criteria for 

evidence-based 

models

Identify manuscripts 

about home visiting 

models through a 

database search and 

submissions to its 

annual call for 

research.

Screens and assigns

manuscript-level points, then 

selects models to review by 

prioritizing ones with the 

highest scores of two tracks; 

Track 1 is for models that 

were not previously found to 

be evidence-based and Track 

2 updates the review of 

literature on models that 

meet the HHS criteria.

Issues a call for research 

every year in August that is 

open through September. 

Research can be submitted at 

any time, but if it is submitted 

outside of the call window, it 

will be screened the following 

year. HomVEE will release 

results for models that are 

not evidence-based in 

September of each year. 

Updates to selected evidencebased models will be released 

by December of each year. 

For models that have 

submitted an appeal, 

HomVEE will issue a final 

decision within 60 days of 

submission.

RCTs and QEDs are accepted 

(single-case, regression 

discontinuity, nonexperimental comparisongroup designs).

For criteria, models must 

meet at least one of the 

following: (1) at least one 

high- or moderate-rated 

impact study of the model 

finds favorable (statistically 

significant) impacts in two or 

more of the eight outcome 

domains; or (2) at least two 

high- or moderate-rated 

impact studies of the model 

(using non-overlapping 

analytic study samples) find 

one or more favorable 

(statistically significant) 

impacts in the same domain.

An electronic 

version of 

manuscript and 

cover email 

including contact 

information, name 

of early childhood 

home visiting model 

being evaluated, and 

study design are 

required. 

Yes; to be 

eligible for 

implementation 

as an evidencebased model 

with MIECHV 

funding, a 

model must 

both meet HHS 

criteria for 

evidence of 

effectiveness 

(as determined 

by HomVEE) 

and meet all 

other statutory 

requirements 

for model 

eligibility (as 

required by 

HRSA).
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 24

What is 

being 

designated 

as effective

Rating scale

Selecting 

programs/approach

es for review

Review Process, including 

reviewer (and Timeline)

Accepted study 

types/Criteria

Required 

materials/rubric Grant maker

NCOA for ACL4

Specific 

packaged 

programs

Evidence-based or 

not

Program developers 

submit applications. 

The review process is a twostage application focused on 

the effectiveness and 

research-based evidence of 

the program as well as 

implementation and 

dissemination readiness.

The Review Council contains 

members with expertise in 

research and program 

evaluation.

(1) Demonstrated through 

evaluation to be effective for 

improving the health and 

well-being or reducing 

disease, disability, and/or 

injury among older adults

(2) Proven effective with 

older adult population, using 

experimental or quasiexperimental design

(3) Research results 

published in a peer-review 

journal

(4) Fully translated in one or 

more community site(s)

(5) Includes developed 

dissemination products that 

are available to the public

Demonstrated 

through evaluation 

to be effective for 

improving the 

health and wellbeing or reducing 

disease, disability 

and/or injury among 

older adults; and

Proven effective 

with older adult 

population, using 

experimental or 

quasi-experimental 

design; and

Research results 

published in a peerreview journal; and

Fully translated in 

one or more 

community 

site(s); and

Includes developed 

dissemination 

products that are 

available to the 

public.

Yes; through 

meeting OAA 

Title III-D 

evidence-based 

requirements.
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 25

What is 

being 

designated 

as effective

Rating scale

Selecting 

programs/approach

es for review

Review Process, including 

reviewer (and Timeline)

Accepted study 

types/Criteria

Required 

materials/rubric Grant maker

Ed (WWC)

5

Programs, 

policies, or 

practices—

intervention 

reports, 

reviews of 

individual 

studies, and 

practice 

guides are 

created to 

present 

findings.

Research rating 

levels: (1) meets 

WWC standards 

without 

reservations; 

(2) meets WWC 

standards with 

reservations; 

(3) does not meet 

WWC Standards

The WWC reviews 

existing, publicly 

available research in 

education. To be 

reviewed by the 

WWC, studies need 

to be publicly 

available and 

eligible for review 

under the current 

version of the Study 

Review Protocol. 

When selecting 

manuscripts for 

review, the WWC 

favors studies with a 

final study report or 

peer-reviewed 

manuscript in ERIC.

Hundreds of trained and 

certified reviewers read and 

rate studies, determine 

whether their methods are 

rigorous; those with rigor are 

summarized.

The WWC uses the Study 

Review Protocol to review

individual studies and studies 

reviewed as part of synthesis

products, such as practice 

guides. Eligible studies 

advance to the next phase in 

the review process, during 

which the eligible findings 

from studies are assessed 

according to WWC standards. 

In the final phase of the 

review process, the WWC 

synthesizes and reports two 

sets of results: (a) a research

rating based on the strength 

of the research design and its 

execution, and (b) an 

effectiveness rating based on 

the evidence of favorable 

effects from the intervention. 

Publicly available 

study and eligible 

for review under 

WWC’s Study 

Review Protocol.

No
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 26

What is 

being 

designated 

as effective

Rating scale

Selecting 

programs/approach

es for review

Review Process, including 

reviewer (and Timeline)

Accepted study 

types/Criteria

Required 

materials/rubric Grant maker

Best Practice Caregiving Methodology6

Specific 

packaged 

programs

Evidence-based or 

not

Programs are 

identified through 

online databases of 

published articles, 

bibliographies in 

published metaanalyses or 

systematic reviews, 

reviewing 

completed grants 

funded by the ACL, 

National Institutes 

of Health, and 

Department of 

Veterans Affairs, 

reviewing 

presentations given 

for dementiaadvocacy 

committees, 

opportunities from 

the project’s expert 

Advisory Committee 

and funding 

organizations, and 

informal discussions 

with experts in 

dementia care and 

services.

After eligible programs are 

identified, program 

developers/distributors 

consent to inclusion and 

provide required materials to 

the project team.

Programs must meet the 

following criteria: One or 

more completed randomized 

or nonrandomized controlled 

trial, or pre/post-test study 

with no control group, that: 

had a sample with at least 

50% of caregivers assisting a 

relative or friend living with 

dementia, was conducted in 

the United States, had at least 

one statistically significant, 

published, beneficial 

caregiver outcome. One or 

more implementations that 

delivered the program as part 

of an organization’s regular 

service portfolio.

Able to offer permission or a 

license to offer the program, 

along with any required 

delivery tools (e.g., manuals, 

training, record keeping 

systems).

Program manuals 

and delivery tools, a 

complete 

bibliography of 

published articles 

and a list of 

completed and 

underway research 

studies, and a list of 

all organizations 

that delivered the 

program in the past 

12 months.

No
Analysis of Evidence-Based Health Promotion and Disease Prevention Review Process

Final Report 27

What is 

being 

designated 

as effective

Rating scale

Selecting 

programs/approach

es for review

Review Process, including 

reviewer (and Timeline)

Accepted study 

types/Criteria

Required 

materials/rubric Grant maker

AAEBI (OAAA)

7

Specific 

packaged 

programs

Evidence-based or 

not

Program developers 

may submit 

programs for 

evaluation. 

The review process is as 

follows: (1) identify programs 

for review and conduct high 

level review that meet 

minimum criteria; (2) recruit 

and screen minimum of three 

external reviewer; 

(3) complete intervention 

summaries and send program 

developers for review and 

updates for OAAA; (4) send 

program materials and 

evaluation forms to 

reviewers, who evaluate and 

submit opportunities; (5) 

engage additional reviewers if 

primary did not unanimously 

agree; (6) determine final 

opportunity. 

Programs should meet the 

following criteria: 

(1) published in a peerreviewed journal; (2) pilot 

study must include 75 or 

more subjects in the 

treatment group; (3) pre- and 

post-test (at least 3 months 

post measurement; 

(4) measure at least two 

arthritis-relevant outcomes; 

(5) statistically significant 

changes in at least two 

arthritis relevant outcomes; 

(6) research has to include 

people with arthritis; 

(7) consistency of evidence; 

(8) reasonable rigor; 

(9) infrastructure for support 

program dissemination. 

The following 

materials are 

required: 

(1) intervention

overview (required), 

(2) research 

article(s) concerning 

the intervention 

(one is required; 

max of three 

permitted).

Additional 

documents are 

optional.

Yes; with CDC 

funding, state 

programs have 

expanded 

access to 

AAEBIs in the 

United States.

1 https://www.thecommunityguide.org/pages/community-guide-methodology.html 2 https://www.samhsa.gov/ebp-resource-center/about 3 https://homvee.acf.hhs.gov/sites/default/files/2021-11/HomVEE-Handbook-v2.1-Nov-2021.pdf 4 https://www.ncoa.org/article/apply-to-become-an-evidence-based-program 5 https://ies.ed.gov/ncee/wwc/WhatWeDo; https://ies.ed.gov/ncee/wwc/Docs/referenceresources/Final_WWC-HandbookVer5_0-0-508.pdf; 

https://ies.ed.gov/ncee/wwc/FAQ

6 https://benrose.org/best-practice-caregiving ; https://bpc.caregiver.org/#methodology 7 https://oaaction.unc.edu/wp-content/uploads/sites/623/2021/11/AAEBI-Review-Criteria.pdf

Note (in alphabetical order): Arthritis-Appropriate, Evidence-Based Interventions (AAEBI); Administration for Community Living (ACL); Centers for Disease Control and 

Prevention (CDC); Community Preventive Services Task Force (CPSTF); Evidence-Based Program (EBP); Evidence-Based Practice Resource Center (EBPRC); U.S. 

Department of Health and Human Services (HHS); Home Visiting Evidence of Effectiveness (HomVEE); Health Resources and Services Administration (HRSA); 

Maternal, Infant, and Early Childhood Home Visiting (MIECHV); National Council on Aging (NCOA); Older Americans Act (OAA); Osteoarthritis Action Alliance 

(OAAA); Quasi-experimental design (QED); Randomized control trial (RCT); Substance Abuse and Mental Health Services Administration (SAMHSA); What Works 

Clearinghouse (WWC)