Analysis of Evidence-Based Health 

Promotion and Disease 

Prevention Program Review 

Processes

September 2023

Contract Number: HHSP233201500039I / 75P00120F37027
Table of Contents

1. Background 2. Methodology 3. Findings 4. Discussion

2
Background

Since 1987, Title III-D of the Older Americans Act (OAA) has authorized 

grants to states and territories to support programs that promote health 

and well-being among older adults (60+). 

An FY2012 congressional appropriations law added a requirement that 

these health promotion programs funded via OAA must be evidencebased. 

The Administration for Community Living (ACL) and key stakeholders worked to 

establish detailed criteria and evidence-based definitions to support Title III-D 

programming and create a formal ACL Evidence-Based Program (EBP) Review 

Process. 

In 2022, ACL sought to examine the existing Review Process, facilitated by the 

National Council on Aging (NCOA), to explore whether any updates may be 

needed and to learn more about EBP review efforts employed across other federal 

entities. 

ACL contracted with RTI International to conduct this exploratory study. 

3
Background: Research Questions

• What processes do other federal agencies use to 

determine evidence-based approval?

• What does the EBP application and review process 

include? 

• What is the history of the Title III-D EBP Review 

Process? 

• What was NCOA’s process? 

• What types of entities are developing programs? 

What are their protocols? What are their 

standards?

4
Methodology

5

Between February and June 2023, the team conducted a 

rapid literature review and key informant interviews to 

address the research questions. 

The rapid literature 

review provided 

details and historical 

context for EBP 

review processes, 

both within ACL and 

across other federal 

agencies and offices.

The key informant 

interviews offered 

specific details 

about how EBP 

applications are 

reviewed and how 

program applicants 

experience the EBP 

review process.
Methodology: Literature Review

The team conducted a rapid literature review to 

investigate the history of the ACL EBP Review 

Process and current processes used to review and 

evaluate EBPs across federal entities. 

The team identified 13 peer-reviewed 

publications and 32 gray literature publications 

for potential inclusion, screening each for 

relevance. 

The team completed a qualitative thematic 

analysis on the final 27 publications selected 

for inclusion. 

6
Methodology: Stakeholder Interviews

The team conducted 33 virtual interviews with stakeholders including ACL 

staff, non-ACL federal staff, nonfederal partners, and program developers 

who either successfully obtained or did not receive EBP designation 

through the ACL-supported EBP Review Process. Interview transcripts 

were coded using NVivo.

Interviewee Group Interviews 

Conducted (N)

ACL Staff 4

Other Federal Partners 6

Nonfederal Partners 7

Successful ACL EBP Developers 8

*Unsuccessful ACL EBP Developers 8

Total 33

*Note: Two types of participants were classified as unsuccessful: ones who applied and did not 

achieve EBP designation (N=4), and ones who considered applying but did not because they felt 

their programs would not be successful in earning EBP designation (N=4). 7
Findings: EBP Application Process

The ACL-supported EBP application consists of two separate stages. 

ACL funds NCOA to review applications at both stages and determine 

which programs should receive EBP designation.

Stage 1 requires program developers to detail their research-based evidence, program 

effectiveness, and associated evaluation results. This stage is further divided by 

program focus, with one application for falls prevention programs and a separate 

application for all other health promotion and disease prevention programs. NCOA’s 

Review Council, composed of experts in research and evaluation, reviews all Stage 1 

applications. 

Stage 2 requires successful Stage 1 applicants to provide additional 

information on program implementation, training, and dissemination. Based 

on material submitted in Stage 2, NCOA’s Review Council makes final 

determinations as to which applicant programs achieve EBP designation. 

8
Findings: EBP Review Process

The ACL-supported EBP Review Process, administered by NCOA, awards EBP 

designation to falls prevention and health promotion programs that demonstrate 

rigor through (1) use of experimental or quasi-experimental research designs, (2) 

demonstrated dissemination, and (3) published program outcomes in peerreviewed journals.

The purpose of the EBP Review Process is to identify programs that 

have proven success in designing and implementing a model that 

meets their stated goals to support their target population, and can 

be replicated in other communities with the same successful results. 

Based on peer-reviewed and gray literature sources, ACL’s EBP Review 

Process is similar to that used by other federal offices and agencies, 

such as CDC, SAMHSA, and HRSA. 

9
Findings: Definitions of EBP 

10

When asked to define “evidence-based,” ACL and non-ACL federal staff 

interviewees cited traditional approaches to EBP review, like the ACL-supported 

EBP Review Process. Interviewees focused on the rigor of the research designs 

(i.e., randomized controlled trials or well-designed quasi-experimental studies). 

Nonfederal partners and EBP program applicants also focused on 

robust research, but many added comments about the importance of 

including contextual factors and community perspectives.

Some interviewees also said that broadening the definition of EBP to 

include more diversity of research designs could increase the number 

and variety of applicant programs, in turn expanding the potential 

reach of EBPs to underrepresented populations. 
Findings: EBP Applicant Characteristics 

ACL, non-ACL federal, and nonfederal stakeholder interviewees 

described the profile of the most successful EBP applicant 

organizations.

Successful EBPs have been designed either by (a) academic institutions with 

research staff leading the programs, or (b) larger community-based 

organizations that have partnered with research experts to design their 

programs and publish program evaluation findings in peer-reviewed journals.

Successful EBP developers tend to have a sizable resource base in terms of 

funding, staff, and population access compared to unsuccessful program 

developers, which tend to be smaller and less resourced and often serve 

smaller subpopulations. 

11
Findings: EBP Review Successes

EBP review successes include growing awareness of the importance of 

evidence in program design and greater priority given to applications that 

may increase support for diverse populations.

Most ACL and non-ACL federal interviewees referred to the evolution 

of EBP reviews as their greatest success, citing gains in efficiency, 

increased transparency, and efforts to make results more relevant to 

practitioners. 

Some federal interviewees said that their EBP application and review 

processes have helped expand awareness of the need for evidence in 

program implementation, in turn expanding the number of EBP 

applicants and successful EBP designations.

Multiple nonfederal partner interviewees described successes in EBP 

application review structures, such as prioritizing applications that 

address gaps in specific health topics or populations served. 

12
Findings: EBP Review Challenges 

Issues of equity and inclusivity were cited by program 

developers as growing challenges with EBP reviews. 

Many small and diverse organizations lack research staff, resources, and 

funding to support all the program implementation steps and analyses 

required to apply for EBP designation—potentially precluding some 

innovative programs from EBP consideration.

Rigorous experimental or quasi-experimental designs are time-consuming and 

expensive to implement and evaluate, and for some entities (e.g., Tribal 

communities), these research designs do not align with community-driven 

methods for implementing and assessing programs.

To achieve EBP designation, many programs include so many 

implementation steps and evaluation components that community 

organizations downstream lack the capacity to implement these programs 

on the same scale.

13
Findings: The Future of EBP Review

Although the ACL-supported EBP Review Process is similar to processes used by 

other federal entities, some federal entities are transitioning to a tiered approach 

that promises to increase the number of EBPs.

Under a tiered EBP review structure, the most rigorous studies (i.e., 

those that meet traditional criteria, including rigorous research design) 

receive full EBP designation. 

Approaches that do not meet all the standard EBP criteria may still 

earn recognition through a lower tier designation that acknowledges 

their potential as promising programs.

14
Discussion

Through the literature review and interviews, the team explored the current 

landscape of EBP review processes and gathered feedback on future 

considerations.

• Overall Structure: There is general alignment between the systematic, multistage 

ACL-supported EBP Review Process that NCOA facilitates and the EBP reviews 

conducted by other federal offices and agencies. This structure ensures that 

designated EBPs are rigorous and well designed to support the needs of their 

intended audiences. 

• Barriers: Although the requirement for experimental or quasi-experimental 

evidence results in a level of rigor that may enhance program credibility, it also 

serves as a barrier for many prospective program developers who cannot achieve 

EBP designation. Current EBP designation tends to be limited to academic 

institutions or larger organizations that have the capacity to meet stringent EBP 

review requirements; smaller organizations or those for whom control- or 

comparison-group designs may not be appropriate have had limited success in 

achieving EBP designation.

15
Discussion, Cont.

16

All interviewees recognized the need to start transitioning EBP reviews 

to reduce barriers. Some federal offices and agencies are already 

making these transitions, most notably with tiered approaches that 

expand the breadth of studies that can apply for EBP designation.

Discussion 
Discussion: Opportunities

17

Federal offices and agencies may consider the following 

opportunities:

• Review or update their definitions of evidence, potentially 

expanding their definitions to include other types of 

program designs (e.g., inclusion of qualitative studies).

• Work together with other federal offices to increase 

collaboration across federal EBP review processes to share 

lessons learned and new developments.

• Develop tiered approaches to retain the existing level of EBP 

rigor, while also extending a lower level of EBP designation 

to a broader range of program developers.