2023 Evaluation Plan

Introduction

All Americans—including people with 

disabilities and older adults—should be 

able to live at home with the supports 

they need, participating in communities 

that value their contributions. To help 

meet these needs, the U.S. Department 

of Health and Human Services (HHS) 

created the Administration for 

Community Living (ACL) in 2012.

ACL brings together the efforts and 

achievements of the Administration on 

Aging (AoA), the Administration on 

Disabilities (AOD), the Center for 

Innovations and Partnership (CIP) and 

the National Institute on Disability,

Independent Living, and Rehabilitation 

Research (NIDILRR) to serve as the 

Federal agency responsible for 

increasing access to community 

supports, while focusing attention and 

resources on the unique needs of older 

Americans and people with disabilities 

across the lifespan. By funding services 

and supports provided by networks of 

community-based organizations, and 

with investments in research, 

education, and innovation, ACL helps 

make this principle a reality for millions 

of Americans.

Planning Activities

ACL will conduct the following planning 

activities in order to ensure that a 

variety of viewpoints and voices are 

represented in the agency’s evaluation 

work moving forward and to guarantee 

that issues of concern for a variety of 

the agency’s stakeholders – both 

internally and externally – are taken 

into account. 

In an effort to make sure that a wide 

variety of vantage points are captured, 

ACL will engage in planned outreach 

and engagement of a variety of 

stakeholder groups. ACL has already 

engaged internal stakeholders at ACL 

through a vigorous Learning Agenda 

process started in 2018 and developed 

a list of priority areas for the agency to 

be revisited every four years. Within 

these priority areas there are specific 

areas of programmatic refinement and 

ACL will be working closely with 

programs and leadership to understand 

where there are additional areas for 

research and evaluative work. 

This close work within ACL will support 

the creation of Research Agendas 

within program offices or around 

particular topics of interest nested 

within the priority areas presented in 

the Learning Agenda. Other Research 

Agendas are renewed attention to 

issues of equity, diversity, and inclusion 

with a specific focus on Lesbian, Gay, 

Bisexual, Transgender, Queer (LGBTQ+) 

populations. ACL is drafting a specific 

LGBTQ+ Research Agenda to guarantee 

that this population receives special 

attention in our work moving forward. 

As these internal working documents 

are implemented, they will be reflected 
in ACL Learning Agenda learning 

activities and ACL’s annual evaluation 

plans, as appropriate.

Other work that is constantly in motion 

at ACL is building a foundation of 

information through annual 

administrative data review and 

snapshot studies. The review of annual 

administrative data is critical for ACL to 

understand the performance of 

programs and activities undertaken by 

the agency with regard to what 

populations we are serving and how we 

are serving them. Snapshot studies 

allow for relatively low-cost, small 

scale, quick turnaround research 

projects that provide rich contextual 

information that is invaluable as a 

foundation for greater in-depth study 

through broader-scale evaluations. 

Using these tools, ACL is able to more 

nimbly engage in understanding 

programs and service delivery in a 

timely manner and to design better 

informed evaluations. 

Beyond the engagement of internal 

stakeholders through research agendas, 

data review, and snapshot research;

ACL intends to engage external 

stakeholders. External stakeholder 

engagement will take place through 

listening sessions, expert panels, 

requests for information (RFIs), and -

field-initiated contracts. By engaging 

the public in these ways – to speak to 

issues important to them, to comment 

officially on ACL priorities and practice, 

and by asking them to submit their own 

research or evaluation projects – ACL is 

ensuring a broad input from the aging 

and disability fields. This input will 

allow ACL to refine its work to better 

reflect the needs and realities of the 

populations we serve. 

Evaluation Questions

ACL develops the high-level questions 

that each evaluation is designed to 

address through an iterative process. 

Staff in ACL’s Office of Performance and 

Evaluation review the authorizing 

legislation and program guidance 

materials to identify each program’s 

goals and objectives. Using this 

information as a starting point, 

program staff and stakeholders, such as 

Congress, the Office of Management and 

Budget, are consulted regarding their 

information needs. To ensure the 

feasibility of each evaluation, service 

providers and recipients are also 

involved through their inclusion in 

expert panels and the process of pilot 

testing data collection tools and 

approaches.

ACL evaluation questions typically fall 

into one of four categories:

1. Describing program inputs and 

activities. These include questions 

about funding, partnerships, the

intended types and level of activities.

2. Implementation and targeting. These 

include questions about actual types 

and level of activities, variations in 

activity over time, geography, recipient 

subgroups.

3. Measuring program outcomes and 

impacts. These include questions 

about the degree to which program 

outcomes have been achieved and the 
impact of program services on the 

target populations, communities, 

and/or systems.

4. Understanding how and why 

programs do or do not produce the 

intended and unintended effects. 

These include questions about program 

context, sustainability, and replicability.

Planned Evaluation Projects

For 2023, OPE is planning for several 

evaluations, some as continuations or 

follow-ons of earlier evaluations and 

some to explore new areas of inquiry. 

These evaluation areas are outlined 

below: 

Evidence Based Programming 

Fidelity Evaluation - The is an 

evaluation of the fidelity with which 

ACL and its grantees under the Older 

Americans Act implement the required 

evidence-based programs. The 

evaluation will help ACL and our 

stakeholders understand the following:

• The process ACL staff use when 

making awards (i.e., discretionary 

grants) or verifying grantees (e.g., 

formula grantees) are using funds for 

evidence-based programming;

• How grantees select evidence-based 

programs they implement and how 

they determine what grants to which 

they will apply;

• How grantees implement those 

programs and verify they are being 

implemented in accordance with their 

original models, including the 

identification of intentional and 

unintentional adaptations;

• The strengths and weakness of the 

current ACL award and 

implementation process; and

• How and why grantees work with 

developers of the evidence-based 

program(s) to identify program 

adaptations that maintain the 

integrity of the evidence-based nature

of the program.

Upon completion, ACL will publish a 

report documenting what ACL and its 

grantees can do to improve the 

selection, implementation, and 

monitoring of evidence-based 

programming.

Aging Network Process Evaluation –

ACL is conducting a process evaluation 

to study the role of the Aging Network 

in implementing Older Americans Act 

programs and ensuring the well-being 

of older Americans. The evaluation 

seeks to determine how the Aging 

Network operates at the local, state, 

and federal levels, as well as how the 

various levels of Aging Network work 

together and how they work with other 

partners. Additionally, ACL is 

interested in learning about the role the 

Network plays in identifying and 

responding to emerging needs, how it 

measures and improves the quality of 

services, and how it measures success.

The evaluation will also investigate 

feasible ways to evaluate the Aging 

Network with regards measuring the 

value of the activities facilitated via 

Aging Network in support of 

individuals, families, and communities, 

specifically with regards to return on 

investment.

Social Determinants of Health 
Evaluability Study– The purpose of 

this contract is to begin looking at how 

the services provided by ACL grantees 

influence the social determinants of 

health (SDOH). The World Health 

Organization describes the social 

determinants of health as the 

“conditions in which people are born, 

grow, live, work, and age, including the 

health system. These circumstances are 

shaped by the distribution of money, 

power, and resources at global, 

national, and local levels, which are in 

and of themselves influenced by policy 

choices.” ACL’s programs address these 

conditions through grants designed to 

improve organizations and systems, 

and to mitigate their effects on 

individuals through the delivery 

of direct services such as providing 

nutrition, linking people to services, 

preventing/addressing violence, health 

education, mobilizing community 

partnerships, providing transportation, 

investing in economic support, social 

integration, and education, among 

many others.

Living Well Cross-site Outcome 

Evaluation – During the last two 

decades, the number of individuals with 

intellectual and developmental 

disabilities (I/DD) living in community 

settings and receiving home- and 

community-based services (HCBS) 

increased dramatically. While this led 

to better outcomes, including people 

with I/DD making more of their own 

choices and being a part of the 

community, there are great variations 

in the accessibility of quality HCBS 

across the country.

In 2017 and 2018 ACL awarded Living 

Well grants to help develop and test 

model approaches for enhancing the 

quality, effectiveness, and monitoring of 

home- and community-based services 

(HCBS) for people with developmental 

disabilities. These Model Approaches 

for Living Well grants, awarded as 

Projects of National Significance by 

ACL’s Administration on Disabilities, 

are focused on building the capacity of 

HCBS systems and enhancing 

community monitoring to prevent 

abuse, neglect, and exploitation. Key 

features of focus of the cross-site 

evaluation are: 

Partnerships, meaningful and active 

engagement with self-advocates and 

families, evidence based practices for 

service improvements, building 

capacity of direct service providers and 

HCBS providers, reducing abuse and 

neglect through community monitoring, 

addressing health and safety with data 

tools, program and outcome evaluation, 

and sustainability. 

Process and Outcome Evaluation of 

the National Paralysis Resource 

Center (NPRC) Grant Program – Using 

a multi-method approach, this project 

will assess the value of the NPRC 

highlighting approaches that are 

working well and identifying areas for 

improvement. It will examine the NPRC 

outcomes of improved health and 

quality of life of individuals living with 

paralysis of all ages, their families, and 

their support system; increased 

awareness of members of the target 

populations about paralysis; increased 
access of members of the target 

populations to services relevant to 

individuals with paralysis; increased 

empowerment, confidence, and 

independence of individuals living with 

paralysis; stronger support networks 

for individuals living with paralysis; 

and changes in community living 

opportunities for individuals living with 

paralysis. The evaluation will be 

implemented concurrently with the

grant period of the resource center.

CLAS Standards Evaluability Study–

This project seeks to explore existing 

ACL documentation and grantee 

understanding of Culturally and 

Linguistically Appropriate Service 

(CLAS) Standards, as well as a separate 

effort concerning how grantees 

operationalize CLAS Standards to 

support the communities and 

individuals they serve. There are three 

separate phases for this work. The first 

phase focuses on ACL and key 

stakeholders, the two additional phases 

target feedback from ACL grantees and 

beneficiaries.

Phase one seeks to address the 

following research questions:

1. What are the CLAS Standards? How 

are they implemented, supported, 

and measured? 

2. How is ACL supporting grantees to 

implement CLAS Standards? 

3. How can ACL enhance existing CLAS 

Standards implementation supports 

for grantees? 

4. How does ACL assess grantee 

adherence to CLAS Standards? 

5. How can ACL help grantees assess 

community needs to ensure CLAS 

Standards are being met? 

Other Evaluation Areas for 

Consideration

The following are areas of evaluation 

interest for ACL and for which ACL is 

starting to develop research and evaluation 

frameworks in order to conduct more 

robust evaluation studies in the future. 

Some of the areas outlined below have 

initial work underway in the shape of small 

studies that may be scaled up or followed 

on with continued inquiries. Others would 

be newer efforts to understand more about 

the needs of the populations that ACL 

serves and the intersections with programs 

that ACL funds. 

Housing – Understanding the impact and 

issues of housing for older adults and 

how this intersects with service delivery. 

Looking at linkages between government 

housing programs and how ACL might 

strengthen or support aging housing 

programming. 

Veterans – Understanding how aging 

Veterans are served by the aging and 

disability networks and how VA and 

network services overlap and 

interconnect. Looking at veterans with 

disabilities, aging, considering housing, 

assistive technology and looking at 

considerations for how ACL might 

strengthen or support Veterans 

programs related to aging and disability 

services.

Unmet Needs –Building from a 

literature review regarding possible 

ways to define unmet need, looking at 
ways in which OAA services can be 

adapted/expanded/differently 

targeted. This project would look to 

identify, develop, and measure the 

effectiveness of practices to address 

unmet need. 

Territories – Looking at how ACL’s 

programs are enacted within the US 

territories and the cultural and other 

considerations involved in providing 

these programs equitably. 

Formal Caregivers – Understanding 

more about this population and their 

intersection with the aging and 

disability networks, the role they play

in supporting older adults and 

individuals with disabilities to remain 

independent and live in the community, 

and identifying possible new ways that 

ACL programs can support this 

population.

Language minorities – Assessing the 

degree to which ACL services are 

accessible to people with a range of 

communication needs including limited 

English proficiency, assessing the 

availability of technical assistance for 

ACL grantees on this topic, and 

identifying areas for improvements 

among ACL programs. 

Volunteers – The aging network 

historically relies on volunteers to help 

with service provision, particularly 

around helping to provide meals and 

transportation. ACL has completed

initial work to understand some of 

these relationships within portions of 

the aging network, but there is 

additional work to be done to 

understand the full impact of 

volunteerism for the aging network; as 

well as to understand what the role of 

volunteers and their impact may be 

within the disability services that ACL 

funds. 

Commitment to Quality 

Improvement

ACL is committed to rigorous evaluation 

practices as defined in the Office of 

Management and Budget memorandum M20-12 and reflected in the ACL Evaluation 

Policy, including committing to a process of 

PDCA (plan, do, check, act). With that in 

mind, the agency is continuously working 

to improve our processes of evidence 

dissemination as well as supporting 

program offices in creating tools to ensure 

evidence-based change as a result of 

evaluation findings. ACL is looking to 

implement a process by which evaluation 

findings and recommendations would be 

revisited with programs to review uptake 

of recommended practices. Further, ACL 

will commit to shifting towards an 

evaluation process that moves beyond the 

presentation of findings and lessons 

learned and begins to work collaboratively 

with programs to find ways to implement 

recommended changes. In this way ACL 

can continue to build stronger programs 

and create an agency-wide PDCA cycle with 

multilateral support between programs 

and performance with evaluation and 

evidence as the foundation. 

For more information about ACL

U.S. Department of Health and Human Services, 

Administration for Community Living, Washington 

DC 20201

Phone: (202) 619 – 0727

Fax: (202) 357 – 3555

Email: ACLinfo@acl.hhs.gov

Web: http://www.acl.gov