1

Evaluation of the ACL Title VI Programs: 

Year 1 Interim Report

January 2018

What are the Title VI Programs? 

In 1965, the Older Americans Act (OAA) was authorized in response to concerns over the 

lack of social service programs and protections for older people. The OAA protects the 

wellbeing of older Americans through community and home-based services to support 

independent living, physical and mental health, suitable housing, restorative services 

for those requiring institutional care, and retirement in “health, honor, and dignity” (OAA 

Title I, Sec. 101: 42 U.S.C. 3001). 

In 1978, the OAA was amended to include Title VI, which established programs to deliver 

home and community-based supportive services for older American Indian, Alaska 

Native, and Native Hawaiian (AI/AN/NH) populations. The program was later expanded 

to include caregiver support services, including support for caregivers of elders and 

grandparents caring for grandchildren. These programs help to reduce the need for 

costly institutional care and medical interventions. They are responsive to the cultural 

diversity of tribal communities and represent an important part of the communities’ 

comprehensive health and social services.

The Administration for Community Living (ACL), Administration on Aging (AoA), 

administers the Title VI Programs with recognition of the strengths and resiliency of AI/

AN/NH elders and with respect for Native communities’ sovereignty, culture, and selfdetermination. ICF has been contracted to complete the Evaluation of ACL’s American 

Indian, Alaska Natives and Native Hawaiian Programs (OAA Title VI) Title VI Grant Program 

under ACL contract # HHSP233201600197A.

Older Americans Act

• Protects the wellbeing of 

 older Americans 

• Promotes independent 

 living

• Provides services that allow 

 for retirement in “health, 

 honor, and dignity” 

Title VI

• Provides home and 

 community-based 

 services for AI/AN/NH elders 

 and caregiver support 

 services

• Recognizes that older AI/

 AN/NH are “a vital resource 

 entitled to all benefits and 

 services available”

• Affirms that “services and 

 benefits should be provided 

 in a manner that preserves 

 and restores their 

 respective dignity, 

 self-respect, and cultural 

 identities” (OAA Title VI, 

 Sec. 602: 42 U.S.C. 3057a)

Title VI Grantees Funded 

in Fiscal Year 2017

 

 

 

270 Part A/B 

 • 

(Nutrition and 

Supportive Services)

269 three-year grants 

awarded to tribes/tribal 

organizations (Part A) 

 • 1 three-year grant awarded 

 

1

to an organization serving 

Native Hawaiian Elders 

(Part B) 

233 Part C 

(Caregiver Services)

Three-year grants awarded for 

the Caregiver Support Program
2

2

Evaluation of the ACL Title VI Programs: Year 1 Interim Report

What is the Evaluation of the ACL Title VI Programs? 

In 2018, the Title VI Program will celebrate 40 years of providing programs and services to older AI/AN/NH communities. 

ACL decided the time was right to better understand how the Title VI program impacts people’s lives and to look for ways 

ACL can do more to make these programs more effective. In 2016, ACL hired a contractor, ICF, to help with:

• 

• 

 

Evaluating the Title VI Programs to learn more about how Tribes organize and manage their Title VI programs and 

Providing grantees participating in the evaluation with training and technical assistance (TTA) to meet their own 

data needs.

Preparing for Evaluation 

In 2014, ICF helped ACL conduct an evaluability assessment (EA) of the Title VI programs. EAs are used to guide and 

inform investments in evaluation and research. The purpose of the EA was: 

• To describe the characteristics, context, activities, processes, implementation, and intended outcomes of the Title VI 

 nutrition, supportive, and caregiver support services and 

• To assess the feasibility of, and best approaches for, a full-scale evaluation of the Title VI Programs. 

As part of this process, a 20-member stakeholder advisory group provided input and guidance on the best way to 

evaluate the Title VI Programs in a culturally responsive way. 

EA Stakeholder Advisory Group Members

• National Indian Council on Aging

• National Resource Centers on Aging

 • National Resource Center for Native Elders 

 (University of Alaska, Anchorage)

 • National Resource Center for Native Hawaiian 

 Elders (University of Hawaii)

 • National Resource Center on Native American 

 Aging (University of North Dakota)

• Title VI grantees

The EA team identified outcomes for the Title VI 

programs in the form of a medicine wheel in order to 

show the outcomes in the four quadrants of indigenous 

practice: Spiritual, Mental, Emotional, and Physical. The 

four quadrants are encircled by community, family, and 

intergenerational connection which highlights their 

importance to the spiritual, mental, emotional, and 

physical well-being of indigenous communities and 

Elders. The medicine wheel guides the evaluation to 

ensure the questions accurately reflect planned and 

achieved program outcomes. 

Title VI Programs Medicine Wheel

A Medicine Wheel was developed as part of the EA, to 

help the team think holistically about the different domains 

that the Title VI programs impact, and how 

Title VI outcomes in each domain could be assessed.
3

Evaluation of the ACL Title VI Programs: Year 1 Interim Report

Designing the Evaluation: A Participatory Approach 

Building on the results of the EA, ICF & ACL began to design the Title VI evaluation in 2016. Similar to the EA, the 

evaluation team continued to involve stakeholders at every step using a Participatory Action Research (PAR) framework. 

As part of this approach, the team promotes the active involvement of ACL Title VI stakeholders in all phases of the 

evaluation including evaluation design, data collection, measure development and piloting, participant recruitment, 

interpretation of data and analysis, and dissemination of evaluation findings.

Title VI Steering Committee

A Participatory Action Research 

Framework

• Promotes a community-driven approach to 

 evaluation design and implementation

• Empowers and involves local stakeholders 

 at every step, helping to identify evaluation 

 questions important to the community and 

 to involve community members in 

 interpretation of findings

• Supports local capacity building to conduct 

 evaluation if desired by the community 

ACL Title VI stakeholders include a wide array of individuals as well as local, state, and federal agencies that connect with 

the ACL Title VI Programs. Stakeholders span three levels of support or involvement with the Title VI Programs. As shown, 

the first level includes those closest to program implementation and includes program recipients (e.g., Elders, caregivers), 

Title VI Program staff and volunteers, and others. The second level includes regional and state partners. Finally, the third 

level includes Federal partners such as the National Indian Council on Aging (NICOA), National Association for Area 

Agencies on Aging (n4a), and the National Association of States United for Aging and Disabilities (NASUAD). Stakeholders 

at each level were consulted for input and guidance on the evaluation design and now serve as part of the key groups 

guiding the implementation of the evaluation—the ACL Title VI Steering Committee and Evaluation Working Group (EWG).

ACL and ICF will continue to use a participatory approach in implementing the evaluation, which means that the 

evaluation team will work closely with grantees to help ensure that it is culturally appropriate and will be useful to tribes 

and programs. To help with that, the evaluation team collaborates with the 17-member Steering Committee representing 

11 tribes and tribal consortia, the three National Resource Centers, and the EWG, as well as two senior services divisions, 

and Meals on Wheels. 
4

4

Evaluation of the ACL Title VI Programs: Year 1 Interim Report

Title VI Evaluation Partners

Training and Technical Assistance 

The evaluation team will provide evaluation training and 

technical assistance (TTA) on topics like data collection, 

data analysis, and how to use data to inform local 

programming. This work will occur directly between ICF 

and the tribe/program. 

ICF’s TTA approach will engage three major groups: the 

Steering Committee, the three Resource Centers, and 

the EWG. The Steering Committee will provide guidance 

on the Evaluation of the ACL Title VI Programs as well as 

important cultural and topical expertise. The Resource 

Centers will provide opportunities for coordination and 

dissemination as well as topical expertise. The EWG will 

provide guidance on local grantee-specific tailoring and 

data collection and analysis TTA needs.

A Participatory TTA Approach
5

Evaluation of the ACL Title VI Programs: Year 1 Interim Report

Evaluation TTA, informed by the PAR framework, is designed to be individually tailored to best meet the needs of Title 

VI grantees participating in the evaluation. TTA will be used to assess evaluation capacity and needs, support data 

collection, build evaluation capacity, and support the use and dissemination of evaluation findings. TTA will be provided 

in multiple ways:

In-person EWG meetings conducted in 2017, 2018, and 2019.

Coordination of TTA resources provided through the National Resource Centers, including peer-to-peer-based 

sharing of resources to support a community of learning, such as a listserv to share lessons learned and strategies 

for data collection.

Virtual skills training for EWG members, including 2 webinars annually, data collection implementation resources, 

and related support via a TTA evaluation liaison. 

An in-person site visit per grantee participating in the evaluation in the first and second year of the evaluation to 

provide intensive and in-person TTA to facilitate and support participation in the evaluation. 

Access to an evaluation liaison who will support grantees in data collection efforts in the community and provide 

TTA related to local data collection and analysis.

Final Evaluation Design

Drawing on input and feedback from Title VI Program stakeholders, the evaluation team created an evaluation design 

including two parts:

1. Implementation Study: a study to look at the context of the Title VI program including barriers and facilitators to 

program implementation. This study aims to learn what stands in the way of the program being successful and the 

work of program directors to try to make the program as successful as possible. 

2. Outcomes Study: a study to assess the impact of the program, particularly to understand the impact that the 

program is having on elders and the community.

The Evaluation of the ACL Title VI Programs will use a mix of information that grantees already collect (secondary data) 

as well as new sources of information (primary data) to describe the implementation of the ACL Title VI Programs and 

outcomes across nutrition services, supportive services, and caregiver support services. The evaluation team worked to 

track down information where it already exists such as the UND Needs Assessment and the n4a survey results, to avoid 

asking grantees for similar information more than once. 

As part of the primary (or new) data collection, grantees will help the evaluation team conduct focus groups (or talking 

circles). The focus groups will be conducted with elders who receive congregate meals. The evaluation also includes 

interviews with a small number of elders who receive home-delivered meals. The evaluation also includes focus 

groups with caregivers and a survey of caregivers. Overall, the interviews, focus groups, and survey will help improve 
6

6

Evaluation of the ACL Title VI Programs: Year 1 Interim Report

understanding of the services people receive as part of Title VI and how those services impact them. In addition, the 

evaluation includes interviews and focus groups with Title VI program staff to improve understanding about how the Title 

VI programs are serving each community and to learn more about areas where programs could use additional support. 

The interviews will be conducted in year 1 and the focus groups in year 3, to see if anything has changed for program staff. 

Evaluation Timeline

Overall, the evaluation project will span five years, as illustrated below. The first year (2016-2017) is a planning and 

recruiting year. In years 2-4 (2018-2020), the project will focus on data collection and TTA. The fifth year will be focused on 

data analysis and reporting. 

Timeline for the Evaluation of the ACL Title VI Programs
7

Evaluation of the ACL Title VI Programs: Year 1 Interim Report

Which Title VI Grantees Will Participate in the Evaluation?

In collaboration with ACL and the Steering Committee, ICF recruited Title VI grantees to participate in the evaluation. 

On May 16, 2017, a Request for Proposals (RFP) emphasizing the participatory nature of the evaluation was released to 

all Title VI program grantees. All Title VI grantees that received awards starting April 1, 2017 could apply to be a part of 

the evaluation. 

• June 2017: 18 applications were received

• July 2017: 13 evaluation grantees were selected 

The evaluation grantees will receive funding each year, for three years, to compensate tribes for participation in the 

evaluation, including costs associated with participant recruitment, data collection, and analysis/dissemination efforts. In 

addition, ACL and ICF will provide grantees with TTA to meet their own data needs. In August 2017, ICF hosted a virtual 

kickoff meeting welcoming evaluation grantees to the project. In September 2017, ICF held a one-day, in-person training 

with evaluation grantees to review evaluation topics and to prepare for participation in the evaluation at the ACL Title VI 

National Training and Technical Assistance Conference.

Title VI Evaluation Grantees

The 13 evaluation grantees represent 5 Title VI regions and 7 states. Together, they 

form the Evaluation Working Group, or EWG, for the Title VI evaluation. Each grantee 

was asked to identify 2-3 people from their program to serve on the EWG throughout 

the 3-year period, including the Title VI program director and one or two other staff 

members who will serve as “community researchers.” The community researcher is 

someone in the community who is interested in learning more about evaluation 

and helping the evaluation team with data collection. Responsibilities for EWG 

members include: 

• Attending the annual in-person meetings and virtual trainings related to the 

 evaluation 

• Participating in quarterly calls with evaluation liaisons

• Helping with recruiting participants to take part in focus groups and surveys, and 

 distribute a caregiver survey 

• Working with the evaluation team to identify what kind of help would be useful to 

 the program related to evaluating the program or managing data

In addition, Title VI directors (and other Title VI staff) will be asked to take part in an 

interview and a focus group about their programs.

Total Evaluation 

Grantees

7

5

13

Regions

Represented

(X, IX, VIII, VI, V)

Grantees

States

Represented

(Alaska, Hawaii, 

Washington, 

Nevada, Montana, 

New Mexico, 

Michigan)
8

8

Evaluation of the ACL Title VI Programs: Year 1 Interim Report

Key Achievements in Year 1 of the Evaluation of Title VI Programs
9

Evaluation of the ACL Title VI Programs: Year 1 Interim Report

What We’ve Learned

While Year 1 of the evaluation project was focused on evaluation planning and grantee recruitment rather than data 

collection, it has provided a rich opportunity for learning about the Title VI program, stakeholder involvement in the 

program and related interests and concerns, and about the 13 evaluation grantees and the populations they serve. The 

evaluation grantees are diverse in many ways, reflecting the following: 

Widely varying 

geographic areas, 

HHS/AOA regions, 

and cultures

A diversity of 

program experiences, 

operations, and 

requirements 

(e.g., tribal approvals)

Differences 

in level of experience 

with evaluation and 

evaluation capacity

To address these differences and needs, the evaluation will need to be flexible and tailored to promote participation. TTA 

will need to include multiple approaches and activities to involve grantees in meaningful ways and support local capacity 

building to conduct evaluation, if desired by the grantee.

Characteristics of Title VI Evaluation Grantees

The grantees also vary in terms of their service populations, types of services offered, and program processes and 

resources. The bar graphs below illustrate: 1) the number of elders in the service area and number of caregivers of elders, 

as reported on grantee evaluation applications; 2) the number of elders at congregate meals and receiving homedelivered meals, as reported on grantee evaluation applications and on the Title VI application; 3) the number of grantees 

that report using paper or electronic records, or both; and 4) the number of grantees that report having a complete list 

of Title VI service recipients; having regular face-to-face contact with Title VI clients; having participated in a previous 

evaluation; and having existing data collection.
10

10

Evaluation of the ACL Title VI Programs: Year 1 Interim Report

Lessons in Collaborating with Evaluation Grantees

While opportunities to collaborate with evaluation grantees have been limited in the first year, evaluation partners 

including ACL, ICF, and the evaluation grantees, had the opportunity to gather at the first in-person meeting for the 

evaluation on September 12, 2017. This full day meeting took place at Mystic Lake Casino Hotel in Minneapolis, MN, 

and provided a chance for all partners to share information, questions and ideas about the evaluation. Related lessons 

learned included:

• Importance of “hands on” learning: information and TTA was more useful to grantees when it included practical, 

 relevant, and relatable language as well as opportunities for active, “hands on” learning, such as participating in a mock 

 focus group. 

• Benefits of peer sharing: the in-person meeting gave grantees a chance to share ideas and experiences about 

 evaluation, allowing more experienced grantees to share many examples of experiences collecting data or 

 participating in other ways in other evaluations.

What’s Next?

The evaluation implementation and TTA will begin when the Office of Management and Budget approves the evaluation 

design and will continue through July 2020. Following is an overview of upcoming events in the first year of the 

evaluation implementation:

• Program Staff Interviews: These interviews will be 60-minute telephone interviews with local Title VI program staff 

 to improve understanding about how the Title VI programs are serving each community and to learn about areas where programs could use additional support.

• Two EWG Virtual Trainings/Webinars: In February or March 2018, the first virtual training or webinar will focus on 

 helping grantees prepare for the first site visit. The time frame for the second training will be determined in the near 

 future. 

• Site Visits: Evaluation liaisons will work with grantees to schedule a 3-day site visit in the spring of 2018. The site visit 

 will be an opportunity for: 1) one-on-one support for grantees; 2) learning more about grantees’ needs around 

 data and evaluation and 3) conducting focus group or talking circles with up to 10 congregate meal elders and 

 interviews with elders receiving home-delivered meals.

• Second Annual EWG In-person Meeting: the second annual EWG in-person meeting will occur in the fall of 2018.

For more information, contact:

Kristen (Krissy) Hudgins, PhD

Social Science Analyst, Office of Performance and Evaluation

Administration for Community Living

Telephone: 202-795-7732

E-mail: Kristen.Hudgins@acl.hhs.gov

Gretchen Clarke, MPH

Technical Specialist, Title VI Evaluation Program Manager

ICF

Telephone: 907-747-7124

E-mail: Gretchen.Clarke@icf.com

References

Older Americans Act (OAA), Public Law 89–73, as amended through P.L. 114–144 (enacted April 19, 2016). 

Title I, Sec. 101 (42 U.S.C. 3001) and Title VI, Sec. 602 (42 U.S.C. 3057a)

http://ltcombudsman.org/uploads/files/issues/Older-Americans-Act-of-1965-Compilation.pdf. 

Tribal Evaluation Workgroup. (2013) “A Roadmap for Collaborative and Effective Evaluation in Tribal Communities.” 

Children’s Bureau, Administration for Children and Families, U.S. Department of Health and Human Services.