The Nutrition Services Program 

Outcomes Evaluation: Effects on 

Participants’ Health Care Utilization

National Evaluation Results Webinar

The National Resource Center on Nutrition and Aging

October 30, 2018
2

Administration for Community Living Team

Holly Greuling, RDN

Heather Menne, Ph.D.
The Nutrition Services Program 

Outcomes Evaluation: Effects on 

Participants’ Health Care Utilization

National Evaluation Results Webinar

The National Resource Center on Nutrition and Aging

October 30, 2018

James Mabli, Ph.D.

Arkadipta Ghosh, Ph.D.

Bob Schmitz, Ph.D.
4

AoA Nutrition Programs Evaluation Objectives

1. Process study

– Provide information to support program planning

– Analyze program structure, administration, staffing, coordination, 

processes, and service delivery

2. Cost study

– Estimate the average costs of congregate and home-delivered meals

– Assess variation in costs by selected characteristics of local providers

3. Outcomes evaluation

– Assess program effectiveness in improving food security, socialization, 

and diet quality

– Assess program effectiveness in improving longer-term health and 

delaying or avoiding institutionalization
5

Objectives of the Outcomes Evaluation 

(First Report, 2017)

1. Describe participants’ demographic and household 

characteristics, health status, mobility, eating behaviors, 

diet quality, food security, and socialization

2. Describe participants’ experiences with and impressions of 

the program and their valuation of meals and supportive 

services received through the program

3. Determine the impact of meals and related services on 

participants’ nutrition, food security, and diet quality

4. Determine the impact of meals and nutrition services on 

overall wellness and well-being
6

Objectives of the Outcomes Evaluation 

(Second Report, 2018)

1. Describe participants’ health care utilization and behavior 

characteristics 

2. Estimate effect of participation on health care utilization outcomes
7

Study Design of Outcomes Evaluation

LSP = Local Service Provider
8

Study Design of Outcomes Evaluation

LSP = Local Service Provider
9

Selected Matched Comparison Group Using 

Medicare Records and Geography
10

Selected Matched Comparison Group Using 

Medicare Records and Geography
11

Selected Matched Comparison Group Using 

Medicare Records and Geography
12

Evaluation Data

• Outcomes survey

– Administered in person by interviewers from October 2015 to April 2016

– Topics included program participation and usage; food security; health status, 

mobility, prescriptions; eating behavior, diet, food preparation; depression, 

loneliness, and social isolation; other topics

• 24-hour dietary recall

– Automated Self-Administered 24-hour dietary recall (ASA-24) module

– Administered in person by interviewers from October 2015 to April 2016

• Medicare administrative claims data

– Used 2013–2014 data for matching participants and nonparticipants

– Used 2015–2017 data for outcomes analysis
13

Outcome Measures (Previous Report)

• Food security

• Socialization

– Loneliness scale

– Depression screener

– Satisfaction with 

socialization opportunities

• Diet quality

– Three measures based on 

nutrient intakes
14

Outcome Measures (Current Report)

• Hospital admissions and readmissions within 30 days of discharge

• Emergency department visits that resulted in an inpatient stay

• Outpatient emergency department visits

• Primary care physician visits in any setting

• Home health episode

• Admittance to a skilled 

nursing facility

• Admittance to a long-term 

care nursing home

• Expenditures on Medicare 

Part A and B services 

(total and by type)
15

Characteristics of Program Participants
16

Demographic Characteristics

• Congregate meal (CM) and home-delivered meal (HDM) 

participants similar in terms of gender, veteran status, whether 

they lived alone, race and ethnicity, and likelihood of having 

been widowed

– More than two-thirds were women

– 16 percent were veterans

– 60 to 63 percent lived alone

– 14 to 18 percent were non-Hispanic black; 9 to 14 percent were Hispanic

– 49 to 51 percent were widowed

• Compared with CM participants, HDM participants were older 

and had less education

– Average age was 77 (CM) versus 82 (HDM)

– 24 and 42 percent, respectively, had not completed high school

Source: AoA NSP outcomes survey, weighted data, Table III.1.
17

Monthly Income Relative to Poverty Guidelines

Source: AoA NSP outcomes survey, weighted data, Table III.1.
18

Dual Enrollment Status in Medicare and Medicaid

Source: Medicare claims and enrollment data matched to 

AoA NSP outcomes survey, weighted data, Table III.3.
19

Dual Enrollment Status in Medicare and Medicaid

Source: Medicare claims and enrollment data matched to 

AoA NSP outcomes survey, weighted data, Table III.3.
20

Dual Enrollment Status in Medicare and Medicaid

Source: Medicare claims and enrollment data matched to 

AoA NSP outcomes survey, weighted data, Table III.3.
21

Health and Prescription Medications

Source: AoA NSP outcomes survey, weighted data, Table III.2.
22

Number and Types of Chronic Conditions

• Congregate meal participants 

– Diabetes with complications (25 percent)

– Specified heart arrhythmias (19 percent)

– Diabetes without complications (16 percent)

– Vascular disease (15 percent)

– Congestive heart failure (15 percent)

• Home-delivered meal participants 

– Diabetes with complications (24 percent)

– Vascular disease (22 percent)

– Congestive heart failure (21 percent)

– Chronic obstructive pulmonary disease (20 

percent)

– Diabetes without complications (15 percent)

Source: Medicare claims and enrollment data matched to 

AoA NSP outcomes survey, weighted data, Tables III.4 and III.5.
23

Health Care Utilization

Source: Medicare claims and enrollment data matched to 

AoA NSP outcomes survey, weighted data, Tables III.6 and III.8.
24

Monthly Medicare Expenditures

Source: Medicare claims and enrollment data matched to 

AoA NSP outcomes survey, weighted data, Table III.9.
25

Effects of Program Participation on 

Health Care Utilization Outcomes
26

Effects of Congregate Meal Program Participation 

on Health Care Utilization

* Difference between participants and nonparticipants is significantly different from zero at the 0.10 level.

Source: Medicare claims and enrollment data matched to 

AoA NSP outcomes survey, weighted data, Table IV.1.
27

Effects of Congregate Meal Program Participation 

on Health Care Utilization (continued)

* Difference between participants and nonparticipants is significantly different from zero at the 0.10 level.

Source: Medicare claims and enrollment data matched to 

AoA NSP outcomes survey, weighted data, Table IV.1.
28

Effects of CM Program Participation on Health Care 

Utilization, by Income and Living Arrangement

Outcome Finding

Participants were less likely than nonparticipants to have a hospital admission

All individuals 

Lower-income individuals 

Higher-income individuals No effect

Individuals who live alone 

Individuals who live with others No effect

Participants were less likely than nonparticipants to have an emergency department visit 

leading to a hospital admission

All individuals 

Lower-income individuals 

Higher-income individuals No effect

Individuals who live alone 

Individuals who live with others No effect

Source: Medicare claims and enrollment data matched to 

AoA NSP outcomes survey, weighted data, Table IV.3.
29

Effects of Home-Delivered Meal Program 

Participation on Health Care Utilization

* Difference between participants and nonparticipants is significantly different from zero at the 0.10 level.

Source: Medicare claims and enrollment data matched to 

AoA NSP outcomes survey, weighted data, Table IV.2.
30

Effects of Home-Delivered Meal Program 

Participation on Health Care Utilization (continued)

* Difference between participants and nonparticipants is significantly different from zero at the 0.10 level.

Source: Medicare claims and enrollment data matched to 

AoA NSP outcomes survey, weighted data, Table IV.2.
31

Effects of HDM Program Participation on Health Care 

Utilization, by Income and Living Arrangement

Outcome Finding

Participants were more likely than nonparticipants to have a home health episode

All individuals 

Lower-income individuals No effect

Higher-income individuals 

Individuals who live alone 

Individuals who live with others No effect

Participants were more likely than nonparticipants to have an emergency department visit 

leading to a hospital admission

All individuals 

Lower-income individuals 

Higher-income individuals 

Individuals who live alone 

Individuals who live with others No effect

Source: Medicare claims and enrollment data matched to 

AoA NSP outcomes survey, weighted data, Table IV.4.
32

Effects of Congregate Meal Program Participation 

on Health Care Utilization in Next 12 Months

• Participants less likely than nonparticipants to be admitted to 

nursing home (4 versus 6 percent)

• Among lower-income individuals:

– Participants less likely than nonparticipants to be admitted to nursing home 

(2 versus 10 percent) 

– Participants less likely than nonparticipants to have ED visit 

(14 versus 24 percent)

• No effects for higher-income individuals

• No differences in effects for individuals who live alone and for 

those who live with other family members

Source: Medicare claims and enrollment data matched to 

AoA NSP outcomes survey, weighted data, Tables IV.5 and IV.7.
33

Effects of Home-Delivered Meal Program Participation 

on Health Care Utilization in Next 12 Months

• Participants more likely than nonparticipants to have 

– Hospital admission (32 versus 22 percent) and readmission (9 versus 3 

percent)

– Outpatient emergency department visit (48 versus 39 percent)

– Nursing home admission (14 versus 5 percent)

• Higher-income participants more likely than nonparticipants to have 

hospital readmission (12 versus 5 percent) and nursing home 

admittance (16 versus 4 percent)

– No effect for lower-income participants

• Individuals who live with other family members more likely to have 

hospital admission (32 versus 16 percent) and readmission (9 versus 

0 percent) and outpatient ED visit (47 versus 35 percent)

– No effect for individuals who live alone

Source: Medicare claims and enrollment data matched to 

AoA NSP outcomes survey, weighted data, Tables IV.6 and IV.8.
34

Conclusion
35

Evaluation Strengths

• Comprehensive data

– Medicare administrative data and in-person survey interview 

data

• Nationally representative samples of NSP participants

– Research design fosters national-level assessment

• Unique research design

– Matching participants to nonparticipants based on geography 

and health and health care utilization profiles
36

Assessing the Health Needs of NSP Participants

• Many NSP participants are in fair or poor health and have 

functional impairments

• Chronic conditions highly prevalent among NSP participants

– Over 75 percent had at least one condition; 13 to 20 percent had at least four

• NSP participants experience many health events

– CM participants

• Outpatient ED visits and PCP visits most common

• Hospital admissions and home health episodes less common but present

– HDM participants

• Chances of experiencing events much higher than CM participants

• Hospital admissions, ED visits, and home health episodes common

• Underscores vulnerability of participants the program serves, 

particularly for HDM participants
37

Assessing Program Effectiveness

• CM participants in comparison to nonparticipants

– Less likely to have hospital admission 

– Less likely to have emergency department visit leading to a hospital admission

– Had fewer home health episodes among those who experienced one

– Less likely in longer run to be admitted to a nursing home

– Effects for lower-income people were large; no effects for higher-income people

– Effects for people who live alone; no effects for people who live with others

• HDM participants in comparison to nonparticipants

– More likely to have emergency department visit leading to a hospital admission

– More likely to have a home health episode

– More likely in longer run to be admitted to a nursing home, have a hospital 

admission, and have an outpatient emergency department visit

– Effects present for higher-income people, but not for lower-income people
38

Summary of Evaluations’ Findings for Key Outcomes

Outcome Finding

Congregate meal program

Participants had greater food security than nonparticipants. 

Participants had higher levels of socialization than nonparticipants. 

Participants had higher diet quality than nonparticipants. Program meals 

made substantial contribution to participants’ diets.



Participants had lower health care utilization than nonparticipants, particularly 

for lower-income people and those who live alone.



Home-delivered meal program

Participants’ food security was similar to nonparticipants. No effect

Participants’ levels of socialization were similar to nonparticipants. Mixed

Participants had higher diet quality than nonparticipants. Program meals 

made substantial contribution to participants’ diets.



Participants had higher health care utilization in short and long run for some 

outcomes, but only among higher-income participants and those who live 

with other family members.

Mixed
39

Potential Areas of Focus

• Congregate meals

– Participants had lower rates of health care utilization than 

nonparticipants, but many participants still experience these events

– Potential areas of focus:

• Who is experiencing these types of health events?

• What role do income and living arrangement play in health care utilization?

• Home-delivered meals

– Findings were less intuitive, potentially reflecting quality of participantnonparticipant matching in research design

– Potential areas of focus:

• What specific types of health events do participants experience?

• Which types of participants experience these events?

• What types of events do new home-delivered meal participants experience?
40

Thank You!

• Mathematica extends our sincere thanks to all of the SUA, AAA, and LSP staff 

who completed study surveys, provided data for the meal cost analysis, and 

helped facilitate a successful outcomes survey

• Holly Greuling (ACL/AoA National Nutritionist)

– Holly.Greuling@acl.hhs.gov

• Heather Menne (ACL/AoA Project Officer)

– Heather.Menne@acl.hhs.gov

• Susan Jenkins (ACL/AoA Director of Performance and Evaluation)

– Susan.Jenkins@acl.hhs.gov

• James Mabli (Evaluation Project Director) 

– JMabli@mathematica-mpr.com

• Arka Ghosh (Evaluation Senior Researcher) 

– AGhosh@mathematica-mpr.com

• Bob Schmitz (Evaluation Senior Fellow)

– BSchmitz@mathematica-mpr.com
41

Evaluation Reports

• Process study report

– www.acl.gov/sites/default/files/programs/2017-02/NSP-Process-EvaluationReport.pdf

• Cost study report

– www.acl.gov/sites/default/files/programs/2017-02/NSP-Meal-Cost-Analysis.pdf

• First outcomes evaluation report

– www.acl.gov/sites/default/files/programs/2017-

07/AoA_outcomesevaluation_final.pdf

• Nutritional quality of program meals issue brief

– https://acl.gov/sites/default/files/programs/2017-

11/IB_NutritionServicesProgramEvaluation.pdf

• Second outcomes evaluation report

– https://acl.gov/news-and-events/announcements