National Family Caregiver Support Program 

Outcome Evaluation

November 28, 2018
Presentation Outline

NFCSP Outcome Evaluation Objectives

Study Design and Methodology

Caregiver Outcome Measures

Key Findings:

1. Relationship between NFCSP service use and other 

HCBS services

2. Difference in outcomes: NFCSP Caregivers vs. 

Comparison caregivers

3. AAA-level processes associated with the caregiver 

outcomes and perception of program effectiveness

4. Association between NFCSP service amount and 

caregiver perception of program effectiveness

2
NFCSP Outcome Evaluation Objectives

• To assess the impact of NFCSP services on 

caregivers’ well-being and their ability to continue to 

provide home-based caregiving.

• To examine the relationship between caregiver 

outcomes and key processes and characteristics of 

the Area Agencies on Aging (AAAs) managing 

and/or providing NFCSP services.

• To provide the aging services network, advocates, 

and policymakers with information that will be 

instrumental in shaping future strategies for 

supporting family caregivers in the U.S. 

3
Participants: NFCSP Client Group

1. Stratified sample of AAAs selected for the 11th 

National Survey of Older Americans Act Participants 

(NSOAAP). 

• Client lists from AAAs

• Removed clients interviewed for NSOAAP and 

those listed as only receiving supplemental 

services

2. Systematic random sampling of NFCSP clients 

within the sample of 176 AAAs

3. Estimated response rates were used to set the size 

of the sampling frames. For example, the baseline 

sampling frame for Client caregivers was 6,024 with 

a target of 1,250 complete interviews.

4
Evaluation Participants with Completed Surveys 

by Interview Cycle

Caregivers Care Recipients

Baseline

(Winter ‘16-17) 1,578 1,222

6-month Follow 

(Summer 2017) 1,005 

(20.5% ineligible*) n/a

12-month Follow 

(Winter ‘17-18)

794 

(19% ineligible*) 212

* Ineligible because they were no longer a caregiver.

Surveys were completed two or more times by 908 caregivers.

5
Caregiver Groups

Group assignment was based on the actual reported 

use of NFCSP services at baseline and 6-months. 

6

In the past 6 months, have you received...

1. Respite care from the provider agency? 

• If yes .... “How many hours per week of respite care do you 

usually receive through this program?” 

2. Caregiver education, training, counseling, or support group 

services from the provider agency?

• If yes .... “What was the number of the sessions attended in 

the past 6 months?”

607 Program Caregivers 

(53%)

545 Comparison Caregivers 

(47%)
Caregiver Outcome Measures

Mental Health score

Physical Health score

Caregiver Burden

Caregiver Satisfaction

Caregiver Confidence

Perception of the effectiveness of NFCSP 

services for Continued caregiving:

“Have the services you received from {Agency} enabled you 

to provide care longer than would have been possible 

without these services?”

7
The NIH Adult PROMIS Health Measures

Short Form v1.1 Global Mental Health score (range 4-20)

Composite and Individual Components Baseline Mean

N=1,568

PROMIS Mental health score 12.54

Mental health rating 3.26

Quality of life rating 3.01

Social activities rating 2.82

Frequency of emotional problems 3.45

Short Form v1.1 Global Physical Health score (range 4-20) 

Composite and Individual Components Baseline Mean

N=1,568

PROMIS Physical health score 14.13

Physical health rating 2.99

Physical activities rating 4.02

Fatigue rating 3.66

Pain rating 3.44

Values for the individual components range from 1 to 5. 8
The Zarit Burden Score

In 1980, Dr. Steven Zarit developed the 22-item Zarit Burden 

Inventory to measure caregiver subjective perceptions of 

burden among ethnically diverse populations. A shortened 

4-item version was developed and validated in 2001.* 

The composite score ranges from 4-20.

Composite and Individual Components Baseline Mean

N=1,568

Zarit Burden score 10.70

Time for self 2.98

Feeling stressed 3.07

Feeling strained 2.40

Feeling uncertain 2.24

* Bedard M, Molloy DW, Squire L, Dubois S, Lever JA, & O’Donnell M. The 

Zarit Burden Interview: A new short version and screening version. The 

Gerontologist. 2001;41, 652-657. 9
I get a great deal of 

satisfaction from being a caregiver.

Overall, how would you rate your 

confidence as a caregiver? 

10
Analysis #1: To what extent do NFCSP service 

participants and care recipients also receive other 

home- and community-based services (HCBS)?

11

 Caregivers were asked if they or their care recipients 

received HCBS from any organization outside of NFCSP 

(including services paid by Medicaid and Medicare).

– Incontinence supplies, home health aides, home 

delivered meals and case management were the most 

reported non-NFCSP services. 

 Model 1 included 163 Program caregivers who, at 

baseline, used NFCSP caregiver services but did not 

receive any HCBS. 

 Model 2 included 330 caregivers who received HCBS at 

baseline, but not NFCSP caregiver services.
Logistic Regression Results

12

 Receiving NFCSP caregiver services at baseline only 

slightly increased the likelihood of receiving non-NFCSP 

HCBS at follow-up (Odds ratio = 1.2; not statistically 

significant - Model 1). 

 Receiving HCBS at baseline significantly increased the 

likelihood of receiving NFCSP services at follow-up (Table 

below - Model 2):

Effect of using HCBS at baseline on receiving NFCSP services at 

follow-up:

Odds ratio P-value

Caregivers who received other HCBS at 

baseline vs. those who did not receive 

HCBS at baseline

1.90 0.04
Analysis #1 Conclusion

13

There is no clear suggestion that NFCSP is serving as 

a gateway to receiving other home- and communitybased services (HCBS). 

However, the caregivers and CRs receiving other 

HCBS from their local AAA or another paid organization 

did appear to initiate NFCSP services subsequently, 

either through discovery of their own or through 

introduction to the NFCSP services by the organization 

providing the other HCBS.

AAAs have an opportunity to increase the awareness of 

HCBS to NFCSP caregiver clients. 
Analysis #2: To what extent do program participants’ 

outcomes differ from those of caregivers who do not 

receive services from the NFCSP?

14

The difference-in-difference (D-i-D) analysis included those 

caregivers who completed surveys at baseline and 6 or 12-

month follow-up. 

Caregivers with Longitudinal Results Program Comparison Total

Full Sample (Any NFCSP service) 491 417 908

Restricted Sample (Respite Care) 370 307 677

Restricted Sample (Educational Services) 71 352 423

Each Program caregiver was matched with one or more 

Comparison caregivers who shared similar characteristics 

as related to the likelihood of receiving NFCSP services, 

using a technique called propensity score matching. 
Characteristics before Propensity Matching

15

Matching covariates (baseline) Program

Mean or %

Comparison

Mean or %

Caregiver age 68.04 65.27**

Caregiver Black/African American 13.03% 17.27%

Caregiver HS graduate or above 92.06% 88.01%*

Caregiver employed 24.44% 29.26%

Caregiver income category (1-8) 4.53 3.97**

Caregiving intensity 45.00% 32.92%***

Also cares for children under 18 years old 7.13% 11.27%*

Caregiver lives with care recipient 87.98% 75.78%***

Caregiver received other respite services 60.49% 38.85%***

Caregiver received other education services 34.22% 23.26%**

Care recipient is a spouse 53.16% 43.41%*

Care recipient diagnosed with dementia 62.93% 39.09%***

Care recipient resists aid 34.83% 25.90%**

*p<.05; ** p<.01;*** p<.001
Example of Balancing a Characteristic with Matching

Propensity score matching substantially reduced average 

differences between Program and Comparison caregivers 

on the selected baseline characteristics.

Example for Care recipient diagnosed with dementia:

16

Program % Comparison %

Before 

matching

After 

matching

CR diagnosed with dementia 62.93% 39.09%*** 62.63%

*** p<.001

 
The Effect of NFCSP Respite or Educational Services 

on Outcomes: Full Sample Results

17

 Among the full sample of caregivers, results did not show 

a significant effect of NFCSP on the five outcomes.

 The difference in differences was less than 0.1 for each 

outcome. 

Covariate findings:

o The model results show that higher income caregivers had 

significantly better physical health but higher burden and 

lower caregiver satisfaction. 

o Caregivers with a CR diagnosed with Alzheimer’s disease 

or dementia had significantly more caregiver burden. 

o Caregivers with a CR who resists aid had significantly 

worse mental health, worse physical health, more 

caregiver burden, and were less satisfied in caregiving. 
NFCSP Respite Care Effect on Outcomes

18

Program Sample: 370 caregivers who reported 4 or 

more average NFCSP respite hours per week. 

It was hypothesized that respite hours would be most 

relevant to improving Caregiver mental health, physical 

health, and burden.

Both groups of caregivers had small improvements in 

mental health over time and both had declines in 

physical health, but the difference in differences 

between the Program and Comparison caregivers was 

not statistically significant for either outcome.
Burden Results Refined by Respite Hours

Caregivers who received more than 4 hours of NFCSP respite 

per week reported a decrease in their burden scores, while 

Comparison caregivers reported an increase in burden.

19
NFCSP Educational Services Effect

20

Program Sample: 71 caregivers who reported using a 

NFCSP educational service at least once in the past 6 

months. 

It was hypothesized that educational services amount 

would be most relevant to improving Caregiver satisfaction 

and confidence.

Both groups experienced a small increase in mean 

caregiver satisfaction scores across time.

Although not statistically significant, the D-i-D analysis 

found that caregiver confidence declined among the 

Comparison caregivers, but not among the Program 

caregivers.
Results Refined by Educational Services Amount

Even though Program caregivers had higher confidence 

scores at baseline than the Comparison group, they did not 

experience a decline in Caregiver confidence.

21
Analysis #2 Conclusion

 The DiD evaluation findings suggest that NFCSP 

caregiver services are effective in reducing caregiver 

burden and improving caregiver confidence. 

 While caregiver burden increased over time for the 

Comparison group, it decreased slightly for the 

Program caregivers who received 4 or more hours a 

week of respite care from the NFCSP. 

 These findings suggest that there may be a certain 

minimum amount of respite care needed to reduce 

caregiver burden. 

22
Analysis #3: What types of organizational structures 

and/or approaches for NFCSP services are associated 

with the best participant-level outcomes?

23

A. To examine the association between AAA assessment 

and reassessment processes with measures of caregiver 

outcomes. 

B. To examine the association between frequency of 

measuring client satisfaction and caregiver perception 

that services help the caregiver continue caregiving 

longer. 

Two datasets linked together:

 AAA-level process survey data

 Survey response data among caregivers who said they 

received NFCSP services in the past 6 months 
A. Assessment Policies

PROCESS QUESTIONS Percent 

Yes

 Does your agency use a standardized assessment 

tool? 69%

 Does your needs assessment include assessing the 

impact of caregiving? 70%

 Does your agency have a policy for client 

reassessment? 75%

 Does your agency use caregiver assessment and 

reassessment to prioritize who receives services? 35%

 Does your agency use needs assessment to 

determine the type and amount of caregiver 

service?

68%

24
Caregiver Outcomes by AAA Process Type

25

Compared to AAAs without the policy, 

significantly lower (worse) mental 

health scores were found among the 

caregivers from AAAs that have 

standardized assessment tools 

and include the impact of caregiving 

in their needs assessment (p = 0.04 

for both).

Significantly higher (worse) mean burden

scores were found among the caregivers 

from AAAs that include the impact of 

caregiving in their needs assessment, in 

comparison to AAAs that do not include 

such evaluation (p = 0.02). 
B. Frequency in Measuring Client Satisfaction

AAAs that assess client satisfaction more often than annually had a 

higher % of caregivers that responded “definitely YES” to “Have the 

services you received from {Agency} enabled you to provide care 

longer than would have been possible without these services?”

Sample: 364 caregivers among 43 Area Agencies on Aging (AAAs) within 31 states. 26
Analysis #3 Conclusion

 Assessing the impact of caregiving may be enabling 

AAAs to target caregivers with emotional stress 

and/or experiencing more burden.

 More frequent AAA measurement of satisfaction may 

offer more opportunity for client feedback to assist 

with customizing support for the purpose of enabling 

the caregiver to care longer. 

 Opportunities exist for ACL and NFCSP to discover 

and share best practices for using client satisfaction 

feedback and assessments to improve the quality of 

caregiver services.

27
Analysis #4: Service Use Amount and Continued 

Caregiving

28

Using a modeling technique called ordinal regression, the 

goal was to determine if the amount of NFCSP services 

received was a significant factor in predicting a higher level 

response category to the Continued caregiving item.

“Have the services you 

received from {Agency} 

enabled you to provide 

care longer than would have 

been possible without these 

services?”

Response Baseline 

Percent

Definitely Yes 42.9%

Probably Yes 29.8%

No, Probably Not 13.5%

No, Definitely Not 13.8%

Four control variables: 

1. Care recipient age

2. Caregiver race

3. If the caregiver lives with the CR or not

4. Caregiving intensity (% of ADLs - assistance daily)
29

Model Results

• As the NFCSP service amount increased so did 

the probability of a more favorable response to the 

Continued caregiving item. 

• The probability of a caregiver responding more 

favorably to Continued caregiving increased 4.7 

percent for each hour increase in respite hours 

(Odds ratio = 1.047; p < 0.001). 

• The probability increased 6.4 percent for each 

additional time that a caregiver used an 

educational service (Odds ratio = 1.064; p < 

0.023). 
30

Analysis #4 Conclusion

• Results suggest that caregiver services 

provided by the NFCSP are effective for 

increasing the caregivers’ perception that the 

services are helping them continue caregiving.

• More research is needed to understand the 

optimal amount of service needed to enable 

caregivers to care longer, thus delaying or 

avoiding the need for their care recipients to be 

placed in a nursing home or other institutional 

care setting. 
Evaluation Summary

31

 Findings suggest that NFCSP could be more 

integrated with HCBS providers.

 A minimum amount of NFCSP respite care (4 

average weekly hours) was effective in reducing 

caregiver burden.

 Education and training services can lead to greater 

caregiver confidence over time. 

 The connections between certain processes and 

caregiver outcomes may help the aging services 

network better design and shape local programs to 

meet the needs of family caregivers. 

 Caregiver use of the services made available by the 

NFCSP helped them continue caregiving
Thank you!

Westat Evaluation Team:

32

Joanne Campione, Project Director

Katarzyna Zebrak, Lead Researcher

Dwight Brock, Senior Statistician

Susan Hassell, Report Manager

Beth Rabinovich, Analyst

Robin Ritter, Recruitment Director

Cecilia Avison, Project Manager

Duck-Hye Yang, Data Analyst

Jackie Severynse, Statistician