DOI: 10.3886/icpsr39080.v1
title: Evaluation of Employment Coaching for Temporary Assistance for Needy Families (TANF) and Related Populations & Long-Term Follow-Up Study, 7 U.S. states, 2016-2026
title: Version 1

description: Employment coaching involves trained staff working collaboratively with participants to help them set individualized goals directly or indirectly related to employment and providing motivation, support, and feedback as participants work toward those goals. Unlike most traditional case managers, coaches work in partnership with participants and do not tell the participants what goals they should pursue or what action steps to take in pursuing them. Recently, there has been growing interest among policymakers, practitioners, researchers, and others in using employment coaching to assist Temporary Assistance for Needy Families (TANF) recipients and other adults with low incomes.
To learn more about the potential of employment coaching, the Administration for Children and Families (ACF) funded an experimental impact study of four employment coaching programs conducted as part of the Evaluation of Employment Coaching for TANF and Related Populations. The impact study evaluated the effectiveness of each program on study participants' self-regulation skills, employment, earnings, and other measures of personal and family well-being during the 21 months after participants enrolled in the study. Data and documentation for the longer term follow-up, 48 to 67 months after study enrollment, are forthcoming.
The four employment coaching programs included in the evaluation are:

Family Development and Self-Sufficiency (FaDSS), which serves TANF recipients and their family members in Iowa. Participation in FaDSS is voluntary and most coaching sessions occur in the participant's home.
Goal4 It!TM, which provides employment coaching to TANF recipients in Jefferson County, Colorado in lieu of traditional case management. Receipt of TANF benefits is conditional on participation in either Goal4 It! or traditional case management.
LIFT, which is a voluntary coaching program operated in four U.S. cities. Most coaching is conducted by unpaid student interns from Master of Social Work programs. 
MyGoals for Employment Success (MyGoals), which is a voluntary coaching program that served recipients of public housing assistance in Baltimore, Maryland, and Houston, Texas. 

The impact study addressed the following primary research questions:
Do the coaching programs improve the outcomes of adults with low incomes? 
Specifically:

Do the coaching programs affect participants' intermediate outcomes related to self-regulation and other skills associated with labor market success? 
Do the coaching programs affect participants' employment and economic security outcomes?
How do the impacts of the coaching programs change over time? 
Are the coaching programs more effective for some groups of participants than others?

Between February 2017 and November 2019, about 4,300 adults who were eligible for one of the four employment coaching programs and who consented to participate in the evaluation were randomly assigned either to (1) a program group that had access to employment coaching, or (2) a control group that did not have access to employment coaching but could receive other services available in the community. The effectiveness of each employment coaching program was assessed based on differences in average outcomes between program and control group members. Impacts were estimated during two follow-up periods: at 9 to 12 months after study enrollment (depending on the program; Moore et al. 2023) and at 21 months after study enrollment (Moore et al. forthcoming).
To estimate the impacts of employment coaching, the study used data from: (1) a baseline survey or form administered to study participants at the time of study enrollment, (2) follow-up surveys administered to study participants approximately 9 to 12 months after study enrollment, and again approximately 21 months after study enrollment, (3) administrative employment and Unemployment Insurance records from the National Directory of New Hires (NDNH), and (4) administrative records from state and local agencies on participation in public assistance programs. 
The employment coaching restricted-use data collection includes nine files with data from these sources, excluding administrative and Unemployment Insurance records from the NDNH. Some of the files include data for a single program, while others combine data for more than one program. A user guide provides documentation for each file.

Long-term data will be added to this study homepage in the future.
description: Policymakers, practitioners, researchers, and others are interested in the potential of employment coaching to help recipients of Temporary Assistance for Needy Families (TANF) and other adults with low incomes to become economically secure. Employment coaching involves trained staff working collaboratively with TANF participants to help them set individualized goals that are directly or indirectly related to employment. Coaches provide motivation, support, and feedback as participants work toward those goals. Unlike most traditional case managers, coaches work in partnership with participants. They do not tell participants what goals to set or what actions to take to work toward them. Rather, participants are given autonomy to identify and pursue their goals, with support from their coach. This nondirective approach to goal setting can help people use and strengthen the skills needed to stay organized, finish tasks, and control emotions, which are referred to as self-regulation skills. These skills are important for obtaining, keeping, and advancing in a job. Yet, poverty and other chronic stressors can hinder the use of these skills. By helping participants practice these skills, coaching could improve employment outcomes, and hence economic security.
To explore the potential of employment coaching for adults with low incomes, the Office of Planning, Research, and Evaluation in the ACF, U.S. Department of Health and Human Services, contracted with Mathematica, Abt Associates, MDRC, and The Adjacent Possible to conduct the Evaluation of Employment Coaching for TANF and Related Populations. This evaluation builds the evidence base by rigorously testing four employment coaching programs designed for adults with low incomes. It assesses the implementation of the four coaching programs and--via an experimental study--their impacts on study participants' self-regulation skills, employment, earnings, self-sufficiency, and other measures of personal and family well-being.
description: Researchers used an experimental design to assess the effectiveness of each employment coaching program in improving participants' outcomes. Between February 2017 and November 2019, about 4,300 adults who were eligible for one of the four employment coaching programs included in the evaluation and who consented to participate in the study were randomly assigned with equal probability either to a program group that was given access to employment coaching or to a control group that did not have access to employment coaching from the program. In the impact study of Goal4 It!, control group members were provided TANF case management. In the study of FaDSS, all program and control group members received TANF case management. In the study of all four programs, all study participants, whether in the program or control group, could receive other services available in the community.
To estimate the impacts of employment coaching, the study collected outcomes from survey and administrative data. The effectiveness of each employment coaching program was assessed based on differences in average outcomes between members of the program and control groups. With random assignment, the program and control group members had similar characteristics and experiences, on average, before participating in the program so any differences in observed outcomes could be attributed to employment coaching. Researchers estimated the impact on each outcome using a statistical model to control for baseline characteristics and improve the precision of the impact estimates. For each impact estimate, researchers reported whether the difference from zero was statistically significant. For impacts on earnings, researchers complemented this by using a Bayesian analysis approach to estimate the probability that the program's impact was than a specified amount.
description: The study team collected baseline data on participants' demographics, employment, education level, family background, household composition, earnings, receipt of public assistance, and goal-setting and self-regulation skills. The study team then contacted participants to complete two follow-up surveys that collected information on an extensive set of outcomes, including service receipt, labor market outcomes such as employment and earnings, economic security, receipt of public assistance, and self-regulation skills. TANF and SNAP administrative records containing information on baseline and follow-up receipt of public assistance benefits were also collected from state agencies. As previously noted, the study team also analyzed administrative earnings and employment data from the NDNH that are not part of the restricted files. 
Although the study examines each program's impact on a broad set of outcomes, it focuses on a few key outcomes that the program is expected to change, referred to as confirmatory outcomes. The main tests of the programs' effectiveness are based on whether the program had a favorable impact on the confirmatory outcomes. The confirmatory outcomes include measures in three domains for all four programs for both follow-up analysis and a fourth domain for FaDSS and Goal4 It! for the second follow-up analysis:

Self-regulation and goal-related skills. Setting goals and working to attain them requires self-regulation skills and is the centerpiece of employment coaching. Researchers used an eight-item scale on goal-setting and attainment skills designed to measure people's ability to set and work toward attaining employment goals as a confirmatory outcome.
Labor market outcomes. Researchers used earnings as the confirmatory measure of labor market success because they encompass three ways that employment coaching could influence labor market success: obtaining a job, working more regularly or more hours, or earning higher wages. Researchers measured earnings using both responses to the follow-up survey and NDNH administrative records. Earnings reported on the survey cover all jobs the study participant may have had, but may be subject to error if study participants remember jobs incorrectly. 
Economic well-being. All programs in the study intend to improve economic well-being. This may be accomplished through improved labor market outcomes, access to other material supports (such as assistance programs), or better financial management. Researchers used a six-item economic hardship scale to assess the extent to which scarce economic resources affected key aspects of material well-being, such as food, housing, and medical care.
Receipt of public assistance. The FaDSS and Goal4 It! programs both include TANF benefit receipt among their enrollment criteria. Reducing participation in TANF is a goal of these programs. Because impacts on TANF benefit receipt should emerge after impacts on earnings, TANF benefit receipt for the second follow-up period is included as a confirmatory outcome for these two programs. Researchers measured receipt of public assistance using public assistance agency administrative records on average monthly TANF benefits.

The study team also examined the impact of each program on other outcomes not deemed as confirmatory. Examples of these outcomes include the receipt of other employment services; participation in, completion of, and receipt of credentials from training and education programs; employment.
Key study variables used in the impact analysis were constructed from one or more raw study variables. Constructed variables include scales that take the average value of responses to multiple survey items, indicator variables using simple combinations of multiple survey items, combinations of variables that provided the same type of information, and earnings and employment variables constructed from participants' reported employment timelines.
description: To account for participants who (1) did not respond to the first follow-up survey, (2) did not respond to the second follow-up survey, and/or (3) did not match to the administrative data, the study team constructed three separate nonresponse weights for the impact analyses. These weights were used to adjust the data by multiplying each respondent's data by their weight. This has the effect of giving more weight to the responses of groups who were less likely to respond to a survey or match to administrative records, and less weight to the responses of groups that had a higher response rate or were more likely to match to administrative records, so that the impact analysis sample is more representative of the full randomly assigned sample.
 The study team constructed each weight by estimating a series of logistic regression models that predicted (1) survey nonresponse or (2) inability to match to the administrative data for each employment coaching program and research group. Each participant was then assigned a weight corresponding to the probability that they (1) responded to a given follow-up survey (and/or (2) were matched to administrative data on public benefit receipt. Participants who did not respond and/or were not matched to the administrative data were assigned a weight equal to zero.
description: Response Rates: FaDSS

First follow-up survey response rate: 63 percent for both program group and control group.
Second follow-up survey response rate: 76 percent for both program group and control group.
Administrative data on TANF and SNAP benefits match rate: 99 percent for both program group and control group.

Goal4 It!

First follow-up survey response rate: 59 percent (program group) and 58 percent (control group).
Second follow-up survey response rate: 71 percent for both program group and control group.
Administrative data on TANF and SNAP benefits match rate: 100 percent for both program group and control group.

LIFT

First follow-up survey response rate: 77 percent (program group) and 73 percent (control group).
Second follow-up survey response rate: 87 percent (program group) and 83 percent (control group).
Administrative data on TANF and SNAP benefits match rate: 75 percent (program group) and 74 percent (control group).

MyGoals

First follow-up survey response rate: 77 percent (program group) and 73 percent (control group).
Second follow-up survey response rate: 82 percent (program group) and 79 percent (control group).
Administrative data on TANF and SNAP benefits match rate: 100 percent for both program group and control group.
description: Presence of Common Scales: The study team constructed scale variables by taking the average value of responses to multiple survey items. The study team coded respondents as missing scale variables if they responded to fewer than two-thirds of the items used to construct a scale.
Constructed scale variables have variable labels that indicate they are scales (for instance, "Goal-setting and attainment scale").
description: The study teams at each employment coaching program location determined participant eligibility and obtained consent before enrolling eligible participants in the study and then conducting random assignment. Study enrollment and random assignment across all programs took place from February 2017 to November 2019, although different programs conducted enrollment at different times. Across the four programs, 4,276 adults who were eligible for one of the four employment coaching programs and who consented to participate in the study were randomly assigned either to a program group that had access to the coaching program or a control group that did not have access to the coaching program.

FaDSS:
To be eligible to enroll in FaDSS, participants must be receiving cash assistance from the Family Investment Program, Iowa's TANF program. FaDSS aims to serve TANF recipients who are determined by their case manager to be at risk of long-term dependency on TANF. Most study participants (80 percent in 2019, the last year of study enrollment) were referred to FaDSS by their TANF case manager. Almost all of the remaining 20 percent contacted the program directly after learning about it from a local service provider, relatives, or friends.
From June 2018 to November 2019, 863 adults enrolled in the study. All study applicants who were found eligible for the program and consented to participate in the study were randomly assigned to either the FaDSS group and offered FaDSS services, or to a control group and not offered FaDSS services. Both the FaDSS and control groups were required to receive case management as part of Iowa's TANF Employment and Training program, and both groups could access other services in the community.
Goal4 It!:
To be eligible to receive Goal4 It!, participants must be deemed eligible for TANF cash assistance in Jefferson County, Colorado, and be subject to TANF work requirements. Participants can receive Goal4 It! only while they are participating in TANF.
From October 2018 to November 2019, 802 adults enrolled in the study. All TANF recipients who were subject to the TANF work requirements and consented to participate in the study were randomly assigned to either the Goal4 It! group or the control group. Members of the Goal4 It! group were required to receive coaching using Goal4 It!. Control group members could not access Goal4 It! coaching but were required to participate in traditional case management from a TANF case manager. Similar to Goal4 It!, traditional case management incorporated setting goals. But unlike Goal4 It!, in which participants determined goals, case managers from the traditional case management approach typically directed goal setting and identified steps for participants. Members of both the Goal4 It! and control groups could access other services in the community.
LIFT:
To enroll in LIFT, applicants must be parents or other caregivers of children younger than age 8, or expectant parents. They also must demonstrate the level of stability needed to work on long- and short-term goals, meaning they must have had stable housing for at least six months and (1) be employed or live with someone who is employed at least part time, or (2) be in an educational program.
From June 2018 to November 2019, 808 adults enrolled in the study. All applicants who were found eligible for the study and consented to participate in the study were randomly assigned to either the LIFT group, who could participate in LIFT, or the control group, who could not participate in LIFT. Members of both the LIFT and control groups could access other services available in the community.
MyGoals: To be eligible to enroll in MyGoals, participants must have been an adult member of a household receiving federal housing assistance (through the Housing Choice Voucher program or living in public housing) and either have been unemployed or working fewer than 20 hours per month. Participants also must have been legally able to work in the United States and must not have been participating in Jobs Plus or the Family Self-Sufficiency Program, which were also managed by the public housing agencies and provided similar services. Study participants were referred to MyGoals by the housing agencies in Baltimore and Houston, or they learned about the program from a MyGoals coach at a recruitment event. MyGoals coaches conducted outreach at community locations such as job fairs, libraries, and TANF offices, and at public housing developments in Baltimore.
From February 2017 to November 2019, 1,803 adults enrolled in the study. All study applicants who were found eligible for the study and consented to participate in the study were randomly assigned to either the MyGoals group, who could participate in MyGoals, or a control group, who could not participate in MyGoals. Both the MyGoals group and control group members could access other services available in the community.
description: Datasets: 
DS0: Study-Level Files
DS1: Study Sample Data
DS2: Baseline Survey Data
DS3: Baseline My Goals Survey Data
DS4: Follow Up 1 Survey Data
DS5: Follow Up 1 Administrative Data
DS6: Follow Up 1 Analysis Data
DS7: Follow Up 2 Survey Data
DS8: Follow Up 2 Administrative Data
DS9: Follow Up 2 Analysis Data
description: TANF recipients and other low-income individuals living in: Iowa; Jefferson County, Colorado; Los Angeles, California; New York City, New York; Baltimore, Maryland; and Houston, Texas. Smallest Geographic Unit: City
description: computer-assisted telephone interview (CATI); web-based surveyFor additional information, please visit the Administration for Children and Families website.