Effects of a Modified

Conditional Cash 

Transfer Program

in Two American Cities 

Findings from Family Rewards 2.0

Cynthia Miller

Rhiannon Miller

Nandita Verma

Nadine Dechausay

Edith Yang

Timothy Rudd

Jonathan Rodriguez

Sylvie Honig

September 2016
MDRC BOARD OF DIRECTORS

Mary Jo Bane

Chair

Thornton Bradshaw Professor of 

Public Policy and Management

John F. Kennedy School of 

Government 

Harvard University

Robert Solow

Chairman Emeritus

Institute Professor Emeritus

Massachusetts Institute of 

Technology

Rudolph G. Penner

Treasurer

Senior Fellow and Arjay and 

Frances Miller Chair in 

Public Policy

Urban Institute

_______________________________

Robert E. Denham

Partner

Munger, Tolles & Olson LLP

Ron Haskins

Senior Fellow, Economic Studies

Co-Director, Center on Children and 

Families

Brookings Institution

James H. Johnson, Jr.

William Rand Kenan Jr. 

Distinguished Professor of 

Strategy and Entrepreneurship

Director, Urban Investment 

Strategies Center

University of North Carolina

Lawrence F. Katz

Elisabeth Allison Professor of 

Economics

Harvard University

Bridget Terry Long

Professor of Education and 

Economics

Graduate School of Education

Harvard University

Josh B. McGee

Vice President of Public 

Accountability

Laura and John Arnold Foundation

Richard J. Murnane

Thompson Professor of Education 

and Society

Graduate School of Education 

Harvard University

Jan Nicholson

President

The Grable Foundation

John S. Reed 

Retired Chairman

Citigroup

Michael Roster

Former General Counsel

Stanford University

Former Managing Partner

Morrison & Foerster, Los Angeles

Cecilia E. Rouse

Dean, Woodrow Wilson School of 

Public and International Affairs

Katzman-Ernst Professor in the 

Economics of Education

Professor of Economics and 

Public Affairs

Princeton University

Isabel V. Sawhill

Senior Fellow, Economic Studies

Co-Director, Center on Children and 

Families

Brookings Institution

_______________________________

Gordon L. Berlin

President, MDRC
Effects of a Modified Conditional Cash Transfer 

Program in Two American Cities 

Findings from Family Rewards 2.0

Cynthia Miller

Rhiannon Miller

Nandita Verma

Nadine Dechausay

Edith Yang

Timothy Rudd

Jonathan Rodriguez

Sylvie Honig

September 2016
Family Rewards 2.0 is one of five evidence-based programs that were implemented as part of the 

Social Innovation Fund (SIF) grant to the Mayor’s Fund to Advance New York City and the New 

York City Center for Economic Opportunity. This report is based upon work supported by the 

SIF, which unites public and private resources to evaluate and grow innovative community-based 

solutions with evidence of results. The SIF is a program of the Corporation for National and 

Community Service, a federal agency that engages more than 5 million Americans in service 

through its AmeriCorps, Senior Corps, SIF, and Volunteer Generation Fund programs, and leads 

the U.S. President’s national call to service initiative, United We Serve.

The SIF grant catalyzes a unique public-private funding model in which each federal dollar must 

be matched by private and local contributions. Matching funds for Family Rewards 2.0 have been 

provided by Bloomberg Philanthropies, Open Society Foundations, The Rockefeller Foundation, 

Benificus Foundation, the City of Memphis, The Kresge Foundation, New York Community 

Trust, W.K. Kellogg Foundation, and the Women’s Foundation of Greater Memphis.

Dissemination of MDRC publications is supported by the following funders that help finance 

MDRC’s public policy outreach and expanding efforts to communicate the results and implications of our work to policymakers, practitioners, and others: The Annie E. Casey Foundation, 

Charles and Lynn Schusterman Family Foundation, The Edna McConnell Clark Foundation, Ford 

Foundation, The George Gund Foundation, Daniel and Corinne Goldman, The Harry and Jeanette 

Weinberg Foundation, Inc., The JBP Foundation, The Joyce Foundation, The Kresge Foundation,

Laura and John Arnold Foundation, Sandler Foundation, and The Starr Foundation.

 

In addition, earnings from the MDRC Endowment help sustain our dissemination efforts. Contributors to the MDRC Endowment include Alcoa Foundation, The Ambrose Monell Foundation, 

Anheuser-Busch Foundation, Bristol-Myers Squibb Foundation, Charles Stewart Mott Foundation, Ford Foundation, The George Gund Foundation, The Grable Foundation, The Lizabeth and 

Frank Newman Charitable Foundation, The New York Times Company Foundation, Jan Nicholson, Paul H. O’Neill Charitable Foundation, John S. Reed, Sandler Foundation, and The Stupski 

Family Fund, as well as other individual contributors.

The findings and conclusions in this report do not necessarily represent the official positions or 

policies of the funders, the Mayor’s Fund to Advance New York City, or the New York City 

Center for Economic Opportunity. 

For information about MDRC and copies of our publications, see our website: www.mdrc.org. 

Copyright © 2016 by MDRC®. All rights reserved.
iii

Overview 

Family Rewards was an innovative approach to poverty reduction in the United States that was 

modeled on the conditional cash transfer (CCT) programs common in lower- and middle-income 

countries. The program offered cash assistance to poor families, contingent on their meeting certain 

criteria related to family health care, children’s education, and parents’ work, in the hope of reducing 

long-term poverty. The first version of Family Rewards was evaluated in New York City starting in 

2007 and had positive effects on some outcomes. The lessons learned from that study led to the next 

iteration and test of the model (“Family Rewards 2.0”), the subject of this report.

Family Rewards 2.0 was launched in July 2011 in the Bronx, New York, and Memphis, Tennessee. 

While still offering rewards in the areas of children’s education, family health, and parents’ work, 

Family Rewards 2.0 offered fewer rewards in each domain, offered the education rewards only to 

high school students, made the rewards more timely by paying them each month, and included 

personalized family guidance. The addition of guidance from staff members, who actively helped

families develop strategies to earn rewards, represented the biggest change to the original model.

MDRC evaluated the program through a randomized controlled trial involving about 1,200 families 

in each city, half of whom could participate in the program and half of whom could not. This report 

presents findings on the program’s effects through four years.

Key Findings

Family Rewards transferred over $6,200, on average, to each participating family during the period 

in which it operated, or just over $2,000 per program year. Through Year 4, it had produced some 

positive effects on some outcomes, but left many other outcomes unchanged.

• After some start-up challenges, the program was generally implemented well in both cities by 

Year 2. The guidance component became more intensive over time, although the amount of interaction between staff and participants was still less than envisioned, especially in Memphis.

• Family Rewards 2.0 increased income and reduced poverty during the program period, and led 

to improvements in parents’ reports of life satisfaction and happiness.

• The program’s primary effect in the health care area was an increase in preventive dental visits, 

although there is some evidence that it also improved adults’ self-reported health status, particularly for those in poorer health at study entry.

• The program led to a reduction in employment covered by the unemployment insurance system, 

driven largely by reductions in work in Memphis.

• The program did not affect students’ school progress through Year 4, either for the full sample 

of students or for the subgroup of academically proficient students.

The findings show that Family Rewards 2.0, while replicating many of the results from the first 

model, did not prove to be more effective. Together, however, both studies provide important 

lessons about the operation and effects of a comprehensive CCT strategy in the United States.

v 

Contents

Overview iii

List of Exhibits vii

Preface xi

Acknowledgments xiii

Executive Summary ES-1

Chapter

1 Introduction 1

Family Rewards 1.0 3 

Family Rewards 2.0 6 

Sample Characteristics 12 

The Evaluation 18 

Organization of This Report 20 

2 Program Implementation and Reward Payments 21

Summary of Implementation Findings 22 

The System for Processing Payments 23 

The Rewards Families Earned 25 

The Advisor-Participant Relationship at the End of the Program 29 

Program Costs 34 

Conclusion 36 

3 Impacts on Income and Material Well-Being 39

Summary of Impacts on Income and Well-Being 39 

Data and Methods 40 

Impacts on Income and Poverty 42 

Impacts on Banking and Financial Services 45 

Impacts on Material Hardship and Well-Being 48 

Conclusion 53 

4 Impacts on Education 55

The Education Rewards 55 

Summary of Impacts on School Progress 56 

Impacts on Attendance, Credits, and High School Exams 57 

Impacts on Parent-Child Interactions and Child Activities 62 

Impacts for Proficient Versus Non-Proficient Students 64 

Conclusion 67 
vi

5 Impacts on Health Care and Health 71

The Health Care Rewards 71

Summary of Impacts on Health 72

Impacts on Receipt of Health Care Services and Health Insurance Coverage 72

Impacts on Health Outcomes 76

Impacts by Self-Rated Health Status at Baseline 77

Impacts on Children’s Health Care and Health 79

Conclusion 80

6 Impacts on Parents’ Education, Training, and Work 83

The Work and Training Rewards 83

Summary of Impacts on Work and Training 84

Impacts on Parents’ Education and Training 84

Impacts on Employment and Earnings 86

Subgroup Impacts of Parents’ Education, Training, and Employment 88

Barriers to Finding and Maintaining Full-Time Work 92

Conclusion 94

7 Impacts by City and for Subgroups 97

Defining Subgroups and Estimating Impacts 98

Impacts by City 99

Impacts by Poverty Level 99

Impacts by Parents’ Education Level 106

Impacts by Parents’ Employment Status 106

Conclusion 113

8 Conclusion 115

Summary of Findings 116

Lessons from Family Rewards 117

Next Steps 120

Appendix

A Rewards Offered in Family Rewards 1.0 and 2.0 123

B Parents’ Understanding of and Experience with Family Rewards 129

C Impacts on Public Benefit Receipt 133

D Impacts on Education Outcomes, by Math Proficiency 137

References 143

Earlier MDRC Publications on Family Rewards 149
vii

List of Exhibits

Table

ES.1 The Family Rewards 2.0 Schedule of Rewards ES-4 

ES.2 Summary of Family Rewards 2.0 Impacts ES-8 

1.1 Family Rewards 2.0 Schedule of Rewards 8 

1.2 Selected Household Characteristics at Enrollment, by City 13

1.3 Selected Adult Characteristics at Enrollment, by City 14

1.4 Select Child Characteristics at Enrollment, by City 16

2.1 Rewards Earned by Families in Years 1 Through 4 26

2.2 Advising Received by Participants in Years 2 and 3, by City 31

2.3 Cost per Family (in 2015 Dollars) 35 

3.1 Impacts on Income and Income Sources 43

3.2 Impacts on Banking, Savings, and Debt 46

3.3 Impacts on Material Hardship, Financial Strain, and Psychosocial Well-Being 50

3.4 How Families Used Rewards, Parent Survey Responses 24 Months After 

Random Assignment 53

4.1 Impacts on Enrollment, Graduation, Attendance, Credits, and Regents or Endof-Course Exams for Students in Grades 9 and 10 at the Time of Random 

Assignment 58

4.2 Impacts on Enrollment, Graduation, Attendance, Credits, and Regents or Endof-Course Exams for Students in Grade 9 at the Time of Random Assignment 61

4.3 Impacts on Parent-Child Interactions and Focal Child’s Educational Outcomes 

and Activities, High School Students 63

4.4 Impacts on Enrollment, Graduation, Attendance, Credits, and Regents or Endof-Course Exams for Students in Grades 9 and 10 at the Time of Random 

Assignment, by English Proficiency Test Score 65

4.5 Impacts on Enrollment, Graduation, Attendance, Credits, and Regents or Endof-Course Exams for Students in Grade 9 at the Time of Random Assignment, 

by English Proficiency Test Score 68
viii

Table

5.1 Impacts on Parents’ Receipt of Services and Health Outcomes 74

5.2 Impacts on Parents’ Health Outcomes, by Self-Reported Health Status at 

Baseline 78

5.3 Impacts on Focal Child’s Receipt of Services and Health Outcomes 80

6.1 Impacts on Educational Attainment and Participation in Educational and 

Employment Activities 85

6.2 Impacts on Unemployment Insurance-Covered Employment and Earnings, 

Years 1 to 3 87

6.3 Impacts on Employment and Job Characteristics 89

6.4 Impacts on Education, Training, and Employment Activities, by Respondent’s 

Education Level at the Time of Random Assignment 91

6.5 Respondents Not Working Full Time 93

7.1 Summary Program Impacts, by City 100

7.2 Summary Program Impacts, by Poverty Level at Random Assignment 103

7.3 Summary Program Impacts, by Adult Education at Baseline 107

7.4 Summary Program Impacts, by Adult Employment at Baseline 110

A.1 Comparison of Rewards Offered by Family Rewards 1.0 and 2.0 125

B.1 Parents’ Understanding of and Experience with the Family Rewards Incentives 

and Procedures 131

C.1 Impacts on Temporary Assistance for Needy Families (TANF) or Safety Net 

Assistance (SNA) and Food Stamp Receipt and Payments, Years 1 to 2 135

C.2 Impacts on Temporary Assistance for Needy Families (TANF) or Safety Net 

Assistance (SNA) and Food Stamp Receipt and Payments, Years 1 to 2 by City 136

D.1 Impacts on Enrollment, Graduation, Attendance, Credits, and Regents or Endof-Course Exams for Students in Grades 9 and 10 at the Time of Random 

Assignment, by Math Proficiency Test Score 139

D.2 Impacts on Enrollment, Graduation, Attendance, Credits, and Regents or Endof-Course Exams for Students in Grade 9 at the Time of Random Assignment, 

by Math Proficiency Test Score 141
ix

Figure

1.1 Theory of Change 10

Box

3.1 How to Read Impact Tables in This Report 41

xi

Preface

Conditional cash transfer (CCT) programs are an innovative approach to poverty reduction. 

They offer cash assistance to poor families, reducing current poverty, but condition this assistance on families’ efforts to get preventive health care and further their children’s education, 

with the hope of reducing longer-term poverty. They are fairly common in low- and middleincome countries but had never been tested in the United States until several years ago, when 

New York City launched Opportunity NYC–Family Rewards (referred to in this report as 

“Family Rewards 1.0”), a three-year demonstration sponsored by the New York City Center for 

Economic Opportunity (CEO). MDRC helped design the initiative based on several existing 

international programs and conducted the implementation and impact evaluations. Low-income 

families in several high-poverty neighborhoods in New York City were offered cash incentives 

(“rewards”) for completing activities related to children’s education, family preventive health 

care, and adults’ work or training. The program had few effects on most behavioral outcomes 

overall, but did reduce hardship, increase receipt of dental care, and improve the educational 

outcomes of more academically prepared high school students. Early lessons from the implementation of that first demonstration, with its large number of incentives, together with patterns 

in its early impact findings suggested to CEO and MDRC that the model should be revised and 

tried again, this time with the support of the federal Social Innovation Fund.

The new version of Family Rewards (“Family Rewards 2.0”) reduced the number of 

rewards the program offered, targeted the education rewards only to high school students, and 

paid the rewards more frequently. Most important, it provided more proactive guidance to help 

families earn rewards, something that the first demonstration suggested many families needed. 

Family Rewards 2.0 was tested again in New York City (the Bronx) as well as in Memphis, 

Tennessee, providing an opportunity to study the concept in a local context very different from 

New York City. 

The findings show that Family Rewards 2.0 replicated some of the effects found in 

Family Rewards 1.0, but fell short in other ways. It increased income and reduced poverty, for 

example, but had few effects on students’ progress in school, even for more academically 

prepared students. Although these results are disappointing, the findings do raise important 

policy questions. Both iterations of Family Rewards did succeed in reducing poverty and 

economic hardship, in part because many families received rewards without changing their 

behavior — most families already used preventive medical care, for example, and many 

students either had high attendance or earned rewards for other education milestones. But the 

program did not increase those rates substantially. In some cases, the attendance and health 

insurance rates among control group families were also high, often leaving less room to make a 

difference. In other cases, the incentives were not enough to change the trajectories of students 
xii

with low attendance and performance. Further complicating interpretation, these tests of 

conditional cash transfers operated on top of the existing safety net, not as an alternative to it, as 

is the case for CCT programs in Mexico and other Latin American countries. Last, in the United 

States, income transfer programs are conditioned on work. When unemployment rates soared

during the Great Recession, demonstrating the limits of an existing policy that conditions 

benefits on work, a strategy that conditioned benefit receipt on a broader array of behaviors had 

appeal.

Although the findings now available from both versions of Family Rewards provide 

important lessons about the use of incentives and CCTs in the United States to combat both 

short- and long-term intergenerational poverty, questions remain about the possible effects of a 

conditional cash transfer program if it were not structured as an add-on to the existing American

system — specifically, whether it would lead to larger behavioral changes while still providing 

an adequate safety net for the most vulnerable families.

Gordon L. Berlin 

President, MDRC
xiii

Acknowledgments

This report reflects the contributions and support of many people. We are especially appreciative of the families participating in the Family Rewards 2.0 evaluation, both in the control and 

program groups. They have generously allowed us to learn from their experiences.

We also recognize the organizations and individuals who implemented the program 

model. Seedco staff, led by Saroya Friedman-Gonzalez and Andrew Stettner, created the 

original systems and materials for the demonstration. The Children’s Aid Society (CAS) 

management team, composed of Dr. William Weisberg, Dan Lehman, Jane Golden, and 

Cathleen Clements, seamlessly took over management of the program after the first year 

without any disruption to program services. The program manager, Ilana Zimmerman, demonstrated tremendous dedication to excellence in directing the program’s staff and updating 

procedures to make the program as simple and attractive to families as possible. She received 

critical support from New York City and Memphis program associates Patricia Hirschler and 

Tonya Melton (replacing Danielle Schonbaum), as well as Brian LaForgia, Lance Petersen, and 

Veronica Cuellar, who led the database design and worked in the central office in Manhattan, 

New York, on payment processing. In Memphis, Meredith Hennessey assisted CAS and 

MDRC as a consultant on issues such as cultivating local partnerships and obtaining administrative data.

Program staff members at each of the neighborhood partner organizations (NPOs) 

played an essential role first in recruiting the sample, and then in providing families with 

ongoing guidance, with support from the leaders of their organizations. We want to express our

appreciation to each of these teams. At CAS in the Bronx, we thank Sandra Romero, Yazmin 

Kelly, Yselly Olivo, and Jessica Schachter. At BronxWorks, we thank John Weed, Julie Spitzer, 

Marie Edwards, Nicauly Andujar, Marlenne Rojas, Adrianne Hosein, Marilyn Pena, and 

Faustino De La Rosa. At Porter-Leath in Memphis, we thank Gwendolyn Price, Michella Crisp, 

Darrell Davis, Teresa Cathey, Veronica Thorns, Sean Lee, and Karen Harrell. At Urban 

Strategies Memphis HOPE, we thank Vicki Jerideau, Susan Glassman, James Franklin, Coasy 

Hale, Astrid Illunga, Rex Harrington, Malarie Yates, Kawanna Poe, and Debra Lamber. 

Temporary outreach staff members were hired at each NPO to help recruit the sample, and we 

appreciate their perseverance and skill as well.

We owe special thanks to our colleagues at the New York City Center for Economic 

Opportunity (CEO), who have nurtured each of the Social Innovation Fund (SIF) projects 

through management oversight, fundraising, report review, SIF Learning Network events, and 

general participation in technical assistance and program development. In particular, we thank 

Sinead Keegan, Emily Apple, and Kate Dempsey. We also acknowledge our former colleagues 
xiv

at CEO, Veronica White, Kristin Morse, Allegra Blackburn-Dwyer, and Emily Firgens, who 

were instrumental in launching the demonstration.

As each SIF project is ultimately a city initiative that requires the support of key members of local government, we thank former Mayor Bloomberg and former Deputy Mayor Linda 

Gibbs in New York City, and former Mayor Wharton in Memphis. Their support opened doors 

for Family Rewards to accomplish its programmatic and evaluation-related goals. We received 

assistance in New York City from the Department of Education and the Human Resources 

Administration. In Memphis, we recognize Robert Lipscomb from the Memphis Housing 

Authority.

We appreciate the continued support of the funders of the Family Rewards 2.0 evaluation. These funders include the Corporation for National and Community Service, Bloomberg 

Philanthropies, Open Society Foundations, The Rockefeller Foundation, Benificus Foundation, 

the City of Memphis, The Kresge Foundation, New York Community Trust, W. K. Kellogg 

Foundation, and the Women’s Foundation of Greater Memphis.

Finally, we want to thank our colleagues at MDRC. First, we thank James Riccio, 

MDRC’s project director for Family Rewards, for guidance throughout the project. We also 

thank other MDRC reviewers — Gordon Berlin, William Corrin, Erin Jacobs Valentine, 

Stephen Nunez, Alice Tufel, Victoria Deitch, and Richard Hendra — for their helpful feedback 

during reviews. We also thank David Greenberg at the University of Maryland for helpful 

suggestions on the benefit-cost analysis. We thank Jared Smith for expertly managing the data 

for the evaluation. He was supported by Natasha Piatniskaia, Sara LaPlante, Anne Warren, 

Mercy Emelike, Shafat Alam, and Anastasia Korolkova. We thank Jennifer Uribe, Rob Mesika, 

and Xavier Alemany for report coordination. Christopher Boland edited the report, and Stephanie Cowell, Carolyn Thomas, and Ann Kottner prepared it for publication.

The Authors
ES-1 

Executive Summary 

Family Rewards was an innovative approach to poverty reduction in the United States that was 

modeled on the conditional cash transfer (CCT) programs common in lower- and middleincome countries. The program offered cash assistance to low-income families, provided that 

they met certain conditions related to family health care, children’s education, and parents’ 

work.

The first version of Family Rewards, called Opportunity NYC–Family Rewards (“Family Rewards 1.0”), was evaluated in New York City beginning in 2007 using a randomized 

controlled trial, in which families were randomly assigned to a program group that was offered 

the program or a control group that was not. While the program did not lead to effects on most 

major outcomes, it did lead to sizable reductions in poverty while the cash assistance was being 

offered and increased graduation rates for a group of ninth-graders who were more academically prepared at study entry.1 Early lessons and insights from that evaluation suggested that such 

an intervention might be made more effective if the model were modified in certain ways.2 The 

lessons learned from that evaluation led to the next iteration and test of the model — “Family 

Rewards 2.0.” 

Family Rewards 2.0, the subject of this report, was launched in July 2011 in the Bronx, 

New York, and Memphis, Tennessee. While still offering rewards in the areas of children’s 

education, family health, and parents’ work, Family Rewards 2.0 refined the original model in 

several ways: it offered fewer rewards in each domain, paid those rewards more frequently, 

offered the education rewards only to high school students, and offered proactive and personalized guidance to help families earn rewards. The addition of guidance from staff members, who

actively helped families develop strategies to earn rewards, represented the biggest change from 

the original model.

This report examines whether those changes led to bigger impacts and whether the program had similar effects in a context different from New York City. The findings show that the 

new program achieved many of the same effects as the original model, but fell short in other, 

important ways. Family Rewards 2.0 met its short-term goals of increasing income and reducing poverty, although the effects were smaller, given that less money was transferred overall. 

 1

James Riccio, Nadine Dechausay, David Greenberg, Cynthia Miller, Zawadi Rucks, and Nandita Verma, 

Toward Reduced Poverty Across Generations: Early Findings from New York City’s Conditional Cash 

Transfer Program (New York: MDRC, 2010); James Riccio, Nadine Dechausay, Cynthia Miller, Stephen 

Nuñez, Nandita Verma, and Edith Yang, Conditional Cash Transfers in New York City: The Continuing Story 

of the Opportunity NYC-Family Rewards Demonstration (New York: MDRC, 2013). 2

Riccio et al. (2010); Riccio et al. (2013).
ES-2 

(The shorter list of rewards meant that families had fewer ways of earning cash transfers.) The 

program also increased dental visits and adults’ self-reported health status, particularly for those 

in poorer health at study entry. Similar to the earlier program, the new model led to reductions 

in work and earnings for some participants. However, the new program did not affect students’ 

school progress through Year 4, neither for the full sample of students nor for a more academically prepared subgroup. Overall, the findings indicate that Family Rewards 2.0 did not lead to 

bigger or more widespread effects. In addition, the failure to replicate the positive effects on 

school progress for more academically prepared students suggests that the model’s effects on 

education were not very robust.

The Social Innovation Fund of the Corporation for National and Community Service 

and private foundations funded the Family Rewards 2.0 demonstration, and The Mayor’s Fund 

to Advance New York City, the New York City Center for Economic Opportunity, and MDRC

managed it. The Children’s Aid Society operated the program in partnership with two community-based organizations in each city (the local chapter of Children’s Aid Society and BronxWorks in the Bronx, and Memphis HOPE and Porter-Leath in Memphis). MDRC conducted the 

evaluation.

The Program Model: 1.0 to 2.0

Similar to its predecessor, Family Rewards 2.0 is based on the assumption that for a variety of 

reasons families may underinvest in their own development even though such investments can 

have long-term benefits. Financial incentives can help change their calculations, encouraging 

them to make extra investments of time and energy in certain educational, health care, and 

work-related efforts. Furthermore, the extra resources can help make it more feasible for lowincome people to undertake such efforts in the short term, functioning not only as incentives for

action but also as enabling resources.

The original Family Rewards model offered families 22 cash rewards, covering activities and outcomes in three domains. In the education domain, rewards were offered for parents’ 

attendance at parent-teacher conferences and for children’s attendance in school, satisfactory 

performance or better on standardized tests, completion of adequate credits per year, passing of 

exams required to graduate, and graduation. In the health domain, rewards were offered for 

maintaining health coverage and for preventive medical and dental checkups for each family 

member. In the work domain, parents were offered rewards for sustaining full-time work and 

for pursuing education or training while working. Families received payments for rewards 

earned every two months. Families were not offered services or counseling, since the goal was 

to test a pure incentives model.

During the three years that Family Rewards 1.0 operated, the average participating family earned nearly $9,000 in rewards, or roughly $3,000 in each year, leading to large reductions 
ES-3 

in poverty and reductions in a number of material hardships. The program did not affect school 

outcomes for elementary or middle school students, but did improve outcomes for the ninthgraders in the study who were performing at a proficient level or better academically when they 

entered, as measured on their prior year’s standardized tests, with sizable effects (for initially 

reading-proficient ninth-graders) on grade promotion and on graduation. In the health area, 

early, positive effects on visits to the doctor and health status faded, although there were 

continued impacts on health coverage and, especially, dental visits. Finally, the program led to a 

small increase in survey-reported employment, but to reductions for some participants in 

employment reported by the unemployment insurance (UI) system.

Family Rewards 2.0 included rewards for the following milestones (Table ES.1):

• Education: Students were rewarded for high attendance, good grades, satisfactory performance or better on state core exams, and taking college entrance exams.

• Health: Families received payments for obtaining medical and dental checkups for each family member.

• Work and training: Parents received payments for full-time work and for

earning General Educational Development (GED) certificates.

The program made several important modifications to the original Family Rewards 

model. In an effort to make the program easier to understand and focus families’ attention on 

a limited number of outcomes, the program offered 8 rewards across the three domains, 

instead of 22 rewards. While all children in a family were eligible to earn health rewards, the 

education rewards were only offered to high school students, given that the original program 

had no effects for younger students. The new program made the rewards more timely, and 

thus more salient to families, in two ways: first, by paying families monthly for rewards 

earned, rather than every two months; and second, by rewarding students for passing grades. 

The rewards for grades were also structured in an attempt to engage less academically 

proficient students, with rewards offered for A, B, and C grades. All education rewards 

earned were also paid directly to the students’ accounts, whereas in the first model parents 

received some portion of education rewards.

Most notably, however, the new model offered proactive and personalized guidance to 

help families earn rewards. In the original program, staff members were purposely restricted 

from engaging in “case management,” so that the experiment would test the effects of the 

incentives offer alone. They were permitted to provide advice and referrals to participants when 

asked for assistance with services, but they did not carry individual caseloads, nor were they

allowed to make proactive efforts to engage individual families in conversations about strategies 
ES-4 

Table ES.1 

Family Rewards 2.0 Schedule of Rewards

Domain Bronx Memphis

Education incentives

(high school students only)

Attends 95% of scheduled $40 per month $40 per month

school days

Takes an SAT or ACT exam $50 $50; must score 19 or more 

(once during program) on the ACT if administered a by Memphis City Schools

Receives grades on an official 

report cardb

$30 per A (90-100); $20 per 

B (80-89); $10 per C (75-79)

$30 per A (93-100); $20 per 

B (85-92); $10 per C (75-84) 

c Passes up to 5 Regents exams

d or 7 End-of-Course exams

$500 per Regents exam for a 

score of 75 or above; $400 

$200 per End-of-Course 

exam for a score of proficient 

per exam for a score of 65-74 or advanced (increased to 

$300 in Year 3)

Health incentives (parents and 

children 19 years and younger)

Annual medical checkup $100 per family member $100 per family member

Preventive dental care every $100 per family member $100 per family member

6 months (once per year for 

children 1-5 years)

Work incentives (parents only)

Sustains full-time employment $150 per month $150 per month

Earns a GED certificate $400 $400

SOURCE: Children’s Aid Society’s Family Rewards program materials.

NOTES: GED = General Educational Development. a

The ACT is scored out of 36. Memphis City Schools officials requested that the minimum score for the 

reward be set at 19 for students taking the test for free in class because this score is considered an indication

that students are ready for college-level work.

b

In Year 1, the amount of each reward for grades was prorated based on the number of official report cards 

issued by a student's school. To simplify verification in Year 2, students were paid the listed amounts for their 

grades regardless of the number of official report cards they received, up to a maximum of $600 per program 

year.

(continued)
ES-5 

Table ES.1 (continued)

c

High school students (grades 9-12) in the Bronx were eligible to earn rewards for the following Regents 

exams: English, one of any math exams (including Math A, Math B, Integrated Algebra, Geometry, and 

Algebra 2/Trigonometry), U.S. History and Government, Global History and Geography, and one of any 

science exams (including Living Environment, Chemistry, Physics, and Earth Science).

d

High school students (grades 9-12) in Memphis were eligible to earn rewards for the following End-ofCourse exams: Algebra 1, Algebra 2, Biology, English 1, English 2, English 3, and U.S. History.

to earn rewards. Findings from the first evaluation suggested that many families needed — and 

wanted — more help to reach the relevant milestones. Consequently, under the new model, staff 

members at the community-based partner organizations were instructed to develop a “Family 

Earning Plan” with every family and meet with them at least twice per year to discuss their 

progress toward earning the rewards. Staff members were also directed to make more aggressive and frequent efforts to engage families who, according to program tracking data, were not 

earning many rewards. Each staff member carried a caseload of about 100 families.

Children’s Aid Society managed the operations of Family Rewards 2.0 and provided 

technical assistance and oversight to the four Neighborhood Partner Organizations (NPOs) 

selected to implement the program. The NPOs were charged with implementing core components of the program. They recruited and enrolled families into the research sample, oriented 

families to the program, and provided continuing guidance to help families earn rewards.

The Study Sample

Family Rewards 2.0 was evaluated using a randomized controlled trial. In each city, about 

1,200 families were recruited for the study. Half were randomly assigned to a program group, 

offered Family Rewards, and half were assigned to a control group, not offered the program. 

The program targeted families with at least one child entering ninth grade or tenth grade. Once 

enrolled in Family Rewards 2.0, however, all of the family’s school-age children became 

eligible for the health-related rewards. The program also targeted recipients of benefits from 

Temporary Assistance for Needy Families (TANF) and the Supplemental Nutrition Assistance 

Program (SNAP, or “food stamps”), in order to get resources to the neediest families and to 

explore how a CCT approach might interact with and complement these safety net programs.

Recruitment began in August 2011 in the Bronx and in September 2011 in Memphis, 

when the NPOs received lists of potentially eligible participants provided by the human services 

agencies and departments of education in each city. The pace of recruitment varied between the 

cities due to delays in receiving some lists and difficulties in reaching some potential participants. Enrollment was originally expected to conclude by October 2011, but in practice the 

majority of families in the study enrolled after that point. Rewards were intended to be offered 
ES-6 

for three years, starting in September 2011. However, because many families entered the study 

after September 2011, the program period was extended beyond three years, to December 2014, 

in order to provide these late-entering families with at least three years of rewards. The evaluation tracked families for up to four years to determine the effects of the program on poverty, 

children’s education, family health, and parents’ work.

The majority of families who enrolled in the study in both cities were single-parent 

families, with a higher percentage in Memphis (91 percent) than in the Bronx (77 percent). A 

large fraction of the sample in the Bronx was Hispanic (74 percent), while nearly all participating families in Memphis were African American (98 percent). Adults in the Bronx had somewhat lower education levels when they enrolled than adults in Memphis. For example, nearly 

half of the adults in the Bronx did not have a high school diploma or equivalency certificate 

when they enrolled in the study, compared with only 31 percent of adults in Memphis. In 

contrast, adults in the Bronx were more likely to be working when they entered the study (57 

percent) than their counterparts in Memphis (44 percent).

Findings

• After some start-up challenges, the program was generally implemented 

well in both cities by Year 2. The guidance component became more intensive over time, although the amount of interaction between staff and 

participants was still less than envisioned, especially in Memphis.

The biggest innovation in the design of Family Rewards 2.0 was the addition of the 

family guidance component, and that component evolved considerably during the first two 

years of the program. Designated staff, or advisors, focused mainly on building relationships in 

the first year. One-on-one sessions at this time were largely transactional or customer serviceoriented. Recognizing that advisors needed more training to go beyond customer service and 

encourage behavioral change, the NPOs introduced a form of counseling called motivational 

interviewing at the start of Year 2. Motivational interviewing is a widely used, client-centered 

directive approach to counseling in which a counselor uses a set of techniques to help participants identify their goals, explore any ambivalence they may have about making changes in 

their lives, and resolve that ambivalence.

It took months of training for advisors to achieve a minimum level of competence in 

motivational interviewing, but the nature and content of family guidance sessions did change 

significantly from Year 1 to the end of Year 2. Participants talked more about their goals, and 

took the lead in creating plans for overcoming barriers to earning rewards. In addition, the 

advisors began conducting targeted outreach to participants with a pattern of earning the fewest 

rewards, a group that proved to be challenging to serve.
ES-7 

This change improved the quality of guidance sessions midway through the program, 

but these sessions did not occur in either city as often as envisioned in the model, especially in 

Memphis. Only one-quarter of adult participants met with their advisors more than once in 

Memphis, and most sessions were conducted over the phone. The families’ transportation 

challenges in Memphis greatly impeded the ability of staff to deliver this program component 

effectively.

• Nearly all families earned rewards from the program, with average 

earnings of just over $2,000 per full program year, or $6,240 for all program years combined.

Over 98 percent of families in the program group earned at least some rewards during 

the program period, and about 95 percent earned rewards in a full program year. During a full 

program year, the average family earned over $2,000 ($2,300 in Year 2, for example, and 

$2,000 in Year 3). Overall, families earned just about $6,200 in rewards over the full program 

period. 

Families earned the most, on average, from the education rewards. Over 97 percent of 

families earned at least one education reward and the average amount earned was just under 

$3,000 over the full program period. Most families also earned at least one health reward over 

the period (at about 90 percent), and those that did earn health rewards earned on average just 

under $2,000 over the full period. Finally, the work rewards contributed the least to the total 

rewards earned because fewer parents earned them. Only about 57 percent of parents earned a 

work or training reward over the full period, and nearly all of these rewards were for work 

rather than training.

• Family Rewards 2.0 increased income and reduced poverty during the 

program period and led to improvements in parents’ reports of life satisfaction and happiness.

Counting the value of the reward payments, Family Rewards 2.0 increased self-reported 

average household income for the program group by $138 in the month before the 24-month 

survey. (See Table ES.2.)3 This extra income reduced the proportion of families living at or 

below the federal poverty level by 5 percentage points, compared with the control group rate of 

78 percent. The program also reduced the proportion of families who were living in severe 

poverty (that is, families with incomes less than 50 percent of the federal poverty level) by 7 

percentage points (not shown). All of the poverty reductions from the program are attributable 

 3

Unless otherwise noted, the impacts cited in this Executive Summary are statistically significant, with less 

than a 10 percent likelihood that they arose by chance.
ES-8 

 

Table ES.2

Summary of Family Rewards 2.0 Impacts

Outcome

Program

Group

Control

Group

Difference

(Impact) P-Value

Effect

Size

Income and poverty 

Average total household income in month prior to

interview (including Family Rewards payments)a($)

Household income at or below the federal poverty

level (including rewards)a,b (%)

Any savings (%)

Children's education

Graduated on time (%)

Attendance rate is 95% or higher, Year 3c (%)

Number of credits earned, Years 1 to 3d

State core exams passed, Years 1 to 3e,f

Family health

Parent had a health checkup (%)

Parent had 2 or more dental checkups (%)

Parent's self-rated health

(1 = poor; 5 = excellent)

Child had health checkup or got shots (%)

Child had 2 or more dental checkups (%)

Parent's work and training

Has any degree, license, or certificate (%)

Has any trade license or training certification (%)

Average quarterly employment, Years 1 to 3g (%)

Total earnings, Years 1 to 3g ($)

1,636

73.5

20.5

63.4

28.9

30.1

2.7

88.4

36.1

3.2

94.9

62.4

78.5

46.6

49.6

 27,684 

1,498

78.3

12.0

63.1

29.1

29.9

2.7

87.6

22.5

3.0

94.4

46.4

77.8

42.7

52.2

29,718

138 ***

-4.8 **

8.5 ***

0.3

-0.2

0.2

0.0

0.8

13.6 ***

0.1 ***

0.5

16.0 ***

0.7

3.9 *

-2.6 **

-2,034 **

0.004

0.012

0.000

0.855

0.910

0.651

0.716

0.569

0.000

0.002

0.614

0.000

0.651

0.073

0.018

0.019

0.126

Sample size (total = 2,016) 1,025 991

(continued)
ES-9 

Table ES.2 (continued)

SOURCE: MDRC calculations using data from the Family Rewards 24-month survey, Children's Aid Society's 

Family Rewards program data, New York City Department of Education and Shelby County Schools 

administrative records, New York State unemployment insurance (UI) wage records, and Tennessee Department 

of Labor and Workforce Development UI wage records.

NOTES: Sample sizes may vary because of missing values. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment 

characteristics of sample members. Standard errors were adjusted to account for multiple observations per 

family. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences.

The effect size is the difference between program and control group outcomes expressed as a proportion of 

the standard deviation of the outcomes for both groups combined. 

Note that all outcomes in the table include zero values for students who were no longer enrolled. Dollar 

averages include zero values for sample members who were not employed.

Years 1, 2, and 3 cover the 2011-2012, 2012-2013, and 2013-2014 school years, respectively. a

Monthly household income amounts equal to or greater than $10,000 were excluded from this 

calculation. About 4.9 percent of the sample was excluded from the income measures because respondents did 

not know or refused to provide the information. An additional 0.2 percent of the sample was excluded because 

the income provided was over $10,000.

bAnnual household income was calculated by multiplying the respondent's income in the month prior to the 

survey interview by 12. For program group members, it includes Family Rewards payments earned during 

program Years 2 and 3. The federal poverty level was calculated based on annual income (monthly income 

multiplied by 12) and the household size at the time of the survey. The poverty threshold was measured 

according to the 2013 or 2014 Poverty Guidelines, depending on when a respondent was interviewed. c

Attendance was calculated as a percentage of total days present divided by total days enrolled according to 

district records. Records provided for students in New York City include enrollment for the regular school year. 

Records for students in Memphis include enrollment during the regular school year, alternative education 

programs, and summer school.

dStudents in New York City earn 1 credit per course per semester completed. Students in Memphis earn 0.5 

credits per course per semester. Credits for students in Memphis were multiplied by two to create a standard 

scale for comparison. To be considered on time to graduate, students in New York City must earn an average of 

11 credits per school year and students in Memphis must earn an average of 5.5 credits per school year. e

The Regents exam measures in this table include the following Regents exams: English, Math A, Math B, 

Geometry, Integrated Algebra, Algebra 2/Trigonometry, U.S. History and Government, Global History and 

Geography, Living Environment, Chemistry, Physics, and Earth Science.

f

The End-of-Course exam measures in this table include the following exams: English 1, English, 2, English 

3, Biology, Algebra 1, Algebra 2, and U.S. History.

gThis outcome only includes employment and earnings in jobs covered by the New York State and 

Tennessee UI programs. It does not include employment outside of either state, or in jobs not covered by the UI 

system (for example, "off-the-books" jobs and federal government jobs). 
ES-10

to the cash transfers that families received, rather than to increased earnings from work. As 

discussed later, the program did not lead to an increase in earnings, and it reduced earnings for

some families.

Although the program did increase income and reduce poverty, the effects were fairly 

modest in size, and a fair proportion of the cash transfers were for education rewards, and thus

deposited in the teenagers’ accounts. As a result, the program did not notably reduce levels of 

material hardship affecting the whole family, such as inability to pay rent or utilities or food 

insecurity. Nonetheless, the extra income families received did lead to improvements in parents’ 

sense of financial security. Parents in the program group, for example, were more likely than 

those in the control group to report that their financial situation was better than it was in the 

previous year and that they could generally afford to buy necessities. Similarly, the program led 

to modest improvements on several measures of psychosocial well-being. For example, parents 

in the program group scored higher than those in the control group on the State of Hope scale, 

which captures respondents’ beliefs in their ability to initiate and sustain action.

Finally, Family Rewards 2.0 increased families’ connection to the banking system and 

their savings. At the time of the survey, only about 45 percent of parents in the control group 

reported having a savings or checking account, compared with 66 percent of parents in the 

program group, an increase of 21 percentage points (not shown). Very few parents in the control 

group (12 percent) reported having any savings at the time of the survey. The program increased that rate to 21 percent for the program group.

• The program’s primary effect in the health area was to increase preventive dental visits, although there is some evidence that it also improved 

adults’ self-reported health status, particularly for those in poorer 

health at study entry.

Most adults and children in the control group had visited a doctor recently for a routine 

checkup, and the program had no effect on this outcome. In contrast, the majority of both 

parents and children in the control group had not visited a dentist for the recommended two 

dental checkups per year, and the program increased this number substantially. Among children

in the control group, for example, only 46 percent of them visited a dentist twice during the 

year, compared with 62 percent of those in the program group, an increase of 16 percentage 

points. The importance of oral health has been well documented.

The program led to a small, positive effect on adults’ self-reported health status, which 

was primarily driven by a sizable effect for those who were in poorer health at study entry. It is 

difficult to know what to make of this effect, given that there were no measured effects on any 

of the other outcomes that might have led to it, such as increased use of preventive care, 
ES-11

increased healthful behaviors, and so on. However, Family Rewards 1.0 led to a similar pattern 

of effects on health status for adults in poorer health.

• The program led to a reduction in employment covered by the UI system, driven largely by reductions in work in Memphis.

The research team collected and analyzed data on employment from the 24-month 

survey and from UI records, which include all employment covered by the UI system. While 

UI records include data from most employment in the United States, the UI system does not 

cover some jobs, such as self-employment, federal government employment, and domestic 

work. The UI system also misses informal (casual or irregular) jobs that are never reported to 

state agencies.

The findings show that Family Rewards 2.0 led to reductions in work according to data 

from UI records and had no effect on work reported in the survey. Whereas the reductions in 

UI-covered work occurred largely in Memphis, in the Bronx the program led to a reduction in 

work reported in the survey. The reason for the difference is unclear.

Other incentives programs that have focused solely on work have found that increasing the reward to work can lead to increased employment.4 However, the work incentives 

here were part of a larger program of rewards. It is likely that the income families received 

from the education and health rewards allowed some parents to reduce work or delay entry 

into the workforce through what is often called an “income effect.” These negative effects on 

work were somewhat larger for parents not working at study entry, and thus more marginally 

connected to work.

• The program did not affect students’ school progress through Year 4, 

for either the full sample of students or the subgroup of academically 

proficient students.

The impact analysis examined a range of education outcomes, including attendance, 

grade progression, credits earned, and college entrance exams taken. The program did not have 

effects on these outcomes for the full sample of entering ninth- and tenth-graders. For example, 

 4

Nada Eissa and Hilary Hoynes, “Behavioral Responses to Taxes: Lessons from the EITC and Labor Supply,” in Tax Policy and the Economy, Volume 20, ed. James M. Poterba (Cambridge, MA: Massachusetts 

Institute of Technology Press, 2006), 74-110; Charles Michalopoulos, Does Making Work Pay Still Pay? (New 

York: MDRC, 2005); Richard Hendra, James Riccio, Richard Dorsett, David H. Greenberg, Genevieve 

Knight, Joan Phillips, Philip K. Robins, Sandra Vegeris, and Johanna Walters, Breaking the Low-Pay, No-Pay 

Cycle: Final Evidence from the UK Employment Retention and Advancement (ERA) Demonstration, UK 

Department for Work and Pensions Research Report No. 765 (Leeds, UK: Corporate Document Service, 

2011). 
ES-12

63.1 percent of entering ninth- and tenth-graders in the control group had graduated on time 

(that is, within four years of entering ninth grade). The corresponding rate for students in the 

program group was 63.4 percent. When the analysis was focused only on entering ninthgraders, the program led to small increases in attendance and credits earned, although it had no 

effects on other outcomes, such as enrollment or graduation.

The research team also examined impacts for the subgroup of students who scored at 

the proficient level or higher on their eighth-grade English language arts exams, since Family 

Rewards 1.0 had its main effects for this group. Family Rewards 2.0 did not affect education 

outcomes for this subgroup. Finally, subgroup analyses suggested that the program led to a 

small increase in graduation rates among students in less disadvantaged families, which included families in which a parent was working at study entry or that had income above 50 percent of 

the poverty line at study entry.

Conclusion

Overall, the results show that the changes to the model, particularly the more intensive guidance, as it was implemented here, did not make the rewards more effective. Family Rewards 2.0

did lead to effects in certain domains that were similar to Family Rewards 1.0’s effects, and 

these findings add to the evidence about how a CCT would work in the United States. However,

the hope was that the new model would have had larger and more widespread effects. In 

particular, the goal was to improve school progress for all students and to help parents move 

into work in order to take advantage of the work rewards. 

There are several possible reasons for the different effects found for Family Rewards 

2.0, compared with those found for Family Rewards 1.0, since the former was not an exact 

replication of the latter. For example, the modified program targeted families receiving TANF 

or SNAP benefits, and it was tested in Memphis. There are also two differences between the 

models to consider. First, Family Rewards 2.0 transferred less money to families than did the 

first program, and there is abundant research suggesting that more income benefits low-income 

children. Perhaps the positive effects found for proficient students in the first program resulted 

from a combination of the rewards acting as incentives and the rewards providing their families 

with more money. Second, Family Rewards 1.0 offered more rewards that families could earn 

regularly, such as school attendance rewards for younger children and rewards for maintaining 

health insurance. While those rewards did not lead to behavioral change in those areas, they 

may have provided a more reliable source of income, which may have contributed to the 

positive effects found for more proficient students.
1 

Chapter 1 

Introduction

Family Rewards was an innovative approach to poverty reduction that was first tested in New 

York City in 2007. Modeled on the conditional cash transfer (CCT) programs common in 

lower- and middle-income countries, the program offered cash assistance to low-income 

families provided that they met certain conditions related to family health care, children’s 

education, and parents’ work. By providing timely cash assistance, CCT programs can reduce 

economic hardship in the short term. By encouraging families to invest in their own development, they have the potential to reduce poverty over the longer term, breaking the intergenerational cycle of poverty.

While research has found CCT programs in other countries to have been successful, reducing poverty and increasing the use of health care services and school enrollment,1 a comprehensive CCT program had never been tested in a high-income country. Family Rewards was 

sponsored by the New York City Center for Economic Opportunity (CEO), and is one of three 

incentives-based programs that are known collectively as Opportunity NYC.2 The first iteration 

of the model, or Family Rewards 1.0, was evaluated using a randomized controlled trial, in 

which families were randomly assigned to a program group that was offered the program or a 

control group that was not. While the program did not lead to effects on most of the main

outcomes, it did lead to notable reductions in poverty while the rewards were being offered and

increased graduation rates for a group of ninth-graders who were more academically prepared at 

study entry.3

The second iteration of the model, or Family Rewards 2.0, is the subject of this report. It 

refined Family Reward 1.0 in several ways: it offered fewer rewards in each of the three areas

(children’s education, family health, and parents’ work) in order to make the program easier to 

understand and to focus families’ attention on a limited number of outcomes; it offered the 

education rewards only to high school students, given the lack of effects found for younger 

students; it offered students rewards for grades earned; and it made the rewards more timely, 

and thus more salient to families, by paying them monthly for rewards earned, rather than every 

 1

Fiszbein and Schady (2009).

2

The other two Opportunity NYC programs are Work Rewards, a program offering incentives for work 

and training to recipients of subsidized housing (see Nuñez et al., 2015), and Spark, a school-based program 

providing fourth- and seventh-graders with incentives for satisfactory performance or better on tests (see Fryer, 

2011). 3

Riccio et al. (2010); Riccio et al. (2013).
2 

two months. Most notably, however, the new model offered guidance, in which staff members 

actively engaged families in conversations about strategies to earn rewards.

This report examines whether those changes led to bigger impacts and whether the program had similar effects in a context different from New York City. Enrollment for the Family 

Rewards 2.0 study started in July 2011 in the Bronx, New York, and Memphis, Tennessee. In 

each city, about 1,200 families were recruited for the study. Half were randomly assigned to a 

program group that was offered the program, and half were assigned to a control group that was

not offered the program. Families in the program group could receive rewards from September 

2011 through December 2014, giving all families three years (or a little longer, depending on 

their enrollment date) to earn them. The evaluation tracked participating families during the 

period the rewards were offered to determine the effects of the program on poverty, children’s 

education, family health, and parents’ work.

An earlier report from the project assessed the implementation of Family Rewards 2.0 

over the first two years, finding that it was operating generally as envisioned in both cities by 

the middle of Year 2, after a few recruitment and start-up challenges.4 The family guidance 

component evolved considerably over time, becoming much more intensive during Year 2. 

Families also seemed to have understood the program fairly well and earned on average more

than $2,000 in rewards in Year 2. Those findings suggested that the changes to the model were 

a step in the right direction, leading to a good understanding of the program among participating

families and higher receipt rates of available rewards.

In brief, the findings presented here indicate that the new program achieved many of the 

same effects as the original model, but fell short in other, important ways. First, Family Rewards 2.0 met its short-term goals of increasing income and reducing poverty, for all families 

and across a range of family types. The program also increased dental visits and adults’ selfreported health status, particularly for those in poorer health at study entry. Similar to the earlier 

program, the new model led to reductions in work and earnings for some participants. However, 

in contrast to the original program, the new model did not affect students’ school progress 

through Year 4, either for the full sample of students or for the subgroup of more academically 

prepared students.

Funded by the Social Innovation Fund of the Corporation for National and Community 

Service and private foundations, the Family Rewards demonstration was overseen by a collaborative, which included staff members from The Mayor’s Fund to Advance New York City, 

CEO, and MDRC. The Mayor’s Fund to Advance New York City is a grant-making institution 

that facilitates public-private partnerships throughout New York City. CEO, a unit in the New 

 4

Dechausay, Miller, and Quiroz-Becerra (2014).
3 

York City Mayor’s Office of Operations, sponsors and manages innovative antipoverty programs and evaluations. MDRC worked in close partnership with CEO to design the Family 

Rewards program and led the evaluation. MDRC is a nonpartisan social policy research firm 

with extensive experience conducting large-scale projects using random assignment research 

designs to build rigorous evidence on what works to improve the well-being of low-income 

families.

The next section presents the results from the Family Rewards 1.0 evaluation. The remainder of the chapter describes the main elements of Family Rewards 2.0, the model, the two 

cities where it was tested, and the characteristics of the families who enrolled. The chapter 

concludes with an overview of the evaluation.

Family Rewards 1.0

Run from 2007 to 2010, the first version of Family Rewards was tested in six of New York 

City’s highest-poverty neighborhoods. Families that were recruited for the study lived in one of 

these neighborhoods, were eligible for the National School Lunch program, and had at least one 

child entering fourth, seventh, or ninth grade. Between July and December 2007, about 4,800 

families enrolled in the study.

The Theory of Change

Family Rewards 1.0 was based on the assumption that, for a variety of reasons, families 

may underinvest in their own development even though such investments can have long-term 

benefits. Financial incentives can help change their calculations while at the same time provide 

cash to help reduce their immediate poverty and hardship. Furthermore, if the rewards are 

sizable, the extra resources can help make it more feasible for low-income people to undertake 

certain educational, health care, and work-related efforts in the short term, by helping them 

cover the cost of, for example, educational materials, tutoring for children, transportation to a 

free dental clinic, or clothes for a job interview. For a more comprehensive discussion of the 

model and its theory of change, see the first Family Rewards 1.0 impact report.

5

Beyond the direct cash payments to families, the program was expected to reduce current and future poverty through a number of avenues. First, children’s school outcomes were a 

central focus of the program. By providing students with incentives for critical educational 

inputs and outcomes, Family Rewards 1.0 was expected to directly affect their school progress, 

as measured by attendance, grade progression, and test scores. The program designers also 

expected that it might indirectly affect school progress in other ways, such as increased parental 

 5

Riccio et al. (2010).
4 

engagement, if parents were encouraged to take a more active role in their children’s schooling, 

or by increased family income. It was theorized that increased income may reduce family stress, 

housing instability, and food insecurity, for example, or allow parents to buy educational 

materials for their children.6

Similarly, effects on the use of health care services and the family’s health status were 

expected to occur directly through the rewards provided for preventive care, encouraging 

parents and children to obtain regular medical and dental checkups, which may also increase the 

diagnosis and treatment of health problems. However, the designers expected that these rewards 

might also encourage families to maintain health insurance, which would also help them 

maintain a regular family physician, instead of relying on costly emergency room services for 

care once a health problem arises. As with school outcomes, it was thought that the use of 

health care services and the family’s heath status might also improve through increased family 

income, if families had the resources to maintain health insurance, for example, to follow up on 

needed treatments, or had adequate and nutritious food at home.

The third targeted domain of the program was parents’ work and training. Family Rewards might increase family income through the direct cash transfers themselves, but only 

temporarily. Sustained reductions in child and family poverty after the payments end require 

that parents maintain regular employment. The program thus included a work component 

designed to promote steady, full-time work and the acquisition of skills to help parents qualify 

for better-paying jobs. It was expected that the program should directly affect employment and 

training through these rewards. However, it was thought that the program might indirectly affect 

training if parents had extra income to pay for that training, or if their increased work effort 

highlighted the benefits of additional training.

Finally, the effects in one domain can reinforce effects in other domains. For instance, 

health care rewards may promote early diagnosis and treatment of certain health and developmental problems that might otherwise make it difficult for students to sustain high attendance 

and perform well in school. As another example, the work rewards, by increasing parents’ 

earnings, may lead to longer-term effects on family income, which can affect children’s health 

and schooling.

 6

Over the last few years, scientific evidence showing that very low income, material hardship, and financial strain have causal influence on children’s life trajectories has grown more convincing. Supporting the 

claim of causal influences are findings from longitudinal studies (Dahl and Lochner, 2012; Duncan and 

Brooks-Gunn, 1997; Gershoff, Aber and Raver, 2003; Mayer, 2002; McLoyd, 1998; Seccombe, 2000) and 

from natural and policy experiments that have effectively raised the income of poor families and evaluated the 

impact of those increases on children (Costello et al., 2003; Morris and Gennetian, 2003; Miller et al., 2008). 

Among the latter studies are evaluations of programs that offered earnings supplements and, in the case of the 

Milwaukee New Hope project, subsidies for child care and health insurance. These evaluations found improvements in young children’s school performance, social behavior, and other developmental outcomes.
5 

The Model and Its Effects

The program offered families 22 cash rewards. In the education area, the program offered rewards for parents’ attendance at parent-teacher conferences and for children’s attendance in school, satisfactory performance or better on standardized tests, completion of adequate 

credits per year, passing of exams required to graduate, and graduation. In the health area, it 

offered rewards for maintaining health coverage and for preventive medical and dental checkups for each family member. In the work area, the program offered parents rewards for sustained full-time work and for pursuing education or training while working.7

A final feature of Family Rewards 1.0 is that it offered no services or active counseling. 

Case management and direct services were deliberately excluded from the model in order to test 

the effectiveness of the incentives alone. In addition, it was expected that families in New York 

City would have access to case management and services through other programs in the 

community.

During the three years the program operated, the average participating family earned 

nearly $9,000 in rewards, or roughly $3,000 in each year. The large majority of families earned 

at least one education reward and one health reward in each year, while just over 40 percent 

earned at least one work reward. The rewards increased families’ incomes by 22 percent on 

average and reduced poverty and severe poverty by about 12 percentage points. However, the 

reductions in poverty and hardship began to diminish after Year 3, when the program ended.8

In terms of children’s school progress, Family Rewards 1.0 improved outcomes for 

ninth-graders who were performing relatively better academically when they entered the study, 

with sizable effects on grade completion and graduation. In contrast, the program had no effect 

for lower-performing ninth-graders or for elementary and middle school students. In the health 

area, early positive effects on visits to the doctor and health status faded, in part because 

families already used preventive care at a fairly high rate, although there were continued 

impacts on health coverage and, especially, dental visits. Finally, the program led to modest 

increases in employment throughout the follow-up period, although much of this new work was 

in jobs not captured by the unemployment insurance system. The program also led to a reduction in unemployment insurance-covered work for the more disadvantaged adults in the study.

 7

At the end of the second year, in an effort to simplify the program and reduce its costs, several rewards 

were eliminated, including those for maintaining health coverage and for school attendance for elementary and 

middle school students.

8

See Riccio et al. (2013) and Riccio and Miller (2016) for the full set of findings.
6 

Family Rewards 2.0

In 2010, the Corporation for National and Community Service made its first set of Social 

Innovation Fund (SIF) grants.9 CEO and the Mayor’s Fund to Advance New York City received a grant to replicate and evaluate five of CEO’s most promising antipoverty programs. 

Family Rewards was one of those programs.

The replication made several important modifications to the original Family Rewards 

model, based on the early experience running the program and its emerging effects. The biggest 

innovation in the design of Family Rewards 2.0 was the addition of the family guidance 

component. Recognizing that many families would need at least some guidance on where they 

could find the kinds of services and assistance that might enhance their success in the program, 

Family Rewards 1.0’s program designers included an information-and-referral component in 

which staff were expected to help educate families about relevant resources that are available in 

the community, through resources guides, marketing materials, and optional workshops. 

However, staff members would generally wait for participants to initiate contact and focus on 

issues directly related to earning rewards rather than reaching out to participants early on and 

walking them through potential strategies.

Based on findings from Family Rewards 1.0, the program designers believed that families would benefit from a regular source of support within the program. The new model would 

use a triage model to provide this guidance, in which all participants would receive some 

guidance, but those who were having the most trouble earning rewards would receive more 

intensive outreach and support, with staff connecting them to the resources and help they 

needed to earn rewards. Findings from Family Rewards 1.0 suggested that there were several 

types of participants who might have benefited from this additional support, such as students 

who started high school scoring below grade level on eighth-grade math and English exams 

(“non-proficient” students).

Program Delivery

A consortium ran the demonstration that included The Mayor’s Fund to Advance New 

York City, CEO, and MDRC. This consortium set the broader design policies and provided 

technical assistance and oversight, largely through the lead program operator. Seedco, a New 

 9

The SIF was an initiative enacted under the Edward M. Kennedy Serve America Act and designed to 

create and fund a catalog of proven policies that can be replicated in communities across the country. For this 

purpose, the SIF uses federal funds to raise additional support for evidence-based solutions to pressing social 

issues.
7 

York-based nonprofit organization, served as the lead program operator until summer 2012, 

when Children’s Aid Society took over the role.10

Four Neighborhood Partner Organizations (NPOs) were selected to operate the program. The NPOs were selected based on the experience they had in program delivery in their 

local communities. In the Bronx, the NPOs were the Children’s Aid Society and BronxWorks. In Memphis, the NPOs were Urban Strategies Memphis HOPE (Memphis HOPE) 

and Porter-Leath.

All NPOs were charged with implementing the program’s core components, such as recruitment, outreach, and family guidance. Each NPO had its own Family Rewards supervisor, 

who was responsible for multiple programs at the NPO, including Family Rewards. The 

supervisor directly managed three advisors, which allowed each NPO some level of autonomy 

in delivering the program elements. The advisors were charged with providing guidance to 

families about how to earn rewards.

The Rewards

Table 1.1 presents the rewards for Family Rewards 2.0. (See Appendix Table A.1 for a 

comparison of rewards in Family Rewards 1.0 and 2.0.) The top panel shows the education 

domain, in which students earned rewards for high attendance, good grades, satisfactory 

performance or better on state core exams, and taking college entrance exams. One addition to 

the list of rewards in the new program was a reward for grades, in an effort to make the rewards 

more timely and more strongly tied to daily performance. In addition, students in Family 

Rewards 2.0 received all the education rewards they earned, deposited directly in their bank

accounts, in contrast to Family Rewards 1.0 in which parents received a portion of those

rewards. The second panel shows rewards for health, in which families received payments for 

medical and dental checkups for each family member. The new program did not include the 

reward for maintaining health insurance, since the original study found that most families 

already had coverage. Finally, the program rewarded parents for maintaining full-time work and 

for earning a General Educational Development (GED) certificate. The original model rewarded 

a range of approved training courses, but few families received the training reward. For this 

reason, the new program focused on one outcome, the GED certificate, which was well known 

to participants and could serve as a stepping-stone to further training.

 10CEO transferred overall management of SIF Family Rewards to Children’s Aid Society in 2012 after 

local and federal investigations uncovered irregularities in Seedco’s administration of workforce services 

unrelated to Family Rewards. 
8 

Table 1.1 

Family Rewards 2.0 Schedule of Rewards

Domain Bronx Memphis

Education incentives

(high school students only)

Attends 95% of scheduled $40 per month $40 per month

school days

Takes an SAT or ACT exam $50 $50; must score 19 or more 

(once during program) on the ACT if administered a by Memphis City Schools

Receives grades on an official 

report cardb

$30 per A (90-100); $20 per 

B (80-89); $10 per C (75-79)

$30 per A (93-100); $20 per 

B (85-92); $10 per C (75-84) 

c Passes up to 5 Regents exams

d or 7 End-of-Course exams

$500 per Regents exam for a 

score of 75 or above; $400 

$200 per End-of-Course 

exam for a score of proficient 

per exam for a score of 65-74 or advanced (increased to 

$300 in Year 3)

Health incentives (parents and

children 19 years and younger)

Annual medical checkup $100 per family member $100 per family member

Preventive dental care every $100 per family member $100 per family member

6 months (once per year for 

children 1 to 5 years)

Work (parents only)

Sustains full-time employment $150 per month $150 per month

Earns a GED certificate $400 $400

SOURCE: Children’s Aid Society’s Family Rewards program materials.

NOTES: GED = General Educational Development. a

The ACT is scored out of 36. Memphis City Schools officials requested that the minimum score for 

the reward be set at 19 for students taking the test for free in class because this score is considered an 

indication that students are ready for college-level work.

b

In Year 1, the amount of each reward for grades was prorated based on the number of official report 

cards issued by a student's school. To simplify verification in Year 2, students were paid the listed amounts 

for their grades regardless of the number of official report cards they received, up to a maximum of $600 

per program year.

(continued)
9 

Table 1.1 (continued)

c

High school students (grades 9-12) in the Bronx were eligible to earn rewards for the following Regents 

exams: English, one of any math exams (including Math A, Math B, Integrated Algebra, Geometry, and 

Algebra 2/Trigonometry), U.S. History and Government, Global History and Geography, and one of any 

science exams (including Living Environment, Chemistry, Physics, and Earth Science).

d

High school students (grades 9-12) in Memphis were eligible to earn rewards for the following End-ofCourse exams: Algebra 1, Algebra 2, Biology, English 1, English 2, English 3, and U.S. History.

As in Family Rewards 1.0, families in Family Rewards 2.0 received payments for meeting reward milestones in one of two ways. The first required no action on the part of the family 

and applied to two of the rewards, for attendance and for performance on state core exams, 

which were automatically verified using school records. The second applied to all other rewards 

and required the family to actively claim the reward by submitting a “coupon,” verifying that 

the milestone has been met, to the program. For example, to claim the reward for full-time work 

the family had to submit a coupon along with other items of verification.

In an effort to make the rewards more timely and thus more salient, families in Family 

Rewards 2.0 received payments for most rewards every month, rather than every two months as 

did those in the original program, along with an earnings statement documenting earnings for 

the month and any rewards that the family was eligible for but did not earn. Finally, families 

were strongly encouraged to open low-fee bank accounts in order to take advantage of the direct 

deposit of rewards. To facilitate this process, the program invited banks that offered no-fee, 

savings accounts to attend orientation sessions and market directly to families.

Family Guidance

Family Rewards 2.0 offered participants active and tailored guidance from designated 

staff members, or advisors. In the new model, advisors at the NPOs developed a Family Earning 

Plan with every family and met with them at least twice a year to discuss their progress toward 

earning the rewards. Advisors also proactively reached out to and engaged those families who 

were not earning many rewards. Staff also had access to a small resource fund, to help families 

with services they might need to earn rewards, such as short-term tutoring, uniforms for work, 

transportation to job interviews, or licensing fees. As discussed in the implementation report, the 

strategy for family guidance shifted during Year 2, as staff members were trained in the use of 

motivational interviewing, a client-centered counseling method that has been found in other 

settings to be effective at eliciting behavioral change.

As in Family Rewards 1.0, the theory of change underlying the new model is that the 

rewards, in combination with the guidance, will lead to a number of effects. Figure 1.1 presents 

a simplified depiction of the overall logic model, showing the main processes through which the 

rewards and guidance were expected to reduce current and future poverty.
10

Figure 1.1

Theory of Change

Rewarded Activities

Work rewards

• Full-time employment

• GED certificate

Health rewards

• Physical exam

• Dental exam

Education rewards

• Attendance

• Grades

• State core exams

• SAT/ACT

Incentives • Guidance

• Family Resource Fund

• Customer service

Mediating Factors

Increase in adult employment

• Greater access to employment resources

• Fewer missed days of work

Improved preventive health care

• Greater access to health and dental services

• Fewer missed days of school or work

Improved education outcomes

• Higher attendance rate

• Better academic performance

• Increase motivation to achieve

• Greater access to academic support

• Change in time use

MiddleTerm 

Outcome

Reduction of 

immediate 

economic 

hardship

Long-Term Outcomes

Sustained 

full-time

employment

Sustained 

good 

health-related 

habits

Higher 

college

attendance

Reduction in 

2ndgeneration 

poverty
11

Target Groups

Family Rewards 2.0 targeted families with at least one child entering ninth or tenth 

grade. Although it only offered the education rewards to students who were ninth- or tenthgraders at study entry, once enrolled in the program, all of the family’s children under 20 years 

of age became eligible for the health-related rewards. The program also targeted families 

receiving Temporary Assistance for Needy Families (TANF) or Supplemental Nutritional 

Assistance Program (SNAP, or “food stamps”) benefits, in order to direct resources to the 

neediest families and to explore how a CCT approach might interact with and complement 

these safety net programs.

The Places

Family Rewards 2.0 was tested in the Bronx, New York, and Memphis, Tennessee. 

The idea behind including another city in the replication effort was to make the findings, 

across both cities together, more relevant to the broader national population. But each city 

also contributed to the analysis, individually. By running the program again in New York 

City, the program designers expected the evaluation could answer whether the changed model 

works better in a similar context. By adding Memphis, they anticipated that the evaluation 

could also assess how the model would work outside of New York City, in a very different 

economic and social context. In this respect, Memphis represented a good addition to the 

evaluation. With its smaller population (700,000 residents in Memphis versus 1.4 million in 

the Bronx) and different racial and ethnic makeup, Memphis provided a very different context 

from the Bronx. Income levels and poverty rates were roughly similar in the two areas in 

early 2013, although the labor markets were fairly different. The unemployment rate was also 

roughly similar in both areas in 2013.11 Median wages were much lower in Memphis (about 

$15 per hour) than they were in the New York City area ($21 per hour), although the cost of 

living is also much lower in Memphis.12

Finally, as noted in the implementation report, there were several differences between 

the two cities that might have affected program implementation and impacts. First, the statesponsored health insurance program for low-income New Yorkers covered routine dental care, 

while the similar plan in Memphis (TennCare) did not. Dental care coverage may have affected

the ability to earn the dental reward. Second, families and students in the Bronx had access to an 

extensive public transportation system, whereas families in Memphis relied heavily on cars. The 

 11U.S. Department of Labor, Bureau of Labor Statistics (2014). 12U.S. Department of Labor, Bureau of Labor Statistics (2015a and 2015b). While this difference in median wages partly reflects the different mix of occupations in the two areas, median wages are also lower in 

Memphis for specific occupations, such as “home health aide” jobs or jobs in food preparation and service.
12

lack of quality public transportation made study recruitment more difficult in Memphis and also 

affected how families participated in the program. Finally, whereas most of the organizations 

and resources listed in the resource guides provided by the staff at the Bronx-based NPOs were 

located in the Bronx, a deliberate decision on their part, most of the organizations and resources

listed in the Memphis guides were located in the downtown or midtown area, not close to where 

participants lived. Transportation challenges may have kept Memphis participants from accessing the services that might have helped them earn rewards. All of these differences suggest that 

Bronx residents had easier access to the resources that could help them earn rewards.

Program Duration

The program was intended to be offered to all participating families for three years, 

starting in September 2011 and ending in August 2014. Because some families entered the 

study after September 2011, the program designers extended the program period beyond three 

years, to December 2014, in order to offer these late-entering families three years of rewards. 

Thus, all families had access to at least three years of rewards, and families who entered the 

study earlier had access to rewards for somewhat longer than three years. Ninth-graders who 

entered the study in August 2011, for example, were eligible for rewards through the fall 

semester of their senior year, assuming they graduated to the next grade every year.

Sample Characteristics

Recruitment began in August 2011 in the Bronx and September 2011 in Memphis, when the 

NPOs received lists of potentially eligible participants provided by the human services agencies 

and departments of education in each city.13 NPO staff members were then required to call 

families on the lists, making multiple attempts to reach eligible families, including additional 

phone calls and home visits. While enrollment was originally expected to conclude by October 

2011, staff faced several challenges to meeting enrollment targets, and as a result the majority of 

families in the study enrolled after October, and many enrolled as late as January or February 

2012.

In total, Bronx NPOs enrolled 1,232 households and Memphis NPOs enrolled 1,226 

households. Tables 1.2, 1.3, and 1.4 present selected characteristics of the samples. There were 

 13In the Bronx, the recruitment lists provided by the New York City Department of Education included 

information on families’ receipt of TANF and food stamps. In Memphis, the recruitment lists did not include 

benefit status. Thus, program staff in Memphis determined whether enrollees were eligible for Family Rewards 

based on self-reported receipt of TANF or food stamps.
13

Table 1.2

Selected Household Characteristics at Enrollment, by City

Characteristic Bronx Memphis

One-parent familya

 (%)

Marital status (%)

Single

Married 

Separated/divorced/widowed

Living with partner or legal domestic partner

Average number of children in household (under 19 years old)

Primary language (%)

English

Spanish

Currently receiving food stamps (%)

Currently receiving TANF or Safety Net Assistance (%)

Living in rental housing (%)

Living in public housing (%)

Receiving Section 8 rental assistance (%)

During the last 12 months, ever unable to pay (%):

Rent or utility bills

Telephone bills

Food or prescription drug costs

77.1

49.8

20.9

27.2

2.0

2.4

48.2

49.8

99.4

5.0

94.3

34.6

23.5

49.3

30.4

25.1

90.5

70.8

8.0

19.7

1.5

3.0

99.1

0.6

98.0

30.5

71.6

10.3

13.6

64.0

32.8

24.2

Sample size 1,230 1,226

SOURCES: MDRC calculations from baseline survey and random assignment module data.

NOTES: Percentages for some categories may not add up to 100 due to rounding or missing values. a

This measure includes families with parents who reported their marital status as single, single but 

living with a boyfriend or girlfriend, separated, divorced, or widowed.

some differences between the city samples. As shown in Table 1.2, for example, a larger 

fraction of participating families were headed by a single parent in Memphis (90 percent) than 

those in the Bronx (77 percent). A much larger fraction of the sample in the Bronx was Hispanic than the sample in Memphis and more sample members in the Bronx spoke Spanish as their 

primary language than those in Memphis. In contrast, nearly all participating families in 

Memphis were African American. Probably reflecting the tighter housing market in New York

City, more families in the Bronx lived in subsidized housing when they entered the study: 35
14

Table 1.3

Selected Adult Characteristics at Enrollment, by City

Characteristic Bronx Memphis

Demographic characteristics

Female (%)

Average age (years)

Race/ethnicity (%)

Hispanic

African American

Other

Citizenship status (%)

U.S. citizen by birth

U.S. citizen by naturalization

Legal permanent resident

Educational characteristics (%)

Highest degree received

High school equivalency certificate 

High school diploma

Technical/associate's/2-year college degree

4-year college degree or higher

None of the above

Trade license or certificate

Employment characteristics

Currently employed (%)

Full time (at least 30 hours per week) 

Part time 

Among adults currently employed

Average weekly wage ($)

Has a physical problem that limits work (%)

Has an emotional or mental health problem that limits work (%)

Receives SSI or SSDI due to disability (%)

87.6

42.6

74.4

22.9

2.7

42.8

31.7

25.5

6.2

32.8

7.3

5.9

47.9

45.0

57.1

39.4

17.6

314

16.0

6.5

13.1

93.9

39.3

0.9

98.1

1.0

98.6

0.8

0.6

7.6

51.2

6.2

4.0

31.0

27.6

44.1

34.0

9.9

332

18.3

7.9

15.1

(continued)
15

Table 1.3 (continued)

Characteristic Bronx Memphis

Medical coverage and care (%)

Source of health insurance coverage

Public health insurance

Employer-provided or other health insurance 

No coverage

Usual source of medical care

Clinic or health center

Doctor's office or health maintenance organization

Hospital emergency room

Hospital outpatient department

Other

Date of last medical checkup

Less than 12 months ago

12 months ago or longer or never had a checkup

Date of last dental checkup

Less than 12 months ago

12 months ago or longer or never had a checkup

Health status (%)

General health status

Excellent/very good

Good

Fair/poor

Felt down, depressed, or hopeless during the past 2 weeks 

Financial characteristics (%)

Has a bank or credit union account

85.9

10.3

3.8

73.2

16.1

3.3

7.2

0.2

87.4

12.6

77.8

22.2

35.3

38.0

26.7

22.1

59.2

81.9

10.1

8.0

49.7

41.0

7.8

1.0

0.5

76.5

23.5

35.7

64.3

35.8

38.4

25.8

23.3

39.7

Sample size 1,311 1,256

SOURCES: MDRC calculations from baseline survey and random assignment module data. 

NOTES: Percentages for some categories may not add up to 100 due to rounding or missing 

Italics indicate measures calculated for a subset of the full sample.

SSI = Supplemental Security Income. SSDI = Social Security Disability Insurance.

values.
16

Table 1.4

Selected Child Characteristics at Enrollment, by City

Characteristic Bronx Memphis

Demographic characteristics

Female (%)

Age (%)

0 to 5 years

6 to13 years

14 years or older

Race/ethnicity (%)

Hispanic

African American

Other

Born in the United States (%)

Educational characteristics (%)

Expected grade level in September 2011

Preschool

Grades 1 to 5

Grades 6 to 8

Grades 9 to 12

Not enrolled

Enrolled as an English Language Learner in the previous school year

Had an Individualized Education Program in the previous school year

Received tutoring help in the previous school year

a Nonproficient on English exam

50.4

10.8

31.0

58.2

72.7

23.8

3.5

87.3

10.0

17.3

13.1

54.3

1.4

21.6

16.3

26.5

75.2

51.6

9.6

36.8

53.6

1.0

97.7

1.4

99.6

9.7

19.7

18.6

50.4

0.9

5.5

13.8

30.0

85.6

(continued)

percent of families in the Bronx reported living in public housing, compared with only 10 

percent of families in Memphis. In contrast, the number of families receiving TANF benefits

was much higher in Memphis, at 31 percent, than the number in the Bronx, at 5 percent. The 

last rows of the table illustrate the material hardships faced by enrolled families. More than half 

of families reported being unable to pay a rent or utility bill within the previous year, and a 

quarter were unable to pay for food or needed medicine.

Table 1.3 shows that enrolled adults in the Bronx had somewhat lower education levels than those in Memphis, reflecting in part the fact that more of the enrolled adults in the
17

Characteristic Bronx Memphis

Medical coverage and care (%)

Has public health insurance 90.2 95.7

Has employer-provided or other health insurance 8.0 3.8

Usual source of medical care

Clinic or health center 79.8 55.3

Doctor's office or health maintenance organization 14.3 41.3

Other 5.9 3.4

Date of last medical checkup

Less than 12 months ago 94.2 88.2

12 months ago or longer or never had a checkup 5.8 11.8

Date of last dental checkup

Less than 12 months ago 85.8 86.3

12 months ago or longer or never had a checkup 14.2 13.7

Health status (%)

General health status

Excellent/very good 73.5 77.4

Good 23.5 19.9

Fair/poor 3.0 2.8

Has a physical problem that limits activities 4.7 5.4

Has an emotional or mental health problem that limits activities 6.0 5.5

Sample size 2,914 3,408

Table 1.4 (continued)

SOURCES: MDRC calculations from baseline survey data and random assignment module data 

and New York City Department of Education and Shelby County Schools administrative 

records.

NOTES: Percentages for some categories may not add up to 100 due to rounding or missing 

values. a

Proficiency level is only reported for high school students who had taken a standardized test 

to determine proficiency within the two years prior to enrollment. Data were available for most 

students who were in ninth or tenth grade at enrollment.

Bronx were immigrants.14 For example, nearly half of the adults in the Bronx sample did not 

have a high school diploma or equivalency certificate at the time of study enrollment, compared with only 31 percent of adults in Memphis. In contrast, adults in the Bronx sample were 

more likely to be working at the time of study entry (57 percent) than their counterparts in 

 14Foreign-born study participants in the Bronx had lower education levels than U.S.-born participants.
18

Memphis (44 percent). This difference is also related to immigrant status, given that foreignborn study participants in the Bronx sample were more likely to be working than their U.S.-

born counterparts.

Finally, Table 1.4 presents selected characteristics of the children in participating families, including the target ninth- and tenth-graders and their siblings. Other than race and ethnicity and English-language learner status, there were no large differences between the city samples. Most children had health coverage at study entry, about 15 percent of children had 

participated in an Individualized Education Program (IEP),15 and most parents reported that 

their children were in excellent or good health.

The Evaluation

The evaluation included a comprehensive study of Family Rewards 2.0, consisting of three 

components.16

1. Implementation analysis. This component includes a description and assessment of 

how families were recruited into the study and how the CCT incentives were marketed and 

communicated to families. This analysis, most of which was presented in the earlier implementation report, examines, for example, how parents and youth understood the Family Rewards 

cash incentives, how many rewards families received, what factors were associated with reward 

receipt, and what strategies the NPOs used to engage families. Because of the extended study 

enrollment period, the first program year (September 2011 through August 2012) was a partial 

year for families who enrolled later in the period. Similarly, the final program period (September 2014 through December 2014) was only four months long. For this reason, the analysis of 

reward receipt focuses on Years 2 and 3, when all families had had a full year to earn rewards.

Data for this analysis include observations of program activities (including family guidance sessions), interviews with advisors at the NPOs and Children’s Aid Society, focus groups 

with adult and high school student participants, a review of all program materials, a case-file 

review to analyze the implementation of the Family Earning Plans, and an analysis of the 

management information system and payment-processing data.

2. Impact analysis. This component examines the program’s effects on a wide range of 

outcomes. The analysis assesses whether Family Rewards 2.0 increased family income and 

 15Mandated by the Individuals with Disabilities Education Act, IEPs are intended to help children with 

disabilities achieve their educational goals. The percentage of students participated in an IEP was similar for 

the high school students and their younger siblings in the sample. 16Appendix E presents more information about the data sources and method used for each of the three 

study components and is available on the MDRC website at www.mdrc.org.
19

reduced material hardship, and whether the program affected a range of outcomes both directly 

and indirectly related to the reward milestones, such as children’s performance in school, health 

care use and outcomes, and parents’ employment and earnings and participation in training. The 

main analysis focuses on effects across the two cities combined, in order to provide a more 

general assessment of the effects. However, the report also examines whether the program’s 

effects differed between the two cities and for different types of families.

Data sources include administrative records on students’ school performance, parents’ 

employment and earnings, and families’ receipt of benefits. A survey was administered to 

families 24 months after study entry, capturing program experiences, income and material 

hardship, and other outcomes.17 Finally, depending on the data source, a family’s “follow-up” 

year does not always align with the “program year.” The education data, for example, were

aligned with the program year, since the school year began in September. Thus, for those who 

entered the study after September, Year 1 of education data includes several months before they 

were eligible for rewards. Impacts on education outcomes in Year 1 should be interpreted with 

this factor in mind. However, the follow-up period for all other data sources, such as unemployment insurance-covered earnings, started once a family entered the study, so that Year 1 

outcomes for these sources reflected a full year of program participation. 

3. Cost analysis. This component estimates the cost to administer rewards and the cost 

of reward payments to participants. Costs are estimated for each reward type. Cost data sources 

include the NPOs’ budgets and expenditure reports as well as reward payment records. The 

analysis shows how much money was transferred to participants in the form of rewards as well 

as how much it cost to administer the transfers. The analysis also explores the potential benefits 

to participants and how the program stands up to a benefit-cost test.

Although this report’s main goal is to present the effects of Family Rewards 2.0 after 

three to four years, comparisons are made throughout between these effects and those of Family 

Rewards 1.0. However, this comparison is not one of “apples to apples.” First, the two programs served different types of families. For example, Family Rewards 2.0 targeted families 

receiving TANF or SNAP benefits, whereas the original program targeted families eligible for 

the free lunch program. Families in the latter group therefore earned on average higher incomes. 

Second, the programs were tested in different places. Family Rewards 2.0 operated in the Bronx 

and Memphis, whereas the original program operated only in New York City neighborhoods 

(the Bronx, Brooklyn, and Manhattan). A more direct assessment of the new program’s added 

value would focus on effects for the same types of families in the same neighborhoods.

 17The 24-month survey achieved response rates of 81 percent for parents in the control group and 83 percent of parents in the program group. Appendix F includes a detailed analysis of survey response and is 

available on the MDRC website at www.mdrc.org. 
20

Organization of This Report

The remainder of the report is organized as follows. Chapter 2 discusses the implementation of 

the program across both cities and presents estimates of its costs. Chapters 3 through 6 present 

the program’s effects in each of the main domains, for both cities combined. Chapter 3 focuses 

on the effects on income and poverty, and Chapters 4 through 6 present the effects on each of 

the three domains of education, health, and work. Chapter 7 presents effects for selected 

subgroups and for each city separately. Chapter 8 summarizes the report’s findings and discusses some broad lessons from both Family Rewards programs.
21

Chapter 2 

Program Implementation and Reward Payments

MDRC’s earlier report on Family Rewards 2.0 described the implementation of the program 

during its first two years of operation, from September 1, 2011, to August 31, 2013.1 The 

program continued to offer rewards and guidance into a partial fourth year, through December 

2014. This chapter provides an update to that analysis, describing the delivery of the program 

during the final 18 months and how much money families earned in rewards overall. It gives 

special attention to the way the guidance component was implemented, as it was the biggest 

change to the program design compared with the earlier version.

Providing tailored, proactive assistance to each adult and high school student participant 

introduced a higher level of operational complexity to the conditional cash transfer (CCT) 

program, and increased the cost of delivering the program relative to Family Rewards 1.0.2 In 

the second year of the program, significant program resources were devoted to training designated staff, or advisors, and developing an approach to providing guidance that incorporated 

motivational interviewing (MI) techniques. The earlier implementation report found that this

shift improved the quality of advisor-participant interactions, although guidance sessions neither

occurred as often as called for by the model’s design (at least twice per year), nor were they 

consistently targeted to those earning the fewest rewards. Still, the earlier report predicted that 

participants would reap the benefit of these investments in program improvement in Years 3 

and 4 in the form of higher reward earnings — especially in challenging categories such as adult 

employment — since all of the program’s components were fully operational at that time. This 

chapter examines whether that prediction bore out.

 1

Dechausay, Miller, and Quiroz-Beccera (2014). 2

Few CCT programs around the world have attempted to combine individualized guidance with incentives. While there is broad recognition that families often face barriers to accessing quality services, it is more 

common to address this problem by facilitating peer support rather than pairing families with trained counselors. Two exceptions to this trend are the programs Solidario in Chile and Bolsa Familia in Brazil. In the 

Chilean model, social workers visit families frequently during the first 6 months after enrollment and intermittently for an additional 18 months (Calvo, 2011). They provide psychological support and facilitate preferential 

access to services in an effort to help families complete 53 conditions and graduate successfully from the 

program. Social workers in the Brazilian model perform the dual role of verifying that families are meeting the 

conditions for transfers and helping families overcome barriers. They can recommend that a family be dropped 

from the program for not meeting conditions (Soares et al., 2009).
22

Summary of Implementation Findings

The implementation analysis of Family Rewards 2.0 covered both cities and was partially 

conducted by Spanish-speaking researchers since some of the program activities in the Bronx 

took place in Spanish. The data collected included: (1) observations of program activities 

including guidance sessions and review of materials throughout the program; (2) annual 

interviews with family advisors at the NPOs and staff from Children’s Aid Society’s central 

office in Manhattan, New York (CAS-Central), which managed the overall program; (3) focus 

groups with 85 adults and 57 high school students who were members of subgroups of interest 

(for example, students who scored proficient or better on their eighth-grade reading exams and 

those who did not) or who earned a certain number of rewards in Years 1 and 2; (3) interviews 

with 21 adults and 18 high school students who were a mix of high and low reward-earners in 

Years 3 and 4; (4) analysis of all payment and case management data from CAS-Central’s

management information system (MIS); and (5) participation and engagement items on the 24-

month survey delivered only to program group members. The analysis of the costs of the 

program was based on data from the CAS-Central’s financial accounting system.

The analysis found that the guidance component evolved during the program, becoming 

more intense (focused on barrier identification rather than issues related to the mechanics of 

claiming rewards) after MI was adopted in Year 2. This shift improved the quality of guidance 

sessions midway through the program, though these sessions did not occur as often as envisioned in the model. In Year 3, strong distinctions emerged in the way that NPOs in the two 

cities delivered the guidance component. Advisors at NPOs in the Bronx found ways to increase 

the frequency and intensity of their interactions with participants, whereas in Memphis only 

one-quarter of adult participants met with their advisors more than once in Year 3 and most 

sessions were conducted over the phone. The transportation challenges in Memphis greatly 

impeded the ability of NPOs to deliver this program component effectively.

The payment processing system was well implemented and parents described it as efficient and accurate. On the other hand, many parents submitted coupons for rewards that were 

rejected, and reported a sense of discouragement with claiming certain incentives.3 Most 

enrolled families (99 percent) earned at least one reward during the program. These families 

received an average of $6,241 overall, about half of which was transferred directly to high 

school students’ bank accounts for education rewards. Most families earned rewards in education and health, but only about 57 percent ever earned a work reward.

 3

As noted in the earlier implementation report, coupons for report card grades and full-time work were 

rejected the most frequently, often because program participants submitted inadequate documentation.
23

The pattern of reward receipt across the two cities was fairly similar, although families 

in Memphis received an average of about $950 less than families in the Bronx. Over approximately three years, the program spent $13,459 on the typical participating family. Nearly half of 

this amount (48.3 percent) was for rewards paid.

The next section highlights the main aspects of the program’s operations. It is followed 

by more detailed descriptions of reward receipt and the guidance component and a benefit-cost

analysis.

The System for Processing Payments

The CAS-Central payment processing system ensured that participants received their earned

incentive payments in a timely fashion. As noted in Chapter 1, Family Rewards 2.0 offered 

eight rewards in three areas: education, health, and work. (See Table 1.1.) While Family 

Rewards 1.0 offered most of these same rewards, the new program sought to make these 

rewards easier to understand and claim for enrolled families. For example, program designers 

changed the criterion for earning the full-time work reward from demonstrating 30 hours of 

work per week to showing 120 hours of work per month. The reward for high school students’ 

grades was new and turned out to be very difficult to administer.4

Children’s Aid Society operated a central office in Manhattan, New York (CASCentral), for all participants in the evaluation. CAS-Central received documents from participants and administrative records from the school districts to verify completion of activities, 

determined whether participants were eligible for payments, and deposited payments in their 

bank or stored value card accounts. Staff who reviewed coupons also answered the program’s 

toll-free helpline, which parents, high school students, and advisors could call with questions 

about paperwork, bank account issues, or payments.

CAS-Central paid families every month for rewards earned about six weeks earlier. The 

lag between earning the reward and receiving payment was due to the time it took to manually 

verify eligibility for payments. The staff provided earnings statements in English and Spanish 

for each family that explained the earned rewards for which the family had been paid, as well as 

the rewards the family had not earned. CAS-Central staff created these statements manually and 

thus mailed them three to four weeks after payments were posted to participants’ accounts. 

Program designers had hoped payments would be received closer to the conditioned behavior to 

amplify the reinforcing effects of the incentives, but that was not achieved because the demonstration’s small scale precluded fully automating verification procedures.

 4

See Dechausay, Miller, and Quiroz-Beccera (2014) for more information.
24

The 24-month survey investigated parents’ understanding of the conditions to earn rewards and their experiences with the payment processing system. Generally, parents demonstrated a strong understanding of the behaviors that were eligible for rewards and the cash value 

of the rewards, although many believed the program rewarded additional behaviors that were 

not eligible for rewards. (See Appendix Table B.1.) The only reward about which there was 

widespread confusion among participants was the one for proficient or above performance on 

the Tennessee End-of-Course exams for high school students. While 94.8 percent of parents in 

the Bronx reported that their children knew the numeric score on the equivalent New York State 

Regents exams they needed to obtain to qualify for the reward, only 19.7 percent of parents in 

Memphis reported that their children understood the level of performance they needed to 

demonstrate on the End-of-Course exams to qualify for it. (Students needed to receive a score of 

“proficient” or above to earn the reward.) The End-of-Course exams in Memphis were new at 

the time Family Rewards 2.0 was launched, and students’ raw scores were scaled to determine 

proficiency. The survey findings suggest that either advisors did not succeed in explaining this 

reward to high school students in Memphis by the time of the survey, or the inherent uncertainty 

of this performance standard made it difficult for families to understand.

Parents gave the payment processing system strongly positive reviews on the survey, 

with about 81 percent agreeing with the statement that they received payments on time and 91 

percent reporting that they understood the reasons when they did not receive a reward or 

payment for a behavior. The majority of survey respondents (about 88 percent) also reported 

that they “remember the deadline for coupons and get them filled out completely.” However, 

despite this strong positive response among survey respondents, most families enrolled in the 

program had coupons rejected in Year 2. An MDRC analysis found that 90 percent of Memphis 

families and 62 percent of Bronx families had at least one rejected coupon that had not been 

resolved by the end of the year.5 In fact, almost every participant who submitted a coupon for 

full-time work had at least one rejected. There is evidence that these rejections led to discouragement. Forty-two percent of survey respondents reported that they had given up on earning 

rewards because “it is hard to get paid for things I do.” As discussed below, most families 

earned rewards throughout the program, but this finding suggests that individuals within 

families may have given up on earning particular rewards entirely or for a period of time due to 

challenges they experienced with the paperwork or criteria.

 5

In this case, “resolved” means that the error was addressed, the family or the staff member resubmitted 

the coupon, and the coupon was approved.
25

The Rewards Families Earned

Table 2.1 presents a summary of rewards earned by families in the Bronx and Memphis in all 

four program years. Years 1 and 4 were partial years for most participants, while Years 2 and 3 

were full years and can be directly compared. The table shows that 99 percent of families 

earned at least one reward, and they earned an average of 100 rewards. The high number of 

rewards is due to those offered for report cards, in which each qualifying grade was counted 

separately. In Years 2 and 3, families earned an average of just over $2,000 per year from 

rewards ($2,314 and $2,039, respectively), with about half of this amount paid to high school 

students for education rewards. Reward receipt was highest in the second year of the program, 

largely because of the high number of students who earned the high-value rewards for state 

exams (about 67 percent).

To put these annual earnings in context, the national average earned income tax credit 

(EITC) received for the 2014 tax year was $2,335.

6 The EITC has been described as “one of the 

nation’s most effective anti-poverty programs.”7 The EITC and Family Rewards incentives

differ in their certainty, salience, and beneficiaries. Family Rewards was less certain or predictable because the amount earned depended on which milestones families reached (and whether 

they made a proper claim for payment). For some milestones, particularly in education, there 

was a high performance standard. The reward money earned from Family Rewards was also, 

arguably, less salient because it was distributed each month in different amounts and intermingled with other money in the participant’s bank account. If the participant opened a special bank 

account to store money from the program, the deposits might have been salient because they 

were clearly marked, or less so if the participant often forgot to check the account or found it 

difficult to access the money. Lastly, Family Rewards distributed the incentives to parents and 

high school students, unlike the EITC, which is paid to the head of household.

The following section summarizes earnings in each of the domains, focusing on the patterns in Years 2 and 3 since these were the only years when all enrolled families were eligible 

for rewards for a full 12 months.

Education

● Almost all families (97 percent) earned at least one education reward due 

mainly to the attendance and report card rewards. In Year 2, for example, of 

families with students who earned an education reward, 88 percent earned at 

least one for attendance and 80 percent for grades. Students continued to 

 6

Internal Revenue Service (2016).

7

Kneebone and Holmes (2015). 
26

 

Table 2.1

 Rewards Earned by Families in Years 1 Through 4

Outcome Year 1a Year 2 Year 3 Year 4b Years 1-4 

Family earned at least 1 reward (%)

Among families who earned at least 1 reward

Average number of rewards earned

Average reward amount earned c

 ($)

Education rewards

Family earned at least 1 education reward (%)

Attendance

State core exam

SAT/ACT

Report card

Among families who earned at least 1 education reward

Average number of education rewards earned

Average education reward amount earned ($)

Health rewards

Family earned at least 1 health reward (%)

Annual physical

Semiannual dental

Among families who earned at least 1 health reward

Average number of health rewards earned

Average health reward amount earned ($)

Work rewards

Family earned at least 1 work reward (%)

Full-time employment (%)

High school equivalency certificate (%) 

Among families who earned at least 1 work reward

Average number of work rewards earned

Average work reward amount earned ($)

95.0

23.1

1,598

91.5

82.5

57.3

1.0

77.7

16.5

777

81.0

70.1

71.6

5.7

570

39.6

39.5

0.4

5.8

872

96.3

36.9

2,314

93.3

88.4

66.5

6.6

80.2

28.5

1,224

78.8

70.4

70.7

6.6

663

46.3

45.9

0.8

8.1

1,216

94.4

35.0

2,039

88.7

86.9

36.3

8.9

73.7

27.4

974

76.8

68.9

68.1

6.3

626

46.1

46.0

0.5

8.4

1,259

77.4

10.5

624

57.9

54.0

0.0

2.8

38.9

9.4

256

46.6

32.5

34.9

3.3

334

37.6

37.3

0.2

3.1

475

98.6

100.1

6,241

97.3

96.0

82.1

19.0

89.9

73.4

2,943

89.8

86.0

84.7

18.0

1,805

56.9

56.2

2.0

19.5

2,926

Sample size 1,230

(continued)
27

 

qualify for the attendance reward at a high level in Year 3 but the percentage 

of families with students who claimed the report card reward declined to 74 

percent. This reward turned out to be more difficult to administer than expected — the two cities had different grading scales, some courses were ineligible, report cards had to be distinguished from “progress reports,” and each 

school distributed different numbers of report cards. CAS-Central issued policy changes to address these challenges, but they may have led to some confusion or disappointment for families as they attempted to claim the reward. 

Further, the report card reward was meant to facilitate ongoing interaction 

between advisors and students as students brought in each newly issued report card to the NPO, but this interaction did not occur in most NPOs.

● More families earned the reward for passing at least one state exam in Year 2 

(67 percent) than in Year 3 (36 percent).8 Almost half of the students entered 

the program in the tenth grade, and very few students took these exams during their senior year (Year 3 for entering tenth-graders).

● Few families (19 percent) ever earned the incentive for taking the SAT or 

ACT college entrance exams. Most of the students who earned it were in the 

Bronx. In Memphis, schools offered the ACT for free but students had to 

earn a certain score to qualify for the reward.

 8

These figures are reported among students who earned any education reward. Ninety-three percent earned 

an education reward in Year 2 and 89 percent in Year 3.

Table 2.1 (continued)

SOURCE: MDRC calculations using Children's Aid Society's Family Rewards program data.

NOTES: Sample sizes may vary because of missing values.

Year 1 covers reward activities that occurred between September 2011 and August 2012. Year 2 

covers reward activities that occurred between September 2012 and August 2013. Year 3 covers reward 

activities that occurred between September 2013 and August 2014. Year 4 covers reward activities that 

occurred between September 2014 and December 2014.

Italics indicate outcomes calculated for a subset of the full sample.

aRandom assignment occurred between September 2011 and February 2012 and prevented many 

participants from receiving a full year of rewards.

bYear 4 only permitted participants to earn during four months of the program year.

cThe lowest and highest amounts earned in Year 1 were $20 and $8,851. The lowest and highest 

amounts earned in Year 2 were $38 and $8,200. The lowest and highest amounts earned in Year 3 were 

$40 and $7,550. The lowest and highest amounts earned in Year 4 were $10 and $3,110.
28

● In general, families in Memphis earned less in education rewards than 

families in the Bronx. As a group, students in Memphis families had lower 

grades (and the amount of the report card incentive was scaled to the 

grade), passed fewer state exams, and did not claim the college entrance 

exam reward as often.9

Health

● Families were highly engaged in the health domain, which offered rewards to 

the entire family including all children 19 years of age and under. Ninety

percent of families earned at least one health reward, and very similar proportions of families claimed the reward in Years 1, 2, and 3. Because Year 4 

was just four months in length, and insurance providers only pay for examinations at a certain periodicity, many participants could not claim this reward 

during this time. 

● Memphis adults faced barriers to claiming the dental reward because these 

visits were not covered by the public health insurance program in Tennessee as they were in New York State. In Year 2 (data not shown), threequarters of Memphis adults did not earn any rewards for dental visits. The 

family advisors made a concerted effort to build relationships with providers who offered free or low-cost cleanings, which increased access to this 

reward in later years.

Work

● Fifty-seven percent of families earned at least one work reward, and almost 

all of these rewards were for full-time employment. The proportion of families that claimed the full-time work reward was the same in Years 2 and 3 at 

around 46 percent. Of the families that earned it, they did so consistently, 

claiming the reward an average of 8 out of 12 months.

● Despite the fact that 48 percent of adults in the Bronx and 31 percent in 

Memphis did not have a high school diploma or equivalency certificate at the 

start of the study, almost no adults claimed the reward for earning the General Education Development (GED) certificate. The GED test was overhauled while the program was in operation. The advisors tried to use this

 9

It should be noted that the grading scales in the two cities were different. In Memphis, the range of scores 

that constituted a C was larger than in New York City, while the range of scores that constituted a B or an A 

was smaller. (See Table 1.1.) 
29

overhaul to motivate adults who were close to passing it to buckle down and 

get it done, but this campaign was not successful.

The pattern of reward receipt shows that the earlier Family Rewards 2.0 implementation

report did not accurately predict the future course of reward earnings in the program. Despite 

improvements to the quality of advising and other changes to outreach strategies and management techniques that will be reviewed in the next section, reward earnings did not increase. In

particular, advisors were not able to increase the number of rewards participants received for 

full-time employment by helping more participants find employment or increase their work 

hours. By Year 2, a larger share of households were claiming this work reward than those in 

Family Rewards 1.0, but the proportion did not increase in Year 3. The next section explores 

how the program set out to engage families in Years 3 and 4 of the program.

The Advisor-Participant Relationship at the End of the Program

Each NPO hired three advisors to work directly with families. Most were women who shared

characteristics with the majority of families in their caseloads in terms of race, ethnicity, and 

spoken language. Most were generalists coming from the social services or nonprofit sectors, 

but two were trained social workers and one was a former teacher. Each NPO had a program 

manager who oversaw the advisor’s work on site, and CAS-Central hired a social worker and 

business professional to provide additional technical assistance.

Each advisor served about 100 families (with at least one parent and one high school 

student per family), though some worked with fewer families because they were part time or 

less skilled, and some had many more for periods of time because of turnover in the position. 

Advisors had three main responsibilities:

1. Outreach: advertising the incentives, planning events, and maintaining a high level 

of general interest in the program.

2. Customer Service: helping participants fill out coupons and understand program 

rules, and serving as an on-site liaison between staff who reviewed documentation 

at the payment processing center and participants. A customized salesforce application that included screens with payment decisions and notes from guidance sessions 

and outreach attempts facilitated this relationship. 

3. Guidance: building personal relationships with the adults and high school students 

to provide a mix of emotional and instrumental support in reaching goals.

The original design model called for a triage approach, whereby advisors were to engage in these modes of interaction with all participants but offer more assistance to those who 
30

were struggling to earn rewards. Each adult and high school student was expected to receive 

guidance sessions twice a year to update an action plan called a Family Earning Plan and, at the 

end of the year, review earnings from the past period. Advisors were to contact participants 

whom CAS-Central had identified as low reward-earners usually for guidance sessions and 

other supports.

As noted in the earlier implementation report, the nature of the advisors’ work evolved 

significantly in the first two years of the program.10 Advisors focused mainly on building 

relationships in the first year, which involved advisors and clients getting to know each other 

and establishing trust and familiarity. One-on-one sessions at this time were largely transactional or customer service oriented. Advisors had few referral partners and did not know how to 

address some of the major life issues that arose, such as housing dislocation, serious health 

problems, domestic violence, infidelity, divorce, substance abuse, teenage pregnancy, and 

incarceration.

Recognizing that advisors needed more training to go beyond customer service and encourage behavioral change, CAS-Central introduced motivational interviewing (MI) in the first 

half of Year 2. MI is a widely used, directive approach to counseling in which the counselor 

uses a set of techniques to explore an individual’s ambivalence about change.11 Evaluations 

have found that MI can be an effective way of encouraging engagement with a program or 

improving the primary outcomes of treatment.12 It took several months of in-service support for 

all advisors to develop a minimum level of proficiency in the techniques. Observations of 

guidance sessions that took place during the second half of Year 2 showed that the content and 

feel of the sessions had in fact improved. Participants talked more about their goals, and took 

the lead in creating plans for overcoming barriers to earning rewards.

This change was positive, but participants did not receive as much of this quality advisement as intended. Table 2.2 summarizes the proportion of adult and high school student 

participants who received guidance as well as the proportion of them who received different 

types of guidance in the Bronx and Memphis in Years 2 and 3. In Year 2, most adults had one 

guidance session in the Bronx and Memphis (86 and 65 percent, respectively), but only a 

minority of these individuals had two or more such sessions (34 and 25 percent, respectively). 

Smaller percentages received referrals (13 and 19 percent, respectively) or discussed barriers or 

next steps (33 and 25 percent, respectively) with their advisors. These last outcomes were 

 10Dechausay, Miller, and Quiroz-Beccera (2014). 11Miller and Rollnick (2002). 12Hettema, Steele, and Miller (2005).
31

Table 2.2

Advising Received by Participants in Years 2 and 3, by City

Outcome

Bronx Memphis

Year 2 Year 3 Year 2 Year 3

Adult participants (%) a Participant had a guidance session

Had at least 1 FEP review

Had 2 or more FEP reviews

Ever received a referral

Ever discussed barriers to earning rewards or next steps

b Participant received customer service

Participant received outreachc

Sample size

87.3

85.7

33.6

13.1

32.7

69.5

87.9

655

90.8

88.7

64.4

62.8

29.0

73.0

81.2

655

72.4

64.8

25.0

18.5

24.7

72.6

93.7

631

76.4

66.4

26.9

38.8

12.8

43.0

68.8

631

Student participantsd

 (%) a,e Participant had a guidance session

Had at least 1 academic/FEP review

Had 2 or more academic/FEP reviews

Ever received a referral

Ever discussed barriers to earning rewards or next steps

b,f Participant received customer service

Participant received outreachc

Sample size

86.6

86.6

67.7

2.4

26.4

69.4

67.1

656

88.6

88.0

64.8

21.2

34.0

75.2

64.8

656

83.4

81.7

20.2

3.5

12.4

63.8

23.6

687

82.8

80.4

25.6

4.1

8.0

39.7

13.1

687

(continued)

indicators of the intensity of the guidance, or the likelihood that the guidance would generat

behavioral change in people who were not at the time on the path to earning certain rewards

Most adult participants received customer service and general outreach.

This trend in Year 2 was similar for high school participants in Memphis, with 82 percent of students having had one guidance session, but only 20 percent having had two or more

The percentage of these students that received referrals or discussed barriers and next steps in

guidance session was very small (4 and 12 percent, respectively). In contrast, in the Bronx

e 

.

. 

, 
32

advisors and high school participants interacted much more. Eighty-seven percent of these 

students received one guidance session, while 68 percent received two or more. These sessions, 

however, clearly did not focus entirely on motivating behavioral change since only 3 percent of 

the students received a referral and 26 percent discussed barriers. That said, a stark difference 

began to emerge in Year 2 between the guidance students received in Memphis and the Bronx. 

The advisors in the Bronx found that the students’ ability to travel to the NPO on their own 

facilitated these interactions, and though not technically required by the program, they asked 

students to come in every time they received a report card to ensure a high level of contact.13

 13In the first year of the program, students were required to bring in the coupon for signature in order to 

receive payment. Program designers intended for this meeting to occur quarterly and for advisors to use the 

students’ report cards to structure a conversation about their academic performance, and remind them about the 

potential reward earnings. This process did not work as well as hoped in practice because (a) many students in 

the Bronx received more than four report cards leading to a large number of meetings; (b) attending the inperson meetings was difficult for many families in Memphis who relied on cars, and so it was typical for 

parents in Memphis to bring several report cards at one time on behalf of their children; and (c) the meetings 

were largely transactional, appointments at which the advisor simply signed report cards and assisted with 

other administrative issues. In Year 2, the program designers removed the report card signature requirement 

because of the problems noted in the first year of the program and replaced it with a single mandatory meeting 

between students and advisors at the beginning of the school year. This meeting, called an “academic review,”

provided an opportunity for advisors to discuss students’ performance in school and plan for the year. Students 

who struggled to earn rewards after that point would be identified in the low-earners reports, which would 

prompt advisors to reach out to these individuals to offer additional support. Advisors in Memphis reported a 

(continued)

Table 2.2 (continued)

SOURCE: MDRC calculations using the Children's Aid Society's Family Rewards program data.

NOTES: FEP = Family Earning Plan.

Sample sizes may vary because of missing values.

Year 2 covers activities that occurred between September 2012 and August 2013. Year 3 covers 

activities that occurred between September 2013 and August 2014. a

Guidance includes creating FEP, reviewing FEP, discussing next steps, discussing barriers to 

earning rewards, discussing public benefits, and making a referral, discussing earnings goals, receiving 

support from the Family Resource Fund, or discussing family finances.

bCustomer service includes group check-ins, group workshops, discussing referral outcomes, 

submitting bank account information, adjusting family composition or eligibility, advisor signature for 

high school equivalency certificate completion, answering earnings questions, help-line calls, and coupon 

submission support. cOutreach includes program campaigning (outreach to particular participants with information or 

reminders), scheduling coupon book pick-up, program reengagement calls, sending e-mails, sending 

letters, and conducting automated calls.

dThis category includes students enrolled in ninth or tenth grade at baseline.

e

This category also includes report card review for students.

f

This category also includes advisor signature for student report cards.
33

Overall, most participants in Year 2 received only one intensive guidance session in 

which the advisor might have used MI techniques to promote engagement, although a lot of 

outreach and customer service occurred in this period (similar to what participants in Family 

Rewards 1.0 received).

In Year 3, there was some improvement in indicators of the intensity of the relationship 

between advisors and participants in the Bronx. More adults in the Bronx received two or more 

guidance sessions (65 percent in Year 3, compared with 34 percent in Year 2), and many more 

adults received referrals (63 percent in Year 3, compared with 13 percent in Year 2). Students in 

the Bronx who received at least one guidance session received more referrals in Year 3 (21 

percent of students in Year 3, compared with 3 percent in Year 2). This improvement was the 

result of a focused effort to develop referral partnerships in New York City, especially those 

with workforce development providers. 

In contrast, advisors in Memphis did not increase contact with participating adults or 

students in Year 3. Advisors in Memphis reported that because the guidance sessions were not 

mandatory and participants had to arrange car rides to get to the NPOs, many stopped visiting 

the NPOs after the first year. As a result, advisors progressively shifted to delivering the 

majority of their guidance sessions over the phone, which they acknowledged was far from 

ideal. Advisors in Memphis did increase their use of referrals (39 percent of adults in Year 3, 

compared with 19 percent in Year 2), but these new referrals were primarily for dental services

for adults. Advisors described a lack of quality workforce development providers. Advisors also 

reported lower rates of outreach and customer service in Year 3. This decrease may have been 

the result of a new performance management approach introduced in Year 3 that resulted in 

advisors in Memphis conducting more home visits. 

In an effort to further refine the guidance component, the program designers established 

quarterly benchmarks that provided specific targets for advisors to meet within the education, 

health, and work domains. Advisors were offered small incentives to motivate them to achieve 

these benchmarks. Each quarter, the program designers and CAS-Central defined a target group 

of adults and high school students based, for the most part, on their reward earnings outcomes, 

as well as provided NPOs with weekly reports on each advisor’s progress toward reaching the 

benchmarks. For example, from March to May 2014, the benchmarks directed advisors to 

increase the number of adults earning work rewards by five each, achieve a 30 percent increase 

in the number of adults earning a medical and dental health reward, and ensure that 75 percent 

of their student caseload had earned at least $75 in grades rewards in Year 3. Advisors in 

Memphis attempted to meet the benchmarks by more aggressively reaching out to eligible 

 

steep decline in visits to the NPO by participants after this policy change. Neither NPO in the Bronx announced 

this change to maintain a high level of contact with students.
34

participants, and, in most cases, offering to personally pick up paperwork participants had not 

turned in or conduct guidance sessions in participants’ homes. The time that advisors spent 

traveling to home visits affected their ability to provide services to participants generally, but it 

was seen as the only way to achieve the benchmarks. In most quarters, the benchmarks were not 

achieved, but the advisors reported that they appreciated them because they brought greater 

focus to their work.

In the end, the guidance component involved a lot of operational challenges related to 

training advisors, making contact with participants, and targeting the right individuals at the 

right times. Student participants in the Bronx had more ongoing contact with advisors than 

student participants in Memphis, who rarely met with their advisors one-on-one. While families 

in the Bronx earned more in education rewards than did families in Memphis, it is impossible to 

conclude that this difference was due to the relationship with their advisors because of many 

confounding factors. At best, it seems that the intensity of the guidance that participants in the 

Bronx received contributed to their engagement in the program but was not transformative. 

Even after advisors began using MI, participants did not experience a significant, positive 

change in the trajectory of their reward earnings. Dosage may have played a role in this result

— participants faced major life challenges and had relatively little contact with their advisors 

even in the best cases.

Program Costs

Over about three years, the program spent $13,459 on the typical participating family (Table 

2.3).14 The largest spending category was rewards paid to families (48.3 percent). It included the 

cash rewards in all three of the domains (education, health, and work), expenses associated with 

an enrollment incentive paid to families when they signed up for the study, and a discretionary 

pot of money called the Family Resource Fund that was available to advisors.

The remaining program costs (51.7 percent) were associated with the administration of 

the program by CAS-Central and the NPOs, including staff salaries, fringe benefits, consultants, 

the administration of payments and support services for participating families, and other costs. 

Staff salaries at CAS-Central accounted for the largest administrative cost ($2,452 per family). 

The costs associated with the NPOs in New York and Tennessee were the second largest

 14The direct cost estimate is based on program expenditure reports from 2011 to 2014. All dollar amounts 

have been converted to 2015 dollars using the Consumer Price Index for all urban consumers. As a result of 

this approach, the cost of “reward payments to participating families” ($6,507) in Table 2.3 is slightly different 

from the “average reward amount earned” per family ($6,241) presented in Table 2.1, which was based on

individual-level MIS data and was not adjusted for inflation.
35

administrative expense ($2,287). Fringe benefits for staff cost $617 per family served. Consultants cost $396 per family, or 3 percent of total costs. The notes to Table 2.3 include additional 

information on each spending category. Finally, the bottom row of the table indicates that for 

every dollar transferred to families for rewards, $1.07 was spent on non-reward costs.

To put the costs of the cash transfer in perspective, it is important to remember that 

Family Rewards was a small and new program. Size and newness increase the amount of nonreward costs, especially the administrative cost of processing payments; more established and 

larger programs typically serve participants at a lower cost. For example, Temporary Assistance 

for Needy Families (TANF) spends approximately 17 cents per dollar of payment, Supplemental Nutrition Assistance Program spends 20 cents per dollar of payment, and Medicaid pays 

5 cents per dollar of payment, although these programs also do not offer guidance to help 

families earn payments.15

 15The research team obtained data on the TANF costs and payments from the financial data that states 

submit to the Administration for Children and Families (data reporting form ACF-196). It obtained Food 

Stamp Program outlays and obligations data from the Food and Nutrition Service in the U.S. Department of 

(continued)

Percentage of

Cost Category Cost ($) Total (%)

Reward payments to participating families

Education rewards 3,030 22.5

Health rewards 1,713 12.7

Work rewards 1,764 13.1

Subtotal 6,507 48.3

Non-reward costs

Central operations

Staff salaries 2,452 18.2

Fringe benefits 617 4.6

Consultants 396 2.9

Other costs 1,200 8.9

Local operations

Partner organizations 2,287 17.0

Subtotal 6,952 51.7

Total cost 13,459 100.0

Non-reward cost per dollar of reward payment 1.07

Sample size 1,230

(continued)

Table 2.3

Cost per Family

(in 2015 Dollars)
36

Table 2.3 (continued)

SOURCES: Program expenditure reports from 2011 to 2015, MDRC calculations from management 

information system data, and phone interviews with program staff. 

NOTES: Estimates have been adjusted for inflation and reflect discounting.

In each domain the cash transfer value from reward payments is presented as a cost. In Tables 3 

and 4, an amount equal to the reward payments is presented as a benefit to participants. These 

amounts cancel each other and are cost and benefit neutral from the perspective of society. 

Description of cost categories: 

Reward payments to participating families include the conditional cash transfer payments (education, 

health, and work), the Family Resource Fund (a discretionary fund intended to assist each family earn 

certain rewards), and small incentive payments for opening a savings account. Resource fund and 

enrollment incentive payments were not tracked separately and therefore cannot be separated from 

conditional cash transfer payments.

Non-reward costs include all resources required to operate the program beyond the reward payments 

to participating families, such as the services to inform participants of available rewards, support 

participants in earning rewards, and process reward payments.

Staff salaries are remuneration paid to senior managerial staff (presidents, directors, and senior 

managers), programmatic staff (program associates, coordinators, assistants, advisors, and reward 

specialists), and administrative staff (financial and business analysts, general counsels, and 

information technology and database staff). Staff were responsible for the development of a payment 

tracking system, processing administrative records to determine if automatically verified rewards had 

been earned, creating coupon books for reward payments that required families to submit documents 

showing that they earned the reward, verifying requirements for coupon payment rewards were met, 

maintaining up-to-date bank account information to make sure payments were disbursed to the correct 

accounts, issuing “earnings statements” each payment period to mail to families, creating and 

maintaining a helpline to answer questions, making payments to families who earned rewards, 

marketing the program, general program management, and oversight of nonprofit organizations.

Fringe benefits – Fringe benefits for central operations staff in New York and Tennessee.

Consultants – Additional administrative support for Children's Aid Society to help the sites with 

managing grants, advertising, supporting audits, and general temporary staffing.

Other costs – Costs other than expenditures for personnel services. These expenditures include the 

cost of travel associated with program staff meeting with participants near their places of residence. 

Other costs also include supplies, outreach materials, insurance, rent for office space, and other 

indirect costs.

Partner organizations – All costs associated with the four partner organizations responsible for 

implementing Family Rewards, including Porter-Leath and Memphis HOPE in Tennessee and 

Children’s Aid Society and BronxWorks in New York. These costs include program orientations, 

refresher sessions, coupon book distribution, customer service, social events, and workshops.

Conclusion

Because few CCT programs include a guidance component, there were no direct comparisons

for the program that CAS-Central and its partners delivered. Family Rewards 2.0 aimed to bring 

about behavioral changes in participants by rewarding them not only for behaviors in which 

 

Agriculture. It obtained financial data on the Medicaid program from the Center for Medicare and Medicaid 

Quarterly Expense Report.
37

they regularly or periodically engaged before entering the program, but for engaging in new or 

sustained behaviors that would increase their human capital. Program developers reduced the 

number of the program’s rewards relative to Family Rewards 1.0 because of initial budget 

constraints, but they expected that the investment in advisors would be a positive addition by 

helping participants create action plans to maximize their receipt of the available cash rewards

and overcome barriers along the way.

CAS-Central delivered most of the incentives for Family Rewards 1.0 and made small 

changes to the verification rules for the new program in an effort to make the incentives easier 

to claim. However, the new rewards for report cards, designed to reward participants’ effort and 

educational input rather than their performance outcomes, raised difficult practical issues and 

did not generate the kind of ongoing interaction between students and advisors that the program 

designers had expected.

The guidance component was completely new and evolved substantially over the 

course of the program. Transportation obstacles in Memphis impeded contact between advisors 

and participants, and thus the component never gained traction. Advisors in the Bronx had 

greater contact with participants, but the frequency of the interaction was still less than anticipated (except, perhaps, for student participants).

Most families earned rewards for education and health during the first three years, and 

the share of families enrolled in Family Rewards 2.0 that claimed the reward for full-time 

employment was greater than that of families enrolled in Family Rewards 1.0. Still, the percentage of families earning work rewards did not increase over time. This result was disappointing 

considering the program’s many components aimed at changing the underlying behaviors in 

participants that would increase reward receipt, including intensive training for advisors, new 

targeting approaches, and the development of referral networks.

39

Chapter 3 

Impacts on Income and Material Well-Being

In the short run, the goal of Family Rewards was to reduce current poverty and material 

hardship through its direct cash reward payments — or conditional cash transfers (CCTs) — 

and through any immediate increases it might generate in parents’ earnings from employment. 

The previous chapter showed that families enrolled in Family Rewards 2.0 earned significant 

cash transfers during the program years, just over $2,000 per year.

This chapter examines the effects of the cash transfers on a broad range of income and 

well-being outcomes. Focusing on Year 2, roughly midway through the program, it considers 

how, and to what extent, participation in Family Rewards 2.0 helped to alleviate poverty and 

hardship. In contrast to Family Rewards 1.0, Family Rewards 2.0 targeted families who were 

receiving Temporary Assistance for Needy Families (TANF) or Supplemental Nutrition 

Assistance Program (SNAP, or “food stamps”), in order to direct resources to the neediest 

families, and offered fewer rewards in each of the three domains. As a result, a central question 

is whether the program led to comparable or even larger impacts on income and well-being for 

these families. The next several sections examine impacts on a range of measures of income and 

well-being. The chapter closes with a brief comparison of the findings from Family Rewards 

1.0 and 2.0. 

Summary of Impacts on Income and Well-Being

Overall, the results show that Family Rewards 2.0 achieved some of its immediate goals. 

Roughly two years after families enrolled in the program, positive effects on various dimensions 

of household income and well-being were evident. Family Rewards 2.0 increased average 

household income, reduced the proportion of families living in poverty, and increased the 

portion of families with savings. It also kept families in the program group connected to 

mainstream banking institutions. In addition, program group members were more likely to 

report higher levels of financial well-being, life satisfaction, and happiness, compared with their 

control group counterparts.

The general pattern of findings in this report are consistent with early results from the 

Family Rewards 1.0 evaluation, although the earlier program produced larger effects on income 

and poverty and financial well-being around 18 months of follow-up (when families were still 

eligible to earn rewards). However, there was no evidence that families in Family Rewards 2.0 

experienced less housing-related material hardship in the initial years, an effect observed in 

Family Rewards 1.0. One likely reason for this difference is that Family Rewards 2.0 offered 
40

fewer rewards and transferred less money to families. In addition, about half of the rewards that 

families earned were earned by — and paid to — high school students, funds that may not have 

contributed as directly to meeting basic family needs.

Data and Methods

The analyses presented here draw on the 24-month survey, as well as administrative records 

data on TANF and SNAP receipt provided by the local administrative agencies. The survey was 

administered to the full study sample and provides extensive data on 2,016 families — or 82 

percent of the full sample.1 The survey interviews were conducted between October 2013 and 

May 2014, the period roughly corresponding to Year 2. Thus, given the timing of the survey, 

the rewards earned in Years 1 and 2 (see Chapter 2) are most relevant for understanding the 

program’s effects on income and well-being while families were in the program. The chapter 

also draws on qualitative data obtained through in-depth interviews with a small sample of 

program participants.

Box 3.1 explains how to interpret the tables that show estimated program impacts that 

are presented throughout the remainder of this report. These tables cover a large number of 

impact estimates that are relevant to family poverty, hardship, and economic security and in the 

three domains in which the program provided rewards: children’s education, family preventive 

health care, and parents’ work and training. The research team calculated the estimates of 

program impacts controlling for a range of pre-random assignment background characteristics, 

such as the parents’ race or ethnicity, education level, marital status, and employment status. 

Appendix E provides more information on the data and analysis methods and is available on the 

MDRC website at www.mdrc.org. 

As the number of outcomes that are examined increases, the probability of obtaining 

impacts that are statistically significant simply by chance also increases. In order to address this 

issue, the impact analysis presented in each of the chapters distinguishes between a limited 

number of primary outcomes in a given domain versus secondary outcomes. If the research 

team found that the program affected primary outcomes in a given domain (for example, 

income and poverty), then it assigned more weight to the effects found for secondary outcomes 

within that domain (material hardship, for example). In contrast, if there were no effects on a 

 1

The survey provides information about sample members on a broad set of topics, including participation 

in employment and education activities, health care status, employment and job characteristics, household 

composition, and child outcomes. Appendix F provides an analysis of survey response and is available on the 

MDRC website at www.mdrc.org. 
41

Box 3.1

How to Read Impact Tables in This Report

In the context of this evaluation, an “impact” is a measure of how much Family Rewards 2.0 

changed outcomes for program participants. All the tables in this report that show impacts use 

a similar format, illustrated in the table excerpt below, which presents data on a material 

hardship and a well-being outcome that were obtained from parents’ reports on the 24-month 

survey. The top row, for example, shows that 41.4 percent of respondents in the program 

group had not paid their rent in full or made a full mortgage payment in the 12 months before 

the time of the survey interview, compared with 42 percent of control group respondents. 

Because families were assigned randomly either to a program group or to a control group, the 

effects of the program can be estimated by the difference in outcomes between the two groups. 

The “Difference” column in the table shows the differences between the two research groups’ 

outcomes — that is, the program’s estimated impacts on the outcomes. For example, the 

estimated program impact on paying full rent or mortgage can be calculated by subtracting 

41.4 percent from 42.0 percent, yielding a reduction, or estimated impact, of -0.6 percentage 

points.

The p-value shows the probability that this difference, or impact, arose by chance. In the table 

below, the difference between the program and control groups in paying the full rent or 

mortgage has a 79 percent probability of arising as a result of chance rather than as a result of 

the Family Rewards program. In contrast, the difference on the measure of financial well-being 

(bottom row) has a 2 percent probability of having arisen by chance. For this evaluation, only 

differences that have a 10 percent probability or less of arising by chance are considered 

“statistically significant” and therefore represent true program effects. The number of asterisks 

indicates whether the impact is statistically significant at the 1 percent (***), 5 percent (**), or 

10 percent (*) level. 

The final column shows the effect size, which is presented for selected impacts. The effect size 

is the difference between the program and control group outcomes divided by the “standard 

deviation” of the outcome (a measure of its variability). Expressing an impact in standard 

deviation units helps to interpret its size, particularly when the outcome is in nonstandard units, 

such as scores on a scale, as in the second row. In contrast, effect sizes are not shown for 

impacts that are easier to interpret, such as impacts on dollar amounts or percentages.

Impacts on Two Material Hardship and Well-being Outcomes,

from the 24-Month Survey

Program Control Difference 

Outcome Group Group (Impact) P-Value

Effect

Size 

Did not pay full rent or

mortgage (%) 41.4 42.0 -0.6 0.791

Financial well-being 

(4 = low; 16 = high) 9.3 9.1 0.2 ** 0.021 0.103
42

primary outcome within a domain, the research team gave the effects on secondary outcomes 

less weight in the analysis.2 Although this strategy helped to avoid placing undue weight on 

false positives, caution should be used when interpreting impacts that do not appear to be part of 

a larger pattern of impacts within a given set of measures. In addition, and as noted in Chapter 

1, the main analysis focuses on the Bronx and Memphis sites combined (or a pooled analysis), 

and Chapter 7 looks at how the effects vary by city and across other subgroups.

Impacts on Income and Poverty

To estimate household income and poverty, families completing the 24-month survey were 

asked to list their sources of income and total household income for the calendar month before 

the month of their survey interview.3 As the top panel of Table 3.1 shows, the estimated average 

monthly household income in that month was $1,636 for the program group (including rewards 

payments) and $1,498 for the control group, representing a statistically significant gain of $138 

— or about a 9 percent increase relative to the control group members’ income. On average, 

families in the program group earned an additional $184 each month in reward payments in the 

year before the interview. If reward payments are excluded, the average monthly household 

income at the time of the interview was similar for the two study groups.

The next several rows present the distribution of income relative to the federal poverty 

line.4 The table shows the distribution of this poverty measure across four levels, including 

“severe poverty,” which is defined as income below 50 percent of the poverty threshold. The 

estimate, which includes reward payments in the calculation of average household income, 

indicates that Family Rewards 2.0 altered the distribution of families across the different levels 

of poverty, in particular reducing the proportion of households in severe poverty and increasing 

the proportion with income at or above 130 percent of the federal poverty threshold. While 33.7 

 2

As a sensitivity test, standard errors for impact estimates for the primary outcomes were adjusted for multiple hypothesis testing. Appendix E (available on the MDRC website at www.mdrc.org) presents the results, 

which are generally similar to those presented in the main report. 3

Respondents were instructed to exclude tax refunds and program reward payments, and to include income from all other sources such as their job(s), jobs of other household members, food stamps, child support, 

TANF, Supplemental Security Income, and unemployment insurance, among other sources, for everyone 

living together in the household. They were also asked to exclude EITC payments because of the difficulty of 

converting them into a monthly amount. The research team combined the reported sources of income and used 

them to calculate household income and poverty relative to the federal poverty threshold, using the 2013 or 

2014 poverty guidelines, depending on when a respondent was interviewed. Given the particular definition, 

readers should be cautious comparing the poverty estimates presented here with estimates from other published 

sources. 4

The research team estimated annual household income based on the one-month household income snapshot gathered at the time of the survey interview.
43

 

Table 3.1

Impacts on Income and Income Sources

Outcome

Program

Group

Control

Group

Difference

(Impact) P-Value

Income and poverty 

Average total household income in month prior to

interview (excluding Family Rewards payments)a

 ($)

Average monthly Family Rewards payments

received during Year 2 ($)

Average total household income in month prior to

interview (including Family Rewards payments)a

 ($)

Household income at or below the federal poverty

level (including rewards)a,b (%)

Total household income in prior year as a percentage

of the federal poverty level (including rewards)a,b (%)

Less than 50% 

50% to 100%

101% to 129%

130% or more

Income sources (%)

Household income source in month prior to interview

Respondent's earnings

Other household members' earnings

Food stamps

Child support

Temporary Assistance for Needy Families (TANF)

or other cash assistance

Unemployment insurance 

Special Supplemental Nutrition Program for Women,

Infants, and Children (WIC)

Heating or cooling assistance

Free or reduced-price school lunch

Supplemental Security Income or Social Security 

Disability Insurance

Other

1,452

184

1,636

73.5

26.6

46.9

14.1

12.3

60.0

23.6

82.5

25.0

7.4

4.7

11.5

7.4

75.8

32.4

4.6

1,497

1,498

78.3

33.7

44.6

13.2

8.5

60.5

23.6

81.7

22.7

8.2

5.7

10.7

5.8

71.6

29.9

4.2

-45

138 ***

-4.8 **

-7.1 ***

2.3

1.0

3.8 ***

-0.5

0.0

0.8

2.3

-0.8

-1.0

0.9

1.6

4.2 **

2.5

0.4

0.332

0.004

0.012

0.001

0.313

0.539

0.006

0.794

0.996

0.632

0.210

0.515

0.311

0.538

0.153

0.033

0.199

0.655

Sample size (total = 2,016) 1,025 991

(continued)
44

percent of control group members’ income fell in the severe poverty category, only 26.6 percent 

of the program group members’ income fell in that category, representing a statistically significant drop of 7.1 percentage points. Similarly, 8.5 percent of the control group members earned 

an income at or above 130 percent of the federal poverty threshold compared with 12.3 percent 

of the program group, a statistically significant gain of 3.8 percentage points. This type of effect

on poverty reduction is generally expected in programs that offer cash transfers and was 

expected in Family Rewards 2.0. Despite being offering fewer opportunities to earn rewards

than participants in the original program, the majority of families in Family Rewards 2.0 earned

cash transfers, and the total annual payments per family averaged around $2,300 during the 

second year of the program. Overall, the effect on poverty reduction observed in Family 

Rewards 2.0 is consistent with, although smaller than, the effect seen in Family Rewards 1.0.5

The 24-month survey also asked respondents whether they had received income or benefits in the prior month from a variety of sources, including job earnings, government benefits, 

and child support. Program and control group members reported similar income sources and 

there was no evidence that the program increased or decreased the likelihood of receiving 

income from particular sources.

In addition to self-reported income sources, the research team examined administrative 

records data to estimate the program’s effects on benefits that participants received from TANF, 

 5

Riccio et. al (2013).

Table 3.1 (continued)

SOURCE: MDRC calculations using data from the Family Rewards 24-month survey.

NOTES: Sample sizes may vary because of missing values. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random 

assignment characteristics of sample members. Standard errors were adjusted to account for multiple 

observations per family. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 

percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences.

Italics indicate outcomes calculated for a subset of the full sample.

a

Monthly household income amounts equal to or greater than $10,000 were excluded from this 

calculation. About 4.9 percent of the sample was excluded from the income measures because 

respondents did not know or refused to provide the information. An additional 0.2 percent of the sample 

was excluded because the income provided was over $10,000.

bAnnual household income was calculated by multiplying the respondent's income in the month 

prior to the survey interview by 12. For program group members, it includes Family Rewards payments 

earned during program Years 2 and 3. The federal poverty level was calculated based on annual income 

(monthly income multiplied by 12) and the household size at the time of the survey. The poverty 

threshold was measured according to the 2013 or 2014 Poverty Guidelines, depending on when a 

respondent was interviewed. 
45

food stamps, and Safety Net Assistance (SNA) programs in New York City. Rewards payments 

were not counted as income or assets when determining eligibility for these benefits.6 Thus, the 

program would have only affected benefit receipt if it had encouraged participants to move into 

employment or earn higher wages. As shown in Appendix Table C.1, Family Rewards 2.0 

overall had no effect on TANF or SNA benefit receipt — on either the likelihood of receiving 

payments or the average dollar amount received.7 Similarly, program and control group members were equally likely to receive SNAP benefits, but program group members received higher 

SNAP payments than their control group counterparts, especially in Year 1 of follow-up — a 

statistically significant difference of $276. Appendix Table C.2 presents these results by site, 

showing that the effects on SNAP receipt were concentrated in Memphis. These effects might 

be related to the reduction in work also observed in Memphis (described in Chapter 7).

Impacts on Banking and Financial Services

Family Rewards 2.0 made payments electronically to families who earned rewards, depositing 

the money into bank accounts or onto stored value cards. Program staff encouraged families to 

open low-fee bank accounts in order to take advantage of the direct deposit of rewards, inviting 

banks that offered no-fee savings accounts to attend orientation sessions and market these 

accounts directly to families. Many families — almost 59 percent in New York City and 40 

percent in Memphis — did not have bank accounts when they entered the study and opted to 

open these new accounts. Direct deposit to bank accounts provided the program with an 

efficient mechanism for paying out cash rewards. In addition, linking families to mainstream 

financial institutions in this way had the potential to spur other positive effects, such as increasing savings and reducing the use of alternative services such as check cashiers or “payday 

loans,”8 often available to low-income households at very high interest rates and in ways that 

many experts contend mislead and exploit the poor.

Table 3.2 shows that almost 66 percent of the program group reported having a bank account at 

the time of the 24-month survey, compared with 44.4 percent of the control group respondents, 

 

 6

For the Bronx sample, the SNA program provides assistance to individuals and families who do not qualify for the time-limited federal TANF program. SNA payments may take the form of direct cash aid to 

beneficiaries or vendor payments (for example, rent payments to landlords) made on the SNA recipients’ 

behalf. 7

Recall that self-reported TANF receipt was much lower among Bronx participants than Memphis participants at study entry, even though the administrative data show similar levels of receipt during each year of 

follow-up for the two cities. These data are not inconsistent but indicate a higher degree of cycling onto and off 

of TANF rolls for the Bronx sample. This pattern is evident by the fact that quarterly TANF receipt rates were

much lower than yearly receipt rates for the Bronx sample. 8

A payday loan is a loan from a check-cashing outlet or other lending institution that must be repaid by the 

next payday. Payday loans are illegal in New York, but may be available through the Internet.
46

a statistically significant difference of 21.1 percentage points. As noted in Chapter 2, more than 

96 percent of families in the program group earned a reward in Year 2 and received payments 

through direct deposit to their bank accounts (since a very small fraction of participants opted to 

receive these payments via stored value cards). It is possible that some enrolled families had 

bank accounts that closed from time to time or changed their bank accounts during the study 

period, so that at any given moment a portion of the participants may have not had a functional 

Table 3.2

Impacts on Banking, Savings, and Debt

Outcome

Program

Group

Control

Group

Difference

(Impact) P-Value

Use of banking/financial services (%)

Currently has any bank account

Currently has checking account

Financial transactions at least once a month

Cash check at check casher

Pay bill at check casher

Family savings and debt

Average savings ($)

$0 (%)

Any (%)

$1 to $250

$251 to $500

More than $500

Average debt ($)

$0 (%)

$1 to $1,000 (%)

$1,001 to $5,000 (%)

$5,001 to $15,000 (%)

More than $15,000 (%)

65.5

44.2

18.4

26.3

145

79.5

20.5

7.9

4.3

6.2

7,308

43.6

8.6

18.0

14.8

14.2

44.4

35.2

22.8

27.6

82

88.0

12.0

4.9

2.1

3.5

7,012

41.5

7.2

18.1

20.0

11.9

21.1 ***

9.0 ***

-4.4 **

-1.3

63 **

-8.5 ***

8.5 ***

3.0 ***

2.2 ***

2.7 ***

295

2.1

1.4

-0.1

-5.2 ***

2.3

0.000

0.000

0.017

0.496

0.012

0.000

0.000

0.007

0.006

0.006

0.636

0.344

0.250

0.971

0.003

0.133

Sample size (total = 2,016) 1,025 991

SOURCE: MDRC calculations using data from the Family Rewards 24-month survey.

NOTES: Sample sizes may vary because of missing values. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random 

assignment characteristics of sample members. Standard errors were adjusted to account for 

multiple observations per family. Statistical significance levels are indicated as follows: *** = 1 

percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences.
47

bank account.9 Program group members were also more likely (by 9 percentage points) to 

report having a checking account than control group members. Family Rewards 2.0 thus clearly 

generated a net increase in participants’ connectedness to mainstream banking institutions.

Fewer program group members reported having used a check casher to cash a check in 

the month before the 24-month survey (18.4 percent of the program group compared with 22.8 

percent of the control group), a statistically significant difference of 4.4 percentage points. As 

noted above, Family Rewards 2.0 had a positive effect on linking families to mainstream 

financial institutions, which may have reduced their reliance on high-cost financial transactions 

through check cashers.

Savings and Debt

Families could use the cash rewards offered through Family Rewards 2.0 as they 

pleased — in other words, the program did not place any restrictions on how these payments

should be spent. The program designers believed that imposing restrictions (such as limiting the 

use of rewards for education, training, or home ownership, as some asset-building programs do, 

or for any other particular purpose) would have reduced the rewards’ real incentive value. 

Consequently, an important question is whether, without such restrictions, the families used the 

additional income from the program to build their financial security. Previous studies suggest 

that resources received in large, lump sums, rather than in smaller, more regular payments, are 

more likely to be saved, used to pay down debt, or invested in human capital.10

At the 24-month point, about 21 percent of the program group reported having some 

family savings, compared with 12 percent of the control group, a statistically significant 

difference of 8.5 percentage points. The program also increased average savings. Program 

group members reported $145 in average savings, compared with $82 by the control group, a 

difference of $63.11

While the 24-month survey asked respondents if they or their spouses or partners had any 

money or savings, it was unclear whether the substantial earnings by student participants were 

contributing to family savings. Nonetheless, the in-depth qualitative interviews suggest that 

parents actively encouraged their children to save. One respondent, for example, said, “[y]ou 

know, my children need to save money, instead of spending it in one day. Save money, all 

 9

A similar pattern was observed in Family Rewards 1.0.

10Chamber and Spencer (2008); Smeeding, Phillips, and O’Connor (2000). 11The averages are for full program and control group samples, respectively, not just for those study group 

members with any savings. The impact on enrolled families’ savings early in the study period was higher in 

Family Rewards 1.0 ($221), but the families in that program were offered and earned more rewards.
48

right? Put $5.00, leave $5.00 in there, as much as you can. And then you’ll see it, like June 

comes, let’s see how much you saved up between now and June.” 

Related to the broader question of how families used the cash rewards is the question of 

whether the additional money was used to pay off debt. Exploring this potential effect in Family 

Rewards 2.0, the 24-month survey asked respondents to indicate whether they had any debt and, 

if they had, the amount owed.12 Table 3.2 shows that both study groups reported relatively high 

amounts of family debt, with no difference on average ($7,308 for the program group versus 

$7,012 for the control group).

Across the range of indicators that this report examines, it appears that Family Rewards 

2.0 had a positive impact on families’ financial security. Families in the program group were 

more likely to maintain bank accounts and have savings. Given that both groups reported 

similar levels of debt, despite the program’s modest effect on savings, it is possible that families 

used much of their extra earnings — and potential savings — to address immediate household 

needs, hardships, and help improve their quality of life. The next section examines these issues.

Impacts on Material Hardship and Well-Being

An important question about Family Rewards 2.0 is whether the program’s cash transfers and 

positive effects on household income translated into reduced hardship and improved well-being. 

This section examines a range of indicators that are likely to be affected by changes in household income. Given that families in Family Rewards 2.0 were offered and earned smaller 

amounts of money from rewards than families in the original program, and given that a significant portion of the rewards were earned by students, it is an open question whether the program 

led to notable effects on material well-being.

Unfortunately, there is neither a common definition of material well-being, nor is there 

a standard approach to its measurement. Most efforts to measure this concept include scales or 

indicators of various dimensions of need — or hardships — across domains such as housing 

and utilities, financial strain, and unmet needs. Drawing on a comprehensive set of measures, 

the 24-month survey examined how families were faring at the time of the interview or in the 

recent past.

 12The survey questions on savings and debt were largely framed around family finances, and it is likely 

that respondents were reporting debt accumulated by the family rather than simply the respondents’ personal 

debt.
49

The top panel of Table 3.3 focuses on housing- and utilities-related hardships that families experienced in the 12 months before the time of the survey. Family Rewards 2.0 had little

effect on any of the measures used to capture housing-related hardships. Both study groups 

reported similar experiences with regard to the housing-related hardships that the survey 

measured — for example, not paying their full rent or mortgage or being evicted for not paying 

their rent or mortgage.13

Related to hardship, the survey asked respondents a series of questions related to unmet needs. Large numbers of program and control group respondents responded affirmatively 

to the question of whether they ever did not have enough money to buy food sometime in the 

past 12 months: 43.3 percent of the program group reported this hardship, compared with 

47.3 percent of the control group, a statistically significant 4 percentage point reduction in this 

hardship for program group families. No notable differences were seen on measures of unmet 

health care needs, such as being unable to receive needed medical care because of cost or 

being unable to fill prescriptions for the same reason. As described later in this report, health 

coverage rates were very high for the study sample, which may have assured families’ access 

to needed health care.

To assess respondents’ perceived financial circumstances more broadly, the survey 

asked respondents to compare their current financial situation with what it had been a year 

before on a financial strain scale. Exactly 60 percent of the program group reported that their 

current financial situation was better than it had been the year before, compared with 51.7 

percent of the control group, a statistically significant increase of 8.3 percentage points. Program group members also scored higher on a financial well-being scale than did control group 

members (9.3 points versus 9.1 points), indicating they had a more positive view of their 

financial well-being. The reader should keep in mind that the survey was conducted during Year 

2, when program group families were still earning cash rewards.

The final set of measures shown in Table 3.3 captures three dimensions of psychological well-being: belief in one’s ability to initiate and sustain action,14 perceived life satisfaction, 

 13On a more technical note, the survey captured the absolute presence or absence of a set of material hardships in a 12-month period and did not ask about the frequency or intensity of the hardships experienced, which 

the cash rewards may have affected. 14The research team used the State of Hope scale to measure “agency,” or whether or not a person feels 

motivated to initiate and sustain action (for example, goal-directed thinking). The team took the listed items 

from Snyder et al. (1996). The Hope Scale scores correlate positively with measures of dispositional 

optimism, self-esteem, and problem-solving ability, and correlate negatively with measures of depression 

and hopelessness.
50

 

Table 3.3

Impacts on Material Hardship, Financial Strain, and Psychosocial Well-Being

Outcome

Program

Group

Control

Group

Difference

(Impact) P-Value

Effect

Size

Any housing/utilities material hardship in 

the past 12 months (%)

Did not pay full rent or mortgage

Evicted from home for not paying 

rent or mortgage

Did not pay full utility billa

 a Utility was turned off

Phone service was disconnectedb

Food security (1 = high; 4 = low)c

Insufficient foodd

 (%)

Strongly or somewhat agree with 

the following (%)

Financial situation is better than last year

Do not worry about having enough 

money in future

Can generally afford to buy needed things

Sometimes have enough money to buy 

something or go somewhere just for fun

Financial well-being (4 = low; 16 = high)e

Did not have enough money to buy food 

sometime in the past 12 months (%)

Did not get needed medical care because 

of cost in past 12 monthsf (%)

Did not fill prescription because

of cost in past 12 months (%)

Psychosocial well-being

Average score on "State of Hope" scale

(6 = low; 24 = high)g

How life today compares to way it was

a year ago (%)

Much or somewhat better

Level of happinessh (%)

Very or pretty happy

64.4

41.4

5.5

42.6

13.3

22.0

3.2

25.4

60.0

20.7

70.9

30.8

9.3

43.3

11.1

18.5

17.8

66.4

76.2

64.1

42.0

5.4

42.9

11.8

23.6

3.2

26.7

51.7

23.3

66.2

31.7

9.1

47.3

11.1

19.5

17.6

58.3

72.2

0.3

-0.6

0.2

-0.3

1.5

-1.6

0.0

-1.3

8.3 ***

-2.6

4.7 **

-0.9

0.2 **

-4.0 *

-0.1

-1.0

0.2 **

8.2 ***

4.0 **

0.879

0.791

0.864

0.876

0.293

0.400

0.495

0.497

0.000

0.160

0.023

0.665

0.021

0.074

0.974

0.573

0.042

0.000

0.037

0.030

0.103

0.090

Sample size (total = 2,016) 1,025 991

(continued)
51

 

and a global level of happiness.15 It is possible that programs that involve regular cash transfers 

can reduce stress and bring about improvements in individual emotional and psychological 

well-being. There is well-established literature supporting a positive correlation between lifesatisfaction measures and income, and one theme in the literature on happiness is the idea that 

life satisfaction may depend on relative rather than absolute income.16

The data show consistently positive effects on all three measures of psychosocial wellbeing. Program group members rated themselves higher on the State of Hope scale, were more 

likely to report that life was better than a year ago, and were also more likely to rate themselves 

as being pretty happy. Although Family Rewards 2.0 increased household income, the impacts 

were modest (a gain of $138, or 9.2 percent), and despite the income gains, program and control 

group participants experienced similar levels of material hardship — in other words, the cash 

transfers allowed for some minor reductions in hardship, but did not lead to notable changes in 

material well-being.

 15The 24-month survey used the same item that has been used on the U.S. General Social Survey. 16Lachowska (2015). 

Table 3.3 (continued)

SOURCE: MDRC calculations using data from the Family Rewards 24-month survey.

NOTES: Sample sizes may vary because of missing values. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment 

characteristics of sample members. Standard errors were adjusted to account for multiple observations per 

family. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 

percent. 

Rounding may cause slight discrepancies in calculating sums and differences.

The effect size is the difference between program and control group outcomes expressed as a proportion of 

the standard deviation of the outcomes for both groups combined. aUtilities include gas, oil, and electricity. 

bThis outcome includes cellular or land service. 

cThe food security question describes food eaten by the family in the prior month: 1 = Enough to eat of the 

kinds of food desired; 2 = Enough to eat but not always the kinds of food desired; 3 = Sometimes not enough 

to eat; 4 = Often not enough to eat.

dInsufficient food is defined as "sometimes" or "often times" not having enough food to eat.

eComponents of the financial well-being scale have been coded such that a lower score implies being 

worse off and a higher score implies being better off. The scale was calculated by summing responses to the 

four component questions, each of which is scored on a four-point scale. Thus, the financial well-being scale 

presented here ranges from 4 to 16 points. 

f

This outcome excludes prescriptions.

gThe "State of Hope" scale measures the level of ongoing goal-directed thinking. The response codes (1 to 

4) of the six items for each person are summed, with lower values representing less goal-directed thinking 

and higher values representing more. The scale was taken from Snyder et al. (1996). 

hHappiness is measured using the U.S. General Social Survey question: "Taken all together, how would 

you say things are these days — would you say that you are very happy, pretty happy, or not too happy."
52

In what ways could Family Rewards 2.0 have made study participants feel happier and 

better about themselves? In-depth interviews conducted with a small number of program group 

families offer a few insights into the lives of study participants and the direct and indirect ways 

in which the cash transfers could have had an impact on their sense of well-being.17

At the most basic level, families enrolled in Family Rewards 2.0 felt supported in the 

program and viewed their advisors as a core source of support in their sometimes unpredictable 

lives. As one participant stated: 

They’re not there physically to help you, but...emotionally, they are. Yeah, and 

if you do need them, they’re a phone call away. You understand? Like I said, and 

it helps me and I just love it and the people that work here that I work with is 

great. Every time I call, it’s a great feeling. You can feel comfortable talking 

with them. 

The in-depth interviews also revealed a perspective common among participants that

the program — with its incentives — functioned as an important parenting tool, something that 

motivated their children to work harder and do better at school. As children performed better in 

school, parents may have experienced some relief and reduced stress as they had one less 

parenting concern. As one participant put it: 

Oh hell yeah. They got him up and out of the house to school. That’s my reward. 

But his grades improved, his attendance has improved and he’s actually opening 

up books. The first day of high school, [I said] “where’s your books?” He’s like, 

“Oh, I leave them at school.” “So how you supposed to study?” That’s my reward. I don’t care about the cash or anything. My reward is that my son improved in his grades and so forth, so on.

The extra money families earned from rewards allowed parents to spend more freely on 

themselves and their children, as they wished. The program designers had hoped that families 

would use the extra money in ways that align with program goals. Drawing on the 24-month 

survey, Table 3.4 presents parents’ accounts of how they used the cash rewards they earned. 

Parents most commonly used the rewards they earned to cover regular expenses (such as rent, 

food, and utilities) and to purchase consumer goods or special treats for the family (such as

eating out). With respect to the rewards that students earned, most parents reported that they 

used those earnings to buy little extras (such as eating out, going to a movie, or buying clothes)

and student-focused investments, and not for households expenses such as rent and utilities 

(although about a fourth of the parents also acknowledged that they used the education rewards 

to help cover basic expenses). Based on their responses, it appears that parents tried to balance 

 17These insights are suggestive and are not meant to explain causal links.
53

their responsibility to provide for the basic needs of their family with a desire to celebrate their 

achievements. For many families, this celebration took the form of a family activity such going 

on a trip or to a movie that would have otherwise been prohibitively expensive, especially for a 

large family with limited means.

Thus, across different measures, the cash rewards themselves and the supportive relationships with advisors, may have moderated and alleviated some of the stress that low-income 

parents commonly experience, which may have been reflected in participants’ improved sense 

of well-being.

Conclusion

This chapter examined Family Rewards 2.0’s short-term effects on household poverty and wellbeing. For the most part, the results support a CCT program’s hypothesized effects. During the 

period in which enrolled families supplemented their household income with reward payments, 

the program produced modest gains in household income; reduced the proportion of families 

who were living in poverty, including severe poverty; increased the proportion of families with 

Table 3.4

How Families Used Rewards, Parent Survey Responses

24 Months After Random Assignment

Use Family Rewards Payments (%)

Parent 

Bank 

Account

Child 

Bank 

Account

Help pay for regular expenses, such as rent, utilities, or food

Pay off bills, such as credit cards or medical bills

Make a major purchase, such as a house, major appliance, or car

Save for some future need, such as college tuition or retirement

Pay for health care, dental care, or health insurance

Pay for things to help children in school, such as special lessons or private schools

Pay for a few extras, such as eating out, going to a movie, or buying electronics or clothes

Help other family members or friends with expenses

Pay for college tuition or other costs for training

65.6

29.8

7.9

15.0

8.2

19.9

44.4

6.6

8.1

23.8

5.8

2.4

23.3

3.2

16.2

71.0

5.5

8.2

Sample size 1,025

SOURCE: MDRC calculations using data from the Family Rewards 24-month survey.

NOTES: This table excludes control group members because it pertains only to the Family Rewards program.

Sample sizes may vary because of missing values. 

Percentages may sum to more than the number participating in any activity because sample members 

could list more than one response.
54

household savings, and reduced reports of financial strain. It also consistently produced positive 

effects on multiple dimensions of perceived well-being. There is less evidence, however, that 

the program reduced more extreme types of hardship measured in this study, which may have 

required more sizeable cash transfers. Finally, the research team examined variation in impacts 

across a range of baseline characteristics, including poverty status at random assignment. As 

Chapter 7 will show, effects on income and poverty were fairly consistent across several types 

of families.

It is important to recall that, unlike Family Rewards 1.0, Family Rewards 2.0 targeted 

families receiving TANF and food stamps. This iteration of Family Rewards also offered fewer 

benefits (8 instead of 22), which meant the total amount that families in the program could have

earned was less than the amount that families enrolled in its predecessor could have earned. 

However, Family Rewards 2.0 offered rewards monthly, potentially making the cash transfers 

more salient to families’ monthly budgets and material well-being. One other design modification relevant to understanding the program’s potential to affect material well-being was that the 

education rewards earned and paid directly to students made up a significant portion of total 

family earnings but were not always used toward meeting basic household needs.

Despite several important design modifications, the findings from the Family Rewards 

2.0 evaluation presented in this report are largely consistent with those from Family Rewards 

1.0. For example, in the short term, both interventions improved enrolled families’ economic 

and financial well-being, reduced poverty, linked families to mainstream banking institutions, 

and increased average family savings. One notable difference is that unlike in Family Rewards 

1.0 there was no evidence from the 24-month survey that families in Family Rewards 2.0 

experienced less housing-related hardship. While the extra income from the rewards helped 

families in both programs improve their material well-being, Family Rewards 2.0 was less 

likely than Family Reward 1.0 to affect more extreme types of hardships, perhaps because 

families in Family Rewards 2.0 earned less than their counterparts in Family Rewards 1.0.
55

Chapter 4 

Impacts on Education

Family Rewards aimed to improve students’ school outcomes by providing students with 

incentives for meeting important milestones, including good grades, high attendance, and 

passing high school exit exams. The original program (Family Rewards 1.0) improved school 

performance and graduation rates for academically proficient ninth-graders. The new program

(Family Rewards 2.0) targeted families with children entering ninth or tenth grade and was 

modified in the hopes that the changes would allow it to have effects on all students, even those 

struggling academically.1 This chapter presents findings over four years on students’ progress in 

school, for the full sample of ninth- and tenth-graders and for subgroups defined by academic 

proficiency. In sum, the findings show that Family Rewards 2.0 had few lasting effects on 

students’ school outcomes, either for the full sample of students or for the more academically 

proficient subgroup. 

The Education Rewards

Students could earn education rewards through four activities. First, they could receive $40 per 

month for maintaining high attendance, defined as 95 percent or higher. Second, they could earn 

$50 for taking college entrance exams, either the SAT or the ACT. In Memphis, students 

routinely take the ACT as part of school district policy. Thus, students earned the rewards for 

scoring above a minimum threshold.2 Third, students could earn money for passing grades on 

report cards, with $10 for each C, $20 for each B, and $30 for each A. Finally, students could 

earn substantial rewards for taking and passing high school exit exams. In the Bronx, students 

could earn $500 for a score of 75 or higher (or $400 for a score between 65 and 74) on each of 

the five New York State Regents exams required to graduate.3 In Memphis, they could earn 

$300 for passing each of the seven End-of-Course exams administered by the school district.

Family Rewards 2.0 differed from Family Rewards 1.0 in several ways. First, as mentioned in Chapter 1, it offered fewer rewards in an effort to streamline the program and make it 

simpler for families to understand. In the case of education rewards, Family Rewards 2.0 did not 

include rewards for credits earned, for example, for graduation, or for parents to attend parent-

 1

The program targeted entering ninth- or tenth-graders since they would have at least three years of exposure to the rewards. 2

If they took the exam independently of the school system, any score was rewarded.

3

Students are only required to pass these exams with a score of at least 65 in order to graduate. However, 

achieving a score of 75 or higher will exempt students from remediation at many New York colleges and is 

used as an admissions cutoff by some universities.
56

teacher conferences. Second, it structured the rewards in such a way as to make them more 

salient to families and children, by paying out benefits monthly rather than every two months, 

and by offering students rewards for grades on report cards. The program designers hoped that 

the rewards for grades, in particular, would be more salient and present in the minds of students 

as they attended and performed in class from week to week. The program also paid all of the 

rewards directly to students’ accounts, whereas some education rewards students earned in 

Family Rewards 1.0 were paid to parents’ accounts. During the first year, students had to visit 

an advisor in order to receive the grade rewards, which meant more interaction with the program.4

 In contrast, the rewards for high school exit exams were paid automatically and did not

require a meeting with an advisor. While still important, these automatic and nonrecurring 

rewards were forgotten by many students in Family Rewards 1.0 during the course of the year. 

Finally, although not specific to the education domain, a big change from the original model 

was the more active guidance provided to families. If a child was struggling to improve school 

performance, the advisor was expected to work with the family to refer the student to tutoring or 

other relevant services.

As described in Chapter 2, each enrolled family earned about $1,200 in Year 2 from the 

education rewards, representing just over 50 percent of the total amount earned from all three 

domains. Over 90 percent of families earned at least one education reward, with the highest 

receipt rates for high attendance and report card grades of C or higher.

Summary of Impacts on School Progress

This section presents impacts for ninth- and tenth-grade students in Memphis and the Bronx 

combined. (As shown in Chapter 7, the findings are generally similar across both cities.) In 

addition, it presents impacts for ninth-graders only, in order to compare the findings more 

directly to those from Family Rewards 1.0. The research team obtained data on key education 

outcomes from administrative records provided by the New York City Department of Education 

(DOE) and the Memphis City Schools. These data were available for all students in the study 

for roughly one year before study entry, or school year 2010-2011, and for four years after study 

entry, or for school years 2011-2012 through 2014-2015. As noted in Chapter 1, the follow-up 

years do not correspond exactly to post-enrollment years, since some families entered the study 

well into the first school year. For example, over 30 percent of families entered the study in 

January or February 2012, meaning that their children could not have responded to the rewards 

 4

The mandate to meet with an advisor for every report card review was lifted in Year 2, in response to 

difficulties students faced in getting to the NPOs. However, as noted in Chapter 2, NPOs in the Bronx thought 

that these face-to-face meetings were important and did not announce that this requirement was dropped in 

Year 2. 
57

until the second half of that school year. Thus, the second and third school years provide a more 

complete account of the program’s effects within a given year.

The DOE and Memphis City Schools data include enrollment status, attendance rates, 

credits earned, and performance on Regents exams (in the Bronx) and End-of-Course exams (in 

Memphis). These data do not provide information for students attending parochial schools, 

private schools, or schools outside of New York City or Memphis, but the survey data indicate 

that few students in the sample (about 5 percent) attended these other types of schools. The 

survey provides information on intermediate outcomes, such as parental effort (parents’ interaction with their children and their children’s teachers), children’s engagement in extracurricular 

activities, and parents’ ratings of their children’s school performance.

In sum, Family Rewards 2.0 had few effects on important markers of school progress 

through Year 4. The research team examined several outcomes, all of which are predictive of 

high school graduation, including whether students were “on grade” in a given year, or in the 

grade they should have been if they had progressed each year, attendance rates, credits earned, 

and the number of high school exams passed. The program did not have effects on these 

outcomes for the full sample of entering ninth- and tenth-graders, or for the subset of students 

who were more academically prepared. (Recall that the main effects of Family Rewards 1.0 

were for proficient ninth-graders.) When the analysis focused only on entering ninth-graders, 

there were small, positive effects observed on attendance and credits earned. However, these 

effects did not persist and did not lead to increased enrollment rates or graduation rates. The 

next sections present effects on these main outcomes and several secondary outcomes.

Impacts on Attendance, Credits, and High School Exams

Table 4.1 presents effects on several outcomes for students who entered the study as ninth- or 

tenth-graders. All of the outcomes, with the exception of graduation, cover three years, since 

many entering tenth-graders would not have been in school during Year 4. Graduation, in 

contrast, was measured through Year 4, capturing entering tenth-graders who took an extra year 

to finish school.

The first panel presents enrollment status. Data for the control group show that 78 percent of entering ninth- and tenth-graders were on grade in Year 3. Of those students not on 

grade (22 percent), most were still in school but in a lower grade (12 percent). The remaining 10 

percent were no longer enrolled in either a Memphis or New York City public school. Survey 

data indicate that most of these students were attending other schools and had not dropped out. 

Family Rewards had no effects on these measures of school progress. Similarly, the program 

had no effect on graduation rates, as shown in the second panel.
58

Table 4.1

Impacts on Enrollment, Graduation, Attendance, Credits, 

and Regents or End-of-Course Exams for Students in Grades 9 and 10 

at the Time of Random Assignment

Grade Level and Outcome

Program

Group

Control

 Group

Difference

(Impact) P-Value

Enrollment, Year 3 (%)

On grade

Not on grade

Not enrolled

Graduation (%)

Graduated on time

Graduated by Year 4a

Attendanceb

 (%)

Attendance rate, Year 1

Attendance rate, Year 2

Attendance rate, Year 3

Attendance rate is 95% or higher, Year 1

Attendance rate is 95% or higher, Year 2

Attendance rate is 95% or higher, Year 3

c Credits

79.4

11.7

8.9

63.4

66.1

86.9

82.5

77.5

44.3

36.8

28.9

78.2

11.8

9.9

63.1

66.1

87.4

80.5

75.8

43.5

35.9

29.1

1.2

-0.1

-1.0

0.3

0.0

-0.4

2.0 *

1.7

0.8

0.9

-0.2

0.456

0.908

0.368

0.855

0.995

0.578

0.057

0.162

0.685

0.645

0.910

Number of credits earned, Year 1

Number of credits earned, Year 2

Number of credits earned, Year 3

Number of credits earned, Years 1 to 3

Earned adequate credits, Years 1 to 3 (%)

d,e Regents or End-of-Course exams

State core exams taken, Years 1 to 3

State core exams passed, Years 1 to 3

10.9

10.0

9.1

30.1

55.4

6.4

2.7

11.0

9.9

8.9

29.9

55.0

6.3

2.7

-0.1

0.1

0.2

0.2

0.4

0.1

0.0

0.704

0.568

0.415

0.651

0.833

0.344

0.716

Sample size (total = 2,676) 1,343 1,333

(continued)
59

The third panel presents data on attendance. The program had no overall effect on attendance, with the exception of a small, positive impact on the attendance rate in Year 2. A 

minority of control group students had high attendance in any given year — only 36 percent in 

Year 2 and 29 percent in Year 3. As noted in the table, all outcomes shown include zero values 

for students who were no longer enrolled in school. 

Although rewarded by the program, a high attendance rate may not give an accurate 

picture of students’ attendance. Chronic absenteeism, for example, is typically defined as less 

than 90 percent attendance. Only 66 percent of students in both research groups attended 90 

percent of school days in Year 1, indicating that just over a third of students were chronically 

absent (not reported in table). The program had no effect on the high attendance rate (as shown 

in Table 4.1) or on attendance of 90 percent or higher (not shown).

The fourth panel in Table 4.1 presents data on credits earned during the year — the

average credits students earned and whether the student passed an adequate number of credits 

Table 4.1 (continued)

SOURCES: MDRC calculations using data from New York City Department of Education and Shelby

County Schools administrative records.

NOTES: Estimates were regression-adjusted using ordinary least squares, controlling for pre-random 

assignment characteristics of sample members. Standard errors were adjusted to account for multiple 

observations per family. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 

percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences. 

Note that all outcomes in the table include zero values for students who were no longer enrolled.

Years 1, 2, 3, and 4 cover the 2011-2012, 2012-2013, 2013-2014, and 2014-2015 school years, 

respectively. aStudents enrolled in tenth grade at the time of random assignment had five years to complete 

graduation in this measure. Students enrolled in ninth grade at the time of random assignment had four 

years.

bAttendance was calculated as a percentage of total days present divided by total days enrolled 

according to district records. Records provided for students in New York City include enrollment for 

the regular school year. Records for students in Memphis include enrollment during the regular school 

year, alternative education programs, and summer school. cStudents in New York City earn 1 credit per course per semester completed. Students in Memphis 

earn 0.5 credits per course per semester. Credits for students in Memphis were multiplied by two to 

create a standard scale for comparison. To be considered on time to graduate, students in New York 

City must earn an average of 11 credits per school year and students in Memphis must earn an average 

of 5.5 credits per school year.

dThe Regents exam measures in this table include the following Regents exams: English, Math A, 

Math B, Geometry, Integrated Algebra, Algebra 2/Trigonometry, U.S. History and Government, 

Global History and Geography, Living Environment, Chemistry, Physics, and Earth Science. e

The End-of-Course exam measures in this table include the following exams: English 1, English, 

2, English 3, Biology, Algebra 1, Algebra 2, and U.S. History.
60

to stay on track for graduation. A student passing adequate classes to stay on track would earn 

11 credits per year in the Bronx and 5.5 credits in Memphis. To standardize the analysis, the 

research team multiplied the data on credits by two for Memphis students. The data show that 

although 78 percent of students were on grade in Year 3, only 55 percent of them were on 

track in terms of credits earned. By rewarding higher grades and attendance, Family Rewards 

2.0 might have affected credits earned. However, there are no impacts on either average or 

on-track credits.

The final panel presents data on high school exit exams, or Regents exams in New York 

and End-of-Course exams in Tennessee. Students in New York must take and pass at least five 

Regents exams covering different subject areas (for example, math, English, science, and 

history) in order to graduate with a Regents diploma. They must take them before graduation, 

although most students take one or two each year of high school. Students in Memphis must 

take End-of-Course exams associated with each of seven core courses. Students typically take 

them after having completed the relevant course. Although it is possible to pass the course 

without passing the End-of-Course exam, the exam accounts for a large fraction of the final 

course grade. The data show that students on average had taken about six exams on average 

during the study period and had passed about three exams. Family Rewards 2.0 had no effect on 

the rate of taking or passing these exams.

Although Family Rewards 2.0 rewarded grades, with a tiered reward structure for 

grades of A, B, and C, the impact analysis did not examine the effects on grades for two 

reasons. First, although it was reasonable to expect that the program might encourage some 

students to improve their grades, which should translate into earning more credits, grades by 

themselves are ultimately not crucial measures of students’ progress toward graduation. More 

predictive outcomes are attendance, credits earned, and grade progression. Second, using grades 

to calculate an overall grade point average might have provided a noisy picture of student 

progress, since it would have required making many assumptions about which course grades 

should contribute to this average and how much they should contribute.

Table 4.2 presents Family Rewards 2.0’s effects on entering ninth-graders over four 

years. It displays impacts separately for this group, to match the analysis for Family Rewards 

1.0. In addition, although it is not obvious that the program’s effects would differ between 

ninth- and tenth-graders, ninth grade is an important transition year into high school, and these 

students might have been more responsive to the rewards. The findings show that the program 

led to small improvements in attendance and credits earned for the ninth-grade sample. Although encouraging, these effects were small and did not lead to effects on enrollment or 

graduation. Just over 60 percent of ninth-graders in the program group had graduated within 

four years, with a similar rate for ninth-graders in the control group.
61

Table 4.2

Impacts on Enrollment, Graduation, Attendance, Credits, 

and Regents or End-of-Course Exams for Students in Grade 9

at the Time of Random Assignment

Grade Level and Outcome

Program

Group

Control

 Group

Difference

(Impact) P-Value

Enrollment and Graduation, Year 4 (%)

On grade

Not on grade

Not enrolled

Graduated on time

Attendancea

 (%)

Attendance rate, Year 1

Attendance rate, Year 2

Attendance rate, Year 3

Attendance rate, Year 4

Attendance rate is 95% or higher, Year 1

Attendance rate is 95% or higher, Year 2

Attendance rate is 95% or higher, Year 3

Attendance rate is 95% or higher, Year 4

b Credits

71.4

10.4

18.2

61.7

86.7

82.4

79.2

61.8

45.0

37.0

33.0

17.4

69.7

10.4

19.9

60.7

86.8

79.7

76.1

58.8

43.1

34.9

32.9

17.7

1.7

0.0

-1.7

1.0

-0.1

2.7 **

3.1 **

3.0

1.9

2.1

0.1

-0.4

0.482

0.990

0.408

0.701

0.921

0.041

0.040

0.160

0.464

0.409

0.968

0.854

Number of credits earned, Year 1

Number of credits earned, Year 2

Number of credits earned, Year 3

Number of credits earned, Year 4

Number of credits earned, Years 1 to 4

Earned adequate credits, Years 1 to 4 (%)

Regents/End-of-Course examsc,d (%)

State core exams taken, Years 1 to 4

State core exams passed, Years 1 to 4

10.8

10.2

9.5

8.6

39.1

54.0

7.6

3.1

10.7

9.7

8.9

8.3

37.6

53.1

7.2

3.1

0.1

0.5 *

0.6 **

0.3

1.5 *

0.9

0.4 *

0.0

0.763

0.071

0.044

0.279

0.094

0.728

0.098

0.976

Sample size (total = 1,471) 734 737

(continued)
62

Impacts on Parent-Child Interactions and Child Activities

Family Rewards 1.0 had hoped to increase parents’ involvement in their children’s education 

through the education rewards in general, as well as through specific rewards for attending 

parent-teacher meetings and checking test results. The program did lead to small increases in 

parents’ involvement, as reported on surveys. It also led to an increase in engagement in 

extracurricular activities (such as sports, non-sports lessons, and tutoring) for children in middle 

school and high school.

Although Family Rewards 2.0 did not reward parental involvement directly, it may 

have led to increased parent-child interactions about schooling through the general education 

rewards. Table 4.3 presents the effects on these interactions, using parents’ responses in the 

24-month survey. Overall, the findings suggest that the program had little effect in this area, 

given the small number of significant differences and their small size. For example, there are 

Table 4.2 (continued)

SOURCES: MDRC calculations using data from New York City Department of Education and Shelby 

County Schools administrative records.

NOTES: Estimates were regression-adjusted using ordinary least squares, controlling for pre-random 

assignment characteristics of sample members. Standard errors were adjusted to account for multiple 

observations per family. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 

percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences. 

Note that all outcomes in the table include zero values for students who were no longer enrolled.

Years 1, 2, 3, and 4 cover the 2011-2012, 2012-2013, 2013-2014, and 2014-2015 school years, 

respectively. aAttendance was calculated as a percentage of total days present divided by total days enrolled 

according to district records. Records provided for students in New York City include enrollment for 

the regular school year. Records for students in Memphis include enrollment during the regular school 

year, alternative education programs, and summer school.

bStudents in New York City earn 1 credit per course per semester completed. Students in Memphis 

earn 0.5 credits per course per semester. Credits for students in Memphis were multiplied by two to 

create a standard scale for comparison. To be considered on time to graduate, students in New York 

City must earn an average of 11 credits per school year and students in Memphis must earn an average 

of 5.5 credits per school year. cThe Regents exam measures in this table include the following Regents exams: English, Math A, 

Math B, Geometry, Integrated Algebra, Algebra 2/Trigonometry, U.S. History and Government, Global 

History and Geography, Living Environment, Chemistry, Physics, and Earth Science.

dThe End-of-Course exam measures in this table include the following exams: English 1, English, 2, 

English 3, Biology, Algebra 1, Algebra 2, and U.S. History. The U.S. History exam was not 

administered for students in Year 4.
63

Table 4.3

Impacts on Parent-Child Interactions and 

Focal Child’s Educational Outcomes and Activities,

High School Students

Outcome

Program Control Difference

Group Group (Impact) P-Value

Effect

Size

Parent-child interactions in past month 

Respondent has done the following

(1 = never; 4 = several times per week)

Talked with child about school

Helped child with homework

Checked to see child's homework was complete

Helped child prepare for test

Respondent discussed child's report card with child

during 2012-2013 school year (%)

Activities since random assignment (%)

Child has done the following since random assignment

Participated in educational programs, such as extra 

classes or tutoring

Enrolled in college exam preparation program

Received special education

Prepared for the ACT or SAT

Took the ACT or SAT

Child participated in any extracurricular activity

Program to help with schoolwork, test preparation, or homework

Sports

Non-sports lessons

Club or youth group

Work for pay

3.7

2.4

3.3

2.4

95.8

60.5

66.1

15.9

84.9

57.7

87.8

59.0

60.5

42.2

46.5

25.8

3.7

2.5

3.3

2.5

93.9

60.6

68.5

15.2

83.5

62.5

85.7

59.4

57.2

42.6

41.1

22.6

0.1 *

0.0

0.0

0.0

1.9 *

-0.1

-2.4

0.8

1.4

-4.9 **

2.1

-0.5

3.3

-0.4

5.4 **

3.2

0.079

0.459

0.692

0.737

0.070

0.950

0.264

0.572

0.418

0.026

0.185

0.830

0.141

0.865

0.018

0.107

0.081

-0.034

0.018

-0.016

Sample size (total = 1,881) 976 905

SOURCE: MDRC calculations using data from the Family Rewards 24-month survey.

NOTES: This table presents outcomes only for focal children who were living in the household at the time of 

the survey and at random assignment. 

Sample sizes may vary because of missing values.

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment 

characteristics of sample members. Standard errors were adjusted to account for multiple observations per 

family. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences.

The effect size is the difference between program and control group outcomes expressed as a proportion of 

the standard deviation of the outcomes for both groups combined.
64

two significant impacts on parent engagement (talking with children about school and reviewing report cards). However, these effects were small and in addition to fairly high levels of 

engagement.

The next panel presents effects on various activities, including college-preparatory and 

extracurricular activities. Family rewards 2.0 had no consistent effect on the five educational 

activities examined, with the exception of an odd negative difference for taking the ACT or 

SAT. Similarly, the results suggest little effect on extracurricular activities. 

Impacts for Proficient Versus Non-Proficient Students

Although Family Rewards 1.0 had modest effects on ninth-graders overall, its main effects were 

for the subgroup of ninth-graders who were academically proficient when they entered the 

study. For students who were proficient in reading (or scored at the proficient level or higher on 

the English language arts exam), for example, the program led to sizable effects on enrollment,

attendance, scores on Regents exams, and, ultimately, graduation. The program’s effect on the 

four-year graduation rate, an 8 percentage point increase, was especially notable and similar in 

size to other more intensive interventions.5 The program designers modified the original model 

with the hopes that it would continue to affect school progress for proficient students but also 

begin to improve school outcomes for those students that were doing less well academically.

This section presents effects for students classified by proficiency status at study entry. 

Using each district’s standards, the research team determined proficiency status by students’ 

scores on their eighth-grade standardized English or math tests. In the Bronx, 25 percent of 

students were determined to be proficient in English and 43 percent were proficient in math.6

Among students in Memphis, proficiency rates were 14 percent in English and 7 percent in 

math.7

Table 4.4 presents Family Rewards 2.0’s effects for entering ninth- and tenth-graders, 

by English proficiency test scores. As expected, data for the control groups show that students 

 5

A longer-term analysis found that the program increased the six-year graduation rate for this group by 9.5 

percentage points (Riccio and Miller, 2016). 6

In Family Rewards 1.0, about 30 percent of entering ninth-graders had scored at the proficient level or 

higher on the English language arts (ELA) exams and 33 percent were proficient in math. 7

The percentage of proficient students enrolled Family Rewards 2.0 was much lower in Memphis, compared with those in the Bronx and compared with the sample in the first program. To compare findings, a 

separate analysis examined impacts for the students who scored in the top third for the ELA exams, compared 

with students who scored in the bottom two-thirds. The findings (not shown here) similarly indicate few 

differences in impacts between the two groups.
Table 4.4

Impacts on Enrollment, Graduation, Attendance, Credits, and Regents or End-of-Course Exams

for Students in Grades 9 and 10 at the Time of Random Assignment, by English Proficiency Test Score

Proficient on Eighth-Grade ELA Test Not Proficient on Eighth-Grade ELA Test

Program Control Difference Program Control Difference

Grade Level and Outcome Group Group (Impact) P-Value Group Group (Impact) P-Value

Enrollment, Year 3 (%)

On grade 89.9 91.8 -2.0 0.446 80.7 78.3 2.4 0.186

Not on grade 7.5 3.3 4.1 ** 0.041 12.0 13.5 -1.5 0.331 ††

Not enrolled 2.7 4.8 -2.2 0.223 7.3 8.3 -1.0 0.439

Graduation (%)

Graduated on time

65

80.9 79.4 1.4 0.700 62.5 62.0 0.6 0.802

Graduated by Year 4a 81.7 81.9 -0.2 0.966 64.9 65.0 -0.2 0.934

Attendanceb (%)

Attendance rate, Year 1 90.3 88.4 1.8 0.363 87.2 88.4 -1.2 0.148

Attendance rate, Year 2 88.2 86.5 1.7 0.418 82.7 81.4 1.3 0.256

Attendance rate, Year 3 85.7 83.8 2.0 0.372 78.5 77.0 1.5 0.264

Attendance rate is 95% or higher, Year 1 57.6 54.6 3.0 0.499 42.4 42.8 -0.4 0.866

Attendance rate is 95% or higher, Year 2 49.2 48.5 0.7 0.870 35.4 34.5 0.9 0.699

Attendance rate is 95% or higher, Year 3 37.9 42.3 -4.4 0.309 28.3 27.2 1.1 0.585

c Credits

Number of credits earned, Year 1 12.3 12.3 0.0 0.935 10.8 11.0 -0.2 0.458

Number of credits earned, Year 2 11.3 11.4 0.0 0.964 10.0 9.9 0.1 0.799

Number of credits earned, Year 3 10.8 10.6 0.2 0.670 9.2 9.0 0.3 0.302

(continued)
66

Table 4.4 (continued)

Proficient on Eigth-Grade ELA Test Not Proficient on Eigth-Grade ELA Test

Grade Level and Outcome

Program Control Difference

Group Group (Impact) P-Value

Program Control Difference

Group Group (Impact) P-Value

Number of credits earned, Years 1 to 3

Earned adequate credits, Years 1 to 3 (%)

d,e Regents or End-of-Course exams

Number of state core exams taken, Years 

1 to 3

Number of state core exams passed, Years

1 to 3

34.5

71.4

6.0

4.4

34.3

70.7

6.2

4.7

0.2

0.7

-0.2

-0.3 *

0.858

0.869

0.371

0.098

30.0

54.3

6.7

2.4

29.8

53.9

6.5

2.3

0.2

0.4

0.2

0.1

0.762

0.876

0.276

0.473 †

Sample size (total = 2,433) 225 248 1,008 952

SOURCES: MDRC calculations using data from New York City Department of Education and Shelby County Schools administrative records.

NOTES: ELA = English language arts. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. 

Standard errors were adjusted to account for multiple observations per family. Statistical significance levels are indicated as follows: *** = 1 percent; 

** = 5 percent; * = 10 percent. Statistical significance levels across subgroups are indicated as follows: ††† = 1 percent; †† = 5 percent; 

† = 10 percent.

Rounding may cause slight discrepancies in calculating sums and differences. 

Note that all outcomes in the table include zero values for students who were no longer enrolled.

Years 1, 2, 3, and 4 cover the 2011-2012, 2012-2013, 2013-2014, and 2014-2015 school years, respectively. aStudents enrolled in tenth grade at the time of random assignment had five years to complete graduation in this measure. Students enrolled in 

ninth grade at the time of random assignment had four years.

bAttendance was calculated as a percentage of total days present divided by total days enrolled according to district records. Records provided for 

students in New York City include enrollment for the regular school year. Records for students in Memphis include enrollment during the regular 

school year, alternative education programs, and summer school. cStudents in New York City earn 1 credit per course per semester completed. Students in Memphis earn 0.5 credits per course per semester. 

Credits for students in Memphis were multiplied by two to create a standard scale for comparison. To be considered on time to graduate, students in 

New York City must earn an average of 11 credits per school year and students in Memphis must earn an average of 5.5 credits per school year.

dThe Regents exam measures in this table include the following Regents exams: English, Math A, Math B, Geometry, Integrated Algebra, Algebra 

2/Trigonometry, U.S. History and Government, Global History and Geography, Living Environment, Chemistry, Physics, and Earth Science. eThe End-of-Course exam measures in this table include the following exams: English 1, English, 2, English 3, Biology, Algebra 1, Algebra 2, 

and U.S. History.
67

who were proficient on eighth-grade tests had better school outcomes than their less proficient 

counterparts — 92 percent of proficient students were on-grade in Year 3, for example, compared with 78 percent of non-proficient students. Similarly, proficient students had earned more 

credits and had passed more high school exams by the end of Year 3. The program, however,

had no effects for either group of students.

Table 4.5 presents Family Rewards 2.0’s effects for entering ninth-graders only, by 

English proficiency test scores. The table shows that the program had small, positive effects on 

attendance and credits earned for the less proficient group. However, these effects were not 

significantly different from the effects for the proficient students. In addition, the effects by 

proficiency status are not consistent. When the research team examined the impacts by math 

proficiency test scores, the results showed a pattern of more positive effects for students

proficient in math, although many of the differences were not statistically significant. (See 

Appendix Tables D.1 and D.2.) 

Conclusion

Family Rewards 2.0 had few lasting effects on school progress for the target group of ninth- and 

tenth-graders. There is some suggestive evidence that ninth-graders may have responded more 

strongly to the program than tenth-graders, but overall the program still had no effect on their 

enrollment or graduation rates. The results by proficiency status were not conclusive and the 

results depended on whether proficiency was defined by reading or math.

This finding stands in contrast to the large, positive findings for proficient ninthgraders in Family Rewards 1.0. Although it is it unclear why the new program had no effects, 

the results suggest that the effects of this type of conditional cash transfer on school progress 

were not very robust. One potential reason for the different effects is that the samples of 

proficient students were somewhat different between the two programs. First, in 2010, New 

York State made several changes to its standardized tests, making the tests more difficult and 

redefining the cutoff required for proficiency. As a result, the percentage of students in New 

York City that tested proficient dropped dramatically, meaning that proficient students in 

Family Rewards 2.0 might have been a more select group. However, additional analysis (not 

shown) that used a less stringent measure of proficiency found similar results. Second, the 

Family Rewards 2.0 results were for students from families receiving TANF or SNAP 

benefits in both Memphis and the Bronx, whereas the earlier findings were just for students in 

New York City who were eligible for the free lunch program. However, Family Rewards 2.0 

similarly produced no effects for proficient students when the sample was restricted to ninthgraders from families living in the Bronx.
Table 4.5

Impacts on Enrollment, Graduation, Attendance, Credits, and Regents or End-of-Course Exams 

for Students in Grade 9 at the Time of Random Assignment, by English Proficiency Test Score

Proficient on Eighth-Grade ELA Test Not Proficient on Eighth-Grade ELA Test

Grade Level and Outcome

Program Control Difference

Group Group (Impact) P-Value

Program Control Difference

Group Group (Impact) P-Value

Enrollment and Graduation, Year 4 (%)

On grade

Not on grade

Not enrolled

Graduated on time

Attendancea (%)

Attendance rate, Year 1

Attendance rate, Year 2

Attendance rate, Year 3

Attendance rate, Year 4

Attendance rate is 95% or higher, Year 1

Attendance rate is 95% or higher, Year 2

Attendance rate is 95% or higher, Year 3

Attendance rate is 95% or higher, Year 4

b Credits

81.4

1.5

17.1

78.9

90.0

87.9

86.7

70.6

54.9

50.9

42.8

23.0

86.7

2.5

10.9

83.5

89.8

86.2

85.3

70.0

56.7

47.3

49.1

23.9

-5.3

-0.9

6.2

-4.6

0.2

1.7

1.4

0.6

-1.9

3.6

-6.2

-0.9

0.272

0.610

0.172

0.400

0.944

0.590

0.637

0.902

0.759

0.576

0.322

0.877

70.5

12.4

17.1

60.0

86.8

82.6

79.3

60.9

43.9

35.3

32.3

17.1

67.7

12.4

19.9

57.9

87.2

79.9

75.9

57.4

41.2

33.1

29.5

16.7

2.8

0.0

-2.8

2.1

-0.4

2.7 *

3.4 *

3.5

2.7

2.2

2.8

0.4

0.315

0.996

0.230

0.476

0.699

0.070

0.051

0.153

0.353

0.433

0.296

0.873

†

Number of credits earned, Year 1

Number of credits earned, Year 2

Number of credits earned, Year 3

Number of credits earned, Year 4

12.3

11.4

10.8

9.8

12.5

11.5

11.5

10.2

-0.2

-0.1

-0.7

-0.4

0.792

0.942

0.309

0.606

10.5

10.1

9.4

8.6

10.4

9.5

8.6

8.2

0.1

0.6 *

0.8 **

0.4

0.698

0.062

0.016

0.238

††

(continued)

68
69

Table 4.5 (continued)

Proficient on Eighth-Grade ELA Test Not Proficient on Eighth-Grade ELA Test

Grade Level and Outcome

Program Control Difference

Group Group (Impact) P-Value

Program Control Difference

Group Group (Impact) P-Value

Number of credits earned, Years 1 to 4

Earned adequate credits, Years 1 to 4 (%)

c,d Regents or End-of-Course exams

Number of state core exams taken, Years 

1 to 4

Number of state core exams passed, Years 

1 to 4

44.3

68.0

6.8

5.2

45.6

74.9

6.8

5.2

-1.3

-6.9

0.0

0.0

0.576

0.246

0.995

0.981

38.7

52.9

7.9

2.7

36.7

50.1

7.5

2.7

2.0 *

2.8

0.4

0.1

0.059

0.347

0.120

0.703

Sample size (total = 1,391) 115 132 588 556

SOURCES: MDRC calculations using data from New York City Department of Education and Shelby County Schools administrative records.

NOTES: ELA = English language arts. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. 

Standard errors were adjusted to account for multiple observations per family. Statistical significance levels are indicated as follows: *** = 1 percent; 

** = 5 percent; * = 10 percent. Statistical significance levels across subgroups are indicated as follows: ††† = 1 percent; †† = 5 percent; † = 10 

percent.

Rounding may cause slight discrepancies in calculating sums and differences. 

Note that all outcomes in the table include zero values for students who were no longer enrolled.

Years 1, 2, 3, and 4 cover the 2011-2012, 2012-2013, 2013-2014, and 2014-2015 school years, respectively. a

Attendance was calculated as a percentage of total days present divided by total days enrolled according to district records. Records provided for 

students in New York City include enrollment for the regular school year. Records for students in Memphis include enrollment during the regular 

school year, alternative education programs, and summer school.

bStudents in New York City earn 1 credit per course per semester completed. Students in Memphis earn 0.5 credits per course per semester. 

Credits for students in Memphis were multiplied by two to create a standard scale for comparison. To be considered on time to graduate, students in 

New York City must earn an average of 11 credits per school year and students in Memphis must earn an average of 5.5 credits per school year. c

The Regents exam measures in this table include the following Regents exams: English, Math A, Math B, Geometry, Integrated Algebra, Algebra 

2/Trigonometry, U.S. History and Government, Global History and Geography, Living Environment, Chemistry, Physics, and Earth Science.

dThe End-of-Course exam measures in this table include the following exams: English 1, English, 2, English 3, Biology, Algebra 1, Algebra 2, 

and U.S. History. The U.S. History exam was not administered for students in Year 4.

71

Chapter 5 

Impacts on Health Care and Health

Conditional cash transfer programs have been an important tool used in low- and middleincome countries to improve low-income families’ access to health care and preventive medical 

services. Historically, these programs have often featured incentives for regular checkups at 

health clinics, participation in health and nutrition education sessions, and engagement in other 

preventive health activities such as vaccinations or pre-natal care.1 Some previous studies using 

health incentives have had positive effects on the receipt of health care services, as well as on 

certain health outcomes among poor families.2 Family Rewards 2.0 incorporated these findings 

into its design and adapted incentives to support low-income families in the context of the 

health care system and public support available in the United States. The program offered 

rewards to low-income families for preventive medical and dental care visits. This chapter 

describes the health impacts of Family Rewards 2.0. In sum, the findings show that the program 

produced some modest effects on outcomes that are generally considered difficult to influence 

through social interventions.3 However, the program did not produce effects on many other 

important measures.

The Health Care Rewards

Each family member in Family Rewards 2.0 was eligible to earn $300 each program year from 

two different rewards.4 Individuals could earn $100 for completing a non-emergency check-up 

once per year and $100 for completing a regular cleaning and dental check-up twice per year. 

Family Rewards 1.0 offered two additional rewards for maintaining health insurance and for 

 1

Fernald, Gertler, and Neufeld (2008); Fiszbein, Schady, and Ferreira (2009). 2

Giuffrida and Torgerson (1997); Kane, Johnson, Town, and Butler (2004); Volpp et al. (2008); Volpp et 

al. (2009). 3

The literature on the effectiveness, or the relative value, of preventive clinical interventions is vast and 

inconsistent. In general, the response to preventive clinical interventions is weak. In a systematic review and 

analysis of recent interventions, insufficient evidence was found to support the effectiveness of a variety of 

counseling and preventive care practices; see Maciosek et al. (2006). While some evidence suggests that health 

insurance alone has beneficial effects on life expectancy, these impacts are small or conclusive only for some 

population groups; see Muennig, Franks, and Gold (2005) and Levy and Meltzer (2008). Recent evaluations of 

the Oregon Health Experiment found that access to Medicaid increased use of services and improved selfreported health status, but they did not find effects on biomarkers associated with physical health; see Finkelstein et al. (2011). On the other hand, the effects of social interventions, such as improved early childhood 

education, have demonstrated effective ways of improving later health outcomes; see Belfield and Levin 

(2007). These interventions require many years to demonstrate effects. 4

Children under 6 years of age were only eligible for one dental reward per year, making them eligible for 

$200 in rewards per year.
72

physician-advised follow-up visits. These rewards were both discontinued during the third year 

of that program, and were not included in Family Rewards 2.0. Increasing access to and use of 

preventive health care was expected to serve two primary functions: (1) to ensure that participants received regular medical attention, which can prevent the escalation of health problems 

that can become more serious and costly when left untreated and (2) to decrease the use of 

costly medical services such as emergency rooms and hospital outpatient services by encouraging participants to establish a relationship with a primary care doctor who serves as a first 

responder.5

 As described in Chapter 2, a large number of families earned health rewards, with 

almost 90 percent of families earning at least one health reward during the program. Families 

that earned at least one health reward earned $1,805 on average in health rewards.

Summary of Impacts on Health

This section presents a summary of the effects of Family Rewards 2.0 on a variety of healthrelated outcomes. The research team obtained data on all outcomes from the 24-month survey 

and designated outcomes as primary (non-emergency medical visits, dental visits, and overall 

health status) or secondary (all others). The program produced impacts on several primary 

outcomes targeted by the program’s reward incentives. Both adult and child participants 

experienced a statistically significant increase in the frequency of dental visits. The program 

marginally increased adult participants’ self-rated health status by 0.1 on a 5-point scale;6 this 

impact was much larger for adults who reported poor health at random assignment (0.3). The 

research team observed no statistically significant effects on any other primary outcomes.

Impacts on Receipt of Health Care Services and

Health Insurance Coverage

The health-related cash incentives offered in Family Rewards 2.0 were directly connected to 

participants’ receipt of preventive medical and dental services and intended to create a direct 

increase in participants’ frequency of preventive service visits.7 As a secondary effect, the 

program designers had hoped that an increase in preventive visits would also reduce reliance 

on costly services such as emergency room and hospital outpatient visits, especially for 

routine, non-emergency care, as participants established a relationship with a regular care 

 5

See Chapter 1 and Riccio et al. (2010) for a discussion of the logic model and the pathways through 

which Family Rewards was expected to influence health outcomes. 6

The research team assessed perceived health status based on an item drawn from the “Short Form 12 

Health Survey,” a widely used scale in national health surveys that provides a generic, multi-dimensional 

measure of physical and mental health; see Hays, Sherbourne, and Mazel (1995). 7

Dechausay, Miller, and Quiroz-Becerra (2014); Riccio et al. (2010)
73

provider at a clinic, health center, or doctor’s office.8 High insurance coverage rates for 

families left little room for Family Rewards 2.0 to improve connection to health care and 

preventive care practices.9

Table 5.1 presents outcomes for preventive care visits and health insurance coverage. 

Program group members did not experience a statistically significant change in annual health 

checkups. Parents in the program group were more likely than those in the control group to have 

had one or two dental checkups in the year before being interviewed (12.3 percentage points 

and 13.6 percentage points, respectively).10 Many dentists in the United States are private 

practitioners and do not participate in Medicaid programs,

11 which can make regular dental 

visits costly and difficult to arrange, particularly for adults who are not guaranteed coverage 

under Medicaid.12 The dental reward in Family Rewards 2.0 offset the costs of copayments and 

out-of-pocket expenses that families might have otherwise incurred.13 A comparison of these 

data with reward receipt data shows that a higher percentage of participating adults reported 

having had both non-emergency medical and biannual dental checkups than did those who 

received rewards for these same activities. The process of documenting the visit and submitting 

a coupon to the program may have been a burden to some participants.

 8

By encouraging participants to have a regular health care provider, the program promoted the “medical 

home” model of health care delivery, which includes an ongoing relationship between a provider and patient 

and a comprehensive approach to care and coordination of care through providers and community services. 

Research suggests that individuals who have continuity of care with a regular practitioner are more likely to 

adhere to prescribed medications and to receive preventive care and well-coordinated, resource-efficient, and 

family-centered care, and are less likely to visit the emergency department and be hospitalized; in addition, 

their practitioners are more likely to recognize their problems and track their information. See Christakis et al. 

(2001); Christakis et al. (2002, 2003); Starfield and Shi (2004). 9

Though Congress passed the Affordable Care Act six months before Family Rewards 2.0 began, it is 

unlikely that it greatly affected participants’ enrollment in health coverage because many families were already 

enrolled in public health insurance at program enrollment. The 24-month survey also indicated high levels of 

enrollment in public insurance and was administered before the first public enrollment period occurred under 

the new law. 10Adults in the Bronx were more likely to report at least two dental checkups than adults in Memphis. This 

finding is consistent with reward earnings. Fewer adults in Memphis had dental insurance coverage than those

in the Bronx at the start of the program, which was cited as a barrier to earning the reward because of the outof-pocket cost of dental care. Though advisors in Memphis emphasized connecting families to low-cost or free 

dental services in the later years of the program, the significant difference in impacts between the two cities 

persisted. See Dechausay, Miller, Quiroz-Becerra (2014). 11Borchgrevink, Snyder, and Gehshan (2008). 12As of 2007, Tennessee did not offer dental coverage for adults under Medicaid; see McGinn-Shapiro 

(2008). Disparities in access to oral health by income level have also been well documented; see Fisher-Owens 

et al. (2008). 13As discussed in Chapter 2, program staff also connected participants with dental practitioners who provided low-cost services or used a sliding scale for their billing. These services reduced the burden of upfront, 

out-of-pocket expenses.
74

Table 5.1

Impacts on Parents' Receipt of Services and Health Outcomes

Outcome

Program

Group

Control

Group

Difference

(Impact) P-Value

Effect

Size

Health care visits in past 12 months (%)

Has seen health professional for any reason 

Had a health checkup

Has seen dentist for any reason 

Had 1 or more dental checkups

Had 2 or more dental checkups

Respondent's health care use (%)

Has a usual source of health care

Clinic or health center

Doctor's office

Hospital emergency room

Hospital outpatient department

Other

Health insurance coverage 

in previous month (%)

Respondent had health insurance

Unmet health needs due to cost 

in past 12 months (%)

Did not get needed medical care

Did not fill prescription

Health status and outcomes

Smokes cigarettes (%)

Has any medical conditiona (%)

Treated for any medical conditiona (%)

Average Body Mass Index (BMI)b

Overweight (%)

Obese (%)

Average self-rated health

(1 = poor; 5 = excellent)

92.7

88.4

71.2

66.3

36.1

94.4

53.9

30.6

5.2

4.3

0.4

92.6

11.1

18.5

15.7

52.5

46.3

31.2

30.7

48.3

3.2

91.7

87.6

61.0

54.0

22.5

91.9

53.4

27.9

5.8

4.6

0.2

92.3

11.2

19.6

18.3

54.2

48.5

30.7

30.6

47.8

3.0

1.0

0.8

10.2 ***

12.3 ***

13.6 ***

2.5 **

0.5

2.6

-0.6

-0.2

0.2

0.3

-0.1

-1.1

-2.6

-1.7

-2.2

0.4

0.1

0.5

0.1 ***

0.397

0.569

0.000

0.000

0.000

0.030

0.836

0.181

0.541

0.798

0.364

0.799

0.937

0.544

0.108

0.403

0.285

0.167 0.061

0.962

0.808

0.002 0.126

Sample size (total = 2,016) 1,025 991

(continued) 
75

Rewards for preventive visits were intended to encourage other health outcomes, such 

as connection to health care and insurance. Though program participants did not experience a 

significant increase in medical checkups, the program appears to have had a small, statistically 

significant effect on participants’ connection to a “regular source of care,” an increase of 2.5 

percentage points over the control group. It did not have a statistically significant impact on 

where program group members sought that regular source of care. It is possible that the program’s overall focus on health encouraged some program members to seek out more regular 

health care, however this outcome is a secondary one and cannot be considered confirmatory. A 

very high number of both program and control group members already had a regular source of 

care, even relative to the general population.14

Family Rewards 2.0 did not have any significant impacts on program participants’ 

health insurance coverage. Although many families in the program already had public health 

coverage, many low-income families move on and off of coverage over time. The consistency 

 14National estimates for this type of measure suggest that 17 percent of adults 18 years of age and over 

were without a usual place of health care. Of those with a usual place of care, 74 percent considered a health 

maintenance organization doctor’s office to be their usual place of health care, 22 percent considered a clinic or 

health center to be their usual place of health care, and 3 percent considered a hospital emergency room or 

outpatient department to be their usual place of health care; see Blackwell, Lucas, and Clarke (2014). Typically, health coverage is much lower among two populations that were not eligible to enroll in the program: 

childless adults and undocumented immigrants.

Table 5.1 (continued) 

SOURCE: MDRC calculations using data from the Family Rewards 24-month survey.

NOTES: Sample sizes may vary because of missing values. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random 

assignment characteristics of sample members. Standard errors were adjusted to account for 

multiple observations per family. Statistical significance levels are indicated as follows: *** = 1 

percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences.

The effect size is the difference between program and control group outcomes expressed as a 

proportion of the standard deviation of the outcomes for both groups combined. aParticipants were asked about the following conditions: asthma, allergies, arthritis, back pain, 

bone or joint problems, cancer, diabetes, depression, digestive problems, blood pressure, high 

cholesterol, lung disease, sinus infections, weight conditions, or other specified problems. The four 

most reported conditions were asthma, diabetes, high blood pressure, and high cholesterol.

bWeight categories are from the National Institutes of Health. Underweight is defined as 

having a BMI of less than 18.5. Normal weight is defined as having a BMI between 18.5 and 24.9. 

Overweight is defined as having a BMI between 25.0 and 29.9. Obesity is defined as having a BMI 

of at least 30. About 4.5 percent of the sample was excluded from this analysis because of missing 

data.
76

of public coverage also varies from state to state,15 and families in Tennessee tend to move on 

and off of coverage more frequently than those in New York. In this case, the program might 

have increased the continuity of coverage by increasing the use of more regular medical visits. 

Overall health insurance coverage proved to be extremely high for all participants, 92.6 percent 

for the program group and 92.3 percent for the control group, and the program had no effect on 

this outcome.16

Impacts on Health Outcomes

Health incentives in Family Rewards 2.0 were intended to encourage participants to seek and 

strengthen their bond with a primary care provider through an annual non-emergency checkup. 

Program designers hoped that this result would improve participants’ health status through 

better diagnosis and management of chronic health conditions. Program designers also hoped 

that a general focus on health could have “spillover” effects onto other risk-taking behavior, 

such as smoking and lifestyle habits that lead to obesity, both of which contribute to negative 

health outcomes and costly health interventions.17

Family Rewards 2.0’s only impact on program participants’ health was on their selfreported health status (Table 5.1). Participants were asked to rate their health on a scale from 

one (poor) to five (excellent). This outcome was reported as an average and was 0.1 points 

higher on this scale for program participants. The effect size better indicates the practical 

significance of this difference between the program and control group;18 effect sizes less than 

 15Medicaid (2015). 16Family Rewards 1.0 also collected and analyzed administrative records on Medicaid coverage for adult 

participants. Data were analyzed separately for parents receiving Temporary Assistance to Needy Families 

(TANF) or Safety Net Assistance (SNA) and indicated that a high percentage of these participants were 

receiving Medicaid coverage consistently throughout the study. While some families may be at risk for 

“churn,” or having gaps in health insurance coverage because of failure to meet administrative obligations to 

reenroll in Medicaid, individuals receiving TANF benefits are automatically enrolled. Additionally, no 

statistically significant difference between participants receiving TANF or SNA in the program and control 

groups was identified at any point during Family Rewards 1.0. Because receipt of TANF or Supplemental 

Nutrition Assistance Program (SNAP) was a condition of program enrollment in Family Rewards 2.0 and 

families were already well connected to health insurance, the research team did not analyze Medicaid data for 

Family Rewards 2.0. 17Fontaine (2003); Mokdad et al. (2003); Mokdad, Marks, Stroup, and Gerberding (2004); Must et al. 

(1999). 18The “Effect Size” column in the table indicates the effect size, or magnitude, of the difference between 

the program and control group outcomes. The research team calculated the effect size by dividing the estimate 

of the difference by the standard deviation for the outcome among the control group members. Because the 

effect size is in uniform, standard deviation units, it is possible to compare the sizes of impact estimates for 

different outcomes in this evaluation and to compare Family Rewards 2.0 impacts with the impacts of other 

programs that have been evaluated. Kazis, Anderson, and Meenan (1989). 
77

0.20 are generally considered small in the behavioral sciences.19 The effect size for parents’ 

self-rated health (0.126) is small by this standard, but indicates a difference worth noting given 

that health is difficult to influence through social interventions.20 Despite positive impacts on 

participants’ self-reported health status, Family Rewards 2.0 did not appear to impact the 

diagnosis or treatment of chronic health conditions. It also did not have any notable impacts on 

risk-related behaviors or conditions.

Impacts by Self-Rated Health Status at Baseline

The effectiveness of Family Rewards 2.0 on improving health outcomes and increasing the 

receipt of health care services may be related to the initial health status of its participants. If 

those who initially suffered from poor health were in that condition because they did not utilize 

access to health care services, Family Rewards 2.0 might lead to improved health outcomes for 

this group. Additionally, participants with poor health at study entry could have more room for 

improvement in certain outcomes than those with good health. Larger effects for those in poorer 

health were found in Family Rewards 1.0.

To examine these hypotheses, Table 5.2 presents important health outcomes by study 

participants’ self-reported health status at random assignment. The research team placed 

participants who rated their health as being excellent, very good, or good in one group, and

those who rated their health as fair or poor in another group. The primary purpose of this 

analysis is to compare whether the impact for one group (those in fair or poor health) is significantly different from the impact for another group (those in better health). If the difference 

between these two impacts is not statistically significant, as indicated by daggers in the rightmost column of the tables, then it is not possible to conclude with confidence that the program 

worked any differently across the two groups.

Table 5.2 shows that Family Rewards 2.0 improved the health status of program participants more for those who had rated their health as poor or fair at random assignment than it did 

for those who had initially rated their health as excellent, very good, or good. The difference 

between the program’s impact on this measure for the two subgroups is statistically significant. 

The program’s impact on self-rated health was a 0.3 increase on a five-point scale for those with 

fair or poor health at random assignment, with the effect size indicating a notable improvement 

for this subgroup.

 19Cohen (1988). 20Belfield and Levin (2007); Maciosek et al. (2006). 
78

Table 5.2

Impacts on Parents' Health Outcomes, by Self-Reported Health Status at Baseline

Outcome

Program Control Difference

Group Group (Impact) P-Value

Effect

Size

Self-rated health status at baseline

(excellent, very good, or good)

Average self-rated health

(1 = poor; 5 = excellent)

Smokes cigarettes (%)

Has any medical conditiona

 (%)

Treated for any medical conditiona

 (%)

Sample size (total = 1,499)

3.4

15.3

43.5

37.8

771

3.3

17.3

44.3

38.3

728

0.1 *

0.0

-0.8

-0.6

0.092

0.280

0.745

0.819

0.086 ††

Self-rated health status at baseline

(fair or poor)

Average self-rated health

(1 = poor; 5 = excellent)

Smokes cigarettes (%)

Has any medical conditiona (%)

Treated for any medical conditiona

 (%)

Sample size (total = 511)

2.4

16.5

78.3

71.2

254

2.1

21.5

83.2

78.6

257

0.3 ***

-4.9

-4.9

-7.4 **

0.000

0.154

0.152

0.044

0.329 ††

SOURCE: MDRC calculations using data from the Family Rewards 24-month survey. 

NOTES: Sample sizes may vary because of missing values. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random 

assignment characteristics of sample members. Standard errors were adjusted to account for multiple 

observations per family. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 

percent; * = 10 percent. 

Statistical significance levels for differences in impacts across subgroups are indicated as follows: 

††† = 1 percent; †† = 5 percent; † = 10 percent.

Rounding may cause slight discrepancies in calculating sums and differences.

The effect size is the difference between program and control group outcomes expressed as a 

proportion of the standard deviation of the outcomes for both groups combined. aParticipants were asked about the following conditions: asthma, allergies, arthritis, back pain, 

bone or joint problems, cancer, diabetes, depression, digestive problems, blood pressure, high 

cholesterol, lung disease, sinus infections, weight conditions, or other specified problems. The four 

most reported conditions were asthma, diabetes, high blood pressure, and high cholesterol.
79

Though the research team observed no other statistically significant differences in impacts by baseline health status, one other exploratory finding should be noted. The program led 

to a statistically significant decrease in the receipt of treatment for any medical condition among 

members of the fair or poor health subgroup. Although this impact is not statistically different 

from the impact for the better health subgroup, it may be associated with the positive effect on 

self-reported health status observed for the fair or poor health subgroup and may be worthy of 

future study.

Impacts on Children’s Health Care and Health

As a two-generational program, Family Rewards 2.0 extended health rewards to parents and all 

children in the family. Using a similar logic to the one applied to parents, program designers 

hoped that emphasizing preventive health services would lead to an improved connection to 

health care and other health outcomes for children. On the 24-month survey, parents were asked 

to answer questions related to these outcomes for one child in their household who was entering 

ninth or tenth grade at study enrollment. Table 5.3 presents outcomes for these focal children. 

Similar to adults, high rates of enrollment in health insurance among children in both the 

program and control groups left little room for Family Rewards 2.0 to improve connection to 

care and preventive care practices for focal children in the program group. Family Rewards 2.0 

did not produce statistically significant changes in focal children’s non-emergency preventive 

health checkups.

The program did lead to an increase in biannual visits to the dentist among children in 

the year before the survey. The magnitude of the impact for a second dental checkup (16 

percentage points) was much larger than just one checkup (3.8 percentage points). This effect

implies that the dental reward was effective at helping children to meet the American Dental 

Association recommendation of maintaining more regular checkups.21 Access to dental care 

also has the potential to help children avoid more serious medical conditions later in life.22

Across both the program and control groups, adults were much less likely to see the 

dentist, even once, compared with their children. The impact on seeing a dentist at least once 

was likely much higher for adults than their children because of this difference. Reported rates 

of reward receipt are consistent with this finding. Family Rewards 2.0 did not produce statistically significant changes in focal children’s source of usual care or any other health outcomes. 

Without an effect on the primary outcome of preventive health checkups, it is unsurprising that 

the program did not show effects on health status or source of care.

 21American Dental Association (2014). 22Glied and Neidell (2010). 
80

Conclusion

Similar to those in the original program, Family Rewards 2.0’s health incentives were designed 

to connect low-income families to preventive health care resources. A high proportion of 

families were already covered by health insurance and were practicing preventive health care, 

which limited the program’s ability to further improve some health care practices and connection to care. Despite this limitation, the program did produce a small number of impacts. The 

program increased dental visits for both adults and children. It also increased adult participants’ 

self-rated health status, particularly for adults who reported being in poor health at study entry.

Table 5.3

Impacts on Focal Child's Receipt of Services and Health Outcomes

Outcome

Program Control Difference

Group Group (Impact) P-Value

Effect

Size

Health care visits in past 12 months (%)

Has seen health professional for any reason

Has seen personal pediatrician for any reason

Had health checkup or got shots

Has seen dentist for any reason 

Had 1 or more dental checkups

Had 2 or more dental checkups

Respondent's health care use (%)

Has usual source of care when sick

Clinic or health center

Doctor's office

Hospital outpatient department 

Hospital emergency room

Other

Health insurance coverage in previous month (%) a All dependent children had health insurance

Health outcomes

Child's health (1 = poor; 5 = excellent)

b Child has any health condition (%)

97.2

74.0

94.9

92.8

92.3

62.4

94.8

55.4

32.1

3.1

3.6

0.5

95.4

4.0

20.8

96.6

74.0

94.4

89.3

88.5

46.4

95.2

56.0

31.5

3.8

3.6

0.3

94.7

3.9

20.4

0.5

0.0

0.5

3.5 ***

3.8 ***

16.0 ***

-0.4

-0.5

0.6

-0.7

0.1

0.2

0.7

0.0

0.5

0.505

0.983

0.614

0.010

0.005

0.000

0.706

0.821

0.782

0.421

0.936

0.570

0.468

0.293

0.788

0.047

Sample size (total = 1,881) 976 905

 (continued)
81

Family Rewards 1.0 produced similar impacts on dental visits and participants’ selfrated health status. It also had some impacts that were not observed in Family Rewards 2.0, 

including small increases in continuous health coverage and the treatment or diagnoses of 

certain conditions for adults and children. It is possible that these additional impacts were not 

replicated in Family Rewards 2.0 because participants were already highly connected to health 

care. Comparing the two programs’ impacts on all of these measures, it seems likely that the 

rewards for preventive health visits, such as dental checkups, can help connect participants to 

better health care resources.

Table 5.3 (continued)

SOURCE: MDRC calculations using data from the Family Rewards 24-month survey.

NOTES: This table presents outcomes only for focal children who were living in the household at 

the time of the survey interview.

Sample sizes may vary because of missing values. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random 

assignment characteristics of sample members. Standard errors were adjusted to account for 

multiple observations per family. Statistical significance levels are indicated as follows: *** = 1 

percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences.

The effect size is the difference between program and control group outcomes expressed as a 

proportion of the standard deviation of the outcomes for both groups combined. a

Child-related health insurance measures were calculated for sample members with at least 

one child at the time of the survey.

bParticipants were asked about the following conditions: learning disability, attention deficit 

disorder or attention deficit hyperactivity disorder, autism, depression or other emotional 

condition, speech disorder or delay, asthma, heart problems, chrinic illness, weight conditions, or 

other specified problem. 

83

Chapter 6 

Impacts on Parents’ Education, Training, and Work

The work incentives in Family Rewards 2.0 were part of a package of rewards intended to 

encourage behavioral changes in the areas of education, health, and work. The program designers expected the incentives structure to affect a family’s decisions about employment in a 

variety of ways. If the reward for employment was large enough, parents might decide to enter 

employment or increase their work hours, since the reward payments could serve as a de facto 

boost to wages. On the other hand, since the program allowed families to earn rewards through 

a range of activities, effects on work and training might depend on how families decide to 

complete health, education, and work-related activities to earn rewards. Parents and children 

could focus on some activities to the exclusion of others. Similarly, additional money that 

families received from the education and health rewards might reduce the need to work and 

offset some or all of the work incentive that these rewards offer.

This chapter examines whether Family Rewards 2.0 prompted parents to pursue any 

further education or training and whether it affected their employment behavior during the three 

years the rewards were offered. It also discusses how the program’s effects compare with those 

found for Family Rewards 1.0. In sum, the program increased participation in training but did 

not result in higher rates of degree or certificate receipt. It also produced small negative effects 

on parents’ employment and earnings.

The Work and Training Rewards

Family Rewards 2.0 provided opportunities for parents to earn rewards for working or for 

earning a General Education Development (GED) certificate. Parents could earn $150 a month 

for working full time (30 or more hours per week) by providing evidence each month that they 

had worked at least 120 hours for the month. Additionally, parents could earn a one-time $400 

reward for earning a GED certificate during the program period. These rewards were slightly 

different from those in Family Rewards 1.0. In that program, parents earned $300 every two

months for averaging 30 or more hours of work per week,1 and they could earn rewards for 

taking training classes, in addition to pursuing and earning a GED certificate.

 1

The Family Rewards 2.0 payment structure for work rewards made it easier for individuals with less 

stable employment to earn rewards for full-time work, since the rewards were paid on a monthly rather than 

a bimonthly schedule. With the bimonthly schedule used in Family Rewards 1.0, parents who worked full 

time for one month but not the other in the same payment period would not have earned a work reward for 

that period.
84

As shown in Table 2.1, families earned an average of $2,926 in work-related rewards 

while enrolled in the program. Just over half of the program group families earned at least one 

reward payment for maintaining full-time work, and 2 percent received a reward for earning a 

GED certificate. 

Summary of Impacts on Work and Training

The research team examined the effects of the program using data from the 24-month survey 

and administrative data from each state’s unemployment insurance (UI) wage records. The UI 

data cover the full study sample for 3 years after random assignment. The survey was fielded 

about 2 years after random assignment, before some of the Year 3 changes were implemented. 

Some changes in advisements — in particular, the full implementation of motivational interviewing — began in Year 3, which ran from September 2013 through August 2014. During 

Year 3, advisors also more actively focused on program participants who had not received 

work-related rewards in the past.

The primary outcomes examined in this chapter are educational attainment, employment rates, and earnings. In sum, the program did not boost educational attainment and produced a reduction in employment and earnings. Two years after random assignment, program 

group members were somewhat more likely to have a trade license or training certificate than 

control group members, but they were no more likely to have a degree. The program produced 

zero to negative impacts on employment and earnings, depending on the data source. The 

reduction in earnings from UI data was particularly large for program group members in 

Memphis, as Chapter 7 discusses, although the negative effects seemed to have diminished

during the second half of Year 3.

Impacts on Parents’ Education and Training

This section presents the program’s effects on parents’ participation in education and training 

activities and on certificate and degree receipt. Education and training data for parents were

only available on the survey, which asked respondents about current education levels as well as 

participation in education and training activities since random assignment.

Table 6.1 shows that at the 24-month point, about 32 percent of the control group reported participating in some education, training, or employment program activity since the time 

of random assignment, 43 percent reported having a trade license or training certificate, and 78 

percent reported having any high school diploma or high school equivalency credential, trade 

license, training certificate, or college degree. Within 24 months of study entry, Family Rewards 2.0 increased participation in education and training programs by 4 percentage points and 
85

Table 6.1

Impacts on Educational Attainment and 

Participation in Educational and Employment Activities

Outcome (%)

Program

Group

Control

Group

Difference

(Impact) P-Value

Has any degree, license, or certificate

Has any trade license or training certification

Home health aide

Nurse's aide/nurse's assistant

Child care/teaching

Medical assistant

Security 

Other

Highest degree or diploma

High school equivalency certificate

High school diploma

Associate's degree

Bachelor's degree or higher

Ever participated in any education, training, 

or employment activity

Adult basic education, high school equivalency,

or high school classes

English as a second language classes

College courses for credit

Vocational training 

Other educational, training, or employment

program activities

Currently participating in any education, training,

or employment activity

78.5

46.6

10.1

5.2

4.7

5.0

1.8

20.0

15.3

33.8

10.8

5.7

36.3

8.5

7.7

13.6

10.3

5.9

12.5

77.8

42.7

11.6

3.9

3.6

2.9

1.2

19.2

15.6

35.2

9.1

6.4

32.2

8.7

6.1

13.1

9.5

4.6

9.6

0.7

3.9 *

-1.5

1.4

1.1

2.1 **

0.6

0.8

-0.3

-1.3

1.7

-0.7

4.1 **

-0.2

1.6

0.5

0.8

1.3

2.9 **

0.651

0.073

0.223

0.145

0.226

0.015

0.299

0.668

0.840

0.486

0.189

0.481

0.049

0.870

0.139

0.759

0.562

0.183

0.039

Sample size (total = 2,016) 1,025 991

SOURCE: MDRC calculations using data from the Family Rewards 24-month survey.

NOTES: Sample sizes may vary because of missing values. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random 

assignment characteristics of sample members. Standard errors were adjusted to account for multiple 

observations per family. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 

percent; * = 10 percent. 

Rounding may cause discrepancies in calculating sums and differences.

Percentages may sum to more than the overall participation rate because sample members could list 

more than one response.
86

certificate receipt by 4 percentage points. It is possible that the reward payments families 

received from the other domains allowed parents to pay for training classes if they were seeking 

some type of certification for future employment. However, the program had no statistically 

significant effect on further educational attainment — including no effect on earning a GED 

certificate, which would have merited a reward payment.

An increase in training activities could have affected how much parents were able to 

work in the short term while participating in their training classes. The next section considers 

whether these effects on training were associated with effects on employment over the threeyear follow-up period.

Impacts on Employment and Earnings

The research team calculated employment impacts from both UI and survey data. UI earnings 

were available quarterly for each sample member in the study, so the follow-up for most sample 

members covered July 2011 through March 2015. The survey asked about employment at the 

time of the survey and in the year before the survey was conducted for all survey respondents. 

The survey also provided rich information about respondents’ job characteristics, such as hourly 

wages, and employer-provided benefits — information that was not available on UI wage 

records. Typically, the survey data captured more employment than UI records because some 

types of jobs, including “off-the-books” jobs, federal government jobs, out-of-state jobs, and 

self-employment, are not reported to the state UI systems.

Table 6.2 shows that about 60 percent of the control group was employed in Year 3, 

with just over half of them employed in each quarter. Family Rewards 2.0 reduced work in UIcovered jobs by about 4 percentage points over three years and reduced earnings from work by 

about $2,000. This amount translates to about 7 percent of the earnings over three years for the 

average control group member.

Economists often refer to a reduction in work (to possibly spend more time in nonpaid 

activities) in response to an increase in household income as an “income effect.” Such an 

income effect was observed in Family Rewards 2.0 for the full study sample and not concentrated in a specific subgroup, as Chapter 7 discusses. Quarterly impacts on employment rates 

and earnings amounts indicate that the negative effects on employment and earnings persisted

only through the middle of Year 3; the negative impacts were no longer statistically significant 

in the last two quarters of the year (not shown). These effects may have been due to implementation changes in Year 3 that focused on conducting motivational interviews with participants 

who were not earning the full-time work reward payments. It is also possible that families were 

increasing their work activities to prepare for the end of the Family Rewards 2.0 program, when 

the reward payments would stop.
87

Table 6.2

Impacts on Unemployement Insurance-Covered

Employment and Earnings, Years 1 to 3

Outcome

Program

Group

Control

Group

Difference

(Impact) P-Value

Ever employed (%)

Year 1

Year 2

Year 3

Years 1 to 3

58.6

58.0

58.5

68.0

61.1

60.9

62.7

71.9

-2.5 *

-2.9 *

-4.2 ***

-3.9 ***

0.057

0.053

0.007

0.005

Average quarterly employment (%)

Year 1

Year 2

Year 3

Years 1 to 3

48.8

49.5

50.6

49.6

51.5

51.9

53.2

52.2

-2.7 **

-2.4 *

-2.6 *

-2.6 **

0.012

0.065

0.071

0.018

Total earnings ($)

Year 1

Year 2

Year 3

Years 1 to 3

8,404

9,124

10,156

27,684

9,054

10,044

10,619

29,718

-650 ***

-921 ***

-463

-2,034 **

0.007

0.006

0.244

0.019

Sample size (total = 2,565) 1,286 1,279

 SOURCES: MDRC calculations using data from New York State unemployment insurance 

(UI) wage records and Tennessee Department of Labor and Workforce Development UI Table 6.6 (continued)

wage records.

NOTES: Estimates were regression-adjusted using ordinary least squares, controlling for prerandom assignment characteristics of sample members. Standard errors were adjusted to 

account for multiple observations per family. Statistical significance levels are indicated as 

follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences.

Dollar averages include zero values for sample members who were not employed.

This tables includes only employment and earnings in jobs covered by the New York 

State and Tennessee UI programs. It does not include employment outside of either state, or 

in jobs not covered by the UI system (for example, "off-the-books" jobs and federal 

government jobs). 
88

Table 6.3 presents employment data as reported on the 24-month survey. The program 

had no effect on survey-reported employment, either at the time of the survey or during the 

previous year. According to the survey, about two-thirds of parents were employed at some 

time in the 12 months before they responded to the survey.

Table 6.3 also shows some characteristics of the jobs that sample members held. Employed control group members were earning, on average, $11 an hour in their current jobs. 

About 43 percent of the control group, or three-quarters of those who were employed, were 

working full time at the time of the survey. Almost 40 percent of the control group, or twothirds of those who said they were employed, had a regular daytime shift. A small proportion of 

the control group was self-employed or working for a “temp” agency at the time of the survey. 

The program did not have effects on any of these job characteristics, even though a larger 

proportion of parents in the program group reported recently looking for work. It is possible that 

the program increased job search efforts that did not result in actual employment.

Subgroup Impacts on Parents’ Education, Training,

and Employment

This section presents subgroup impacts on education, training, and employment defined by 

parents’ educational level at the time of study entry. Chapter 7 discusses impacts for additional 

subgroups. In Family Rewards 1.0, families who were less disadvantaged — as defined by 

those in which parents had at least a high school diploma or equivalency certificate, or were 

employed when they enrolled in the study — experienced a small increase in educational 

attainment but no effect on earnings.

Family Rewards 2.0 had similar impacts on training. The program’s positive impacts on 

certificate receipt were concentrated among the parents who were more educated when they 

enrolled in the study, as shown in Table 6.4.2 It is possible that parents who had not yet passed 

their GED exams before 2014 felt less inclined to take advantage of the GED reward offer 

because of the changes to the GED exam in 2014 that made it more difficult.3 About 60 percent 

of the survey respondents who were interviewed in 2014 did not have a high school diploma or 

equivalency certificate when they enrolled in the study.

 2

The difference of the impact on trade license and certificate receipt across parents with differing education levels was statistically significant. The difference of the impact on participation in training was not

statistically significant, but the pattern of impacts is similar — the program effects were larger for parents who 

were more educated and smaller for parents without a high school diploma or equivalency certificate at the 

time of random assignment. 3

Porter (2015).
89

However, the program’s effects on earnings were somewhat different. Family Rewards 

1.0 reduced earnings only among parents without a high school diploma or equivalency certificate. Family Rewards 2.0, in contrast, reduced earnings among the full program group sample. 

A pattern of statistically significant and negative effects on earnings and employment were

consistently larger for parents who were more educated at the time of random assignment, 

although the impact differences across the less educated and more educated subgroups were not 

statistically significant. While it is possible that some parents cut back on work hours to participate in training, as some program participants indicated in field interviews, a separate calculaTable 6.3

Impacts on Employment and Job Characteristics

Outcome

Program

Group

Control

Group

Difference

(Impact) P-Value

Employment status (%)

Employed at the time of the survey

Employed in past year

Characteristics of current joba

Average hourly wage ($)

Less than $7.00 (%)

$7.00 to $8.99 (%)

$9.00 or more (%)

Not reported (%)

Worked at least 30 hours per week (%)

Average weekly earnings ($)

Usual work schedule (%)

Regular daytime shift

Regular evening/night shift

Self-employed (%)

Worked for a temporary employment agency (%)

Received any employer-provided benefitb (%)

Employment search (%)

Looked for work in the previous 4 weeks

55.9

66.4

11.31

5.6

9.8

32.6

7.9

41.9

193

37.3

6.2

9.5

6.0

37.7

34.2

57.1

67.4

11.39

7.0

9.4

30.4

10.3

43.2

199

39.2

7.7

8.3

6.1

38.7

30.6

-1.2

-1.0

-1.4

0.5

2.1

-2.4 *

-1.3

-6

-2.0

-1.5

1.3

-0.1

-1.0

3.5 *

0.528

0.586

0.195

0.722

0.266

0.055

0.520

0.554

0.316

0.176

0.301

0.910

0.564

0.081

Sample size (total = 2,016) 1,025 991

(continued)
90

tion of these subgroup impacts by city showed that the subgroups who experienced positive 

training effects were not the ones who experienced negative employment effects (not shown).4

The research team also estimated impacts by city (discussed more fully in Chapter 7) to 

assess whether the reductions in work occurred in each of the different labor markets. The 

findings show that reductions in UI-covered work for three years after the time of random 

assignment were more pronounced in Memphis, while reductions in work reported at the time 

of the survey were more pronounced in the Bronx. The reason for the differences across cities is 

not clear.

The reduction in work and earnings was likely a response to the substantial rewards that 

these families were earning from the program’s health and education rewards. Since families 

were receiving reward payments, they could reduce work hours and, in some cases, stop 

working entirely, without lowering the amount of financial resources they had before the 

 4

The analysis showed that in the Bronx the program increased training for those with less education at 

baseline without affecting their work behavior. Families with more educated adults cut back on work hours and 

did not participate in more training. In Memphis, the education subgroups showed no statistically significant 

movement in either training or work. While the sample sizes in this separate investigation were too small to 

draw conclusive findings, and differences in impacts across these subgroups were not statistically significant, 

the analysis does show that sample members were generally not leaving work to pursue additional training.

Table 6.3 (continued)

SOURCE: MDRC calculations using data from the Family Rewards 24-month survey.

NOTES: Sample sizes may vary because of missing values. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random 

assignment characteristics of sample members. Standard errors were adjusted to account for multiple 

observations per family. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 

percent; * = 10 percent. 

Italic type indicates comparisons that are nonexperimental. Statistical tests were not performed.

Rounding may cause slight discrepancies in calculating sums and differences.

a

If a respondent worked multiple jobs at the time of the interview, then only the characteristics of the 

primary job are reported. (The job at which the respondent worked the most hours is considered 

primary.) Respondents who were not employed at the time of the survey are included in all the current 

job characteristics measures, except for average hourly wage. The average hourly wage measure 

includes only respondents who were employed at the time of the survey.

bThis category includes benefits that are or eventually will be offered, regardless of whether the 

respondent received them. Sample members were asked whether they received paid sick days, paid 

vacation days, paid holidays, dental benefits, retirement benefits, or health or medical insurance from 

their current employers.
91

Table 6.4

Impacts on Education, Training, and Employment Activities,

 by Respondent's Education Level at the Time of Random Assignment

No High School Diploma or High School Diploma or 

Equivalency Certificate Equivalency Certificate

Program Control Difference Program Control Difference

Outcome Group Group (Impact) P-Value Group Group (Impact) P-Value

Survey responses (%)

Has any degree, license, or certificate 49.2 49.8 -0.7 0.855 97.0 95.5 1.6 0.142

Has any trade license or 37.7 39.1 -1.4 0.674 52.5 45.1 7.5 *** 0.009 ††

training certification

Ever participated in any education, 

training, or employment activity 32.2 29.3 2.9 0.377 39.1 33.7 5.4 ** 0.047

Employed at the time of the survey 48.8 50.5 -1.8 0.542 60.4 60.7 -0.3 0.909

Worked at least 30 hours per week 36.0 36.7 -0.8 0.805 45.9 46.8 -0.9 0.735

Sample size (total = 769) 396 373 618 609

Unemployment insurance records

Employment in Years 1 to 3 (%) 59.5 62.3 -2.8 0.222 73.7 78.0 -4.3 ** 0.014

Total earnings in Years 1 to 3 ($) 20,735 21,715 -980 0.318 32,287 34,941 -2,655 ** 0.040

Sample size (total = 1,005) 510 495 760 770

SOURCE: MDRC calculations using data from the Family Rewards 24-month survey, New York State unemployment insurance (UI) wage 

records, and Tennessee Department of Labor and Workforce Development UI wage records.

NOTES: Sample sizes vary because of missing values. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. 

Standard errors were adjusted to account for multiple observations per family. Statistical significance levels are indicated as follows: *** = 1 

percent; ** = 5 percent; * = 10 percent. 

Rounding may cause discrepancies in calculating sums and differences.

Statistical significance levels across subgroups are indicated as follows: ††† = 1 percent; †† = 5 percent; † = 10 percent.
92

program started. Although it is possible that some parents decided to forego immediate employment opportunities to pursue education and training, very few parents who were not 

working when they responded to the survey listed education or training as a primary reason for 

not working, instead citing either their own or a family member’s health issues.

Barriers to Finding and Maintaining Full-Time Work

About 40 percent of study participants were not working at the time of the survey, and about 60 

percent were not working at least full time. Table 6.5 presents reasons that survey respondents 

in the program group reported for either not working full time or not working at all. Although 

the reasons do not explain why program group members cut back more on work than did the 

control group members, they provide some context as to why parents could not work as much 

as they as they might have wanted. Among program group members who were working part

time at the time of the survey, about a third of them were unable to increase their hours or find a 

full-time job.

In field interviews, participants expressed difficulty in obtaining the minimum 120 

hours per month of work that were required to earn the full-time work reward payment. One 

participant described the instability of her hours as a home health aide and how difficult it was 

for her to accrue enough hours to earn the reward:

[T]his year has been very bad for me, very bad, because I missed three months’ 

worth of $150.00. Three months. And they called me, “What happened? You 

know, you’re not sending in your pay stubs”... I said, “I didn’t work a week, and 

then I had to wait another week, and then I only had like 18 hours.” I mean, it 

was really frustrating. But as home care attendants, if you’re not working, you’re 

not getting paid, you don’t have any hours. So it messes you up.

Program advisors, despite encouraging families in the program to increase their work 

hours so that they could earn the full-time work reward, indicated that they felt limited in 

providing concrete guidance to individuals on how to do so:

We have ... so many people who come so close. You know, they’re in between 

100 and 119 ... so they’re working, you know, they’re working part time, but 

they’re working consistently, but they’re not meeting the 120 ... It’s hard for 

people to hit the 120. We talked to people about ... talking to your employer 

about getting more hours, or we’ve had some people getting second jobs.”

About 12 percent of program group respondents working part-time reported that their 

caretaking responsibilities at home limited their ability to work full time. Interviews with 

participants reflected this reality — one participant had to spend a lot of time taking care of her 

sick mother; another participant had to care for a grandmother whose leg was amputated.
93

Table 6.5

Respondents Not Working Full Time

Outcome (%)

Program

Group

Among respondents working part time at the time of the survey

Main reason for not working full time

Inability to a find full time job or work more hours

Need to take care of home or family

Attendance at school or training

Respondent's illness or disability

Someone else's illness or disability

Inability to find adequate child care

Pregnancy

Other

Sample size

31.2

11.6

4.4

4.4

2.9

2.9

0.7

35.5

138

Among respondents who were not working at the time of the survey

Main reason for not working

Respondent's illness or disability

Inability to find work

Need to take care of home or family

Someone else's illness or disability

Being temporarily laid off or on temporary leave

Inability to find adequate child care

Attendance at school or training

A criminal record that makes finding work hard

Pregnancy

Retirement or aging

Other

Average wage that respondents would consider before taking a job offer

Less than $7.00 

$7.00 to $8.99 

$9.00 or more 

Not reported

Sample size

40.8

18.7

8.6

3.5

2.8

4.2

3.3

0.9

0.7

0.2

16.3

4.5

10.0

44.7

40.8

441

SOURCE: MDRC calculations using data from the Family Rewards 24-month survey.

NOTES: Sample sizes may vary because of missing values. Statistical tests were not 

performed.
94

Program group respondents who were not employed at the time of the survey most frequently reported that the reason for not working was the respondents’ illness or disability; about 

41 percent of nonworking respondents reported this reason as the main one. About one-fifth of 

nonworking respondents also reported that they were unable to find work.

Conclusion

Through its first two years, Family Rewards 2.0 increased the proportion of parents who earned 

a trade license or training certificate and the proportion of parents who participated in an 

education or training activity. The education and training impacts were concentrated primarily 

among adults who already had a high school diploma or equivalency certificate when they 

enrolled in the study. The program reduced UI-covered work and did not produce statistically 

significant impacts on survey-reported work. The average program group adult earned about 

$2,000 less from working than the average control group member over three years, which is 

about one-third of what each family earned in reward payments, on average, over the full 

program period. The reduction in work may have had additional implications for families, 

including a possible reduction in Earned Income Tax Credit benefits.

These findings are similar to those in Family Rewards 1.0, which produced a small increase in college course participation and a small increase in getting a trade license or training 

certification. Family Rewards 1.0 also increased employment, mostly in jobs that were not 

reported to the UI system. However, it led to reductions in UI-covered work for the less “workready,”5 with no or small positive effects on survey-reported employment.6

Even though the reductions in work among families enrolled in Family Rewards 2.0 

were not completely unexpected based on findings from the earlier program, it is not immediately clear why this replication program would have produced a broader income effect than did 

Family Rewards 1.0, given the smaller amount of other rewards that families earned. However, 

all sample members in the Family Rewards 2.0 evaluation had to be receiving food stamps or 

Temporary Assistance for Needy Families to be eligible, which was not a requirement for 

Family Rewards 1.0. Thus, the broader income effect on a more disadvantaged population 

makes sense in this context. The finding, however, is a concern, particularly since employment 

and training incentives have been shown to work in other places. Earlier studies have found that 

 5

The research team defined the subgroups that were less “work-ready” by (1) parents who did not have a 

high school diploma or equivalency certificate when they enrolled in the program, and (2) parents who were 

not working when they enrolled in the program. 6

Riccio et al. (2013).
95

providing work and training incentives in the absence of the other incentives can increase work 

and, in some cases, training.7

A reduction in work, particularly among more disadvantaged families, has now been 

observed in Family Rewards 1.0 and replicated in Family Rewards 2.0. This finding suggests 

that work and training incentives may be more effective in employment-focused programs 

rather than packaged with other incentives that target family or child well-being. It also serves 

as a general caution for conditional cash transfer (CCT) programs with generous incentives 

structures that may improve education and health outcomes for participants who earn the 

incentives — although there is little evidence that CCT programs in developing countries 

reduce work.8 Program designers and administrators interested in developing and structuring 

large-scale CCT programs should weigh the benefits of education and health improvements 

against the decisions that low-income families might make about work. Without the work 

incentives, however, the program might have produced even larger negative employment 

effects.

 7

Riccio et al. (2008); Martinson and Hendra (2006); Michalopoulos et al. (2002). 8

Fiszbein and Schady (2009).

97

Chapter 7 

Impacts by City and for Subgroups

Earlier chapters of this report provided some evidence that Family Rewards 2.0’s effects vary 

across different types of families. The program’s effects on health outcomes, for example, 

varied by participants’ health status at study entry. It is reasonable to expect that a program such 

as Family Rewards 2.0 might have varying effects for different types of families, particularly as 

defined by their level of disadvantage or ability to meet the milestones to earn the rewards. 

Varying effects were also found to some extent in Family Rewards 1.0.

This chapter presents Family Rewards 2.0’s effects for several subgroups of families, 

those defined by family income, parents’ education level, and parent’s work status at study 

entry. Although the earlier chapters present effects in certain domains for a few of these groups, 

this chapter presents effects across outcomes in all domains, to provide a fuller picture of the 

program’s effects.

The chapter also presents effects by city. The report has presented overall effects for 

both cities combined, in order to provide a more generalizable assessment of the program’s 

effects. However, effects might have varied by city because of differences in the types of 

participants across the two areas and differences in the context. The implementation findings 

also suggest that the program’s effects might have varied by city. Transportation, for example, 

posed a challenge for Memphis participants in meeting with their advisors. In addition, families 

earned on average more from the rewards in the Bronx than in Memphis, indicating that the 

program may have led to bigger changes in behavior for the former group.

In sum, the findings show that the program had fairly similar effects across the two cities. In both cities, the program reduced poverty, improved health status, and led to more dental 

visits. Also in both cities, the program had little overall effect on children’s progress in school 

and led to some reductions in parents’ work. The results also suggest that the program had 

broadly consistent effects across more versus less disadvantaged families, with a few exceptions. The program seemed to work the least well for families without an employed parent, with 

those families earning the least from the program and seeing relatively large reductions in work 

and earnings during the follow-up period. In addition, there is some suggestion that the program 

increased graduation rates among students in the study’s more advantaged families, or those 

whose families had higher incomes or whose parents were working at study entry.
98

Defining Subgroups and Estimating Impacts

The research team specified the subgroups — defined by city, family income, parents’ educational level, and parents’ work status — before conducting the analysis. The team selected them

based on the understanding that measures of well-being vary greatly across groups defined by 

socioeconomic characteristics and that groups defined by those characteristics might have 

differential responses to the program’s cash incentives. In other words, different capacities or 

starting points might affect how individuals respond to the incentives. For example, the more 

“advantaged” participants — those with higher income or those who were employed or who 

had higher levels of education at study entry — may have been more likely to earn rewards and 

thus experience greater improvements in income, poverty, and material well-being. Alternatively, these same participants might have been more likely to have already met the conditions 

necessary to earn rewards, so that they earned rewards with little change in behavior. Descriptive analyses presented in the implementation report show that families who earned more in 

rewards differed substantially from those who earned much less. Among other distinguishing 

characteristics, the top earning families were more likely to have been employed at baseline and 

to have held a high school diploma or equivalency certificate. This chapter examines whether 

the program had bigger effects on their education, health, and work outcomes.

The research team conducted the analysis by estimating impacts separately for each 

subgroup and the then assessing the differences in effects. In general, impacts were expected to 

vary to some extent across subgroups, simply as a result of natural variation around the average 

impact for the full sample. This chapter assesses whether that variation in impacts across 

subgroups is statistically significant, or beyond what would be expected to occur naturally. For 

that reason, the focus is not on whether a given impact for, say, the less-educated subgroup is 

statistically significant, but whether the difference between that impact and the impact for the 

more educated subgroup is statistically significant (indicated by daggers in the rightmost 

column of the tables). If the difference between these two impacts is not statistically significant, 

the results suggest that the effects observed for the full sample generally held across both groups 

being compared. However, differences that are notable but not statistically significant may be 

highlighted in the text that follows as exploratory and worthy of further analysis. Finally, 

because subgroup samples tended to be smaller and because the test for the difference between 

two impacts required more statistical power than a test of one impact, it is important to keep in 

mind that differences in impacts between any two subgroups needed to be fairly large in order 

to be found statistically significant.
99

Impacts by City

Table 7.1 presents effects separately for the Bronx and Memphis. As shown in the bottom 

panel, families in Memphis earned on average $5,765 from the program over the full period,

while families in the Bronx earned an average of $6,719. The reason for this difference is due 

largely to lower education rewards earned among Memphis families.

Overall, as indicated by the daggers, the main differences in effects are in the area of 

work and training. As mentioned briefly in Chapter 6, the program led to a reduction in employment reported on the survey in the Bronx, with no effects on this measure in Memphis. In 

contrast, the program led to a reduction in employment and earnings covered by unemployment 

insurance (UI) in Memphis, with less of an effect in the Bronx. The reason for this difference is 

not clear. The program’s positive effects on training, found for the full sample, were also driven 

entirely by positive effects in the Bronx. 

The remainder of the table suggests that the program’s effects in the other domains 

were fairly consistent across cities, although there are differences in the size of the effects. For 

example, the program led to a smaller reduction in poverty in Memphis than in the Bronx, 

which is probably related to the lower rewards earned on average for families in Memphis. The 

program reduced severe poverty (or the percentage of families below 50 percent of the federal 

poverty line) by similar amounts in both cities (not shown).

Impacts by Poverty Level

Table 7.2 presents effects separately for families who entered the study with income below 50 

percent of the federal poverty line versus families with income above that threshold. Lowerincome families earned substantially less in rewards ($5,328 on average) than higher-income 

families ($7,103 on average), due to lower reward payments in education and work. Nonetheless, the program’s effects on poverty reduction were roughly the same for both types of 

families.

Even though lower-income families earned less in rewards, the program might nonetheless have had larger effects on their behavior. For these families, the rewards represented a 

larger potential fraction of their current income and thus may have been more salient and 

powerful incentives. The table also presents effects on education, health, and work. Overall, 

there are few significant differences, with the exception of the health domain. The program 

increased health coverage for lower-income families, but had smaller effects for this group on 

two or more dental visits. Finally, there is a difference in impacts on graduation rates, with a 4.3 

percentage point increase in graduation rates for students in the higher income group.
100

Table 7.1

Summary Program Impacts, by City

Outcome

Program

Group

Bronx Memphis

Control

Group

Difference

(Impact) P-Value

Program

Group

Control Difference

Group (Impact) P-Value

Income, poverty, and well-being (%)

Household income at or below the federal poverty

level (including rewards)a,b

Any savings

Any housing/utilities material hardship in 

the past 12 months

Did not have enough money to buy food 

sometime in the past 12 months

c Very or pretty happy

Education

Number of state core exams passed, Years 1 to 3d,e

Attendance rate is 95% or higher, Year 3f (%)

Graduated on time (%)

Health 

All dependent children had health insuranceg (%)

Parents' average self-rated health

(1 = poor; 5 = excellent)

Focal child had 2 or more dental checkups in past

12 months (%)

71.0

18.3

59.7

45.0

72.3

3.7

25.6

61.4

94.3

3.1

60.5

78.8

11.6

60.1

50.2

67.1

3.7

26.8

57.2

94.6

3.0

42.7

-7.8 ***

6.7 ***

-0.5

-5.2

5.2 *

0.0

-1.2

4.2

-0.3

0.1

17.8 ***

0.005

0.003

0.88

0.101

0.075

0.867

0.630

0.122

0.823

0.154

0.000

76.1

22.8

69.1

41.8

80.2

1.8

32.7

66.4

96.5

3.2

64.3

77.9

12.5

67.9

44.3

77.3

1.7

30.6

67.5

94.8

3.0

50.0

-1.8

10.3 ***

1.2

-2.5

2.9

0.1

2.1

-1.1

1.7

0.2 ***

14.3 ***

0.492

0.000

0.678

0.417

0.261

0.106

0.412

0.660

0.190

0.003

0.000

(continued)
101

Table 7.1 (continued) 

Outcome

Program

Group

Bronx Memphis

Control

Group

Difference

(Impact) P-Value

Program

Group

Control Difference

Group (Impact) P-Value

Work 

Ever participated in any education, training, 

or employment activity (%)

Currently employed at the time of the survey (%)

Unemployment insurance-covered employment

earnings, Years 1 to 3 ($)

Reward participation, Years 1 to 4 ($)

Among families who earned at least one reward

Average reward amount earned h

Average reward amount earned, by domain

Education

Health

Work

40.0

58.1

31,270

 6,719

 3,344

 1,728

 3,022

32.3

63.0

32,427

—

—

—

—

7.7 **

-4.9 **

-1,157

—

—

—

—

0.011

0.048

0.354

—

—

—

—

32.5

53.8

23,933

 5,765

 2,554

 1,885

 2,818

32.2

51.3

26,893

—

—

—

—

0.3

2.5

-2,960 **

—

—

—

—

0.920

0.360

0.013

—

—

—

—

†

††

Sample size (total = 2,016) 516 491 509 500

(continued)
102

Table 7.1 (continued) 

SOURCES: MDRC calculations using data from the Family Rewards 24-month survey, Children's Aid Society's Family Rewards program data, New York 

City Department of Education and Shelby County Schools administrative records, New York State unemployment insurance (UI) wage records, and 

Tennessee Department of Labor and Workforce Development UI wage records. 

NOTES: This table presents survey outcomes only for focal children who were living in the household at the time of the interview and at random 

assignment.

Sample sizes may vary because of missing values.

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Standard 

errors were adjusted to account for multiple observations per family. The p-value indicates the likelihood that the difference between the program and control 

groups arose by chance. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Statistical significance levels across subgroups are indicated as follows: ††† = 1 percent; †† = 5 percent; † = 10 percent.

Rounding may cause slight discrepancies in calculating sums and differences.

Education measures include all students who were enrolled in ninth or tenth grade at baseline (2,676 sample members). Employment measures calculated 

from unemployment administrative records include all enrolled parents (2,565 sample members). a

Monthly household income amounts equal to or greater than $10,000 were excluded from this calculation. About 4.9 percent of the sample was excluded 

from the income measures because respondents did not know or refused to provide the information. An additional 0.2 percent of the sample was excluded 

because the income provided was over $10,000. 

bAnnual household income was calculated by multiplying by 12 the respondent's income in the month prior to the survey interview. For program group 

members, it includes Family Rewards payments earned during Years 2 and 3. The federal poverty level was calculated based on annual income (monthly 

income multiplied by 12) and the household size at the time of the survey. The poverty threshold was measured according to the 2011 Poverty Guidelines, 

depending on when a respondent was interviewed. c

Happiness was measured using the U.S. General Social Survey question: "Taken all together, how would you say things are these days — would you say 

that you are very happy, pretty happy, or not too happy?"

dThe Regents exam measures in this table include the following Regents exams: English, Math A, Math B, Geometry, Integrated Algebra, Algebra 

2/Trigonometry, U.S. History and Government, Global History and Geography, Living Environment, Chemistry, Physics, and Earth Science. e

The End-of-Course exam measures in this table include the following exams: English 1, English, 2, English 3, Biology, Algebra 1, Algebra 2, and U.S. 

History.

f

Attendance was calculated as a percentage of total days present divided by total days enrolled according to district records. Records provided for students 

in New York City include enrollment for the regular school year. Records for students in Memphis include enrollment during the regular school year, 

alternative education programs, and summer school.

gChild-related health insurance measures were calculated for sample members with at least one child at the time of the survey.

hThe lowest and highest amounts earned in in all years combined were $40 and $19,521 in the Bronx and $40 and $18,400 in Memphis. 
103

Table 7.2 

Summary Program Impacts, by Poverty Level at Random Assignment

Income Less Than 50% of FPL at Baseline Income at or Above 50% of FPL at Baseline

Program Control Difference

Outcome Group Group (Impact) P-Value

Program Control Difference

Group Group (Impact) P-Value

Income, poverty, and well-being (%)

Household income at or below the federal poverty

level (including rewards)a,b

Any savings

Any housing/utilities material hardship in 

the past 12 months

Did not have enough money to buy food 

sometime in the past 12 months

c Very or pretty happy

Education

Number of state core exams passed, Years 1 to 3d,e

Attendance rate is 95% or higher, Year 3 (%)f

Graduated on time (%)

Health 

All dependent children had health insuranceg (%)

Parents' average self-rated health

(1 = poor; 5 = excellent)

Focal child had 2 or more dental checkups in past

12 months (%)

79.4

16.6

62.5

41.9

75.9

2.3

26.2

57.9

97.6

3.0

60.2

83.4

10.3

64.5

48.1

69.5

2.4

27.5

61.5

94.7

2.9

48.0

-4.1

6.4 ***

-2.0

-6.2 *

6.3 **

-0.1

-1.3

-3.5

2.8 **

0.1 *

12.2 ***

0.110

0.004

0.516

0.052

0.026

0.329

0.604

0.192

0.024

0.093

0.000

68.0

24.3

66.2

44.8

76.6

3.0

31.4

68.8

93.1

3.3

65.2

73.3

13.8

63.5

46.4

74.7

2.9

30.7

64.6

94.9

3.1

44.2

-5.3 *

10.5 ***

2.7

-1.7

1.9

0.1

0.7

4.3 *

-1.9

0.2 **

21.0 ***

0.065

0.000

0.371

0.590

0.468

0.522

0.787

0.098

0.213

0.010

0.000

††

††

†

(continued)
 

104

Table 7.2 (continued)

Income Less Than 50% of FPL at Baseline Income at or Above 50% of FPL at Baseline

Program Control Difference

Outcome Group Group (Impact) P-Value

Program Control Difference

Group Group (Impact) P-Value

Work 

Ever participated in any education, training, 

or employment activity (%)

Currently employed at the time of the survey (%)

Unemployment insurance-covered employment 

earnings, Years 1 to 3 ($)

Reward participation, Years 1 to 4 ($)

Among families who earned at least one reward

Average reward amount earned h

Average reward amount earned, by domain

Education

Health

Work

33.5

36.9

 8,239

 5,328

 2,767

 1,907

 2,515

31.9

37.6

11,323

—

—

—

—

1.6

-0.7

-3,083 ***

—

—

—

—

0.590

0.799

0.003

—

—

—

—

39.1

74.1

45,955

 7,103

 3,111

 1,713

 3,101

32.2

75.8

47,185

—

—

—

—

6.9 **

-1.7

-1,230

—

—

—

—

0.021

0.497

0.360

—

—

—

—

Sample size (total = 2,016) 497 490 528 501

(continued)
 

105

Table 7.2 (continued)

SOURCES: MDRC calculations using data from the Family Rewards 24-month survey, Children's Aid Society's Family Rewards program data, New York 

City Department of Education and Shelby County Schools administrative records, New York State unemployment insurance (UI) wage records, and 

Tennessee Department of Labor and Workforce Development UI wage records. 

NOTES: FPL = federal poverty level.

This table presents survey outcomes only for focal children who were living in the household at the time of the interview and at random assignment.

Sample sizes may vary because of missing values. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Standard 

errors were adjusted to account for multiple observations per family. The p-value indicates the likelihood that the difference between the program and 

control groups arose by chance. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Statistical significance levels across subgroups are indicated as follows: ††† = 1 percent; †† = 5 percent; † = 10 percent.

Rounding may cause slight discrepancies in calculating sums and differences.

Education measures include all students who were enrolled in ninth or tenth grade at baseline (2,676 sample members). Employment measures 

calculated from unemployment administrative records include all enrolled parents (2,565 sample members). aMonthly household income amounts equal to or greater than $10,000 were excluded from this calculation. About 4.9 percent of the sample was 

excluded from the income measures because respondents did not know or refused to provide the information. An additional 0.2 percent of the sample was 

excluded because the income provided was over $10,000.

bAnnual household income was calculated by multiplying by 12 the respondent's income in the month prior to the survey interview. For program group 

members, it includes Family Rewards payments earned during Years 2 and 3. The federal poverty level was calculated based on annual income (monthly 

income multiplied by 12) and the household size at the time of the survey. The poverty threshold was measured according to the 2011 Poverty Guidelines, 

depending on when a respondent was interviewed. cHappiness was measured using the U.S. General Social Survey question: "Taken all together, how would you say things are these days — would you 

say that you are very happy, pretty happy, or not too happy?"

dThe Regents exam measures in this table include the following Regents exams: English, Math A, Math B, Geometry, Integrated Algebra, Algebra 

2/Trigonometry, U.S. History and Government, Global History and Geography, Living Environment, Chemistry, Physics, and Earth Science. e

The End-of-Course exam measures in this table include the following exams: English 1, English, 2, English 3, Biology, Algebra 1, Algebra 2, and U.S. 

History.

f

Attendance was calculated as a percentage of total days present divided by total days enrolled according to district records. Records provided for 

students in New York City include enrollment for the regular school year. Records for students in Memphis include enrollment during the regular school 

year, alternative education programs, and summer school.

gChild-related health insurance measures were calculated for sample members with at least one child at the time of the survey.

hThe lowest and highest amounts earned in in all years combined were $40 and $20,700 for families with income less than 50 percent of the federal
106

Impacts by Parents’ Education Level

Table 7.3 presents effects by parents’ educational level, or by whether at least one parent had a 

high school diploma or equivalency certificate or higher at study entry. Families with more 

educated parents earned more than those with less educated parents ($6,515 versus $5,819), 

with the difference due to higher reward payments in the education and work domains.

The rates for the control groups show that educational level is a strong predictor of 

many of the outcomes shown. More educated parents had much lower poverty rates, their 

children did better in school, they were more likely to work, and they reported somewhat better 

health. Overall, Family Rewards 2.0 had very similar effects across the two types of families. 

The program led to a larger reduction in UI-covered earnings for the more educated group, but 

the difference between the two impacts is not statistically significant.

Impacts by Parents’ Employment Status

Table 7.4 presents effects by parents’ employment status, or whether they were working at 

study entry. The research team defined two-parent families as employed if either parent was 

working at study entry. The table shows that employment status is an important predictor of 

how much families earn from the program. Families with employed parents earned over $7,500 

on average from the program, compared with only $4,800 for other families. This difference is 

due largely to earnings from work rewards. However, families with employed parents also 

earned more in education rewards than those with unemployed parents ($3,180 and $2,685, 

respectively).

Consistent with the higher reward earnings, the program led to a large reduction in poverty for the employed group (reducing the rate by 9 percentage points) and had little effect on 

this outcome for the unemployed group. (Although not shown, the program similarly had no 

effect on average household income for the unemployed group.) Another difference in effects is 

that the program led to larger effects on children’s health coverage for the unemployed group. 

This pattern matches the positive effects for the low-income group shown in Table 7.2. In the 

education domain, the program led to a small, positive effect on graduation rates for the employed group, similar to the positive effects on this outcome for the higher-income group shown 

in Table 7.2. These two impacts were related, given the substantial overlap between the higherincome group and the employed group at study entry.1 Finally, effects on work were roughly 

similar, with a pattern of reductions in UI-covered earnings for both groups.

 1

For example, over 75 percent of higher-income group members were also employed at study entry.
107

Table 7.3 

Summary Program Impacts, by Adult Education at Baseline

Outcome

No High School Diploma or 

Equivalency Certificate

High School Diploma or 

Equivalency Certificate

Program Control Difference

Group Group (Impact) P-Value

Program

Group

Control Difference

Group (Impact) P-Value

Income, poverty, and well-being (%)

Household income at or below the federal poverty

level (including rewards)a,b

Any savings

Any housing/utilities material hardship in 

the past 12 months

Did not have enough money to buy food 

sometime in the past 12 months

c Very or pretty happy

Education

Number of state core exams passed, Years 1 to 3d,e

Attendance rate is 95% or higher, Year 3f (%)

Graduated on time (%)

Health 

All dependent children had health insuranceg (%)

Parents' average self-rated health

(1 = poor; 5 = excellent)

Focal child had 2 or more dental checkups in past

12 months (%)

80.9

15.2

59.0

42.4

72.8

2.5

24.4

57.8

95.8

3.0

61.9

86.7

9.0

64.1

53.7

67.7

2.4

23.5

59.0

95.6

2.9

47.0

-5.8 **

6.2 ***

-5.1

-11.3 ***

5.1

0.1

0.8

-1.2

0.2

0.2 **

14.9 ***

0.032

0.009

0.151

0.002

0.123

0.479

0.765

0.703

0.881

0.041

0.000

68.5

24.3

67.6

44.2

78.4

2.7

31.4

66.8

95.2

3.2

62.9

73.0

14.1

64.2

43.5

74.9

2.8

32.4

65.7

93.8

3.1

45.7

-4.5 *

10.2 ***

3.5

0.7

3.5

-0.1

-1.1

1.1

1.4

0.1 **

17.2 ***

0.086

0.000

0.204 †

0.817 †††

0.142

0.211

0.654

0.640

0.296

0.025

0.000

(continued)
108

Table 7.3 (continued)

Outcome

No High School Diploma or 

Equivalency Certificate

High School Diploma or 

Equivalency Certificate

Program Control Difference

Group Group (Impact) P-Value

Program

Group

Control Difference

Group (Impact) P-Value

Work 

Ever participated in any education, training, 

or employment activity (%)

Currently employed at the time of the survey (%)

Unemployment Insurance-covered employment 

earnings, Years 1 to 3 ($)

Reward participation, Years 1 to 4 ($)

Among families who earned at least one reward

Average reward amount earned h

Average reward amount earned, by domain

Education

Health

Work

32.2

48.8

20,735

 5,819

 2,789

 1,858

 2,734

29.3

50.5

21,715

—

—

—

—

2.9

-1.8

-980

—

—

—

—

0.377

0.542

0.318

—

—

—

—

39.1

60.4

32,287

 6,515

 3,021

 1,776

 3,033

33.7

60.7

34,941

—

—

—

—

5.4 **

-0.3

-2,655 **

—

—

—

—

0.047

0.909

0.040

—

—

—

—

Sample size (total = 1,996) 396 373 618 609

(continued)
109

Table 7.3 (continued)

SOURCES: MDRC calculations using data from the Family Rewards 24-month survey, Children's Aid Society's Family Rewards program data, New York 

City Department of Education and Shelby County Schools administrative records, New York State unemployment insurance (UI) wage records, and 

Tennessee Department of Labor and Workforce Development UI wage records. 

NOTES: This table presents survey outcomes only for focal children who were living in the household at the time of the interview and at random 

assignment. For students with two parents in the household, no high school diploma or equivalency certificate is defined as having no adults in the 

household with a degree.

Sample sizes may vary because of missing values. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Standard 

errors were adjusted to account for multiple observations per family. The p-value indicates the likelihood that the difference between the program and 

control groups arose by chance. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Statistical significance levels across subgroups are indicated as follows: ††† = 1 percent; †† = 5 percent; † = 10 percent.

Rounding may cause slight discrepancies in calculating sums and differences.

Education measures include all students who were enrolled in ninth or tenth grade at baseline (2,676 sample memebers). Employment measures 

calculated from unemployment administrative records include all enrolled parents (2,565 sample members). a

Monthly household income amounts equal to or greater than $10,000 were excluded from this calculation. About 4.9 percent of the sample was 

excluded from the income measures because respondents did not know or refused to provide the information. An additional 0.2 percent of the sample was 

excluded because the income provided was over $10,000.

bAnnual household income was calculated by multiplying by 12 the respondent's income in the month prior to the survey interview. For program group 

members, it includes Family Rewards payments earned during Years 2 and 3. The federal poverty level was calculated based on annual income (monthly 

income multiplied by 12) and the household size at the time of the survey. The poverty threshold was measured according to the 2011 Poverty Guidelines, 

depending on when a respondent was interviewed. c

Happiness was measured using the U.S. General Social Survey question: "Taken all together, how would you say things are these days — would you 

say that you are very happy, pretty happy, or not too happy?"

dThe Regents exam measures in this table include the following Regents exams: English, Math A, Math B, Geometry, Integrated Algebra, Algebra 

2/Trigonometry, U.S. History and Government, Global History and Geography, Living Environment, Chemistry, Physics, and Earth Science. eThe End-of-Course exam measures in this table include the following exams: English 1, English, 2, English 3, Biology, Algebra 1, Algebra 2, and U.S. 

History.

f

Attendance was calculated as a percentage of total days present divided by total days enrolled according to district records. Records provided for 

students in New York City include enrollment for the regular school year. Records for students in Memphis include enrollment during the regular school 

year, alternative education programs, and summer school.

gChild-related health insurance measures were calculated for sample members with at least one child at the time of the survey.

hThe lowest and highest amounts earned in in all years combined were $40 and $20,700 for families with no adults with a high school diploma or 

equivalency certificate at baseline and $40 and $22,185 for families who had at least one adult with a high school diploma or equivalency certificate at 

baseline. 
110

Table 7.4

Summary Program Impacts, by Adult Employment at Baseline

Outcome

Program

Group

Not Employed at Baseline Employed at Baseline

Control Difference

Group (Impact) P-Value

Program

Group

Control Difference

Group (Impact) P-Value

Income, poverty, and well-being (%)

Household income at or below the federal poverty

level (including rewards)a,b

Any savings

Any housing/utilities material hardship in 

the past 12 months

Did not have enough money to buy food 

sometime in the past 12 months

c Very or pretty happy

Education

Number of state core exams passed, Years 1 to 3d,e

Attendance rate is 95% or higher, Year 3f (%)

Graduated on time (%)

Health 

All dependent children had health insuranceg (%)

Parents' average self-rated health

(1 = poor; 5 = excellent)

Focal child had 2 or more dental checkups in past

12 months (%)

82.1

16.7

62.7

41.3

74.7

2.2

24.8

58.3

96.9

3.0

58.4

82.3

8.9

64.9

46.4

69.6

2.3

24.8

62.0

94.6

2.9

43.6

-0.2

7.9 ***

-2.2

-5.1

5.2 *

-0.1

0.0

-3.8

2.3 *

0.1

14.8 ***

0.932

0

0.48

0.105

0.072

0.405

0.994

0.172

0.078

0.153

0.000

65.6

24.1

65.4

45.6

77.6

3.1

32.6

68.9

94.2

3.3

66.3

74.7

15.2

63.1

47.7

74.5

3.0

33.0

63.9

95.1

3.1

48.7

-9.0 ***

8.9 ***

2.3

-2.1

3.2

0.1

-0.4

5.0 *

-1.0

0.2 ***

17.6 ***

0.002

0.000

0.449

0.503

0.224

0.348

0.862

0.052

0.494

0.005

0.000

††

††

†

(continued)
111

Table 7.4 (continued)

Outcome

Program

Group

Not Employed at Baseline Employed at Baseline

Control Difference

Group (Impact) P-Value

Program

Group

Control Difference

Group (Impact) P-Value

Work 

Ever participated in any education, training, 

or employment activity (%)

Currently employed at the time of the survey (%)

Unemployment Insurance-covered employment

 earnings, Years 1 to 3 ($)

Reward participation, Years 1 to 4 ($)

Among families who earned at least one reward

Average reward amount earned h

Average reward amount earned, by domain

Education

Health

Work

32.8

30.0

10,055

 4,839

 2,685

 1,822

 1,888

32.3

29.1

13,304

—

—

—

—

0.5

0.9

-3,249 ***

—

—

—

—

0.875

0.758

0.003

—

—

—

—

39.3

80.4

44,918

 7,517

 3,180

 1,795

 3,282

32.4

83.2

45,477

—

—

—

—

6.9 **

-2.8

-559

—

—

—

—

0.021

0.249

0.667

—

—

—

—

Sample size (total = 2,006) 479 493 541 493

(continued)
 

112

Table 7.4 (continued)

SOURCES: MDRC calculations using data from the Family Rewards 24-month survey, Children's Aid Society's Family Rewards program data, New York 

City Department of Education and Shelby County Schools administrative records, New York State unemployment insurance (UI) wage records, and 

Tennessee Department of Labor and Workforce Development UI wage records. 

NOTES: This table presents survey outcomes only for focal children who were living in the household at the time of the interview and at random 

assignment. For students with two parents in the household, not employed at baseline is defined as neither adult being employed at baseline.

Sample sizes may vary because of missing values. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Standard 

errors were adjusted to account for multiple observations per family. The p-value indicates the likelihood that the difference between the program and 

control groups arose by chance. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Statistical significance levels across subgroups are indicated as follows: ††† = 1 percent; †† = 5 percent; † = 10 percent.

Rounding may cause slight discrepancies in calculating sums and differences.

Education measures include all students who were enrolled in ninth or tenth grade at baseline (2,676 sample memebers). Employment measures 

calculated from unemployment administrative records include all enrolled parents (2,565 sample members). aMonthly household income amounts equal to or greater than $10,000 were excluded from this calculation. About 4.9 percent of the sample was 

excluded from the income measures because respondents did not know or refused to provide the information. An additional 0.2 percent of the sample was 

excluded because the income provided was over $10,000.

bAnnual household income was calculated by multiplying by 12 the respondent's income in the month prior to the survey interview. For program group 

members, it includes Family Rewards payments earned during Years 2 and 3. The federal poverty level was calculated based on annual income (monthly 

income multiplied by 12) and the household size at the time of the survey. The poverty threshold was measured according to the 2011 Poverty Guidelines, 

depending on when a respondent was interviewed. c

Happiness was measured using the U.S. General Social Survey question: "Taken all together, how would you say things are these days — would you 

say that you are very happy, pretty happy, or not too happy?"

dThe Regents exam measures in this table include the following Regents exams: English, Math A, Math B, Geometry, Integrated Algebra, Algebra 

2/Trigonometry, U.S. History and Government, Global History and Geography, Living Environment, Chemistry, Physics, and Earth Science. eThe End-of-Course exam measures in this table include the following exams: English 1, English, 2, English 3, Biology, Algebra 1, Algebra 2, and U.S. 

History.

f

Attendance was calculated as a percentage of total days present divided by total days enrolled according to district records. Records provided for 

students in New York City include enrollment for the regular school year. Records for students in Memphis include enrollment during the regular school 

year, alternative education programs, and summer school.

gChild-related health insurance measures were calculated for sample members with at least one child at the time of the survey.

hThe lowest and highest amounts earned in in all years combined were $40 and $16,1400 for families with no adults employed at baseline and $40 and 

$22,185 for families who had at least one adult employed at baseline. 
113

Conclusion

The results indicate that overall the program had similar effects across the two cities, reducing 

poverty, improving health status and increasing dental visits, and having little effect on school 

progress. The program led to slight reductions in work in both cities, although only in surveyreported work in the Bronx and only in UI-covered work in Memphis.

The results also suggest that the program had broadly consistent effects across more 

versus less disadvantaged families, with a few differences. For most groups, the program 

increased incomes and reduced poverty. And for all groups, the program increased health status 

and health care use, primarily dental visits. Although the effects on health were fairly similar,

findings presented in Chapter 5 suggest that the program may have had larger positive effects 

on health for parents who entered the study in poorer health.

The most notable difference found in the subgroup analysis was the one between employed and unemployed parents at study entry. Families without a working parent earned 

substantially less from the program than other families, from both work and education rewards. 

If a main goal of the program is poverty reduction, these parents in particular may need additional assistance to earn rewards. Finally, there is some suggestion that the program increased 

graduation rates for a subset of students from more advantaged families.

115

Chapter 8 

Conclusion

In the ongoing effort to determine what works to help improve the lives of low-income families 

in the United States, conditional cash transfer (CCT) programs represented an exciting and 

innovative approach. The model has had success in lower- and middle-income countries. But 

until 2007, when Family Rewards 1.0 was launched, no comprehensive CCT program had been 

attempted in a higher-income country. Modeled on the programs in other countries, and tested 

in six of New York City’s highest-poverty neighborhoods, Family Rewards 1.0 provided cash 

assistance to low-income families if they met certain conditions related to family health care, 

children’s education, and parents’ work. While the program left the most important outcomes 

unchanged, it did lead to notable reductions in poverty in the short term and increased graduation rates for a group of ninth-graders who were more academically prepared at study entry.

The early findings from the Family Rewards 1.0 evaluation — about how families experienced the program, which types of families earned rewards, and which types of families and 

students benefited — suggested ways in which the model could be modified to be more effective. Family Rewards 2.0 was the next iteration of the model. The new model offered fewer 

rewards in each of the three areas of children’s education, family health, and parents’ work (8 

rewards total, instead of 22), in order to make the program easier to understand and to focus 

families’ attention on a limited number of outcomes. It offered the education rewards only to 

high school students, given the lack of effects found for younger students, and it made the

rewards more timely, and thus more salient to families, by paying them monthly for rewards 

earned, rather than every two months, and by rewarding students for grades earned. The biggest 

change, however, was that the new model offered guidance, in which staff members actively 

engaged families in conversations about strategies to earn rewards.

In short, the model moved from one with a large set of incentives but little guidance to 

one with a smaller set of rewards, that were paid on a timelier basis, and that combined the 

rewards with family guidance. To assess whether this new approach was more effective, the 

program was implemented and tested in the Bronx, New York, and Memphis, Tennessee in July 

2011. Among the group of low-income families who were recruited for and volunteered to 

participate in the study, a randomly chosen subset was offered the new program for three years. 

This report presents the effects on families for three to four years after study entry.
116

Summary of Findings

After some recruitment and start-up challenges in Year 1, the program was operating generally 

as envisioned in both cities by the middle of Year 2. The family guidance component changed 

the most over time, starting out as fairly modest and transactional during Year 1 and becoming 

more intensive during Year 2, with the introduction of motivational interviewing. In addition, 

the advisors began targeting low-earning participants to provide them with more guidance 

sessions in the second half of Year 2. Reaching out to these participants involved a tremendous 

amount of effort from advisors.1

Further changes during Year 3 included a greater push to have advisors make referrals. 

It was anticipated at the outset that advisors would function as “resource experts” who developed a knowledge of the services available in their communities and connected participants to 

appropriate agencies, when such services might help participants meet the conditions to earn 

rewards. However, through Year 2, the fraction of guidance sessions that included referrals 

remained quite low. After the increased effort, referrals increased dramatically in the Bronx in 

Year 3, but remained low in Memphis.

Overall, participants reported good relationships with their advisors, suggesting that the 

quality of advisement improved over the three years. However, the frequency of advisement 

was not as high as program designers had hoped, particularly in Memphis. Across both cities, 

for example, most participants did not attend the anticipated two formal guidance sessions in 

Year 2 or 3, with rates of just over 60 percent in the Bronx and less than 25 percent in Memphis. 

Finally, interviews suggest that most families understood the program fairly well. Reward 

earnings rates by different types of families also suggest that the outreach later in the program 

targeting low reward-earners did lead to increased reward receipt among more disadvantaged 

families, something with which administrators in the first program struggled. Families earned 

on average more than $2,000 per year in rewards in Years 2 and 3.

In terms of effects, the findings show that the new program achieved many of the same 

effects as the original model, but fell short in other, important ways. Family Rewards 2.0 met its 

short-term goals of increasing income and reducing poverty for all families and across a range 

of family types. It also increased dental visits and adults’ self-reported health status, particularly 

for those in poorer health at study entry. Similar to Family Rewards 1.0, the new model led to 

reductions in work and earnings for some participants. However, in contrast to the original 

program, it did not affect students’ school progress through Year 4, either for the full sample of 

students or for the subgroup of academically proficient students. Although there were some 

 1

See Dechausay et al. (2014) for more details about program implementation during the first two years.
117

small, positive effects on attendance and credits for the ninth-graders, these effects did not 

persist or lead to lasting effects on enrollment or graduation.

The addition of Memphis as a site in the study also provided information on how the 

program might work in a different context, since the original program had been tested only in 

New York City. In general, the program was successfully implemented in both cities and had 

similar effects on participating families’ poverty, health, education, and work. However, the 

context did matter for how the program was implemented and how families experienced it. In 

some cases, the differences were due to specific policies. For example, students in the Bronx 

were less likely to earn rewards for attendance during the first two years because New York

City public schools count tardiness toward absences, and program staff followed that convention until Year 3. In contrast, more students in the Bronx earned the high-value reward for 

passing the state core exams because they were well informed about the exams and took 

practice tests in class as preparation. In Memphis, these exams were relatively new. As another 

example, fewer adults claimed the reward for dental checkups in Memphis than in the Bronx, 

largely because the public health insurance program in Tennessee does not cover dental cleanings, as it does in New York State. 

While the program might have been easily adjusted for these types of local differences, 

what proved to be more challenging in Memphis was transportation. In that city, the lack of 

good public transit was an important barrier to participation in the guidance component. Staff 

members adapted by conducting guidance sessions over the phone or by making home visits, 

but they did not have much direct contact with students. Participants in Memphis tended to feel 

that they were not as close to their advisors as did participants in the Bronx. While the findings 

suggest that these differences in the quality and quantity of advisement did not lead to different 

effects, it is nonetheless an important issue to consider in a model in which guidance is a central

component. 

Finally, an analysis of program expenditures found that the program cost just under 

$13,500 per family over the full program period. Reward payments accounted for about half of 

this amount, and the costs of operating the program, which included administering the payments 

and providing the advisement component, accounted for the other half. As noted in Chapter 2, 

these administrative costs were undoubtedly higher than what such costs would have been had 

the program been operated on a much larger scale and not as part of a relatively small demonstration project.

Lessons from Family Rewards

Although a somewhat modified replication of the original model that targeted different types of 

families, Family Rewards 2.0 adds to the evidence about how a CCT program might work in 
118

the United States. By comparing the findings from the two Family Rewards evaluations, it is 

possible to draw some general conclusions about this type of holistic CCT program. 

Most families earn rewards, especially in education and health. In terms of reward 

receipt, nearly all enrolled families earned at least some rewards in both programs, and most of 

their earnings came from the education and health domains. For example, over 97 percent of 

families in each program earned at least one education reward, about 98 percent in the first 

program and 90 percent in the second program earned at least one health reward, and just over 

50 percent in each program earned at least one work reward. Family Rewards 2.0 did not 

include rewards for maintaining health insurance, so the relatively high receipt rate in that 

domain is encouraging.

More disadvantaged families earn less than other families. One of the goals of the 

targeted outreach strategy in Family Rewards 2.0, in which staff reached out to families whose 

reward earnings were low, was to increase reward receipt among the types of families who had

earned less in Family Rewards 1.0. Comparing the receipt rates of similar rewards across both 

programs, the data on reward earnings suggest that this outreach succeeded. However, the 

general pattern that more advantaged families earned substantially more in rewards than their 

disadvantaged counterparts still held in both programs. Parents of families that earned more 

rewards, for example, had higher education levels, were more likely to be working, and were 

more likely to be married than those of families that earned less.

Family Rewards reduces poverty and material hardship and increases savings 

while the rewards are offered. But a program with fewer rewards transfers less cash to 

families and leads to smaller effects. Family Rewards 2.0 included fewer rewards in order to 

make the program easier to understand and focus families’ effort on key milestones. The 

qualitative data suggest that families understood the more streamlined program more easily, 

although the result of offering fewer rewards was that families earned on average much less per 

year (just over $2,000 per year in Family Rewards 2.0, compared with about $3,000 per year in 

Family Rewards 1.0). Not surprisingly, the program reduced poverty by lesser a degree (by 5 

percentage points in Family Rewards 2.0 compared with 12 percentage points in Family 

Rewards 1.0) and had fewer effects on several measures of material hardship.

Family Rewards increases dental visits and seems to improve some parents’ health 

status. The two programs had fairly similar effects on health. Both had the largest and most 

consistent effects on dental visits, with both parents and children much more likely to have 

completed the two recommended visits per year. Both programs also seemed to improve 

parents’ self-rated health status. Family Rewards 1.0 had a marginally significant effect on this 

outcome for the full sample, but a statistically significant effect for those in poor health at study 
119

entry. Family Rewards 2.0 replicated this finding. Neither program led to changes in health 

coverage or preventive medical visits.

Family Rewards leads to modest reductions in work, particularly among more 

disadvantaged groups. Although other work incentives programs have had positive effects on 

work, there was a concern about Family Rewards from the outset that families’ reward earnings 

in the other two domains might create an “income effect” and lead to reductions in work. An 

income effect was observed to some extent in both programs, although in somewhat different 

ways. The first program led to a modest increase in employment for the sample as a whole, but 

largely in work not captured by the unemployment insurance (UI) system. It also led to notable 

reductions in UI-covered work and earnings for parents without a high school diploma and 

those with no recent work experience. The second program led to reductions in UI-covered 

earnings for the full sample, although negative effects in Memphis largely drove this change. 

The reasons for the differences across the two cities are not entirely clear, but the findings add 

to the evidence that CCT programs with generous incentives have the unintended effect of 

reducing work to some extent. This reduction in work conflicts with the intent of CCT programs 

to increase longer-term self-sufficiency.

Family Rewards can sometimes improve school outcomes. The original program led 

to notable increases in school progress and graduation rates for ninth-graders who entered the 

study already proficient in English language arts. The hope with the modified model, particularly through its rewards for grades and the addition of family guidance, was that it would lead to 

similar effects, but for all students, including those who were struggling academically. With the 

exception of some small effects for entering ninth-graders, Family Rewards 2.0 did not generally affect school progress for the full group of students and was not able to replicate the positive 

effect for proficient students.

Although the students in Family Rewards 2.0 were from different types of families (for 

instance, they were from families receiving Temporary Assistance for Needy Families or 

Supplemental Nutrition Assistance Program benefits and from families living in Memphis in 

addition to the Bronx), these differences did not seem to account for the lack of effects. Another

way in which students in the two programs differed is that students in the Family Rewards 2.0 

study were doing better in school in the absence of the program, with higher rates of attendance 

and grade-to-grade retention. By the end of Year 2, for example, only 44 percent of the ninthgraders in the Family Rewards 1.0 control group had earned enough credits to stay on track for 

an on-time graduation, compared with 60 percent of ninth-graders in Family Rewards 2.0

control group. (A later paper will present a more formal analysis of the differences between the 

two programs.) Nonetheless, the findings across both programs show that a CCT program of 

this design, with or without guidance, does not have robust and consistent effects on students’ 

school outcomes.
120

Adding more intensive guidance to the model is challenging and does not necessarily lead to bigger effects. Not surprisingly, providing more intensive guidance to families 

about how to earn rewards was the most challenging part of the program. Although advisors 

built relationships with participants during Year 1, they largely relied on a light-touch and 

transactional method of guidance, reporting that they often felt unqualified to address some of 

the personal issues that families brought up. In Year 2, the advisors received training in motivational interviewing and were pushed to provide more intensive guidance and extra outreach to 

the lowest-earning families. Overall, they succeeded on both fronts, although it required 

significant time and effort on their part. The more intensive advisement contributed to higher 

program costs. One way to measure program costs is to calculate how much every dollar of 

reward payments transferred to families cost each program. That cost was $0.47 in Family 

Rewards 1.0 and $1.07 in Family Rewards 2.0.

However, a comparison of effects across both programs suggests that the more intensive guidance and advising did not lead to bigger or more widespread effects. This result was 

found in Memphis, where transportation barriers prevented advisors from meeting with families 

as often as hoped, but also in the Bronx, where advisors met with parents and students more 

frequently.

Next Steps

Evidence from the two Family Rewards programs points to what a CCT model can and cannot

do for families in the United States. A CCT program can transfer significant sums of money to 

low-income families and reduce poverty. And it can increase dental visits and parents’ health 

status. In terms of the other domains, an important question is: are there any modifications to the 

current model that might lead to stronger and more consistent effects on students’ schooling or 

that might reduce the incentives it creates for parents to reduce their work effort?

In terms of work, the answer is not necessarily to remove the work rewards, since the 

reductions in work might have been even larger in their absence. Should all or some fraction of 

the rewards (in all domains) be conditional upon work? Doing so might minimize the reductions

in work, but it would also decrease the income transfer going to the most disadvantaged 

families. Or, is the answer to provide more support for families who want to work? The value of 

more guidance seems to be less clear, given that Family Rewards 2.0 provided it and that many 

families, especially in Memphis, had a hard time taking advantage of it.

More guidance might be one answer to making the education rewards more effective. 

But the issue of participation remains, since many students would probably still have trouble 

attending guidance sessions. If more guidance is the answer, and it is not clear that it is, then one 

option might be to make the program school based. If the program were offered to an entire 
121

school, the incentives themselves might be more salient (through group and classroom dynamics), and students who needed tutoring or other assistance might be more likely to get it (assuming the school has the resources). However, enrolling an entire school in a holistic CCT program — with rewards for reaching milestones in education, health, and work milestones — 

might mean offering rewards to some families who are not low income. This prospect would 

limit the program’s ability to reduce short-term poverty, which is a primary program goal.

Finding effective ways to invest in families and break the cycle of poverty is critical, 

and the results presented in this report indicate that perhaps Family Rewards is not part of the 

answer. Nonetheless, the benefits of making even small improvements in health and education 

suggest that the continued search is worth the effort.

Appendix A

Rewards Offered in Family Rewards 1.0 and 2.0

Appendix Table A.1

Comparison of Rewards Offered by Family Rewards 1.0 and 2.0 

Family Rewards 2.0

Reward Type Family Rewards 1.0 Bronx Memphis

Education incentives

Elementary and middle school students (paid to parent)

Attends 95% of scheduled school days every month $25 per month (discontinued after Year 2) Not offered Not offered

Scores at proficiency level or improves

on annual math and English tests

Elementary school students $300 per math test; $300 per English test Not offered Not offered

Middle school students $350 per math test; $350 per English test Not offered Not offered

Parent reviews low-stakes interim tests $25 for parents to download, print, and Not offered Not offered

up to 5 times per year review results (discontinued after Year 1)

Parent discusses up to 2 annual math and $25 (discontinued after Year 2) Not offered Not offered

English test results with teachers

a High school students

Takes up to 2 PSAT tests $50 per test Not offered Not offered

Accumulates 11 course credits per year $600 Not offered Not offered

Graduates from high school $400 Not offered Not offered

Attends 95% of scheduled school days $50 per month $40 per month $40 per month

Takes an SAT or ACT exam (once during program) Not offered $50 $50; must score 19 or 

better on the ACT if 

administered by MCSb

(continued)

125
Appendix Table A.1 (continued)

Family Rewards 2.0

Reward Type Family Rewards 1.0 Bronx Memphis

Education incentives (continued)

Receives grades on an official report card Not offered $30 per A (90-100); $30 per A (93-100); 

$20 per B (80-89); $20 per B (85-92); 

$10 per C (75-79) $10 per C (75-84)c

d Passes up to 5 Regents exams or 7 End-of- $600 per Regents exam $500 per Regents $200 per End-of- e Course exams exam for a score of Course exam for a 

75 or above; $400 score of proficient or 

per exam for a advanced (increased 

score of 65-74 to $300 in Year 2)

All grades

Parent attends parent-teacher conferences $25 per conference Not offered Not offered

up to 2 times per year

Child obtains library card $50 once during program Not offered Not offered

(discontinued after Year 2)

Health incentives

Maintaining public or private health insurance Not offered Not offered

For each parent covered Per month: $20 (public); $50 (private)

If all children are covered Per month: $20 (public); $50 (private)

(discontinued after Year 2)

Annual medical checkup $200 per family member $100 per family $100 per family 

member member

A doctor-recommended follow-up visit $100 per family member Not offered Not offered

(discontinued after Year 2)

(continued)

126
Appendix Table A.1 (continued)

Family Rewards 2.0

Reward Type Family Rewards 1.0 Bronx Memphis

Early intervention evaluation for child under 30

months old, if advised by pediatrician $200 per child Not offered Not offered

Preventive dental care every 6 months $100 per family member $100 per family $100 per family member

(once per year for children 1-5 years old) member

Adult work efforts

Sustained full-time employment $150 per month $150 per month $150 per month

Education and training while employed at least Amount varied by length of course, Not offered Not offered

10 hours per week up to a maximum of $3,000 over 3 years 

(employment requirement discontinued 

after Year 2)

GED certificate Not offered $400 for successful $400 for successful 

completion completion

SOURCE: Children’s Aid Society's Family Rewards 2.0 program materials and Seedco's Family Rewards 1.0 program materials.

NOTES: MCS = Memphis City Schools; GED = General Education Development.

 a

In Family Rewards 1.0, the following education rewards were paid entirely to students: taking up to two PSAT Tests ($50), passing five Regents exams 

($600 per exam), and obtaining a library card ($50). The following education rewards were split between parents and students: accumulating 11 course credits 

per year ($600) and attending 95 percent of scheduled school days. In Family Rewards 2.0, all education rewards were paid entirely to students.

 b

The ACT is scored out of 36. Memphis City Schools officials requested that the minimum score for the reward be set at 19 for students taking 

the test for free in class because this score is considered an indication that students are ready for college-level work.

 c

In Year 1, the amount of each reward for grades was prorated based on the number of official report cards issued by a student's school. To 

simplify verification in Year 2, students were paid the listed amounts for their grades regardless of the number of official report cards they 

received, up to a maximum of $600 per program year.

 d

High school students (grades 9-12) in the Bronx were eligible to earn rewards for the following Regents exams: English, one of any math exams 

(including Math A, Math B, Integrated Algebra, Geometry, and Algebra 2/Trigonometry), U.S. History and Government, Global History and Geography, 

and one of any science exams (including Living Environment, Chemistry, Physics, and Earth Science).

 e

High school students (grades 9-12) in Memphis were eligible to earn rewards for the following End-of-Course exams: Algebra 1, Algebra 2, Biology, 

English 1, English 2, English 3, and U.S. History.

127

Appendix B 

Parents’ Understanding of and Experience with 

Family Rewards

131

Program

Outcome (%) Group

a Across all domains

Average proportion of rewarded activities that respondents correctly identified

b,c as eligible for rewards

Average proportion of unrewarded activities that respondents correctly identified 

d as not eligible for rewards

Payment process 

Agreed or strongly agreed with the following statements

My payments from the program are always on time

I have given up trying to earn certain rewards because it is hard to get paid for things I do

I remember the deadlines for the coupons and get them filled out completely

When I don't earn a reward for an activity, I understand why

89.4

35.3

81.2

41.6

87.9

91.1

Sample size 1,025

Appendix Table B.1

Parents' Understanding of and Experience with the Family Rewards Incentives and 

Procedures

SOURCE: MDRC calculations using data from the Family Rewards 24-month survey.

NOTES: This table excludes control group members because it pertains only to the Family Rewards 

program. 

Sample sizes may vary because of missing values. 

a

Responses of "don't know" are considered a wrong answer.

bThis measure refers to the average number of correctly identified items divided by the total number 

of items and multiplied by 100. c

There are eight rewarded activities that respondents could identify as eligible.

dThere are three unrewarded activities that respondents could identify as not eligible.

Appendix C 

Impacts on Public Benefit Receipt 

135

Appendix Table C.1

Impacts on Temporary Assistance for Needy Families (TANF) or Safety Net

 Assistance (SNA) and Food Stamp Receipt and Payments, Years 1 to 2

Outcome

Program

Group

Control

Group

Difference

(Impact) P-Value

Ever received TANF/SNA (%)

Years 1 to 2

Year 1

Year 2

Amount of TANF/SNA received ($)

Years 1 to 2

Year 1

Year 2

Ever received food stamps (%)

Years 1 to 2

Year 1

Year 2

Amount of food stamps received ($)

Years 1 to 2

Year 1

Year 2

22.8

18.1

14.6

897

449

448

95.8

91.2

90.1

9,993

5,292

4,700

23.0

17.6

13.9

834

414

420

95.9

91.5

89.1

9,589

5,016

4,573

-0.2

0.5

0.7

64

35

28

-0.2

-0.3

1.0

404 *

276 **

128

0.880

0.729

0.601

0.562

0.525

0.658

0.835

0.777

0.389

0.068

0.018

0.273

Sample size (total = 2,456) 1,230 1,226

SOURCE: MDRC calculations using administrative records data from the New York City and 

Memphis human resources administrations.

NOTES: Estimates were regression-adjusted using ordinary least squares, controlling for prerandom assignment characteristics of sample members. Standard errors were adjusted to account 

for multiple observations per family. Statistical significance levels are indicated as follows: *** = 1 

percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences.

Dollar averages include zero values for sample members who were not receiving TANF or SNA 

benefits or food stamps.
136

Appendix Table C.2 

Impacts on Temporary Assistance for Needy Families (TANF) or Safety Net

 Assistance (SNA) and Food Stamp Receipt and Payments, Years 1 to 2 by City

Bronx Memphis

Program Control Difference P-Value Program Control Difference P-Value

Ever received TANF/SNA (%)

Years 1 to 2

Year 1

Year 2

Amount of TANF/SNA received ($)

Years 1 to 2

Year 1

Year 2

Ever received food stamps (%)

Years 1 to 2

Year 1

Year 2

Amount of food stamps received ($)

Years 1 to 2

Year 1

Year 2

22.1

14.6

17.0

1,290

559

731

98.3

90.3

90.9

9,479

5,015

4,465

21.3

13.2

17.2

1,205

521

684

98.3

90.2

92.4

9,290

4,845

4,444

0.9

1.3

-0.2

86

39

47

0.0

0.0

-1.5

190

169

21

0.684

0.440

0.926

0.675

0.698

0.701

1.000

0.995

0.329

0.498

0.269

0.890

23.6

21.6

12.2

506

338

168

93.3

92.1

89.3

10,489

5,561

4,929

24.7

21.9

10.5

459

307

152

93.5

92.7

85.7

9,908

5,198

4,710

-1.1

-0.3

1.7

47

31

16

-0.2

-0.6

3.6 *

581 *

363 **

219

0.600

0.876

0.313

0.437

0.429

0.597

0.879

0.700

0.053

0.085

0.036

0.220

††

Sample size (total = 2,456) 617 613 613 613

SOURCE: MDRC calculations using administrative records data from the New York City and Memphis human resources administrations.

NOTES: Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. 

Standard errors were adjusted to account for multiple observations per family. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 

5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in calculating sums and differences.
Appendix D 

Impacts on Education Outcomes, by Math Proficiency

139

Appendix Table D.1

Impacts on Enrollment, Graduation, Attendance, Credits, and Regents or End-of-Course Exams

for Students in Grades 9 and 10 at the Time of Random Assignment, by Math Proficiency Test Score

Proficient on Eighth-Grade Math Test Not Proficient on Eighth-Grade Math Test

Grade Level and Outcome

Program Control Difference

Group Group (Impact) P-Value

Program Control Difference

Group Group (Impact) P-Value

Enrollment, Year 3 (%)

On grade

Not on grade

Not enrolled

Graduation (%)

Graduated on time

Graduated by Year 4a

Attendanceb

 (%)

Attendance rate, Year 1

Attendance rate, Year 2

Attendance rate, Year 3

Attendance rate is 95% or higher, Year 1

Attendance rate is 95% or higher, Year 2

Attendance rate is 95% or higher, Year 3

c Credits

91.1

6.1

2.8

77.9

78.6

89.3

87.6

84.3

60.1

48.7

36.8

85.5

9.6

4.9

74.6

76.2

86.6

83.6

80.1

52.0

47.9

40.1

5.5 **

-3.5

-2.1

3.3

2.4

2.7

4.0 *

4.1 *

8.1 *

0.8

-3.3

0.035

0.115

0.201

0.350

0.491

0.204

0.070

0.078

0.050

0.837

0.404

79.5

12.8

7.7

61.9

64.6

87.4

82.7

78.4

41.0

34.8

28.0

79.6

11.9

8.5

62.2

65.7

88.9

82.1

77.7

42.8

33.8

27.0

-0.1

0.9

-0.8

-0.3

-1.2

-1.5 **

0.7

0.7

-1.9

1.0

1.1

0.952

0.530

0.518

0.881

0.599

0.042

0.546

0.595

0.428

0.650

0.612

†

†

†

††

Number of credits earned, Year 1

Number of credits earned, Year 2

Number of credits earned, Year 3

Number of credits earned, Years 1 to 3

Earned adequate credits, Years 1 to 3 (%)

12.3

11.4

10.7

34.3

73.5

11.7

11.3

10.1

33.1

68.9

0.5

0.1

0.6

1.2

4.6

0.203

0.858

0.202

0.308

0.224

10.7

9.9

9.2

29.8

52.5

11.1

9.8

9.1

30.0

53.4

-0.3

0.1

0.1

-0.2

-0.9

0.123

0.837

0.679

0.785

0.715

†

(continued)
 

140

Appendix Table D.1 (continued)

Proficient on Eighth-Grade Math Test Not Proficient on Eighth-Grade Math Test

Grade Level and Outcome

Program Control Difference

Group Group (Impact) P-Value

Program Control Difference

Group Group (Impact) P-Value

d,e Regents or End-of-Course exams

State core exams taken, Years 1 to 3

State core exams passed, Years 1 to 3

7.5

5.0

7.1

4.9

0.4

0.1

0.108

0.452

6.3

2.0

6.3

2.1

-0.1

0.0

0.760

0.728

Sample size (total = 2,450) 284 308 955 903

SOURCES: MDRC calculations using data from New York City Department of Education and Shelby County Schools administrative records.

NOTES: Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. 

Standard errors were adjusted to account for multiple observations per family. Statistical significance levels are indicated as follows: *** = 1 

percent; ** = 5 percent; * = 10 percent. Statistical significance levels across subgroups are indicated as follows: ††† = 1 percent; †† = 5 percent; 

† = 10 percent.

Rounding may cause slight discrepancies in calculating sums and differences. 

Note that all outcomes in the table include zero values for students who were no longer enrolled.

Years 1, 2, 3, and 4 cover the 2011-2012, 2012-2013, 2013-2014, and 2014-2015 school years, respectively. a

Students enrolled in tenth grade at the time of random assignment had five years to complete graduation in this measure. Students enrolled in 

ninth grade at the time of random assignment had four years.

bAttendance was calculated as a percentage of total days present divided by total days enrolled according to district records. Records provided for 

students in New York City include enrollment for the regular school year. Records for students in Memphis include enrollment during the regular 

school year, alternative education programs, and summer school. cStudents in New York City earn 1 credit per course per semester completed. Students in Memphis earn 0.5 credits per course per semester. 

Credits for students in Memphis were multiplied by two to create a standard scale for comparison. To be considered on time to graduate, students in 

New York City must earn an average of 11 credits per school year and students in Memphis must earn an average of 5.5 credits per school year.

dThe Regents exam measures in this table include the following Regents exams: English, Math A, Math B, Geometry, Integrated Algebra, 

Algebra 2/Trigonometry, U.S. History and Government, Global History and Geography, Living Environment, Chemistry, Physics, and Earth 

Science. eThe End-of-Course exam measures in this table include the following exams: English 1, English, 2, English 3, Biology, Algebra 1, Algebra 2, 

and U.S. History.
141

Appendix Table D.2

Impacts on Enrollment, Graduation, Attendance, Credits, and Regents or End-of-Course Exams 

for Students in Grade 9 at the Time of Random Assignment, by Math Proficiency Test Score

Proficient on Eighth-Grade Math Test Not Proficient on Eighth-Grade Math Test

Grade Level and Outcome

Program Control Difference

Group Group (Impact) P-Value

Program Control Difference

Group Group (Impact) P-Value

Enrollment and Graduation, Year 4 (%)

On grade

Not on grade

Not enrolled

Graduated on time

Attendancea

 (%)

Attendance rate, Year 1

Attendance rate, Year 2

Attendance rate, Year 3

Attendance rate, Year 4

Attendance rate is 95% or higher, Year 1

Attendance rate is 95% or higher, Year 2

Attendance rate is 95% or higher, Year 3

Attendance rate is 95% or higher, Year 4

b Credits

84.2

5.1

10.8

74.1

90.0

86.7

83.9

72.5

60.9

46.7

37.0

27.5

76.9

11.2

11.9

72.8

87.8

83.6

80.1

65.7

52.6

47.5

43.0

21.1

7.3 *

-6.2 **

-1.1

1.3

2.2

3.1

3.8

6.7 *

8.2

-0.8

-6.0

6.4

0.082

0.028

0.749

0.786

0.367

0.248

0.196

0.097

0.116

0.883

0.228

0.154

68.7

12.5

18.8

59.4

86.5

82.5

79.4

59.7

41.1

35.0

32.8

15.0

69.1

10.4

20.5

58.8

87.6

80.3

76.8

57.6

41.1

31.7

29.9

16.8

-0.4

2.1

-1.7

0.5

-1.2

2.2

2.6

2.1

0.0

3.3

3.0

-1.8

0.896

0.274

0.490

0.863

0.284

0.142

0.139

0.423

0.989

0.270

0.294

0.454

††

Number of credits earned, Year 1

Number of credits earned, Year 2

Number of credits earned, Year 3

Number of credits earned, Year 4

Number of credits earned, Years 1 to 4

Earned adequate credits, Years 1 to 4 (%)

12.5

11.3

10.9

9.4

44.0

70.8

11.7

11.2

10.3

9.0

42.3

67.4

0.8

0.1

0.5

0.3

1.8

3.4

0.136

0.862

0.360

0.559

0.348

0.479

10.3

10.0

9.2

8.7

38.3

50.7

10.5

9.4

8.8

8.4

37.1

50.4

-0.2

0.6 *

0.5

0.3

1.2

0.4

0.459

0.076

0.194

0.437

0.286

0.912

†

(continued)
 

142

Appendix Table D.2 (continued)

Proficient on Eighth-Grade Math Test Not Proficient on Eighth-Grade Math Test

Grade Level and Outcome

Program Control Difference

Group Group (Impact) P-Value

Program Control Difference

Group Group (Impact) P-Value

c,d Regents or End-of-Course exams

State core exams taken, Years 1 to 4

State core exams passed, Years 1 to 4

8.8

5.8

8.0

5.5

0.8 **

0.3

0.044

0.341

7.4

2.3

7.2

2.3

0.2

0.0

0.594

0.961

Sample size (total = 1,400) 171 190 534 505

SOURCES: MDRC calculations using data from New York City Department of Education and Shelby County Schools administrative records.

NOTES: Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. 

Standard errors were adjusted to account for multiple observations per family. Statistical significance levels are indicated as follows: *** = 1 percent; 

** = 5 percent; * = 10 percent. Statistical significance levels across subgroups are indicated as follows: ††† = 1 percent; †† = 5 percent; † = 10 percent.

Rounding may cause slight discrepancies in calculating sums and differences. 

Note that all outcomes in the table include zero values for students who were no longer enrolled.

Years 1, 2, 3, and 4 cover the 2011-2012, 2012-2013, 2013-2014, and 2014-2015 school years, respectively. a

Attendance was calculated as a percentage of total days present divided by total days enrolled according to district records. Records provided for 

students in New York City include enrollment for the regular school year. Records for students in Memphis include enrollment during the regular school 

year, alternative education programs, and summer school.

bStudents in New York City earn 1 credit per course per semester completed. Students in Memphis earn 0.5 credits per course per semester. Credits 

for students in Memphis were multiplied by two to create a standard scale for comparison. To be considered on time to graduate, students in New York 

City must earn an average of 11 credits per school year and students in Memphis must earn an average of 5.5 credits per school year. c

The Regents exam measures in this table include the following Regents exams: English, Math A, Math B, Geometry, Integrated Algebra, Algebra 

2/Trigonometry, U.S. History and Government, Global History and Geography, Living Environment, Chemistry, Physics, and Earth Science.

dThe End-of-Course exam measures in this table include the following exams: English 1, English, 2, English 3, Biology, Algebra 1, Algebra 2, and 

U.S. History. The U.S. History exam was not administered for students in Year 4.
143

References

American Dental Association. 2014. “Mouth Healthy.” Website: www.mouthhealthy.org/en. 

Belfield, Levin. 2007. The Price We Pay: Economic and Social Consequences of Inadequate 

Education. Washington, DC: Brookings Institution Press. 

Blackwell, Debra L., Jacqueline W. Lucas, and Tainya C. Clarke. 2014. “Summary Health 

Statistics for U.S. Adults: National Health Interview Survey, 2012.” Data from the National Health Survey. Vital and Health Statistics 10, 260: 1-161. 

Borchgrevink, Alison, Andrew Snyder, and Shelly Gehshan. 2008. The Effects of Medicaid 

Reimbursement Rates on Access to Dental Care. Portland, ME: National Academy for 

State Health Policy. 

Calvo, Cindy. 2011. “Social Work and Conditional Cash Transfers in Latin America.” Journal 

of Sociology and Social Welfare 38, 3: 53-72. 

Chambers, Valerie, and Marilyn Spencer. 2008. “Does Changing the Timing of a Yearly 

Individual Tax Refund Change the Amount Spent Vs. Saved?” Journal of Economic Psychology 29, 6: 856-862. 

Christakis, Dimitri A., Loren Mell, Thomas D. Koepsell, Frederick J. Zimmerman, and Frederick A. Connell. 2001. “Association of Lower Continuity of Care with Greater Risk of 

Emergency Department Use and Hospitalization in Children.” Pediatrics 107, 3: 524-529. 

Christakis, Dimitri A., Jeffrey A. Wright, Frederick J. Zimmerman, Alta L. Bassett, and 

Frederick A. Connell. 2002. “Continuity of Care Is Associated with High-Quality Care by 

Parental Report.” Pediatrics 109, 4: e54. 

Christakis, Dimitri A., Jeffrey A. Wright, Frederick J. Zimmerman, Alta L. Bassett, and 

Frederick A. Connell. 2003. “Continuity of Care Is Associated with Well-Coordinated 

Care.” Ambulatory Pediatrics 3, 2: 82-86. 

Cohen, Jacob. 1988. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: 

Lawrence Erlbaum Associates.

Costello, E. Jane, Scott N. Compton, Gordon Keeler, and Adrian Angold. 2003. “Relationships 

Between Poverty and Psychopathology: A Natural Experiment.” Journal of the American 

Medical Association 290, 15: 2023-2029. 

Dahl, Gordon, and Lance Lochner. 2012. “The Impact of Family Income on Child Achievement: Evidence from Changes in the Earned Income Tax Credit.” American Economic Review 102, 5: 1927-1956. 

Dechausay, Nadine, Cynthia Miller, and Victoria Quiroz-Becerra. 2014. Implementing a 

Conditional Cash Transfer Program in Two American Cities: Early Lessons from Family 

Rewards 2.0. New York: MDRC.
144

Duncan, Greg J., and Jeanne Brooks-Gunn. 1997. Consequences of Growing Up Poor. New 

York: Russell Sage Foundation. 

Eissa, Nada, and Hilary Hoynes. 2006. “Behavioral Responses to Taxes: Lessons from the 

EITC and Labor Supply.” Pages 74-110 in James M. Poterba (ed.), Tax Policy and the 

Economy, Volume 20. Cambridge, MA: Massachusetts Institute of Technology Press.

Fernald, Lia C. H., Paul J. Gertler, and Lynnette M. Neufeld. 2008. “Role of Cash in Conditional Cash Transfer Programmes for Child Health, Growth, and Development: An Analysis of 

Mexico’s Oportunidades.” The Lancet 371, 9615: 828-837. 

Finkelstein, Amy, Sarah Taubman, Bill Wright, Mira Bernstein, Jonathan Gruber, Joseph P. 

Newhouse, Heidi Allen, and Katherine Baicker. 2011. The Oregon Health Insurance Experiment: Evidence from the First Year. NBER Working Paper No. 17190. Cambridge, 

MA: National Bureau of Economic Research. 

Fisher-Owens, Susan A., Judith C. Barker, Sally Adams, Lisa H. Chung, Stuart A. Gansky, 

Susan Hyde, and Jane A. Weintraub. 2008. “Giving Policy Some Teeth: Routes to Reducing Disparities in Oral Health.” Health Affairs 27, 2: 404-412. 

Fiszbein, Ariel, and Norbert Schady. 2009. Conditional Cash Transfers: Reducing Present and 

Future Poverty. Washington, DC: The World Bank.

Fontaine, Kevin, David T. Redden, Chenxi Wang, Andrew O. Westfall, and David B. Allison. 

2003. “Years of Life Lost Due to Obesity.” Journal of the American Medical Association

289, 2: 187-193. 

Fryer, Roland. 2011. “Financial Incentives and Student Achievement: Evidence from Randomized Trials.” The Quarterly Journal of Economics 126, 4: 1755-1798. 

Gershoff, Elizabeth T., J. Lawrence Aber, C. Cybele Raver. 2003. “Child Poverty in the United 

States: An Evidence-Based Conceptual Framework for Programs and Policies.” Pages 269-

324 in Richard M. Lerner, Francine Jacobs, and Donald Wertlieb (eds.), Handbook of Applied Developmental Science: Promoting Positive Child, Adolescent, and Family Development Through Research, Policies, and Programs, Vol. 2. Thousand Oaks, CA: Sage 

Publishing, Inc.

Giuffrida, Antonio, and David J. Torgerson. 1997. “Should We Pay the Patient? Review of 

Financial Incentives to Enhance Patient Compliance.” British Medical Journal 315, 7110: 

703-707. 

Glied, Sherry, and Matthew Neidell. 2010. “The Economic Value of Teeth.” Journal of Human 

Resources 45, 2: 468-496. 

Hays, Ron D., Cathy D. Sherbourne, and Rebecca Mazel. 1995. User’s Manual for the Medical 

Outcomes Study (MOS) Core Measures of Health-Related Quality of Life. Santa Monica, CA: Rand Corporation. 
145

Hendra, Richard, James Riccio, Richard Dorsett, David H. Greenberg, Genevieve Knight, Joan 

Phillips, Philip K. Robins, Sandra Vegeris, and Johanna Walters. 2011. Breaking the LowPay, No-Pay Cycle. Final Evidence from the UK Employment Retention and Advancement 

(ERA) Demonstration. UK Department for Work and Pensions Research Report No. 765.

Leeds: Corporate Document Service. 

Hettema, Jennifer, Julie Steele, and William R. Miller. 2005. “Motivational Interviewing.” 

Annual Review of Clinical Psychology 1, 1: 91-111.

Internal Revenue Service. 2016. “Earned Income Tax Credit and Other Refundable Credits.” 

Website: www.eitc.irs.gov/EITC-Central/eitcstats.

Kane, Robert L., Paul E. Johnson, Robert J. Town, and Mary Butler. 2004. Economic Incentives 

for Preventive Care. Report/Technology Assessment No. 101 (Prepared by the University 

of Minnesota Evidence-based Practice Center under Contract No. 290-02-0009). AHRQ 

Publication No. 04-E024-2. Rockville, MD: Agency for Healthcare Research and Quality.

Kazis, Lewis E., Jennifer J. Anderson, and Robert F. Meenan. 1989. “Effect Sizes for Interpreting Changes in Health Status.” Medical Care 27, 3: S178-S189.

Kneebone, Elizabeth, and Natalie Holmes. 2015. “Strategies to Strengthen the Earned Income 

Tax Credit.” Washington, DC: Brookings Institution. Website: www.brookings.edu/

research/papers/2015/12/09-earned-income-tax-credit-strategies-kneebone-holmes.

Lachowska, Marta. 2015. “The Effect of Income on Subjective Well-Being:Evidence from the 

2008 Economic Stimulus Tax Rebates.” Upjohn Institute Working Paper 15-238. 

Kalamazoo, MI: W. E. Upjohn Institute for Employment Research. 

Website: dx.doi.org/10.17848/wp15-238.

Levy, Helen, and David Meltzer. 2008. “The Impact of Health Insurance on Health.” Annual 

Review of Public Health 29, 1: 399-409.

Maciosek, Michael V., Ashley B. Coffield, Nichol M. Edwards, Thomas J. Flottemesch, 

Michael J. Goodman, and Leif I. Solberg. 2006. “Priorities Among Effective Clinical Preventive Services: Results of a Systematic Review and Analysis.” American Journal of Preventive Medicine 31, 1: 52-61.

Martinson, Karin, and Richard Hendra. 2006. The Employment Retention and Advancement 

Project: Results from the Texas Site. New York: MDRC.

Mayer, Susan E. 2002. The Influence of Parental Income on Children’s Outcomes. Wellington, 

New Zealand: Knowledge Management Group, Ministry of Social Development.

McGinn-Shapiro, Mary. 2008. “Medicaid Coverage of Adult Dental Services.” State Health 

Policy Monitor 2, 2: 1-6.

McLoyd, Vonnie C. 1998. “Socioeconomic Disadvantage and Child Development.” American 

Psychologist 53, 2: 185-204.
146

Medicaid. 2015. “State Medicaid and CHIP Profiles.” Website: www.medicaid.gov/medicaidchip-program-information/by-state/by-state.html.

Michalopoulos, Charles. 2005. Does Making Work Pay Still Pay? New York: MDRC.

Michalopoulos, Charles, Doug Tattrie, Cynthia Miller, Philip K. Robins, Pamela Morris, David 

Gyarmati, Cindy Redcross, Kelly Foley, and Reuben Ford. 2002. Making Work Pay: Final 

Report on the Self Sufficiency Project for Long-Term Welfare Recipients. New York: 

MDRC.

Miller, Cynthia, Aletha C. Huston, Greg J. Duncan, Vonnie C. McLoyd, and Thomas S. 

Weisner. 2008. New Hope for the Working Poor: Effects After Eight Years for Families 

and Children. New York: MDRC.

Miller, William R., and Stephen Rollnick. 2002. Motivational Interviewing: Preparing People 

for Change. 2nd Edition. New York: The Guilford Press.

Mokdad, Ali, Earl Ford, Barbara Bowman, William Dietz, Frank Vinicor, Virginia Bales, and 

James Marks. 2003. “Prevalence of Obesity Diabetes, and Obesity-Related Health Risk 

Factors, 2001.” Journal of the American Medical Association 289, 1: 76-79.

Mokdad, Ali, James Marks, Donna Stroup, and Julie Gerberding. 2004. “Actual Causes of 

Death in the United States, 2000.” Journal of the American Medical Association 291, 10:

1238-1245.

Morris, Pamela A., and Lisa A. Gennetian. 2003. “Identifying the Effects of Income on Children’s Development Using Experimental Data.” Journal of Marriage and Family 65, 3:

716-729.

Muennig, Peter, Peter Franks, and Marthe Gold. 2005. “The Cost-Effectiveness of Health 

Insurance.” American Journal of Preventive Medicine 28, 1: 59-64.

Must, Aviva, Jennifer Spadano, Eugenie Coakley, Alison Field, Graham Colditz, and William 

Dietz. 1999. “The Disease Burden Associated with Overweight and Obesity.” Journal of 

the American Medical Association 282,16: 1523-1529.

Nuñez, Stephen, Nandita Verma, and Edith Yang. 2015. Building Self-Sufficiency for Housing 

Voucher Recipients: Interim Findings from the Work Rewards Demonstration in New York 

City. New York: MDRC.

Porter, Caroline. 2015. “High-School Equivalency Degree Loses Its Dominant Position.” The 

Wall Street Journal (February 9).

Riccio, James, Helen Bewley, Verity Campbell-Barr, Richard Dorsett, Gayle Hamilton, Lesley 

Hoggart, Alan March, Cynthia Miller, Kathryn Ray, and Sandra Vegeris. 2008. Implementation and Second-Year Impacts for Lone Parents in the UK Employment Retention and 

Advancement (ERA) Demonstration. UK Department for Work and Pensions Research Report No 489. Leeds, UK: Corporate Document Services.
147

Riccio, James, Nadine Dechausay, David Greenberg, Cynthia Miller, Zawadi Rucks, and 

Nandita Verma. 2010. Toward Reduced Poverty Across Generations: Early Findings from 

New York City’s Conditional Cash Transfer Program. New York: MDRC.

Riccio, James, Nadine Dechausay, Cynthia Miller, Stephen Nunez, Nandita Verma, and Edith 

Yang. 2013. Conditional Cash Transfers in New York City: The Continuing Story of the 

Opportunity NYC-Family Rewards Demonstration. New York: MDRC.

Rudd, Timothy, and Jonathan Rodriguez. 2016. Examining the Costs and Benefits of Family 

Rewards 2.0. New York: MDRC.

Seccombe, Karen. 2000. “Families in Poverty in the 1990s: Trends, Causes, Consequences, and 

Lessons Learned.” Journal of Marriage and Family 62, 4: 1094-1113.

Smeeding, Timothy M., Katherine Ross Phillips, and Michael O’Connor. 2000. “The EITC: 

Expectation, Knowledge, Use, and Economic and Social Mobility.” National Tax Journal

53, 4: 1187-1210.

Snyder, Charles R., Susie C. Sympson, Florence C. Ybasco, Tyrone F. Borders, Michael A.

Babyak, and Raymond L. Higgins. 1996. “Development and Validation of the State Hope 

Scale.” Journal of Personality and Social Psychology 70, 2: 321-335.

Soares, Sergel, Rafael Guerreiro Osório, Fábio Veras Soares, Marcelo Medeiros, and Eduardo 

Zepeda. 2007. “Conditional Cash Transfers in Brazil, Chile and Mexico: Impacts Upon Inequality.” International Poverty Centre Working Paper Number 35. Brasilia, Brazil: International Poverty Centre.

Starfield, Barbara, and Leiyu Shi. 2004. “The Medical Home, Access to Care, and Insurance: A 

Review of Evidence.” Pediatrics 113, Supplement 4: 1493-1498.

U.S. Department of Labor, Bureau of Labor Statistics. 2014. “Economic News Release, Table 1. 

Civilian Labor Force and Unemployment by State and Metropolitan Area.” Website: 

www.bls.gov/news.release/archives/metro_05292013.htm.

U.S. Department of Labor, Bureau of Labor Statistics. 2015a. “May 2015 Metropolitan and 

Non-metropolitan Area Occupational Employment and Wage Estimates.” Website: 

www.bls.gov/oes/current/oessrcma.htm.

U.S. Department of Labor, Bureau of Labor Statistics. 2015b. “CPI Detailed Report” (various 

years). Website: www.bls.gov/cpi.

Volpp, Kevin G., Leslie K. John, Andrea B. Troxel, Laurie Norton, Jennifer Fassbender, and 

George Loewenstein. 2008. “Financial Incentive-Based Approaches for Weight Loss: A 

Randomized Trial.” Journal of the American Medical Association 300, 22: 2631-2637.
148

Volpp, Kevin G., Andrea B. Troxel, Mark V. Pauly, Henry A. Glick, Andrea Puig, David A. 

Asch, Robert Galvin, Jingsan Zhu, Fei Wan, Jill DeGuzman, Elizabeth Corbett, Janet 

Weiner, and Janet Audrain-McGovern. 2009. “A Randomized, Controlled Trial of Financial Incentives for Smoking Cessation.” New England Journal of Medicine 360, 7: 699-

709.
149

Earlier MDRC Publications on Family Rewards

New York City’s First Conditional Cash Transfer Program: What Worked, What Didn’t 

2016. James Riccio and Cynthia Miller 

Implementing a Conditional Cash Transfer Program in Two American Cities 

Early Lessons from Family Rewards 2.0 

2014. Nadine Dechausay, Cynthia Miller, Victoria Quiroz-Becerra

Conditional Cash Transfers in New York City

The Continuing Story of the Opportunity NYC−Family Rewards Demonstration 

2013. James A. Riccio, Nadine Dechausay, Cynthia Miller, Stephen Nuñez, Nandita Verma,

Edith Yang

Using Incentives to Change How Teenagers Spend Their Time: The Effects of New York City’s 

Conditional Cash Transfer Program

2012. Pamela Morris, J. Lawrence Aber, Sharon Wolf, Juliette Berg

Learning Together: How Families Responded to Education Incentives in New York City’s 

Conditional Cash Transfer Program

2011. David Greenberg, Nadine Dechausay, Carolyn Fraker

Toward Reduced Poverty Across Generations: Early Findings from New York City’s Conditional 

Cash Transfer Program

2010. James Riccio, Nadine Dechausay, David Greenberg, Cynthia Miller, Zawadi Rucks,

Nandita Verma

____________________________ 

NOTE: A complete publications list is available from MDRC and on its website (www.mdrc.org), from which 

copies of reports can also be downloaded. 
About MDRC

MDRC is a nonprofit, nonpartisan social and education policy research organization 

dedicated to learning what works to improve the well-being of low-income people. Through 

its research and the active communication of its findings, MDRC seeks to enhance the 

effectiveness of social and education policies and programs.

Founded in 1974 and located in New York City and Oakland, California, MDRC is best 

known for mounting rigorous, large-scale, real-world tests of new and existing policies and 

programs. Its projects are a mix of demonstrations (field tests of promising new program 

approaches) and evaluations of ongoing government and community initiatives. MDRC’s 

staff bring an unusual combination of research and organizational experience to their work, 

providing expertise on the latest in qualitative and quantitative methods and on program 

design, development, implementation, and management. MDRC seeks to learn not just 

whether a program is effective but also how and why the program’s effects occur. In addition, 

it tries to place each project’s findings in the broader context of related research — in order 

to build knowledge about what works across the social and education policy fields. MDRC’s 

findings, lessons, and best practices are proactively shared with a broad audience in the 

policy and practitioner community as well as with the general public and the media.

Over the years, MDRC has brought its unique approach to an ever-growing range of policy 

areas and target populations. Once known primarily for evaluations of state welfare-to-work 

programs, today MDRC is also studying public school reforms, employment programs 

for ex-offenders and people with disabilities, and programs to help low-income students 

succeed in college. MDRC’s projects are organized into five areas:

•	 Promoting Family Well-Being and Children’s Development

•	 Improving Public Education

•	 Raising Academic Achievement and Persistence in College

•	 Supporting Low-Wage Workers and Communities

•	 Overcoming Barriers to Employment

Working in almost every state, all of the nation’s largest cities, and Canada and the United 

Kingdom, MDRC conducts its projects in partnership with national, state, and local 

governments, public school systems, community organizations, and numerous private 

philanthropies.