The First-Time 

Homebuyer Education 

and Counseling 

Demonstration:

Early Insights

U.S. Department of Housing and Urban Development | Office of Policy Development and Research
Visit PD&R’s website

huduser.gov

to find this report and others sponsored by HUD’s Office of Policy Development and Research (PD&R). 

Other services of HUD User, PD&R’s research information service, include listservs, special interest 

reports, bimonthly publications (best practices, significant studies from other sources), access to public 

use databases, and a hotline (800-245-2691) for help accessing the information you need.
The First-Time 

Homebuyer Education 

and Counseling 

Demonstration:

Early Insights

Prepared for:

U.S. Department of Housing and Urban Development 

Office of Policy Development and Research 

Washington, D.C.

Prepared by:

Donna DeMarco

Nichole Fiore

Debbie Gruenstein Bocian

Shawn Moulton

Laura Peck

Abt Associates

June 2016
Acknowledgments/Disclaimer

Acknowledgments 

The authors of this report gratefully acknowledge the efforts of many individuals who have assisted in 

carrying out the First-Time Homebuyer Education and Counseling Demonstration since it began in 2011. We 

particularly thank the Government Technical Reviewer, Dr. Marina Myhre, and other staff members at the 

U.S. Department of Housing and Urban Development (HUD), Office of Policy Development and Research. 

We also extend our appreciation to Sarah Gerecke from the HUD Office of Housing Counseling.

The study team is grateful for the many contributions of participating lenders and homebuyer education and 

counseling agencies. We specifically thank NeighborWorks® America, ClearPoint Credit Counseling Solutions, 

Community Ventures Corporation, and the National Council of La Raza.

This study has benefited from the contributions of a number of technical experts, including Doug Dylla 

(Doug Dylla Consulting), Lucy Gorham (University of North Carolina), Stephanie Moulton (The Ohio State 

University), Roberto Quercia (University of North Carolina), and Jonathan Spader (Joint Center for Housing 

Studies of Harvard University). We also thank the members of our expert panel—Kenneth Brevoort 

(Consumer Financial Protection Bureau), J. Michael Collins (University of Wisconsin-Madison), Chris Herbert 

(Joint Center for Housing Studies of Harvard University), Genevieve Melford (Consumer Financial Protection 

Bureau), Carolina Reid (University of California-Berkeley), Kris Rengert (Federal Deposit Insurance 

Corporation), and Peter Zorn (Freddie Mac).

The project staff at Abt Associates includes Sarah Ballinger, Edward Bein, Emily Bell, Valerie Benson, 

Yolanda Borquaye, Lionel Combat, Alison Comfort, Elizabeth Copson, Karen Cuenca, Zachary Epstein, 

Elizabeth Giardino, Cora Goldston, Jill Hamadyk, Anna Jefferson, Marble Karuu, Utsav Kattel, Renee 

Lamoreau, Stephanie Mills, Amar Patel, Lily Rosenthal, Louise Rothschild, Galen Savidge, Brian Sokol, 

Stephen Whitlow, and Tara Wommack. Abt SRBI staff members who carried out participant survey data 

collection include Nick Bertoni, Molly Caldaro, Sandra Hernandez, Robert Magaw, Julie Pacer, Yasmin 

Pluss, Amanda Reilly, Emily Segura, Estella Sena, Brian Shepard, Chintan Turakhia, and a large number of 

telephone and field interviewers. 

The study’s project quality advisor, Jill Khadduri (Abt Associates), provided technical guidance, and Meryl 

Finkel (Abt Associates) provided management oversight. 

Finally, special thanks are due to the individuals participating in the First-Time Homebuyer Education 

and Counseling Demonstration who have shared their experiences of the homebuying process with the 

study team. 

Disclaimer

The contents of this report are the views of the contractor and do not necessarily reflect the views or policies 

of the U.S. Department of Housing and Urban Development or the U.S. government.

The First-Time Homebuyer Education and Counseling Demonstration: Early Insights
Foreword

Foreword

The U.S. Department of Housing and Urban Development (HUD) has sought for nearly 40 years to measure 

the impact of pre-purchase counseling. Although a substantial amount of research has been done on 

homebuyer counseling and education, many studies have suffered from methodological and resource 

constraints and have failed to provide a definitive assessment of counseling’s impact. Researchers and 

Congress have been calling for a large-scale randomized experiment for a long time.

In preparation for designing and implementing such an experiment, HUD’s Office of Policy Development and 

Research published an overview of the housing counseling industry in 2008 (Turnham, Herbert, and Rodger, 

2008) and completed small-scale outcome evaluations of both pre-purchase (Turnham and Jefferson, 2012) 

and foreclosure (Jefferson et al., 2012) counseling in 2012. In 2014, HUD successfully launched a largescale, randomized experiment to assess the impact of homebuyer education and counseling reliably for a 

diverse sample of over 5,800 low-, moderate-, and middle-income prospective first-time homebuyers in 28 

U.S. metropolitan areas. 

This report, The First-Time Homebuyer Education and Counseling Demonstration: Early Insights, presents 

the preliminary findings from early stages of this effort. Like most firsts, this first large-scale randomized 

study has already shed light on key issues for replicating such research and for policy design. In particular, 

the very low take-up rate for in-person education and counseling suggests substantive barriers to that 

delivery mechanism as implemented in this study. The considerably higher take-up rate of online education 

and remote counseling shows promise. In terms of impacts, the very preliminary results are encouraging. 

Moreover, now that the randomized experiment has been implemented successfully, we are hopeful that, over 

the next 4 years, the study will produce the long-sought answers about the impact of homebuyer education 

and counseling on mortgage literacy and preparedness, homebuyer outcomes, and loan performance. 

Katherine M. O’Regan

Assistant Secretary for Policy 

Development and Research

Department of Housing and Urban 

Development

The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

v The First-Time Homebuyer Education and Counseling Demonstration: The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

Contents

Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2. Study Methodology and Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Study Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Study Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

The Study’s Homebuyer Education and Counseling Agencies and Their Services. . . . . . . . . . . . . . 7

Outcome Domains. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Data Sources. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

3. Characteristics of the Study’s Early Enrollees . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Early Enrollees’ Demographic and Financial Characteristics. . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Early Enrollees’ Stage in the Homebuying Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

4. Experiences With Homebuyer Education and Counseling . . . . . . . . . . . . . . . . . . . 19

Take-up Rates and Reasons for Take-up and Completion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Insights From Focus Group Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

5. Outcomes and Impacts for Early Enrollees After 12 Months . . . . . . . . . . . . . . . . . 25

Outcomes for the Early Enrollee Sample After 12 Months. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Early Impacts of Homebuyer Education and Counseling on Selected Outcomes . . . . . . . . . . . . . . 27

6. Conclusion and Future Study Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

Appendix A: Supplemental Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Appendix B: Description of Measures and Missing Data Procedures . . . . . . . . . . . . . . 41

Appendix C: Technical Supplement to the Impact Analysis . . . . . . . . . . . . . . . . . . . . 49

Study Design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

Model for Early Enrollee Sample Impact Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

Impact Estimates: Pooled Remote and Choice Versus Control Groups . . . . . . . . . . . . . . . . . . . . 53

Impact Estimates for Remotely Provided Homebuyer Education and Counseling Services . . . . . . . 55

Follow-up Survey Nonresponse Weights. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Appendix D: Works Cited . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

vii The First-Time Homebuyer Education and Counseling Demonstration: The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

Executive Summary

For generations, homeownership has been a gateway to the middle class and a cornerstone of the 

American Dream. As the recent collapse and uneven recovery of the housing market revealed, however, 

homeownership poses risks, both to borrowers and to the broader economy. Homebuyer education and 

counseling are intended to help individuals navigate the complexities of the housing market, make good 

home purchase and mortgage decisions, improve their financial management skills, and achieve sustainable 

homeownership and enhanced financial health.

Previous research indicates that pre-purchase homeownership counseling has benefits, but the extent of 

those benefits has been questioned because of concerns that those who choose to get counseling may 

be different from those who do not choose to get counseling (selection bias). The First-Time Homebuyer 

Education and Counseling Demonstration (hereafter, the Demonstration) was designed as a randomized 

experiment—the gold standard for evaluation—to definitively answer the question about the relative 

efficacy of homebuyer education and counseling on higher-risk borrowers. The Demonstration, which began 

enrollment in September 2013, is expected to follow study members for 42 months (subject to funding 

availability) to determine the impact of offering homebuyer education and counseling on a wide variety of 

outcomes related to homeownership preparedness; financial literacy, capability, and management; and 

homeownership sustainability. 

This report has four main objectives: to present (1) a summary of the Demonstration’s progress to date, (2) 

preliminary information on the early enrollee sample, (3) preliminary data on take-up rates and early enrollee 

experiences, and (4) preliminary information about the outcomes of early enrollees.1

Successful Implementation

The Demonstration has successfully recruited and enrolled more than 5,800 low-, moderate-, and middleincome prospective first-time homebuyers across 28 large metropolitan areas and has randomly assigned 

them to treatment and control groups. The Demonstration used a unique recruitment and referral process, 

drawing on prospective first-time homebuyers who had requested preapproval, requested prequalification, or 

applied for a home loan from one of three large national lenders. Enrollment began with a pilot in September 

2013 and continued with full study enrollment from January 2014 through January 2016. Study members 

were randomly assigned to one of three treatment groups or a control group. Members assigned to the first 

treatment group have been offered online homebuyer education and telephone-based counseling (that is, 

“remote” education and counseling). Members assigned to the second treatment group have been offered 

“in-person” education and counseling from 1 of 63 local U.S. Department of Housing and Urban 

1 Individuals who enrolled in the study before December 1, 2014, are referred to as the “early enrollee” sample.
Executive Summary

viii The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

Development (HUD)-approved homebuyer counseling agencies. Members of the third treatment group have 

been offered a choice of remote or in-person services. Study members assigned to the control group have 

not been offered any services.2

Demographic and Financial Characteristics of the Early Enrollee Sample 

The early enrollee sample is racially and ethnically diverse, with 59.1 percent self-identifying as Hispanic, 

African-American, Asian, or other race and 40.9 percent self-identifying as White non-Hispanic. These 

study participants also reflect a wide range of educational attainment levels, with most (54 percent) having 

achieved a bachelor’s degree or higher. The median income for early enrollees and their co-borrowers during 

the 12 months before enrollment was $55,000, with 10 percent of early enrollees having made more than 

$100,000 in the previous year and 10 percent having made less than $27,000 in the previous year. This 

report also discusses at what stage the early enrollees were in the homebuying process at the time they were 

recruited into the study. Reflective of the process used to enroll these individuals into the study, the early 

enrollee sample includes both those who had not yet started the home search (9 percent) and those who 

had already purchased a home (14 percent), with the balance falling in between.

Initiating and Completing Services: Take-up Rates and Participant Experiences

Overall, approximately one-half of those offered homebuyer education and counseling actually initiated 

services, but the take-up rates varied meaningfully according to the mode of the intervention. Whereas about 

two-thirds (65 percent) of study participants who were offered remote homebuyer education and counseling 

had initiated services by February 2016, only about one-fourth (25 percent) of those who were offered inperson homebuyer education and counseling had initiated services by February 2016. 

To increase service initiation and completion rates and to improve the study’s chances of detecting the 

impacts of the intervention, the study team, in consultation with HUD and an expert panel, modified what 

was the in-person treatment group to become a choice treatment group. Study participants assigned to 

the choice treatment group could choose how to access services, either remotely or in person, on the 

premise that this choice would increase take-up of both modes of service. Early enrollees assigned to the 

choice treatment group had a clear preference for remote services; 85 percent of study participants who 

were offered their choice of remote or in-person services stated a preference for receiving remote services. 

Initial findings from implementing the choice treatment group indicate that this change was positive to the 

Demonstration.

This report also shares insights from focus groups conducted with treatment group members to understand 

take-up rates and experiences with services. Focus group participants who did take up services were 

primarily motivated by the perceived value in those services and reported gaining important information that, 

for many, positively influenced their decisions and behaviors. For those who did not take up services, the 

reasons for not taking up services or not completing services tended to vary by service delivery mode. Those 

2 Study members initially were randomly assigned to the control group, the remote group, or the in-person group. In September 2014, the 

study was redesigned and new enrollees were assigned to the control group, the remote group, or the choice group.
Executive Summary

ix The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

who had been referred to in-person homebuyer education and counseling services spoke about scheduling 

difficulties, the length of the course, and the agency’s location. Those who had been offered remote services 

cited competing priorities on their schedules and the length of the course as their main reasons for not 

taking up or completing services.

Preliminary Impacts on Select Outcomes for the Early Enrollee Sample

We find positive and statistically significant impacts on three of the four short-term outcomes tested,3 which 

provides a preliminary indication that offering individuals education and counseling services may lead to 

some favorable results. The findings on these outcomes are summarized as follows—

1. Improved mortgage literacy. Treatment group members performed better than their control group 

counterparts on a four-question mortgage literacy quiz. This result indicates that the intervention is 

improving financial literacy, which has been shown to be associated with better mortgage outcomes (An, 

Bostic, and Yao, 2015). 

2. Greater appreciation for communication with lenders. Treatment group members are more likely to 

report that they would contact their lender before missing a mortgage payment. This finding indicates 

that education and counseling are successfully encouraging participants to engage productively with 

their lenders in times of distress.

3. Improved underwriting qualifications. Treatment group members are more likely than their control 

group counterparts to have a credit score of 620 or higher. This threshold of 620 is a proxy for the 

minimum credit score needed to qualify for conventional mortgages.4 The finding implies that 

education and counseling are helping treatment group members correct inaccuracies in their credit 

reports, reduce bad credit events such as late or missed payments, or both to push their credit scores 

over the 620 threshold.

4. No evidence of improved budgeting practices. Treatment group members are no more or less likely 

than their control group counterparts to compare a budget with their actual spending. This finding 

indicates that counseling and education are not helping treatment group members improve their 

budgeting practices.

Next Steps

With the enrollment phase now complete, this Demonstration has successfully laid the groundwork to provide 

empirical evidence of the effects of homebuyer education and counseling. Although the results discussed 

in this report are preliminary and only for the early enrollee sample, these signs of improvements in the 

treatment groups’ outcomes are encouraging, because they could represent important steps along the path 

toward homeownership sustainability and financial health. 

3 Given the recommendations of HUD and the expert panel, these outcomes were preselected based on their close relationship to the 

objectives of homebuyer education and counseling services. 

4 See Fannie Mae’s “General Requirements for Credit Scores” at https://www.fanniemae.com/content/guide/selling/b3/5.1/01.html.
Executive Summary

x The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

This study has three future reports planned that, together, will allow for more definitive conclusions about 

the impact of housing counseling and education services. First, a baseline report (expected to be released 

in the fall of 2016) will describe in detail the implementation of the study and the characteristics of the 

full sample at the time of enrollment. An interim report (expected to be released in early 2018) will present 

12-month impacts on a wider variety of outcomes for the full study sample. If funded, the final report 

(expected to be released in 2020) will address the extent to which homebuyer education and counseling 

improve long-term outcomes.
1 The First-Time Homebuyer Education and Counseling Demonstration: The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

Introduction 

Homeownership has traditionally been one of the most important pathways to financial security for U.S. 

families. Homeownership helps households achieve financial security in a variety of ways. First, homeowners 

systematically build equity through the portion of their regularly scheduled mortgage payments that is 

applied directly toward principal reduction. Second, homeownership is a “leveraged investment”; although 

homeowners typically invest only a small fraction of the home’s value when purchasing a home, they 

realize returns on the entire value of the property during periods of appreciation in housing prices. Finally, 

homeowners can receive federal tax deductions for property taxes, mortgage interest, and private mortgage 

insurance. In addition, homeownership can bestow nonfinancial benefits. For example, studies have shown 

that homeowners are more engaged in social and political activities and exhibit better psychological health 

(for example, Rohe and Lindblad, 2013). For these reasons, homeownership has been an important part of 

the American Dream for generations.5

Homeownership also has risks, however. Unaffordable mortgage payments, the opportunity costs of 

not investing in alternative assets, home maintenance responsibilities, and the higher transaction costs 

associated with moving can make homeownership more a burden than an asset in certain circumstances. 

Housing price fluctuations can also make investment in homeownership precarious, particularly during short 

time horizons.6 When homeownership becomes unsustainable, it can lead to financial distress and other 

adverse consequences—studies have shown that families who are forced to move from their homes often 

suffer negative educational and health effects (Isaacs, 2012; Tsai, 2015). Therefore, for homeownership to 

be an effective tool for achieving favorable financial health outcomes, individuals must have the knowledge, 

skills, and resources to make good decisions about whether and how to purchase and finance a home. 

Homebuyer education and counseling7 are designed to help individuals think critically about these benefits 

and risks, understand how to select affordable homes and appropriate mortgage products, and build 

the financial knowledge, resources, and behaviors needed for sustainable homeownership and long-term 

financial health. Despite extensive research on the benefits of homebuyer education and counseling, 

however, the limitations of previous studies prevent causally linking homebuyer education and counseling to 

desired outcomes.8 This link is impossible because, in the absence of randomized experiments, it is difficult 

to isolate the impact of homebuyer education and counseling from other factors, such as borrower self5 Even in the wake of the foreclosure crisis, Americans still hold a strong desire to own homes. The MacArthur Foundation’s 2014 How 

Housing Matters survey found that 70 percent of respondents aspired to homeownership (Hart Research Associates, 2014). 

6 For a discussion of the costs and benefits of homeownership in the aftermath of the housing crisis, see Herbert, McCue, and SanchezMoyano (2013).

7 Homebuyer education is general training about buying a home, whereas counseling is one-on-one guidance that is tailored to the 

particular needs of the recipient. 

8 A summary of 18 evaluations of homebuyer education and counseling concluded that, although the research is “suggestive” of favorable 

impacts, a “profound lack of randomized experiments” hinders conclusive evidence in this field (Collins and O’Rourke, 2011).

1.
2

2 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

1 Introduction

selection (for example, more self-motivated homebuyers may use counseling services) or lender 

practices (for example, certain types of mortgage products are available only to borrowers who have 

completed counseling).9

As the first large-scale national experimental evaluation of homebuyer education and counseling, the 

First-Time Homebuyer Education and Counseling Demonstration attempts to fill this critical research void. 

The study is a randomized experimental evaluation of education and counseling for first-time homebuyers 

recruited through three large, national lenders. By randomly assigning study participants to either a 

treatment group or a control group, the study is able to isolate the unbiased experimental impact of being 

offered access to free homebuyer education and counseling. The study will follow participants to determine 

the impact of homebuyer education and counseling on a wide variety of short-, intermediate-, and long-term 

outcomes, including those related to homeownership preparedness and housing search; financial literacy, 

capability, and management; and homeownership sustainability. 

This report has four main objectives. First, the report summarizes the study’s implementation, which began 

in the fall of 2011. Implementation included the recruitment of 3 national lenders, 63 in-person homebuyer 

education and counseling agencies in 28 large metropolitan areas, 2 remote service providers, and more 

than 5,800 study participants across 28 large metropolitan areas. Second, this report describes the 

demographic and financial characteristics of the early enrollee sample, which includes study participants 

who were randomly assigned before December 1, 2014 (hereafter referred to as the “early enrollee” 

sample).10 Third, the report presents take-up rates for the early enrollee sample and insights from focus 

groups with some treatment group members regarding their program experiences. Fourth, the report 

provides preliminary insights into the short-term impacts of homebuyer education and counseling. These 

insights are based on an analysis of 12-month outcomes for the early enrollee sample. 

9 Collins and O’Rourke (2011) also concluded that the field is in need of evaluations that (1) examine the impact of housing education and 

counseling on outcomes beyond default and foreclosures; (2) track data longitudinally over time and examine standardized interventions; 

and (3) include and compare alternative modes of services that are available, such as group versus individual education or telephone 

versus in-person counseling.

10 The early enrollee sample is the study subsample of 2,377 individuals for whom 12-month follow-up data, from both survey and 

administrative sources, were requested as of January 2016.
3 The First-Time Homebuyer Education and Counseling Demonstration: The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

Study Methodology and 

Implementation

The study completed its enrollment phase in February 2016. This section describes the study’s design 

and implementation, provides a conceptual framework that defines how the intervention is hypothesized to 

impact study participant outcomes, and summarizes the data sources used for the study. 

Study Design 

Central to the study is the following research question: What is the impact of homebuyer education and 

counseling on low-, moderate-, and middle-income first-time homebuyers? In other words, to what extent 

does homebuyer education and counseling affect outcomes related to homeownership preparedness and 

housing search; financial literacy, capability, and management; and homeownership sustainability following 

home purchase?

To answer these questions, the study employs a randomized experimental design. Eligible individuals were 

randomly assigned to a control group or to one of three treatment groups, the members of which were 

offered access to free homebuyer education and counseling services (“the intervention”). The intervention 

included two modes of education and counseling: (1) in-person services (education and counseling that were 

delivered in person at a counseling agency location) and (2) remote services (online education and telephone 

counseling). These two modes of services reflect the predominant models of homebuyer education and 

counseling in the field. Treatment group members were offered in-person services, remote services, or 

their choice of remote or in-person services (more details are given subsequently in this section). Study 

participants who were assigned to the control group were not offered any services. By creating treatment 

and control groups that are similar in all ways, both measurable and unmeasurable, differences in the mean 

outcomes between the treatment and control groups can be attributed to the intervention.11

Study Implementation 

The study enrolled 5,854 first-time homebuyers between September 2013 and January 2016: 87 during the 

pilot period (September through December 2013) and 5,767 thereafter. Enrollment began with a 3-month, 

three-city pilot period, expanding in January 2014 to 28 large metropolitan areas, as shown geographically 

in exhibit 2.1. This section describes recruitment, enrollment and random assignment. 

11 To validate that our randomization of individuals into the treatment and control groups was carried out with integrity, we tested the 

balance of characteristics across the groups for the early enrollee sample used in this report. See appendix A for more detail. 

2. 
2 Study Methodology and Implementation

4 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

2 Study Methodology and Implementation

Exhibit 2.1: Map of Study Locations

Atlanta

Boston

Chicago

Houston

Las Vegas

Dallas

Detroit

Los Angeles

Raleigh

Portland

Phoenix

Philadelphia

Orlando

New York

Minneapolis-St. Paul

Miami

St. Louis 

Seattle

San Jose

Virginia Beach

Washington

 Tampa

Stockton 

San Francisco

San Diego

San Antonio

Sacramento

Riverside

400+ Participants

140 - 399 Participants

<140 Participants

Key:

Study Recruitment

Three major national lenders partnered with the U.S. Department of Housing and Urban Development (HUD) 

and the study team to recruit participants into the study.12 During the recruitment process, lenders screened 

their national databases for customers who met the following criteria—

• The customer resided in 1 of the study’s 28 locations.13

• The customer requested preapproval or prequalification or applied for a home loan. 

• The customer would be a first-time homebuyer (that is, had not owned a home in the previous 3 years).

• The customer was a low-, moderate-, or middle-income household (that is, household income of less 

than 120 percent of the local Area Median Income).

The study team worked extensively with each participating lender to determine the best process for 

identifying and recruiting customers who met these criteria. Lenders referred customers who were interested 

and who consented to having their contact information shared to the study team. 

12 These lenders were chosen based on their large volume of loan originations and their willingness to partner for the study. 

13 The 28 metropolitan areas were chosen based on the large amount of origination activity by the study’s three lenders.

Atlanta

Boston

Chicago

Houston

Las Vegas

Dallas

Detroit

Los Angeles

Raleigh

Portland

Phoenix

Philadelphia

Orlando

New York

Minneapolis-St. Paul

Miami

St. Louis 

Seattle

San Jose

Virginia Beach

Washington

 Tampa

Stockton 

San Francisco

San Diego

San Antonio

Sacramento

Riverside

400+ Participants

140 - 399 Participants

<140 Participants

Key:
2 Study Methodology and Implementation

5 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

2 Study Methodology and Implementation

Study Enrollment

The study team contacted the individuals who had been referred by the lenders and conducted an additional 

assessment of eligibility to participate in the study.14 Customers who met these eligibility requirements and 

consented to be in the study completed a baseline survey, which is the main source of information on study 

participants’ characteristics at the time of enrollment. 

After enrolling, participants were randomly assigned to a control group or one of three treatment groups. Based 

on their assignment, participants either received or did not receive access to services, as described below.

• Control group (no referral to services). Study participants assigned to this group were not referred to 

education and counseling services. 

• Treatment groups (referral to homebuyer education and counseling services). Treatment group members 

received access to one of the following: (1) remote services, (2) in-person services, or (3) their choice of 

remote or in-person services. All services provided to those in the treatment groups were free of charge 

and covered similar content, although they differed in format, time, and intensity.

Although this report analyzes the treatment groups collectively (that is, it aggregates all individuals offered 

access to any mode of homebuyer education and counseling services), note that individuals were randomly 

assigned to three distinct treatment groups—in-person, remote, and choice. At the inception of the study, 

individuals were randomly assigned to the control group, the in-person group, or the remote group. Takeup of in-person services remained very low, however, even a year after random assignment. Only about 

20 percent of study participants assigned to the in-person treatment group had initiated education or 

counseling, which raised concerns that the large proportion of in-person treatment group members not 

receiving services would dilute the treatment effects. In response, the in-person treatment group was 

changed to a choice treatment group in the fall of 2014, in hopes of increasing the rate of service take-up. 

Starting in September 2014, all individuals were asked before enrollment whether they would prefer remote 

or in-person services; those who were assigned to the choice group were then offered the type of treatment 

for which they had expressed a preference.15 We present the initial take-up rates for each intervention in 

section 4. 

Exhibit 2.2 shows the timing of randomization to these three treatment groups, including the sample sizes in 

each for the entire study and for the subsample of early enrollees that is used in this report. As the exhibit 

makes clear, for the early enrollees for whom 1 year of follow-up data are available at this time, the study 

14 The additional eligibility screen ensured that customers: (1) had not previously completed homebuyer education and counseling; (2) 

were not participating in a downpayment assistance program that would require homebuyer education and counseling; and (3) could 

complete the work of receiving homebuyer education and counseling services offered through the study (that is, the customer was willing 

to participate in the services, spoke English or Spanish, had access to transportation to in-person services, and had computer and internet 

access allowing for participation in online services).

15 The study’s interim and final reports will use this information to test for the effects of the mode of service delivery by estimating the 

impact of homebuyer education and counseling on (1) those who were offered remote services (including remote treatment group 

members and choice treatment group members who stated a preference for remote services) and (2) those who were offered in-person 

services (including in-person treatment group members and choice treatment group members who stated a preference for in-person 

services).
2 Study Methodology and Implementation

6 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

2 Study Methodology and Implementation

does not yet have sufficient sample to consider the relative effectiveness of service delivery mode. As a 

result, we pooled study participants randomized to all the treatment arms, including the remote arm, the 

in-person arm (in the study’s initial phase), and the choice arm (in the study’s redesign phase, which began 

in September 2014). The control group covers the periods before and after study redesign and represents a 

valid counterfactual for the pooled treatment groups. This report provides early insights about the efficacy of 

offering access to homebuyer education and counseling services, defined broadly to include the combined 

sample of treatment group members. Future reports will analyze the treatment groups separately.

Exhibit 2.2: Study Enrollment, by Group and Time Period

Note: The numbers in this exhibit exclude study participants who chose to withdraw. 

Source: Random Assignment and Service Tracking System

The study enrolled 87 participants during the pilot period (September through December 2013) and 5,767 

during the full enrollment period (January 2014 through January 2016). After study participant enrollment, 

those randomly assigned to the treatment groups were referred to the appropriate homebuyer counseling 

agency, and the study team monitored their initiation and completion of education and counseling services. 

Those in the control group received an explanation that the study would not refer them to education or 

counseling services but would still keep in touch with them and encourage them to complete surveys.16

Exhibit 2.3 summarizes this enrollment process.

16 Control group members were not prohibited from accessing homeownership education and counseling on their own. Instead, they did 

not receive referrals to the free services offered through the study. Future reports will document the extent to which control group members 

sought out homeownership education and counseling on their own.
2 Study Methodology and Implementation

7 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

2 Study Methodology and Implementation

Exhibit 2.3: Study Enrollment Process

• 28 large housing markets

• 3 national lenders' customers 

(first-time homebuyer, low-, moderate- and middle-income, applied for 

pre-approval/pre-qualification or a loan)

RECRUITMENT

• Eligibility criteria 

(no prior counseling, has transportation, has computer access)

• Consent to participate in study

• Baseline survey completion

ENROLLMENT

• Control group: no services

• Treatment groups: offer of remote services (online education and 

telephone counseling); offer of in-person education and counseling; 

or offer of choice of remote or in-person services 

RANDOM ASSIGNMENT

REFERRAL AND OUTREACH

• Notification of random assignment group

• Monitoring of treatment group members’ completion of services

• Outreach to encourage take-up and completion of services

The Study’s Homebuyer Education and Counseling Agencies and Their Services

All 63 service delivery organizations participate in HUD’s homebuyer counseling program and are required 

to meet standards governing the content, delivery, and reporting of education and counseling services. In 

addition, the content of the education and counseling, regardless of mode or location, adhere to the National 

Industry Standards (NIS) for Homeownership Education and Counseling. Neither NIS nor HUD requires 

a specific curriculum for homebuyer education but provide core topic areas to be included in education 

courses. The five NIS content areas are (1) assessing readiness to buy a home, (2) budgeting and credit, 

(3) financing a home, (4) selecting a home, and (5) maintaining a home and finances. These key topics are 

elaborated with their content in exhibit 2.4. 
2 Study Methodology and Implementation

8 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

2 Study Methodology and Implementation

Exhibit 2.4: National Industry Standards for Homeownership Education and Counseling’s Core Area Topics

• The pros and cons of homeownership

• The home purchase process

• Housing affordability

• The “4 Cs” of credit

ASSESSING READINESS TO BUY A HOME

• Importance of goal setting

• Tracking expenses

• Setting up a spending plan

• Budgeting and saving tips

• Importance of good credit

• Understanding credit and how 

to protect credit ratings

• Credit bureaus, reports and scores

• How to fix credit problems

• Debt management tips

BUDGETING AND CREDIT

• How a lender decides 

whether or not to lend

• Housing affordability and qualification

• Sources for mortgage loans

• Predatory loans and 

how to avoid them

• Types of mortgage loans

• Special financing products

• Steps in the mortgage loan process

• Loan application and approval process

• Common lending documents

• What to do if the loan is denied

• The closing process

FINANCING A HOME

SELECTING A HOME

MAINTAINING A HOME AND FINANCES

• The homebuying team

• Real estate professionals

• Types of homes and ownership

• How to select a home and neighborhood

• How to make an offer

• Negotiating tips

• The purchase contract

• Inspections

• Escrow and the closing process

• How to maintain and protect 

a home after moving in

• Home safety and security

• Energy efficiency

• Preventive maintenance

• Home repairs and improvements

• Working with a contractor

• Community involvement

• Record-keeping

• Taxes

• Insurance

• What to do if you might miss a payment

• Predatory lending and other

financial pitfalls

Source: National Industry Standards for Homeownership Education and Counseling, available at http://www.homeownershipstandards.

com/Home/Standards.aspx
2 Study Methodology and Implementation

9 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

2 Study Methodology and Implementation

For participants offered access to remote services, eHome America (a program run by Community Ventures 

Corporation) provided online education. The eHome America online curriculum takes approximately 7 to 8 

hours to complete. In addition to informational text, the course features videos, worksheets, and quizzes 

to engage participants in the learning process. Participants can log in and out at their leisure, allowing for 

them to complete the course at their convenience. ClearPoint Credit Counseling Solutions provided the 

telephone counseling sessions. The telephone counseling services follow a protocol to cover the same core 

elements for each client while allowing for some individualization based on client needs. Discussions about 

finances, budgeting, and the mortgage process are individualized to the homebuyer and vary depending on 

the homebuyer’s needs and questions at the time of counseling.

For the provision of in-person services, the study team partnered with 63 local housing counseling agencies 

across the study’s 28 cities.17 Participants completing services in person attended group education 

workshops and individual counseling sessions offered by a housing counseling agency in their community. 

The order in which study participants completed services depended on the local agency’s set protocol. 

In-person education workshops typically total 8 hours in length and are offered as a single 8-hour class 

on a Saturday or four to five weeknight sessions. Most organizations provide workshops at least once per 

month. Workshops are typically provided in an interactive format using a variety of tools, with guest speakers 

on specific topics such as the roles and services of lenders, real estate agents, insurance agents, home 

inspectors, and closing attorneys. In-person counseling is typically done on a one-to-one basis with clients 

following a standard protocol. Housing counselors provide individualized, objective information and advice 

to clients about homeownership. Counseling sessions typically occur in 1- to 2-hour time blocks, with the 

counselor reviewing the client’s information in advance in order to address the client’s particular situation.

The remote and in-person services offered by the study both adhere to NIS, but the two modes differ in 

several respects. Exhibit 2.5 summarizes those differences, focusing on format, time and intensity, variation 

in content, and transaction costs for the participant.

17 Most of these housing counseling agencies were affiliated with NeighborWorks® America. Given the limited number of NeighborWorks®

America affiliates across the country and the small size of some of the organizations, however, it was impossible or impractical in some 

study locations—or portions of metropolitan areas that constitute the study locations—for clients to obtain in-person services from 

NeighborWorks® America organizations. In those cases, other HUD-approved housing counseling agencies provided homebuyer education 

and counseling services comparable with those provided by NeighborWorks® America organizations.
2 Study Methodology and Implementation

10 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

2 Study Methodology and Implementation

Exhibit 2.5: Differences Between In-Person and Remote Education and Counseling

Features In-Person Group Remote Group

Education

Format Participants attend one or more in-person 

classroom sessions at a local HUDapproved homebuyer counseling agency. 

Participants access the training on 

line from a home or public computer. 

Participants can complete training in as 

few or as many sessions as desired.

Hours 8+ hours. Typically 7–8 hours.

Variation in 

content

Moderate variation: In-person providers 

cover generally similar topics but may 

vary curricular materials and emphasize 

different topics. 

Low variation: All participants experience 

the same online learning environment. 

Some variation exists in effort expended 

and time to complete.

Participant 

transaction costs

Higher transaction costs: Participants 

must schedule in advance and travel to 

sessions.

Lower transaction costs: Participants 

must invest time but can complete the 

process at their own pace, when and 

where they want.

Counseling

Format Participants attend one or more in-person, 

one-on-one sessions with a certified 

housing counselor at a HUD-approved 

homebuyer counseling agency.

Participants hold one or more telephone 

calls with a ClearPoint Credit Counseling 

Solutions counselor from their homes or 

location of choice. 

Hours 1+ hours 1+ hours

Variation in 

content

Moderate variation: The amount 

and content of counseling will vary 

depending on the client’s readiness for 

home purchase and specific issues or 

challenges.

Moderate variation: The amount 

and content of counseling will vary 

depending on the client’s readiness for 

home purchase and specific issues or 

challenges.

Participant 

transaction costs

Higher transaction costs: Participants 

must schedule in advance and travel to 

sessions.

Lower transaction costs: Participants 

may complete counseling via telephone 

at a scheduled time but at a location of 

choice. 

Outcome Domains

This study seeks to measure the extent to which the homebuyer education and counseling services affect 

outcomes in three domains: (1) preparedness and search; (2) financial literacy, capability, and management; 

and (3) homeownership sustainability. The preparedness and search domain covers outcomes related to 

the tenure decision (that is, the decision whether to purchase or defer), selection of a home, and qualifying 

for and terms of mortgages. The financial literacy, capability, and management domain covers outcomes 

related to participants’ financial health, including budget discipline, debt and savings, access to affordable 

credit, and credit profile. The homeownership sustainability domain covers outcomes related to mortgage 

performance, ability to avoid foreclosure, and the accrual and protection of home equity. 
2 Study Methodology and Implementation

11 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

2 Study Methodology and Implementation

Exhibit 2.6 depicts the mechanisms through which homebuyer education and counseling are expected 

to affect these domains. Within the domain of preparedness and search, in the short and medium terms, 

homebuyer education and counseling should increase service recipients’ awareness and knowledge of 

the pros and cons of homeownership, homeowners’ responsibilities, mortgage products and terms, and 

underwriting criteria. In addition, education and counseling should enhance a recipient’s ability to determine 

the affordability of homes and the appropriateness of financing options. The added knowledge and skills are 

expected to inform a host of behaviors and decisions, starting with whether or not to purchase a home. For 

service recipients who decide to purchase a home, the services should help them to search for and select 

affordable homes and to select and qualify for appropriate mortgage products. In the domain of financial 

literacy, capability, and management, the short-term outcomes of homebuyer education and counseling 

should be improved understanding of financial terminology and the importance of good credit, as well as 

service recipients’ ability to create budgets, track expenses, and correct credit reports. These enhanced 

skills and knowledge should translate to better financial behaviors, such as improved budgeting practices, 

better money and debt management, and increased savings.18

Exhibit 2.6: Logic of Intervention’s Influence on Outcomes

18 Improvements in behaviors and decisions may also increase knowledge and skills. For example, in the homeownership preparedness 

domain, if service recipients shop more critically for mortgage products as a result of housing education and counseling, then the shopping 

process may result in a more solid understanding of mortgage financing than what they developed directly from the education and 

counseling sessions. In the financial literacy, capability, and management domain, if service recipients begin checking their credit reports 

regularly, then their understanding of how their financial behavior is impacting their credit scores may increase. 
2 Study Methodology and Implementation

12 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

2 Study Methodology and Implementation

In the longer term, the central goal of homebuyer education and counseling is homeownership sustainability, 

with homeowners making timely mortgage payments, avoiding foreclosure, and building wealth. If 

homebuyer education and counseling have improved home preparedness and search decisions (for example, 

better tenure decisions, more-affordable purchase and financing decisions) and financial management 

behaviors (for example, improved money management, savings, budgeting), then recipients of education and 

counseling services will be more likely to meet their monthly payments and accrue home equity. Recipients 

will be more likely to sustain homeownership, enabling them to build wealth in the long term. 

Homebuyer education and counseling have another longer-term goal. For all service recipients, regardless 

of whether they decide to purchase a home, the intervention should lead to enhanced long-term financial 

health by improving financial management, budgeting, and saving decisions.

Data Sources

The evaluation will use a wide variety of primary and secondary data sources in its analyses, including the 

following— 

• Three surveys of study participants: at baseline, at 12 months after enrollment, and at 42 months after 

enrollment. The 12-month follow-up survey data are still being collected for most study participants, and 

the 42-month follow-up survey is pending future funding.

• Loan origination and servicing data from participating lenders and the Federal Housing Administration.

• Study member credit data from one of the three major credit bureaus.

• Data on services received from counseling agencies.

• Focus groups with treatment group members in four locations, supporting a qualitative exploration 

of study participants’ experiences with homebuyer education and counseling services and the home 

purchase process.

• Qualitative data from study team members’ interactions with the lenders and from site visits to 

participating homebuyer education and counseling service-providing agencies.
13 The First-Time Homebuyer Education and Counseling Demonstration: The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

Characteristics of the Study’s 

Early Enrollees

The preliminary impacts presented in section 5 of this report cover less than one-half of the study participants 

and are based on the 12-month survey and credit bureau data for those who were enrolled on or before 

December 1, 2014. This section describes the demographic and financial characteristics of these early 

enrollees, which mirror the characteristics of the larger study sample.19 The section then discusses at what 

stage the early enrollees were in the homebuying process at the time they were recruited into the study.20

Early Enrollees’ Demographic and Financial Characteristics

Exhibit 3.1 illustrates the early enrollees’ key sociodemographic characteristics, as measured at the time 

of their enrollment. The early enrollee sample is racially and ethnically diverse, with 59.1 percent selfidentifying as Hispanic, African-American, Asian, or other race and 40.9 percent self-identifying as White 

non-Hispanic. These study participants also reflect a wide range of educational attainment levels, with the 

most (54 percent) having achieved a bachelor’s degree or higher. Also, 89 percent worked full-time (at least 

30 hours per week), 59 percent are men, and 41 percent are women. As shown in exhibit 3.2, median 

income for early enrollees and their co-borrowers during the 12 months before enrollment was $55,000, 

with 10 percent having made more than $100,000 and 10 percent having made less than $27,000.

19 To determine whether baseline characteristics in the early enrollee sample are statistically the same as in the full study sample, we 

compared the two along 50 baseline traits. Only 3 traits were statistically different (at a 95-percent confidence level), which is not more 

than what we would expect because of random chance. Therefore, the early enrollees examined in this report are reflective of the full study 

sample that will be used in future analyses. Appendix exhibits A.1 and A.2 show this comparison.

20 The extent to which the study sample is typical of all low-, moderate-, and middle-income first-time homebuyers will be addressed in the 

study’s baseline report (forthcoming in 2016).

3.
2

14 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

3 Characteristics of the Study’s Early Enrollees

Exhibit 3.1: Baseline Sociodemographic Characteristics, Early Enrollee Sample 

Notes: All measures are defined for the entire early enrollee sample. The early enrollee sample used in this exhibit includes 2,377 study 

participants who enrolled in the study before December 1, 2014, excluding withdrawals. Measure-specific sample sizes may vary because of 

item nonresponse. Appendix B details variable construction.

Source: Baseline survey of study participants

Exhibit 3.2: Baseline Income Distribution of Early Enrollees and Their Co-Borrowers

Note: The early enrollee sample used in this exhibit includes 2,364 study participants with nonmissing income data who enrolled in the 

study before December 1, 2014, excluding withdrawals.

Source: Baseline survey of study participants
2

15 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

3 Characteristics of the Study’s Early Enrollees

Next, we consider the enrollees’ household composition. As shown in exhibit 3.3, a plurality of participants 

in the early enrollee sample has children (27.2 percent of married households and 11.7 percent of single 

households, or 38.9 percent in total). 

Exhibit 3.3: Household Composition, Early Enrollee Sample

Notes: The early enrollee sample used in this exhibit includes 2,362 study participants with nonmissing data on measures used to 

construct household composition who enrolled in the study before December 1, 2014, excluding withdrawals. Although study participants 

were not explicitly asked about whether they had children living with them at the time of enrollment, they were asked how many household 

members younger than age 18 would live with them after the purchase of a home. For the purpose of this exhibit, we assume that these 

projections reflect the household composition at the time of enrollment. 

Source: Baseline survey of study participants
2

16 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

3 Characteristics of the Study’s Early Enrollees

Another interesting feature of household composition is whether the potential homebuyer is single or married 

(or living with a partner). The earnings of a second adult could be important for sustaining homeownership, 

even if the second adult is not a co-borrower. As the exhibit shows, about one-half of the participants in the 

early enrollee sample (37.6 plus 11.7 percent, or 49.3 percent) are single. As shown on the bottom portion 

of exhibit 3.3, men were much more likely to be married or living with a partner than women (59.9 versus 

37.3 percent). 

Overall, the early enrollee sample exhibited favorable financial management and savings behaviors at the 

time of enrollment. As shown in exhibit 3.4, about 3 out of every 4 used a written budget (76 percent) and 

reported paying their credit card balance in full most of the time (78 percent), and 9 out of 10 regularly set 

aside money for retirement, education, or a financial cushion (90 percent). According to credit bureau data, 

the early enrollee sample had a median credit score of 708 (the average credit score was 705), and threefourths had credit scores of 655 or higher, as shown in exhibit 3.5. Next, as shown in exhibit 3.6, one-half 

of the early enrollee sample had at least $15,000 available for a downpayment at the time of enrollment. 

Exhibit 3.4: Baseline Measures of Financial Responsibility and Creditworthiness, Early Enrollee Sample

Notes: All measures are defined for the entire early enrollee sample. The early enrollee sample used in this exhibit includes 2,377 study 

participants who enrolled in the study before December 1, 2014, excluding withdrawals. Measure-specific sample sizes may vary because 

of item nonresponse. Appendix B details variable construction.

Source: Baseline survey of study participants
2

17 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

3 Characteristics of the Study’s Early Enrollees

Exhibit 3.5: Baseline Distribution of Credit Scores, Early Enrollee Sample

Note: The early enrollee sample used in this exhibit includes 2,045 study participants with nonmissing credit score data who enrolled in 

the study before December 1, 2014, excluding withdrawals.

Source: Credit bureau data

Exhibit 3.6: Baseline Distribution of Select Assets and Liabilities, Early Enrollee Sample

Notes: A measure of nonmortgage debt is provided by credit bureau data and includes student loan, automobile, and credit card debt 

(sample size: 2,070). Total savings and investments are self-reported in the baseline survey and include balances in all checking, savings, 

retirement, and other investment accounts (sample size: 2,377). Cash on hand for downpayment and closing costs is also self-reported 

in the baseline survey and may be lower than total savings and investments (if the former is a subset of the latter) or higher (if study 

participants have access to outside funds for downpayment and closing costs, for example, from a family member). Cash on hand 

for downpayment and closing costs refers to the funds that were available at the time of purchase for those participants who already 

purchased at the time of enrollment (sample size: 2,365).

Sources: Baseline survey of study participants; credit bureau data
2

18 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

3 Characteristics of the Study’s Early Enrollees

Financial vulnerabilities nonetheless were indicated within the group of early enrollees. For example, 

18.3 percent of the early enrollees reported that they were short on money sometimes or often during the 

year before enrollment, and 10 percent had credit scores of 615 or less at baseline. As shown in exhibit 

3.6, these early enrollees also reported an average of close to $20,000 in nonmortgage debt (which 

includes student loan, automobile, and credit card debt), and 10 percent had more than $50,000 in 

nonmortgage debt. 

Early Enrollees’ Stage in the Homebuying Process

Despite targeting potential homebuyers who are early in the homebuying process, the study enrolled 

individuals at every stage of home purchase. As shown in exhibit 3.7, the early enrollee sample includes 

those who had not yet started home search (9 percent) and those who had already purchased a home 

(14 percent). 

Exhibit 3.7: Stage in the Homebuying Process, Early Enrollee Sample

Note: The early enrollee sample used in this exhibit includes 2,331 study participants with nonmissing data on their stage in the 

homebuyer process who enrolled in the study before December 1, 2014, excluding withdrawals.

Source: Baseline survey of study participants. Recruitment of study participants through lenders, along with the time required to introduce 

the study and complete the enrollment process, may explain why some study participants were in later stages of the home purchase 

process. Because study participants first became aware of the study after their initial contact with a lender for prequalification or 

preapproval, some study participants had progressed into later stages of the home purchase process before enrollment in the study. The 

further along study participants were in this process, the more decisions they would have already made. This fact has implications for how 

homebuyer education and counseling may affect their outcomes. For example, those who complete homebuyer education and counseling 

after they have already chosen a home and secured financing will be less likely to experience changes in their preparedness and search 

outcomes (for example, affordability of homes, appropriateness of financing). We would still, however, expect these treatment group 

members to experience changes in their financial literacy, capability, and management and, ultimately, homeownership sustainability, 

because the content of homebuyer education focuses on these issues as well. 
19 The First-Time Homebuyer Education and Counseling Demonstration: The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

Experiences With Homebuyer 

Education and Counseling

This section describes the early enrollees’ experiences with education and counseling, including their 

experiences with taking up, engaging in, and completing education and counseling services. After reporting 

on the experience of the early enrollee sample, the section then provides some insights from focus groups of 

small numbers of treatment group members from the full sample.

Take-up Rates and Reasons for Take-up and Completion

The study’s impact analysis considers the effect of being offered access to free homebuyer education 

and counseling. Not everybody assigned to the treatment groups actually took up the services that were 

offered, however. Overall, approximately one-half (48.9 percent) of those offered homebuyer education and 

counseling actually initiated services.

Is this take-up rate lower than might be expected? Most previous studies are based on those who received 

homebuyer education and counseling, rather than on a treatment group offered the opportunity to take up 

services as part of an experiment. That makes comparisons with other research imperfect. That said, the 

rates of service take-up for the enrollees in this study are in line with the scant other research. For example, 

Smith, Hochberg, and Greene (2014) reported that about one-half of study participants who were offered 

education and counseling through their study initiated services. A recent randomized experiment on inperson financial coaching had similar take-up rates, ranging between 37 and 56 percent (Theodos et 

al., 2015). 

Note that the take-up rates in this study varied according to the mode of service. As shown in exhibit 4.1, 

about two-thirds (64 percent) of the early enrollee sample who were offered remote education and telephone 

counseling had initiated those services as of February 2016 (14 months after the end of their enrollment). 

By contrast, only one-fourth of those offered in-person education and counseling had started to receive 

services. Furthermore, not all treatment group members who started services completed them. About onefourth of participants who were offered remote services had completed them as of February 2016, but less 

than 14 percent of those who were offered in-person services had completed all the offered education and 

counseling services. 

4. 
2

20 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

4 Experiences With Homebuyer Education and Counseling

Exhibit 4.1: Take-up Rates, by Service Mode, for Early Enrollee Sample, as of February 2016

Treatment 

Offered

Number 

Randomized

Initiated Any 

Services (%)

Completed 

Education 

(%)

Completed 

Counseling 

(%)

Completed 

Any Services 

(%)

Completed 

All Services 

(%)

Remote 

education and 

counseling

708 64.6 30.6 44.1 48.7 25.9

In-person 

education and 

counseling

542 24.5 22.7 15.3 24.4 13.6

Choice: prefer 

remote services 142 67.6 33.0 45.8 49.3 29.6

Choice: prefer 

in-person 

services

26 30.7 30.8 23.1 30.8 23.1

Overall: All 

modes of 

service 

1,418 48.9 27.9 32.9 39.1 21.6

Note: The number randomized by treatment arm in this exhibit includes people who ultimately withdrew from the study, and so the totals 

provided do not match what is reported in exhibit 2.2.

Sources: Study participant data from eHome America, ClearPoint Credit Counseling Solutions, and local housing counseling agencies 

As explained in section 2, the low take-up and service completion rates for in-person services led the 

study team to redesign the original in-person treatment group as a choice treatment group, in which 

participants could choose in-person or remote services. Exhibit 4.1 shows take-up rates for the choice 

group, separately for those who expressed a preference at baseline for receiving remote services and for 

those who expressed a preference at baseline for receiving in-person services. Although the sample size is 

quite small for the choice group within the early enrollee sample, some patterns already become apparent 

by looking at this group by the mode of service chosen. First, study participants had a clear preference for 

remote services; 142 of them (85 percent) stated a preference for receiving remote services, whereas 26 

of them (15 percent) stated a preference for receiving in-person services. This finding is consistent with 

previous research, which found that people prefer online education and telephone counseling to in-person 

services (Barron and Staten, 2012). Second, even those choice treatment group members who expressed 

a preference for in-person services had a much lower take-up rate (31 percent) than the choice treatment 

group members who expressed a preference for remote services (68 percent).

According to responses to questions asked of study participants who did not complete services in the 

12-month follow-up survey, a major reason for not completing services was inconvenience. The specific 

reasons selected by those who were offered access to different service modes vary somewhat. 

As shown in exhibit 4.2, 52 percent of those offered in-person services and 44 percent of those offered 

remote services reported that “homebuyer education did not work for my schedule.” Study participants 

also found the scheduling of counseling services inconvenient, with 34 percent of those offered in-person 
2

21 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

4 Experiences With Homebuyer Education and Counseling

services and 23 percent of those offered remote services reporting that “counseling was not offered on days 

or times that worked for my schedule.” Of those offered in-person services, 25 percent reported “the agency 

was too far away” as an obstacle to their participation in homebuyer education, and 16 percent reported 

“the agency was too far away” as an obstacle to their participation in homebuyer counseling. A few study 

participants who were offered access to remote services also gave reasons related to location, which may 

reflect a misunderstanding (or faulty memory) of the services they had been offered.

Exhibit 4.2: Survey Respondents’ Reasons for Not Completing Services, Early Enrollee Sample

* The difference between the in-person and remote groups is statistically significant (p < 0.05).

Notes: Survey respondents could select more than one response to the questions about why they did not complete services. The 

in-person bar includes all participants who were referred to in-person services (that is, both members of the in-person treatment group 

and those members of the choice group who stated a baseline preference for in-person services). For the “Reasons for not completing 

education” panel, the in-person services bar sample includes 315 early enrollee treatment group members who did not complete 

education and answered the interim survey. For the “Reasons for not completing counseling” panel, the in-person services bar sample 

includes 347 early enrollee treatment group members who did not complete counseling and answered the interim survey. The remote bar 

similarly includes all participants who were referred to remote services (that is, both members of the remote treatment group and those 

members of the choice group who stated a baseline preference for remote services). For the “Reasons for not completing education” 

panel, the remote services bar sample includes 376 early enrollee treatment group members who did not complete education and 

answered the interim survey. For the “Reasons for not completing counseling” panel, the remote services bar sample includes 272 early 

enrollee treatment group members who did not complete counseling and answered the interim survey. 

Source: 12-month follow-up survey of study participants
2

22 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

4 Experiences With Homebuyer Education and Counseling

Note that those participants offered remote education were more than twice as likely as those offered inperson services to report that the length of the course was too long, 26 versus 12 percent. It appears that, 

although a day-long workshop was generally perceived as acceptable by those offered in-person services, a 

larger percentage were reluctant to spend the same amount of time in front of a computer.

Finally, less than 10 percent of those who did not complete counseling services reported that unappealing 

content was the reason. A somewhat larger percentage said that they thought education did not apply to 

them because they had already bought a house. 

Insights From Focus Group Research 

To better understand the experiences of study participants, the study team hosted 14 focus groups in four 

cities. The focus groups generally were limited to treatment group members who were offered the same 

mode of service and either had completed services or had not. These focus groups explored topics related 

to both study implementation and experiences with homebuyer education and counseling. This section first 

describes focus group participants’ decisions about whether to take up the services provided through the 

study, helping to further explain the take-up rates described in the previous subsection. It then highlights 

some of the reported effects of homebuyer education and counseling services on treatment group members’ 

knowledge, behaviors, and decisions. 

Reasons for Engaging in Homebuyer Education and Counseling Services

Regardless of whether they were assigned to remote or in-person education and counseling, focus group 

participants cited similar reasons for taking up services. First and foremost, focus group participants took 

up the services they were offered because they believed the information would be valuable. By contrast, the 

reasons given by focus group participants for not taking up education and counseling tended to differ based 

on whether they had been assigned to in-person or remote services. Consistent with the survey results, 

those assigned to the in-person treatment group reported more logistical barriers in completing services 

than those assigned to the remote group.

Reasons for Taking Up Services

Many focus group participants said that they took up services because they knew the homebuying process 

would be complicated and they thought the information provided by education and counseling would 

help them through the process. A few participants noted that their past attempts to buy a home were 

unsuccessful and said that they thought completing education and counseling might help them succeed 

this time. 

Focus group participants expressed the view that the decision to buy a home was important and would have 

a big impact on their future. As expressed by a focus group participant in Atlanta, Georgia: “I think it was 

just a priority issue...for me: like you are going to make something happen if it is the number one priority.... 

So when you look out and say this will affect my life in tangible ways for the foreseeable future...I will make 

the time to get to the study on Saturday because this is important.” 
2

23 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

4 Experiences With Homebuyer Education and Counseling

Another study participant in Atlanta spoke about the housing and economic crisis the country has gone 

through during the past 10 years and said that he would not only learn something from homebuyer 

education but also be contributing to something greater than himself. 

A handful of focus group participants also noted that access to free services and the study’s incentive 

payment for completing education and counseling encouraged them to take up services.

Reasons for Not Taking Up Services

Reasons given for not taking up services or not completing services included busy schedules, childcare needs, 

and the course being too long. The reasons tended to vary by service delivery mode. Those who had been 

referred to in-person education and counseling services spoke about scheduling difficulties, the length of the 

course, and the agency’s location. Those who had been offered remote services also cited their schedules and 

the length of the course. They also said that they already spend too much time on a computer throughout the 

day and did not want to spend another 6 to 8 hours on a computer completing homebuyer education. Some 

participants said they “just haven’t gotten around to it yet.” 

Some focus group participants who had not engaged in the services offered explained that they relied on 

other sources of information about the homebuying process, including friends, family, or their realtor. For 

example, a focus group participant in Chicago stated: “My wife and I have a realtor friend, and most of the 

information that we needed to know about, you know, preparing us for buying a house, came through her.... 

So maybe that’s the reason why I decided not to participate in one of those workshops, because most of the 

information [we were getting] was very reliable, especially coming from her.”

Reported Effects of Homebuyer Education and Counseling on Study 

Participant Decisions

The focus groups also shed light on whether the services received 

by study participants are affecting their knowledge, skills, behaviors, 

and decisions. Focus group participants, named in this report with 

aliases to protect privacy, reported that homeownership education and 

counseling taught them about the steps in the homebuying process, 

including how to apply for a loan, how to examine a credit report and 

score, and what occurs during the closing process. Some focus group 

participants worked with counselors to develop new financial behaviors 

to help them get ready for homeownership. Several participants said 

that counseling helped them create a new budget or modify an existing 

budget to account for the new expenses homeownership would bring. 

The counselors often highlighted ways to save money, cut expenses, 

and achieve financial goals. 

“The homebuying process 

can be overwhelming....

Maybe if someone went in 

knowing what they were 

getting into, they would 

have a clearer picture of 

what they could afford.

Rather than some starryeyed concept of, oh, I am 

going to have this huge 

home and...it is going to be 

great and then once I buy 

it, we are done. And just 

understanding the costs 

going into, costs of closing, 

the cost of maintaining.”

–Denise, a focus group 

participant in Dallas, Texas
2

24 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

4 Experiences With Homebuyer Education and Counseling

In the preparation and search outcome domain, focus group participants 

who were in the early stages of the homebuying process when they 

completed services noted that learning specialized terminology was 

extremely helpful. Those who were further along in the homebuying 

process said that they had learned about energy efficiency, home 

maintenance, foreclosure prevention, and creating a budget for a 

new home.

Some focus group participants said explicitly that homebuyer education 

and counseling can lead people to the conclusion that they are not 

yet ready to purchase a home. A focus group participant in Chicago, 

Illinois, explained that she wanted to stay in her current neighborhood 

because of her daughter’s school but could not afford the type of 

house she wanted to buy in that neighborhood. She stated, “just 

talking to the young lady, it made me realize...what I could afford and, 

well, what I was preapproved for.... She was adding on other expenses 

that I had totally forgot to add, you know, ’cause I thought I had it all 

together. And I hadn’t. So I ran into a few things talking to her that 

made me realize that I probably need to just, you know, wait.” 

In the financial literacy, capability, and management outcome domain, 

focus group participants suggested how people’s behavior might have 

changed in response to homebuyer education and counseling. Focus 

group participants reported that they were taught how to take action 

to decrease their debt and increase their credit scores. In Dallas, the 

homebuyer workshop taught a study enrollee how to manage credit 

card debt successfully. Another focus group participant explained that, 

when reviewing his credit report with the housing counselor, he found 

errors. With assistance from the counselor, he sent letters to the credit 

bureau agencies to fix the errors. 

In summary, the focus groups corroborated the findings from the followup survey, with scheduling convenience a primary barrier for those who 

did not take up services. Those who did take up homebuyer education 

and counseling reported gaining important information that, in some 

cases, favorably influenced decisions and behaviors during the 

homebuying process.

The education and 

counseling “gave us the 

idea of whether we should 

go for it right now or not. It 

is really telling us what the 

timing if we are not really 

prepared and if we don’t 

have enough credit or other 

issues... you know, maybe 

it is not the right time for 

us. So it is really helping us 

to make the decision of go 

or no go.”

–Yang, a focus group 

participant in Dallas, Texas

“Since that meeting, I paid 

off four major credit cards.

And actually, I maybe got 

two now which is probably 

Sam’s and Home Depot, 

but I need that to operate.

But the credit card thing 

I saw it paying a lot of 

interest and stuff like 

that.... I doubled up on 

what I was paying.... I put 

it together and eventually I 

got four of them paid off.

–Walter, a focus group 

participant in Dallas, Texas
25 The First-Time Homebuyer Education and Counseling Demonstration: The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

5.

Outcomes and Impacts for Early 

Enrollees After 12 Months

This section reports selected outcomes for the early enrollee sample, measured 1 year after they entered the 

study, describing how the entire sample was faring in the homebuying process, what their financial capability 

and skills were, and, for those who had bought homes, whether homeownership appeared sustainable in 

the short term. For a limited set of outcomes, the section also reports the impact of being offered access 

to homebuyer education and counseling—that is, the differences in outcomes between the treatment and 

control groups.

Outcomes for the Early Enrollee Sample After 12 Months

Exhibit 5.1 presents mean values of 12-month outcomes in the domains of preparedness and search;

financial literacy, management, and capability; and homeownership sustainability for the early enrollee 

sample. These values are the average outcomes for the entire early enrollment sample, including both 

treatment and control group members. They do not tell us about the effectiveness of homebuyer education 

and counseling, but they do describe key features of this sample of low-, moderate-, and middle-income 

homebuyers 12 months after enrollment, including their financial literacy, their credit scores, and their 

mortgage performance.21

21 In the section titled “Early Impacts of Homebuyer Education and Counseling on Selected Outcomes,” we report the impact of being 

offered access to homebuyer education and counseling—that is, the differences in outcomes between the treatment and control groups. 
2

26 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

5 Outcomes and Impacts for Early Enrollees After 12 Months

Exhibit 5.1: Mean Outcomes, Early Enrollee Sample, 12 Months After Enrollment

Outcome Domain and Definition Early Enrollee Sample

Homeownership preparedness and search

Respondent purchased a homea

 (%) 57.8

Number of lenders from which the respondent received price quotesb 1.6

Respondent was very confident that he/she could find the information he/she 

needed about the home purchase processb (%) 31.6

Respondent was very satisfied with the home purchase processb (%) 30.0

Number of correct answers (out of 4) to mortgage literacy quizb 2.79

If in financial difficulty, the respondent would contact his/her lender for 

assistance before missing a mortgage paymentb,c (%) 40.5

Respondent has a credit score greater than or equal to 620d (%) 87.5

Financial literacy, management, and capability

Respondent knows how to correct inaccurate information in his/her credit 

reportb

 (%) 78.6

Respondent occasionally does not have enough money at end of the month to 

cover all billsb

 (%) 10.0

Respondent has electronic or written budgetb

 (%) 57.2

Respondent has a budget and often compares it with actual spendingb (%) 36.1

Credit scored 706.6

Sustainable homeownership

Ever 30 days delinquent on mortgage loana,e (%) 1.8

Ever 60 days delinquent on mortgage loana,e (%) 0.6

Sample sizes

Study participants randomly assigned before December 1, 2014 2,377

a Outcome was constructed using follow-up survey data, credit bureau data, lenders’ loan origination and servicing data, and Federal 

Housing Administration data and covers 2,353 (99.0 percent) of the early enrollee sample.

b Outcome was constructed using follow-up survey data. These data cover 1,926 (81.0 percent) of the early enrollee sample. Outcomespecific sample sizes may vary because of item nonresponse. Outcomes defined using follow-up survey data are weighted using survey 

nonresponse weights (see appendix C for details).

c This outcome is set equal to 1 if the respondent has a mortgage and would contact his or her lender for assistance before missing a 

mortgage payment and is set equal to 0 if the respondent has a mortgage but would not contact his or her lender for assistance regarding 

missed payments or would wait to contact the lender until after a missed payment. The outcome is also set to 0 if the respondent does not 

have a mortgage loan, and as a result it is less than if only mortgage holders were included.

d Outcome was constructed using credit bureau data. These data cover 2,180 (91.7 percent) of the early enrollee sample.

e This outcome is set equal to 0 if the enrollee does not have a mortgage loan (and therefore was not observed as delinquent on mortgage 

loan), and as a result it is less than if only mortgage holders were included.

Note: Appendix B provides detail on the construction and source of each outcome measure.
2

27 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

5 Outcomes and Impacts for Early Enrollees After 12 Months

Preparedness and Search

As shown in exhibit 5.1, about 58 percent of early enrollees reported purchasing a home within a year of 

entering the study. Early enrollees received price quotes from an average of 1.6 lenders during their housing 

search. A minority of early enrollees (32 percent) were “very confident” that they could find the information 

they needed about the home purchase process, and a similar proportion (30 percent) were very satisfied 

with the homebuying process, regardless of whether they purchased a home.

Financial Literacy, Management, and Capability

Financial capability and management skills were mixed at the 1-year follow-up point. On the one hand, 

close to 4 out of 5 early enrollees reported knowing how to correct inaccurate credit report information. The 

average credit score was more than 700, with 88 percent having a score of more than 620, approximately 

the score needed to qualify for a conventional mortgage. Although most early enrollees (57 percent) reported 

having a budget, 36 percent reported comparing actual spending with their budgets. Furthermore, although 

most respondents appear to be relatively financially stable, 1 in 10 reported occasionally not having enough 

money to cover their monthly bills. The early enrollees scored an average of 2.8 out of a possible 4 points on 

a financial literacy quiz at the 1-year follow-up.

Homeownership Sustainability

On the mortgage performance metrics—critical indicators of homeownership sustainability—less than 

2 percent of the early enrollees had been 30 days late on a mortgage payment at least once during the 

12-month period following their enrollment (averaged across all individuals, regardless of whether they 

purchased a home), and less than 1 percent had been 60 days late on a mortgage.22

Early Impacts of Homebuyer Education and Counseling on Selected Outcomes 

This study evaluates the effectiveness of offering homebuyer education and counseling services using a 

randomized experimental design, in which study participants were randomized to a control group that was 

not offered access to services or to one of three treatment groups that were offered access to services. The 

three treatment groups for the early enrollee sample are a remote group (offered online homebuyer education 

and telephone counseling), an in-person group (offered in-person education workshops and in-person 

counseling), and a choice group (randomized after expressing a preference for remote or in-person services). 

With small sample sizes, particularly for the choice group that began only in September 2014 (when 

randomization to the in-person group ended), the estimates provided in exhibit 5.2 are based on pooling 

outcomes for the members of the three treatment groups and comparing the means for those outcomes with 

the mean outcomes for the control group.23,24 Thus, these estimates are of the impacts of being offered 

22 We observe that 58 percent of the early enrollee sample purchased a home at follow-up. Therefore, for early enrollees who purchased a 

home, the 30- and 60-day mortgage delinquency rates are 3.1 percent and 1.0 percent, respectively. 

23 For a detailed description of how measures used in this analysis were constructed, including handling of missing data, see Appendix B: 

Description of Measures and Missing Data Procedures. 

24 For a detailed description of the impact model and survey nonresponse weighting procedures, see Appendix C: Technical Supplement 

to the Impact Analysis. 
2

28 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

5 Outcomes and Impacts for Early Enrollees After 12 Months

any mode of homebuyer education and counseling. The study team preselected the four outcome measures 

before the impact estimates were calculated.25

Exhibit 5.2: Impact Estimates, Early Enrollee Sample

Outcome Treatment 

Group

Control 

Group

Difference 

(Impact)

Number of correct answers (out of 4) to mortgage literacy quiza 2.82 2.74 0.08** 

If in financial difficulty, the respondent would contact lender for 

assistance before missing a mortgage paymenta,b (%) 41.8 37.9 3.9* 

Respondent has a credit score greater than or equal to 620c

 (%) 88.4 86.1 2.3** 

Respondent has a budget and often compares it with actual 

spendinga

 (%) 35.4 35.6 – 0.2

** Difference is statistically significant (two-sided t-test) at the p < 0.05 level. 

* Difference is statistically significant (two-sided t-test) at the p < 0.10 level.

a

 Outcome was constructed using follow-up survey data. These data cover 1,926 (81.0 percent) of the early enrollee sample. Outcomespecific sample sizes may vary because of item nonresponse. We applied sample weights that adjust for follow-up survey nonresponse for 

analyses of outcomes collected from follow-up surveys. (See appendix C for details.)

b

 This outcome is set equal to 1 if the respondent has a mortgage and would contact his or her lender for assistance before missing a 

mortgage payment and is set equal to 0 if the respondent has a mortgage but would not contact his or her lender for assistance regarding 

missed payments or would wait to contact the lender until after a missed payment. The outcome is also set to 0 if the respondent does 

not have a mortgage loan, and as a result it is less than if only mortgage holders were included.

c

 Outcome was constructed using credit bureau data. These data cover 2,180 (91.7 percent) of the early enrollee sample.

Notes: Because of rounding, reported impacts (treatment-control differences) may differ from differences between reported means for 

the treatment and control groups. Exhibit C.2 provides additional detail on the construction of the impact estimates. To maintain the 

integrity of the experimental design, all outcomes reported in this exhibit are defined for all enrollees with nonmissing outcome data. 

Appendix B provides additional detail on the construction of measures.

These estimates are for the early enrollees who were randomly assigned before December 1, 2014, and who 

comprise 41 percent of the study’s full sample. They also are early in the sense that more time is needed to 

gauge the impact of homebuyer education and housing counseling on all the study’s outcomes of interest. 

Exhibit 5.2 presents mean outcomes of both treatment and control group members within the early enrollee 

sample, as well as the difference in these two values, which is the estimated impact of being offered 

homebuyer education and counseling. If the impact estimate has asterisks next to it, then the impact is 

statistically different from 0. These impact analyses show the following— 

25 The study team prespecified these four outcomes during the evaluation’s design phase. Restricting the preliminary impact analysis to 

these prespecified outcomes helps guard against “fishing” for statistically significant impacts.
2

29 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

5 Outcomes and Impacts for Early Enrollees After 12 Months

• Improved mortgage literacy. The treatment group members perform 0.08 points (or 2.8 percent)26

better, relative to their control group counterparts, on a four-question mortgage literacy quiz. Although 

this impact is small in magnitude, it indicates that the intervention is improving mortgage literacy. 

• Better reported communication with lenders. The treatment group members are 3.9 percentage points 

(or 10.4 percent) more likely to report that they would contact their lender before missing a mortgage 

payment.27 This finding indicates that homebuyer education and counseling successfully encourage 

participants to communicate proactively and productively with their lenders in times of financial distress. 

Research shows that the longer borrowers wait to reach out to their lenders when they face difficulties 

meeting their mortgage obligations, the less likely they are to recover from a delinquency (Cutts and 

Merrill, 2008). 

• Improved credit and underwriting risk. The treatment group members are 2.3 percentage points (or 2.7 

percent) more likely, relative to their control group counterparts, to have a credit score of more than 

620.28 This difference may indicate that the intervention is helping participants repair damaged credit 

or correct inaccuracies in their credit reports, bringing their scores above an important underwriting 

threshold. 

• No evidence of improved budgeting practices. We find no evidence that treatment group members 

were more or less likely, relative to their control group counterparts, to compare a budget with their 

actual spending. This finding is surprising because budgeting activity is often a point of emphasis in 

homebuyer education and counseling. One possible explanation for this null finding is that homebuyer 

education and counseling may increase budgeting only temporarily during the home search process. 

For example, those who are still looking for a home may be particularly conscientious about staying 

within a predetermined budget as they determine how much house they can afford or as they save 

for a downpayment. Because this outcome is measured 12 months after respondents enrolled in the 

study, many study members (77 percent) had either purchased a home or ended their search. Future 

work (using the full sample) may analyze this outcome separately for those at different stages of the 

homebuying process.

The results reported in exhibit 5.2 should be interpreted as the impact of being offered homebuyer education 

and counseling to those who volunteered to be part of the study. The impacts represent overall access 

to homebuyer education and counseling services provided, and not to a particular mode of service. As 

26 On average, the treatment group scored 2.82 compared with an average score of 2.74 for the control group, which represents a 

2.8-percent increase in average score for the treatment group relative the control group.

27 This outcome was coded as 1 if the respondent had a mortgage and would contact his or her lender for assistance before missing a 

mortgage payment and was coded as 0 if the respondent would not contact his or her lender for assistance before missing a mortgage 

payment or if the respondent did not have a mortgage loan. To determine whether this finding is driven by differential mortgage loan rates 

across treatment groups, we estimated the impact of the intervention on an indicator for whether the respondent had a mortgage loan. We 

found no evidence that treatment group members are more likely to have a mortgage loan, alleviating concerns that this finding is driven by 

differential mortgage loan rates across treatment groups.

28 As a point of reference, Fannie Mae’s underwriting guidelines published in February 2015 required borrowers to have a minimum credit 

score of 620. See https://www.fanniemae.com/content/guide/selling/b3/5.1/01.html.
2

30 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

5 Outcomes and Impacts for Early Enrollees After 12 Months

detailed in section 6, future analysis will estimate impacts of alternative modes of homebuyer education and 

counseling as well as the impacts of actually taking up services as opposed to being offered them. 

To preview that analysis, appendix exhibits C.2 and C.3 present impacts for two subsets of early enrollees. 

The impact analysis presented in exhibit C.2 uses the remote and choice treatment groups, excluding the inperson group, which includes substantial no-shows. Exhibit C.3 uses only the remote treatment group. The 

size of the sample at this early stage precludes separate analysis for in-person services or for the 

choice group. The magnitudes of the impact estimates reported in these exhibits are in line with the 

magnitude of the impacts observed for the entire early enrollee sample. The impact estimates for credit 

scores are in the same direction but not statistically significant, which may reflect the smaller sample sizes 

for these estimates. 

Indeed, it is encouraging to see that, even at this early time with a limited sample, evidence suggests that 

the intervention is moving the needle in a favorable direction for these individuals. Underpinning the 

analysis is the study’s strong evaluation design, which permits interpretation of the results as causal and 

unbiased. As discussed in the conclusion, forthcoming study reports will have more information about the 

impacts of homebuyer education and counseling, including the extent to which differences in impacts exist 

by service delivery mode.
31 The First-Time Homebuyer Education and Counseling Demonstration: The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

6. 

Conclusion and Future 

Study Activities

In light of the recent collapse of the U.S. housing market, understanding whether first-time homebuyer 

education and counseling can help improve outcomes is critical to policymakers as well as to private lenders. 

Indeed, homeownership offers both potential benefits and potential risks, and homebuyer education and 

counseling are intended to help buyers reap the benefits and minimize the risks. Informed consumers 

are expected to be more deliberate in their decisions about whether and when to purchase a home and 

more selective in choosing homes and mortgages that are appropriate for their financial situations and 

lifestyles. In addition, education and counseling are intended to help improve homebuyers’ overall financial 

management, leading not only to lower levels of mortgage delinquency but to better overall financial health. 

Although some research suggests benefits of homebuyer education and counseling services, little evidence 

is conclusive because of the difficulty in isolating the impact of services from other correlated factors. 

Through its experimental evaluation design, HUD’s First-Time Homebuyer Education and Counseling 

Demonstration overcomes weaknesses in previous research, which has tended to suffer from limited sample 

sizes, nonexperimental design, or both. The study successfully enrolled 5,854 individuals in 28 metropolitan 

areas and successfully randomized them to create comparable control and treatment groups. The study is 

groundbreaking in its scale and scope.

Although it is still too early to know the extent to which homebuyer education and counseling improve 

long-term homeownership sustainability, early signs indicate that offering individuals access to services has 

favorable impacts. The treatment group’s mortgage literacy, credit scores, and communication with lenders 

demonstrably improved, and all signal steps on the path to sustainable homeownership. In the early enrollee 

sample, study participants across all modes of homebuyer education and counseling scored about 3 percent 

better than control group members on mortgage literacy test questions. Treatment group members were 

about 10 percent more likely to contact their lender for assistance before missing a mortgage payment. They 

were 3 percent more likely to have a credit score of 620 or higher, a proxy for qualifying for conventional 

mortgages. These early favorable signs imply a solid foundation for improved long-term homeownership 

sustainability outcomes for the treatment group. 

Additional analysis shows that these findings largely hold up in a smaller sample that compares the remoteonly treatment group with the control group. In that comparison, we can be more assertive about the 

specific characteristics of the intervention that are associated with treatment group impacts: offering access 

to remotely provided services improves the treatment group’s mortgage literacy and response to financial 

stress. The magnitudes of these impacts are not detectably different from those of the impacts observed in 

the entire early enrollee sample, which pools treatment group members across all modes (in-person, remote, 

and choice) to show the effect of offering access to any kind of homebuyer education and counseling. In the 
2

32 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

6 Conclusion and Future Study Activities

future, larger samples will enable us to more precisely estimate the effect of offering in-person homebuyer 

education and counseling services.

This report provides early insights from a subset of the study participants and focuses on only a few shortterm outcomes. Future reports will provide more-detailed analyses of the implementation and impact of the 

homebuyer education and counseling intervention, enabling us to answer a wealth of additional research 

questions. In brief, future impact analyses will push forward our understanding of the effects of homebuyer 

counseling and education in the following ways— 

• Examine more varied outcomes. Future analyses will include many more outcomes, including those 

that we expect to surface only after a greater passage of time, providing a more complete picture of the 

impact of homebuyer education and counseling. Long-term homeownership sustainability and personal 

financial health are among the goals of the intervention that will be examined.

• Understand the effects of service delivery mode. The design offers the potential to provide rigorous 

evidence of (1) the effect of remotely provided homebuyer education and counseling services; (2) 

the effect of in-person homebuyer education and counseling services; and (3) the differences, if any, 

between the impact of remote services and that of in-person services. Understanding the impact of 

these alternative modes of education and counseling is of central interest to both policymakers and the 

homebuyer education and counseling field.29

• Understand the effect of taking up program services. It is important to recognize that the overall impact 

of homebuyer education and counseling on the treatment group is diluted by treatment group members 

who do not initiate education and/or counseling services—which we know to be about one-half of the 

enrolled treatment group. The study will analyze the effect of the treatment on the treated, or TOT, by 

following the conventional approach of assuming that those who did not take up the offer experienced 

none of the benefit. Because of the differential take-up rates across our three treatment groups, this 

analysis will be particularly policy relevant, revealing the impacts on those who initiated services. 

• Understand the effects of homebuyer education and counseling for subpopulations. Future reports will 

consider whether the intervention’s impacts vary for specific demographic and socioeconomic groups, 

such as Spanish speakers, or groups defined by traits such as education, income, and credit score. 

These types of analyses will provide deeper insights into how policymakers and homebuyer education 

and counseling practitioners can target services toward particular populations.

• Evaluate the role of “dosage” on impacts. Related to the examination about the effect of taking up 

the offer of treatment, the study will also consider the relative effects of variation in the intensity of 

treatment. Because we expect impacts to be greater among those treatment group members who 

experienced more education and counseling, future analyses will examine the relative effectiveness for 

29 The post-redesign baseline survey asked all study participants whether they would prefer to receive remote or in-person services. This 

question allows for us to make an experimental comparison between those in the choice group with a preference for in-person services and 

their counterparts in the control group with a preference for in-person services. Capitalizing on this design will enable us create a pooled 

sample with which we can experimentally estimate the impact on those who were offered in-person services (including in-person treatment 

group members and choice treatment group members who stated a preference for in-person services).
2

33 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

6 Conclusion and Future Study Activities

treatment group members who completed part or all of the offered education and counseling activities. 

Although the planned method for this analysis relies on experimental data, the method requires 

additional assumptions to produce estimated dosage effects. The estimated dosage effects are therefore 

interpreted as nonexperimental. 

Exhibit 6.1 summarizes the planned reports, their anticipated release dates, and the topics they will cover. 

Funding permitting, the study will document the long-term impact of first-time homebuyer education and 

counseling through a 42-month follow-up survey. In addition to revealing whether the intervention can 

indeed improve long-term outcomes, the final report will also provide analyses of the relative effectiveness 

of in-person versus remote delivery approaches, as well as subgroup and dosage analyses. In doing 

so, the report will provide an evidence base for future policy decisions regarding homebuyer education 

and counseling. 

Exhibit 6.1: Future Study Products

Report Release Date Topics Covered

Baseline report Fall 2016 Study implementation, intervention description, participant 

experiences, full sample descriptive characteristics, analysis of 

representativeness of sample. 

Interim report Early 2018 Analysis of all 12-month outcomes and impacts, preliminary analysis 

of alternative service delivery modes’ impacts, preliminary subgroup 

analyses, policy implications.

Final reporta 2020 Analysis of long-term (42-month) outcomes and impacts, analysis of 

alternative service delivery modes’ impacts, subgroup analyses, policy 

implications.

a

 Contingent on funding for the study’s 42-month survey and associated analyses and report.
34 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights
35 The First-Time Homebuyer Education and Counseling Demonstration: The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

A

Appendix A: Supplemental Tables

This appendix includes two sets of supplemental tables. Exhibits A.1 and A.2 compare the characteristics of 

early enrollees with those of the full sample. Exhibits A.3 and A.4 report treatment-control baseline balance 

tests for the Early Enrollee Sample. 

Exhibit A.1: Comparison of Full Sample and Early Enrollee Sample: Demographic Characteristics

Baseline Variable Full Sample Early Enrollee 

Sample

Statistically 

Significant 

Difference

Race/ethnicity of study participant

Hispanic (%) 25.0 23.8

White non-Hispanic (%) 38.5 40.9 *

African-American non-Hispanic (%) 20.4 19.7

Asian non-Hispanic (%) 12.1 11.3

Other race (%) 3.9 4.3

Gender of study participant

Male (%) 60.2 58.8

Marital status of study participant

Married (%) 38.1 39.3

Divorced, widowed, or separated (%) 14.8 15.5

Single and never married (%) 47.1 45.3

Plans to purchase the home with a co-borrower 

Co-borrower (%) 26.2 28.0

Household size

One (%) 22.8 22.2

Two (%) 32.0 32.7

Three (%) 19.8 20.0

Four (%) 15.2 15.2

Five (%) 6.7 6.9

Six or more (%) 3.6 3.0

Education of study participant

Bachelor’s degree or higher (%) 53.5 54.2

Associate’s degree (%) 12.9 12.6

Some college, but no degree (%) 16.1 16.5

High school diploma or less (%) 17.5 16.7

* Full sample is statistically significantly different from the early enrollee sample at the p < 0.05 level. 

Source: Baseline survey of study participants

(continued)
2

36 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

A Appendix A

Exhibit A.1: Comparison of Full Sample and Early Enrollee Sample: Demographic Characteristics

Baseline Variable Full Sample Early Enrollee 

Sample

Statistically 

Significant 

Difference

Employment

Full-time employment (30+ hours per week) (%) 89.9 89.3

Part-time employment (1–29 hours per week) (%) 4.1 4.2

Unemployed and looking for work (%) 0.5 0.4

Not working, homemaker, retired, student, or other 

(%) 5.5 6.1

Income received by study participant and any co-borrowers in past 12 months

$24,999 or less (%) 8.1 7.8

$25,000 to $49,999 (%) 34.0 33.6

$50,000 to $74,999 (%) 32.6 33.9

$75,000 to $99,999 (%) 14.6 14.2

$100,000 or more (%) 10.7 10.5

* Full sample is statistically significantly different from the early enrollee sample at the p < 0.05 level. 

Source: Baseline survey of study participants

Exhibit A.2: Comparison of Full Sample and Early Enrollee Sample: Measures of Homebuyer Status, 

Financial Responsibility, and Creditworthiness

Baseline Variable Full Sample Early Enrollee 

Sample

Statistically 

Significant 

Difference

Stage in the homebuyer process

Not yet started home search (%) 11.2 9.4 *

Started home search, but no visits (%) 13.2 12.4

Visited homes, but no offer (%) 23.9 22.6

Made an offer on a home, but no purchase (%) 13.2 13.1

Signed a purchase agreement (%) 25.3 28.2 *

Purchased a home (%) 13.2 14.3

Measures of financial responsibility

Uses a written budget (%) 74.7 76.4

Usually pays credit card balance in full to avoid 

interest charges (%) 77.4 77.9

During the past year, was short on money sometimes 

or often (%) 16.4 18.3 *

* Full sample is statistically significantly different from the early enrollee sample at the p < 0.05 level.

Sources: Baseline survey of study participants; credit bureau data

(continued)
2

37 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

A Appendix A

Exhibit A.2: Comparison of Full Sample and Early Enrollee Sample: Measures of Homebuyer Status, 

Financial Responsibility, and Creditworthiness

Baseline Variable Full Sample Early Enrollee 

Sample

Statistically 

Significant 

Difference

Sets aside extra money for retirement, education, or 

to build a financial cushion sometimes or often (%) 91.1 90.2

Everyone in household has health insurance (%) 88.4 87.1

Level of total savings and investments ($) 55,149 56,611

Creditworthiness

Credit score

 Less than 580 (%) 4.0 4.2

 580 to 619 (%) 6.8 7.3

 620 to 659 (%) 15.3 15.5

 660 to 699 (%) 17.6 18.5

 700 to 739 (%) 19.9 19.2

 740 or more (%) 36.5 35.2

Cash on hand for downpayment and closing costs ($) 33,758 34,475

Amount of nonhousing debt ($) 19,606 19,971

Monthly payment nonhousing debt ($) 355 351

* Full sample is statistically significantly different from the early enrollee sample at the p < 0.05 level.

Sources: Baseline survey of study participants; credit bureau data
2

38 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

A Appendix A

Exhibit A.3: Baseline Balance Testing, Early Enrollee Sample: Demographic Characteristics

Baseline Variable Treatment 

Group Control Group

Statistically 

Significant 

Difference

Race/ethnicity of study participant

Hispanic (%) 23.6 24.1

White non-Hispanic (%) 40.1 42.0

African-American non-Hispanic (%) 20.0 19.3

Asian non-Hispanic (%) 11.4 11.1

Other race (%) 4.8 3.6

Gender of study participant

Male (%) 58.7 59.0

Marital status of study participant

Married (%) 38.2 40.7

Divorced, widowed, or separated (%) 15.2 15.9

Single and never married (%) 46.6 43.3

Plans to purchase the home with a co-borrower 

Co-borrower (%) 26.4 30.2 *

Household size

One (%) 23.1 21.0

Two (%) 32.1 33.4

Three (%) 20.6 19.1

Four (%) 14.9 15.6

Five (%) 6.2 7.8

Six or more (%) 3.0 3.1

Education of study participant

Bachelor’s degree or higher (%) 56.2 51.5 *

Associate’s degree (%) 12.5 12.6

Some college, but no degree (%) 16.8 16.1

High school diploma or less (%) 14.4 19.8 *

Employment

Full-time employment (30+ hours per week) (%) 90.3 87.9

Part-time employment (1–29 hours per week) (%) 3.6 5.0

Unemployed and looking for work (%) 0.5 0.3

Not working, homemaker, retired, student, or other 

(%) 5.6 6.8

* Pooled treatment group is statistically significantly different from control group at the p < 0.05 level.

Source: Baseline survey of study participants

(continued)
2

39 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

A Appendix A

Exhibit A.3: Baseline Balance Testing, Early Enrollee Sample: Demographic Characteristics

Baseline Variable Treatment 

Group Control Group

Statistically 

Significant 

Difference

Income received by study participant and any co-borrowers in past 12 months

$24,999 or less (%) 7.6 8.1

$25,000 to $49,999 (%) 33.5 33.6

$50,000 to $74,999 (%) 34.1 33.6

$75,000 to $99,999 (%) 14.9 13.4

$100,000 or more (%) 9.9 11.3

* Pooled treatment group is statistically significantly different from control group at the p < 0.05 level.

Source: Baseline survey of study participants
2

40 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

A Appendix A

Exhibit A.4: Baseline Balance Testing, Early Enrollee Sample: Measures of Homebuyer Status, Financial 

Responsibility, and Creditworthiness

Baseline Variable Treatment 

Group Control Group

Statistically 

Significant 

Difference

Stage in the homebuyer process

Not yet started home search (%) 9.1 9.8

Started home search, but no visits (%) 12.0 13.0

Visited homes, but no offer (%) 23.1 22.0

Made an offer on a home, but no purchase (%) 12.9 13.3

Signed a purchase agreement (%) 28.6 27.7

Purchased a home (%) 14.3 14.3

Measures of financial responsibility

Uses a written budget (%) 74.2 79.3 *

Usually pays credit card balance in full to avoid 

interest charges (%) 79.4 75.8 *

During the past year, was short on money sometimes 

or often (%) 16.7 20.5 *

Sets aside extra money for retirement, education, or 

to build a financial cushion sometimes or often (%) 90.8 89.5

Everyone in household has health insurance (%) 87.8 86.2

Level of total savings and investments ($) 52,556 62,194

Creditworthiness

Credit score

 Less than 580 (%) 3.3 5.5 *

 580 to 619 (%) 7.3 7.3

 620 to 659 (%) 14.5 17.0

 660 to 699 (%) 18.6 18.4

 700 to 739 (%) 19.9 18.3

 740 or more (%) 36.5 33.5

Cash on hand for downpayment and closing costs ($) 33,132 36,323

Amount of nonhousing debt ($) 18,955 21,342

Monthly payment nonhousing debt ($) 343 363

* Pooled treatment group is statistically significantly different from control group at the p <0.05 level.

Sources: Baseline survey of study participants; credit bureau data
41 The First-Time Homebuyer Education and Counseling Demonstration: The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

B

Appendix B: Description 

of Measures and Missing 

Data Procedures

This appendix provides additional details on the construction of baseline and outcome measures presented 

in this report. Exhibit B.1 shows how the baseline covariates included in the impact model were constructed. 

Exhibit B.2 shows how the outcomes examined in section 5 were constructed. After each exhibit, we report 

the strategy for handling missing data of each variable type, baseline covariates, and outcomes, respectively.

Exhibit B.1: Operationalization of Baseline Covariates

Domain Variable Description Operationalization Data Source(s) Items

Demographic 

characteristics

Race/ethnicity of study 

participant

Series of mutually exclusive binary 

variables:

Hispanic. 

White non-Hispanic. 

African-American non-Hispanic. 

Asian non-Hispanic. 

Other race. 

Baseline survey H10, H11

Demographic 

characteristics

Gender of study 

participant

Binary variable that takes on value: 

0 if female.

1 if male.

Baseline survey H1

Demographic 

characteristics

Marital status of study 

participant

Series of mutually exclusive binary 

variables:

Married. 

Divorced, widowed, or separated. 

Single and never married. 

Baseline survey H2

Demographic 

characteristics

Plans to purchase the 

home with a co-borrower

Binary variable that takes on value: 

0 if does not plan to purchase the home 

with a co-borrower.

1 if plans to purchase the home with a 

co-borrower.

Baseline survey CB1

(continued)
2

42 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

BA Appendix B

Exhibit B.1: Operationalization of Baseline Covariates

Domain Variable Description Operationalization Data Source(s) Items

Demographic 

characteristics

Household size Series of mutually exclusive binary 

variables:

One. 

Two. 

Three. 

Four. 

Five. 

Six or more. 

Baseline survey H3

Demographic 

characteristics

Education of study 

participant

Series of mutually exclusive binary 

variables that capture educational 

attainment:

Bachelor’s degree or higher. 

Associate’s degree. 

Some college, but no degree. 

High school diploma or less. 

Baseline survey H8

Stage in the 

homebuyer 

process

Stage in the homebuyer 

process

Series of mutually exclusive binary 

variables that capture the stage in the 

homebuying process:

Not yet started home search.

Started home search, but no visits.

Visited homes, but no offer.

Made an offer on a home, but no 

purchase.

Signed a purchase agreement.

Purchased a home.

Baseline survey A1, A3a, 

A3b, A3f, A3g

Employment and 

income

Employment Series of mutually exclusive binary 

variables for employment status of the 

study participant:

Full-time employment (30+ hours per 

week).

Part-time employment (1–29 hours per 

week).

Unemployed and looking for work.

Not working, homemaker, retired, 

student or other.

Baseline survey H5

(continued)
2

43 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

BA Appendix B

Exhibit B.1: Operationalization of Baseline Covariates

Domain Variable Description Operationalization Data Source(s) Items

Employment and 

income

Income received by 

study participant and 

any co-borrowers in past 

12 months

Series of mutually exclusive binary 

variables defined based on the reported 

income received by study participant 

and any co-borrowers in past 12 

months:

$24,999 or less. 

$25,000 to $49,999. 

$50,000 to $74,999. 

$75,000 to $99,999. 

$100,000 or more. 

Note: If the respondent reported an 

income range rather than a specific 

value (for example, $40,000 to less 

than $55,000), we used the midpoint 

of the reported range to determine the 

appropriate income category. If neither 

a value nor a range was reported, then 

the variable was set to missing.

Baseline survey F2, F2b

Financial 

responsibility

Uses a written budget Binary variable that takes on value: 

0 if does not have a budget of monthly 

household expenses.

1 if has a budget of monthly household 

expenses.

Baseline survey F4

Financial 

responsibility

Usually pays credit card 

balance in full to avoid 

interest charges 

Binary variable that takes on value: 

0 if does not usually pay credit card 

balance in full to avoid interest 

charges. 

1 if does usually pay credit card balance 

in full to avoid interest charges.

Baseline survey C1f

Financial 

responsibility

During the past year, 

was short on money 

sometimes or often 

Binary variable that takes on value: 

0 if reports being short on money rarely 

or never. 

1 if reports being short on money 

sometimes or often.

Baseline survey F7

Financial 

responsibility

Sets aside extra 

money for retirement, 

education, or to build 

a financial cushion 

sometimes or often 

Binary variable that takes on value: 

0 if reports setting aside extra money 

rarely or never. 

1 if reports setting aside extra money 

sometimes or often.

Baseline survey F8

(continued)
2

44 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

BA Appendix B

Exhibit B.1: Operationalization of Baseline Covariates

Domain Variable Description Operationalization Data Source(s) Items

Financial 

responsibility

Everyone in household 

has health insurance 

Binary variable that takes on value: 

0 if at least one household member 

does not have health insurance. 

1 if everyone in the household has 

health insurance.

Baseline survey F16, F17

Financial 

responsibility

Level of total savings 

and investments 

Sum of reported values for checking 

accounts, savings accounts, retirement 

accounts, and other savings and 

investment accounts (continuous 

variable).

Note: The responses were capped at 

999,999 for each of these separate 

categories.

Baseline survey F9a, F10a, 

F11a, F12a, F13a

Creditworthiness Credit score Series of mutually exclusive binary 

variables defined based on the baseline 

credit score of the study participant:

Less than 580.

580 to 619. 

620 to 659. 

660 to 699. 

700 to 739. 

740 or more.

Note: For each study participant, we 

capture the person’s baseline credit 

data within 2 months before their 

enrollment. 

Credit data

Creditworthiness Cash on hand for 

downpayment and 

closing costs 

Total cash on hand for downpayment 

and closing costs (continuous variable).

Note: If respondent did not provide an 

exact amount but reported a range, 

we used the midpoint of the range and 

included this value in the continuous 

measure. Responses were capped at 

999,999.

Baseline survey B3, B3b

Creditworthiness Amount of nonhousing 

debt 

Amount of nonhousing debt equals 

the total balance on open installment 

accounts plus open revolving accounts 

minus the balance on open mortgage 

accounts. This measure is set equal to 

0 if the study member was included 

in the credit file but had no open debt 

accounts on file.

Credit data

(continued)
2

45 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

BA Appendix B

Exhibit B.1: Operationalization of Baseline Covariates

Domain Variable Description Operationalization Data Source(s) Items

Credit worthiness Monthly payment 

nonhousing debt 

Monthly scheduled payments for 

nonhousing debt equals the total 

scheduled monthly payments for all 

open accounts besides mortgage 

accounts. This measure is set equal to 

0 if the study member was included 

in the credit file but had no open debt 

accounts on file.

Credit data

To address item nonresponse in the baseline covariate data, we use the dummy variable adjustment

approach. This strategy sets missing cases to a constant and includes a set of missing data flags in the 

impact model. As detailed by Puma et al. (2009), this method is appropriate for handling missing baseline 

data from experimentally designed evaluations. The method involves the following three steps for each 

baseline covariate X with missing data—

• Step 1: Create a new variable Z that is set equal to X for all cases in which X is nonmissing and set to a 

constant value, C = the mean of X, for those cases in which X is missing.

• Step 2: Create a new variable D, which is set equal to 1 for cases in which X is missing and set equal to 

0 for cases in which X is not missing. 

• Step 3: Replace the baseline covariate X in the impact analysis model with Z and D. Doing so will 

allow for the impact model to estimate the relationship between Y and X when X is not missing and to 

estimate the relationship between Y and D when X is missing. 

Exhibit B.2: Operationalization of Outcomes

Domain Variable Description Operationalization Data Source(s) Items

Preparedness 

and search

Purchased a home Binary variable that takes on value: 

1 if observed that respondent purchased 

a home in either interim survey, credit 

bureau data, lender data, or FHA data.

0 if did not observe that respondent 

purchased a home in the interim survey, 

credit bureau data, lender data, or FHA 

data, but respondent has data from at 

least one of these data sources.

Interim survey A1; credit 

bureau; lender data; FHA 

data

FHA = Federal Housing Administration. (continued)
2

46 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

BA Appendix B

Exhibit B.2: Operationalization of Outcomes

Domain Variable Description Operationalization Data Source(s) Items

Preparedness 

and search

Number of lenders from 

which the respondent 

received price quotes

Total number of lenders from which 

respondent received price quotes 

(continuous variable). Variable equals 0 if 

respondent did not contact any lenders or 

if respondent contacted lenders but did 

not receive any quotes.

Interim survey C4, C5, 

C5a

Preparedness 

and search

Respondent was very 

confident that he or she 

could find the information 

he or she needed about 

the home purchase 

process 

Binary variable that takes on value: 

1 if reports being very confident that 

respondent could find information about 

the home purchase process. 

0 if reports being confident, somewhat 

confident, or not confident at all that 

respondent could find information about 

the home purchase process.

Interim survey C9

Preparedness 

and search

Respondent was very 

satisfied with the home 

purchase process 

Binary variable that takes on value: 

1 if reports being very satisfied with the 

home purchase process. 

0 if reports being somewhat satisfied, 

somewhat dissatisfied, or very 

dissatisfied with the home purchase 

process.

Interim survey C10

Financial 

literacy, 

capability, and 

management

Number of correct 

answers (out of 4) to 

mortgage literacy quiz

Sum of correct answers to mortgage 

literacy quiz (ranges from 0 to 4).

Interim survey B2, B3, 

B4, B5

Financial 

literacy, 

capability, and 

management

Respondent knows how 

to correct inaccurate 

information in credit 

report 

Binary variable that takes on value: 

1 if agrees or strongly agrees that knows 

how to correct inaccurate information in 

credit report.

0 if disagrees or strongly disagrees 

that knows how to correct inaccurate 

information in credit report.

Interim survey B1h

FHA = Federal Housing Administration. (continued)
2

47 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

BA Appendix B

Exhibit B.2: Operationalization of Outcomes

Domain Variable Description Operationalization Data Source(s) Items

Financial 

literacy, 

capability, and 

management

Respondent occasionally 

does not have enough 

money to cover all bills at 

the end of the month 

Binary variable that takes on value: 

1 if agrees or strongly agrees that 

occasionally does not have enough 

money to cover all bills at the end of the 

month. 

0 if disagrees or strongly disagrees that 

occasionally does not have enough 

money to cover all bills at the end of the 

month.

Interim survey B1b

Financial 

literacy, 

capability, and 

management

Respondent has an 

electronic or written 

budget 

Binary variable that takes on value: 

1 if has an electronic or written budget.

0 if does not have an electronic or written 

budget.

Interim survey B9

Financial 

literacy, 

capability, and 

management

Respondent has a budget 

and often compares it 

with actual spending 

Binary variable that takes on value: 

1 if has a budget and often compares it 

with actual spending.

0 if has a budget but sometimes, rarely, or 

never compares it with actual spending 

or respondent does not have a budget.

Interim survey B9, B10

Financial 

literacy, 

capability, and 

management

Credit score Credit score measured in September 2015 

(continuous variable).

Credit bureau 

Financial 

literacy, 

capability, and 

management

Respondent has a credit 

score greater than or 

equal to 620

Binary variable that takes on value:

1 if credit score measured in September 

2015 is greater than or equal to 620.

0 if credit score measured in September 

2015 is less than 620.

Credit bureau 

Homeownership 

sustainability

Ever 30 days delinquent 

on mortgage loan 

Binary variable that takes on value: 

1 if was observed as being 30 days 

delinquent on mortgage loan in interim 

survey, credit bureau data, lender data, 

or FHA data.

0 if was not observed as being 30 days 

delinquent on mortgage loan in interim 

survey, credit bureau data, lender data, 

or FHA data, but respondent has data 

from at least one of these data sources.

Interim survey D9, E9, 

E9a; credit bureau; lender 

data; FHA data

FHA = Federal Housing Administration. (continued)
2

48 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

BA Appendix B

Exhibit B.2: Operationalization of Outcomes

Domain Variable Description Operationalization Data Source(s) Items

Homeownership 

sustainability

Ever 60 days delinquent 

on mortgage loan 

Binary variable that takes on value: 

1 if was observed as being 60 days 

delinquent on mortgage loan in interim 

survey, credit bureau data, lender data, 

or FHA data.

0 if was not observed as being 60 days 

delinquent on mortgage loan in interim 

survey, credit bureau data, lender data, 

or FHA data, but respondent has data 

from at least one of these data sources.

Interim survey D9, E9, 

E9a; credit bureau; lender 

data; FHA data

Homeownership 

sustainability

If in financial difficulty, 

the respondent would 

contact his or her lender 

for assistance before 

missing a mortgage 

payment

Binary variable that takes on value: 

1 if would contact lender for assistance 

before missing a mortgage payment. 

0 if would not contact lender for 

assistance regarding missed payments 

or would wait to contact lender until 

after missed payment. 

Note: The follow-up survey questions 

used to construct this outcome were 

asked only of respondents who have a 

mortgage loan. Therefore, to ensure that 

this outcome is defined for all enrollees 

(thereby maintaining the integrity of the 

experimental design), this outcome was 

set equal to 0 if the respondent does not 

have a mortgage loan. 

Interim survey E12, E12a, 

E14, E14a

FHA = Federal Housing Administration.

When outcome data are missing for a given individual, we follow the recommendation of Puma et al. (2009) 

to case-delete observations with missing outcome data. This method has the additional benefit of ease of 

implementation and interpretation. 
49 The First-Time Homebuyer Education and Counseling Demonstration: The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

C

Appendix C: Technical Supplement 

to the Impact Analysis

This appendix serves as a technical supplement to the impact analysis presented in section 5. The first 

section provides more detail related to the study’s design. The second section describes the model used 

to estimate the impact of homebuyer education and counseling. The analytic strategies described in this 

section are relevant to the current report and also will provide the foundation for the study’s interim report, 

which will present analyses for the full sample and many more outcomes. The third section previews future 

impact analyses by presenting impact estimates using the sample that will be used for the study’s primary 

confirmatory analyses, a sample that excludes enrollees randomized to an in-person treatment group before 

September 2014. The fourth section presents impact estimates for the subsample of study participants 

who were offered remote services. The final section describes the study team’s approach to adjusting for 

survey nonresponse.

Study Design

This section describes the study’s experimental design, including modifications that took place in September 

2014 to ensure the study’s later ability to detect the impact of the intervention. 

To evaluate the effectiveness of homebuyer education and counseling, the study randomized enrollees to 

a control group, which was not offered access to services, or to one of three treatment groups (although 

to only two treatment arms at any given time). The random assignment ratio was 42:29:29 in order to 

maximize the study’s ability to detect selected effects of interest, both between each treatment arm and the 

control group and between the pooled treatment group and the control group. 

In the first year of implementation, the two treatment arms referred study participants to remote or inperson services, as illustrated in exhibit 2.2. During that time, a relatively small proportion (about 20 

percent) of those offered in-person services took up the offer. Having such a large share of no-shows in this 

treatment arm implies very low power to detect the effect of offering in-person services. To improve the 

study’s chances of detecting impacts, the study team, in consultation with HUD, stopped assigning enrollees 

to an in-person treatment group, replacing that group with a choice treatment group. As the name implies, 

the choice group allowed for enrollees to choose how to access services, either remotely through online 

education and telephone counseling or in person at a local agency. It is important to note that all individuals 

randomized to treatment or control are part of the sample for this report. That said, the study’s interim and 

final reports will pool the remote and choice treatment groups but exclude the in-person treatment group. In 

addition, we will test for mode of service delivery effects by estimating the impact of homebuyer education 
2

50 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

AC Appendix C

and counseling on (1) those who were offered remote services (including remote treatment group members 

and choice treatment group members who stated a preference for remote services) and (2) those who 

were offered in-person services (including in-person treatment group members and choice treatment group 

members who stated a preference for in-person services). 

In addition, the interim and final reports will focus on a single confirmatory outcome variable—an indicator 

for whether the enrollee was ever 60 days delinquent on a mortgage loan—but also will provide analyses of 

many other outcomes. As described in this report’s section 6, these future exploratory analyses will estimate 

the impact of homebuyer education and counseling on a wide range of outcomes, test for differential 

impacts across service delivery modes, estimate subpopulation effects, and estimate effects by dosagedefined subgroups. 

This report focuses on all enrollees who were randomly assigned before December 1, 2014, referring to 

them as the study’s “early enrollees.” Future reports will use the full sample, which includes individuals who 

enrolled through the end of January 2016. 

Model for Early Enrollee Sample Impact Analysis 

The difference in treatment and control group outcomes is an unbiased estimate of the impact of offering 

access to treatment. We operationalize this concept by using multiple regression, which uses baseline 

variables to increase the precision with which we estimate the treatment’s impact. The regression model 

used to estimate the impact of being offered homebuyer education and counseling is as follows— 

(1) Yis = α + δTis + βXis + πTimeis + μs + eis,

where

Yis is the outcome interest for individual i in site s.

30

Tis is a dummy variable that equals 1 if individual i in site s is assigned to a treatment group and equals 0 if 

the individual is assigned to the control group.

Xis is a vector of individual background characteristics for individual i in site s.

31

Timeis is a dummy variable equal to 1 if individual i in site s was randomly assigned before September 16, 

2014, and equal to 0 if the individual enrolled on or after this date. This measure controls for differences 

across time periods that could influence the outcome.

μs is a set of site fixed effects.

eis is a random error term.

30 In this case, “site” refers to the 28 large metropolitan areas where study participants enrolled. 

31 Exhibit B.1 provides a detailed description of the baseline covariates included in the impact analysis model. 
2

51 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

AC Appendix C

The coefficient δ provides an intent-to-treat, or ITT, estimate of the impact of being offered homebuyer 

education and counseling—compared with being in the control group—and is the parameter of central 

interest.32,33 Future analyses will consider the effect of the treatment on the treated, or TOT, which provides 

an estimate of the impact of receiving homebuyer education and counseling services. In earlier design plans, 

the study team prespecified the four outcomes examined in this impact analysis. Restricting the preliminary 

impact analysis to these four prespecified outcomes helps guard against fishing for statistically significant 

impacts. Impacts on these select outcomes are presented in exhibit 5.2 in section 5. Exhibit C.1 provides 

additional detail corresponding to these impact estimates. 

32 We cluster standard errors at the site level.

33 For both continuous and binary outcomes, we estimated the equation (1) model using weighted least squares regression, so that the 

interpretation of impact estimates is comparable for different types of outcomes.
C Appendix C

52 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

Exhibit C.1: Impact Estimates, Early Enrollee Sample, Expanded Results

Outcome

Treatment 

Sample 

Size

Control 

Sample 

Size

Treatment 

Group

Control 

Group

Difference 

(Impact)

Percent 

Impact

Standard 

Error p-Value

90 Percent 

Confidence 

Interval

Number of correct 

answers (out of 4) 

to mortgage literacy 

quiza

1,068 858 2.82 2.74 0.08** 2.8% 0.03 0.012 (0.03, 0.12)

If in financial 

difficulty, the 

respondent would 

contact lender for 

assistance before 

missing a mortgage 

paymenta,b (%)

1,056 844 41.8 37.9 3.9* 10.4% 2.31 0.099 (0.1, 7.8)

Respondent has a 

credit score greater 

than or equal to 

620c (%)

1,254 926 88.4 86.1 2.3** 2.7% 0.96 0.022 (0.8, 3.9)

Respondent has a 

budget and often 

compares it with 

actual spendinga

 (%)

1,065 854 35.4 35.6 – 0.2 – 0.6% 1.75 0.896 (– 3.1, 2.6)

** Difference is statistically significant (two-sided t-test) at the p < 0.05 level.

* Difference is statistically significant (two-sided t-test) at the p < 0.10 level.

a

 Outcome was constructed using follow-up survey data. We applied sample weights that adjust for follow-up survey nonresponse for analyses of outcomes collected from follow-up surveys 

(see ”Follow-up Survey Nonresponse Weights” section for details).

b

 Outcome is set equal to 1 if respondent has a mortgage and would contact lender for assistance before missing a mortgage payment and is set equal to 0 if respondent has a mortgage but 

would not contact lender for assistance regarding missed payments or would wait to contact lender until after missed payment. The follow-up survey questions used to construct this outcome 

were asked only of respondents who have a mortgage loan. Therefore, to ensure that this outcome is defined for all enrollees (thereby maintaining the integrity of the experimental design), this 

outcome was set equal to 0 if the respondent does not have a mortgage loan. 

c

 Outcome was constructed using credit bureau data.

Notes: Because of rounding, reported impacts (treatment-control differences) may differ from differences between reported regression-adjusted means for the treatment and control groups. 

To maintain the integrity of the experimental design, all outcomes reported in this exhibit are defined for all enrollees with nonmissing outcome data. Appendix B provides additional detail 

related to construction of measures.

Sources: Baseline survey; 12-month follow-up survey; credit bureau data 
2

53 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

AC Appendix C

Impact Estimates: Pooled Remote and Choice Versus Control Groups 

To understand the impact of homebuyer education and counseling at this early stage of follow-up, we use 

the entire early enrollee sample (with nonmissing data) and report those impact estimates in exhibit 5.2. 

As described previously in this appendix, the in-person treatment group, which experienced comparatively 

low take-up of services, will not be included in the sample used for the study’s future confirmatory impact 

analysis, although it will be included in other analyses. Exhibit C.2 reports impact estimates for the subset 

of the early enrollee sample that includes study participants assigned to the remote and choice treatment 

groups or to the control group. That is, it excludes the in-person treatment group. 

For all four outcomes of interest, we find that the impact estimates reported in exhibit 5.2 for the entire 

early enrollee sample are statistically similar in magnitude to those reported in exhibit C.2 and lead to 

the same general conclusions. Excluding the in-person subsample increases the magnitude of the lender 

contact outcome, but tests of statistical significance show that the two estimates are not different from one 

another. That is, we cannot reject the null hypothesis that the estimated impacts are the same across the 

two samples. 

That said, we observe in exhibit C.2 that the credit score outcome is not statistically significant when the inperson treatment group is excluded from the sample, whereas it is statistically significant when we analyze 

the entire early enrollee sample. This apparent difference might be because of the difference in sample 

size or differences in the composition of study participants analyzed, but the difference in the two point 

estimates is not statistically significant. When outcome data are available for the full sample, future reports 

will examine impacts for the service delivery mode-defined treatment groups separately, comparing them 

explicitly to ascertain which might be relatively more effective at influencing any of these and other impacts. 
C Appendix C

54 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

Exhibit C.2: Impact Estimates, Early Enrollee Sample Excluding the Pre-Redesign In-Person Group, Expanded Results

Outcome

Treatment 

Sample 

Size

Control 

Sample 

Size

Treatment 

Group

Control 

Group

Difference 

(Impact)

Percent 

Impact

Standard 

Error p-Value

90 Percent 

Confidence 

Interval

Number of correct answers 

(out of 4) to mortgage 

literacy quiza

647 858 2.80 2.74 0.06* 2.1% 0.03 0.087 (0.00, 0.11)

If in financial difficulty, the 

respondent would contact 

lender for assistance 

before missing a mortgage 

paymenta,b (%)

640 844 43.2 37.8 5.4** 14.3% 2.45 0.036 (1.4, 9.4)

Respondent has a credit 

score greater than or equal 

to 620c

 (%)

781 926 86.9 86.1 0.9 1.0% 1.08 0.432 (– 0.9, 2.6)

Respondent has a budget 

and often compares it with 

actual spendinga (%)

645 854 35.5 35.1 0.4 1.2% 1.95 0.833 (– 2.8, 3.6)

** Difference is statistically significant (two-sided t-test) at the p < 0.05 level.

* Difference is statistically significant (two-sided t-test) at the p < 0.10 level.

a Outcome was constructed using follow-up survey data. We applied sample weights that adjust for follow-up survey nonresponse for analyses of outcomes collected from follow-up surveys 

(see ”Follow-up Survey Nonresponse Weights” section for details).

b Outcome is set equal to 1 if respondent has a mortgage and would contact lender for assistance before missing a mortgage payment and is set equal to 0 if respondent has a mortgage but 

would not contact lender for assistance regarding missed payments or would wait to contact lender until after missed payment. The follow-up survey questions used to construct this outcome 

were asked only of respondents who have a mortgage loan. Therefore, to ensure that this outcome is defined for all enrollees (thereby maintaining the integrity of the experimental design), this 

outcome was set equal to 0 if the respondent does not have a mortgage loan. 

c

 Outcome was constructed using credit bureau data.

Notes: Because of rounding, reported impacts (treatment-control differences) may differ from differences between reported regression-adjusted means for the treatment and control groups. 

To maintain the integrity of the experimental design, all outcomes reported in this exhibit are defined for all enrollees with nonmissing outcome data. Appendix B provides additional detail 

related to construction of measures.

Sources: Baseline survey; 12-month follow-up survey; credit bureau data
2

55 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

AC Appendix C

Impact Estimates for Remotely Provided Homebuyer Education and Counseling Services 

In this section, we report the estimated impact of being offered remote homebuyer education and counseling 

services. To compute the impact of being offered remote services, we restrict the sample to study 

participants randomly assigned to the remote treatment group or to the control group. Regression-adjusted 

estimates of the effect of remote homebuyer education and counseling are presented in exhibit C.3. Remote 

treatment group members performed better than their control group counterparts on a four-question 

mortgage literacy quiz. Although the point estimates are positive, the estimated impacts on the indicator for 

whether the study participant has a credit score greater than or equal to 620 and the indicator for whether 

the study participant reports that he or she would contact the lender before missing a mortgage payment are 

not statistically significant. These findings are similar in magnitude to those reported in exhibit C.2 for the 

sample that includes the choice group but excludes the in-person group.34 Future reports will estimate the 

impact of remote services on additional outcomes and will test whether the impact of being offered remote 

services is statistically different from the impact of being offered the choice of remote or in-person services. 

34 This finding is perhaps not surprising given that the only difference between the samples used to compute the impact estimates in 

exhibit C.2 versus exhibit C.3 is that the sample used in exhibit C.2 includes the relatively small set of early enrollee sample choice group 

members. 
C Appendix C

56 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

Exhibit C.3: Impact Estimates, Early Enrollee Sample, Remote Treatment Group Versus Control Group Comparison

Outcome

Treatment 

Sample 

Size

Control 

Sample 

Size

Treatment 

Group

Control 

Group

Difference 

(Impact)

Percent 

Impact

Standard 

Error p-Value

90 Percent 

Confidence 

Interval

Number of correct answers 

(out of 4) to mortgage 

literacy quiza

520 858 2.82 2.75 0.07* 2.7% 0.04 0.056 (0.01, 0.13)

If in financial difficulty, the 

respondent would contact 

lender for assistance 

before missing a mortgage 

paymenta,b (%)

513 844 42.4 38.5 3.9 10.1% 2.69 0.161 (– 0.6, 8.3)

Respondent has a credit 

score greater than or equal 

to 620c

 (%)

629 926 86.7 85.9 0.8 0.9% 1.07 0.463 (– 1.0, 2.5)

Respondent has a budget 

and often compares it with 

actual spendinga (%)

519 854 35.7 35.7 0.0 0.1% 2.24 0.991 (– 3.7, 3.7)

* Difference is statistically significant (two-sided t-test) at the p < 0.10 level

a Outcome was constructed using follow-up survey data. We applied sample weights that adjust for follow-up survey nonresponse for analyses of outcomes collected from follow-up surveys 

(see ”Follow-up Survey Nonresponse Weights” section for details).

b Outcome is set equal to 1 if respondent has a mortgage and would contact lender for assistance before missing a mortgage payment and is set equal to 0 if respondent has a mortgage but 

would not contact lender for assistance regarding missed payments or would wait to contact lender until after missed payment. The follow-up survey questions used to construct this outcome 

were asked only of respondents who have a mortgage loan. Therefore, to ensure that this outcome is defined for all enrollees (thereby maintaining the integrity of the experimental design), this 

outcome was set equal to 0 if the respondent does not have a mortgage loan. 

c Outcome was constructed using credit bureau data.

Notes: Because of rounding, reported impacts (treatment-control differences) may differ from differences between reported regression-adjusted means for the treatment and control groups. 

To maintain the integrity of the experimental design, all outcomes reported in this exhibit are defined for all enrollees with nonmissing outcome data. Appendix B provides additional detail 

related to construction of measures. 

Sources: Baseline survey; 12-month follow-up survey; credit bureau data
2

57 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

AC Appendix C

Follow-up Survey Nonresponse Weights

To help guard against imbalances caused by attrition from follow-up survey nonresponse, the analysis 

applies sample weights that adjust for follow-up survey nonresponse for analyses of outcomes collected from 

follow-up surveys. Applying nonresponse weights adjusts only for differences in observed characteristics 

between respondents and nonrespondents. As is true for all survey-based random assignment analyses, we 

cannot rule out the possibility of differential nonresponse bias being introduced in the impact estimates 

because of differences in unobserved characteristics between respondents in the treatment and control 

groups. We generated nonresponse weights as follows.35

• Step 1: We computed the predicted probability that a sample member responded to the follow-up survey 

by modeling response to the follow-up survey as a function of treatment group indicator and the same 

set of covariates included in the equation (1) preliminary impact analysis model described previously. 

For missing baseline covariate data, we again used the dummy variable adjustment approach.

• Step 2: Because the weights created in step 3 are inversely proportional to the probability of survey 

response, weights associated with near-zero probabilities become extremely large, thereby contributing 

to reduced precision. To guard against reduced precision that may result from inclusion of small or large 

weights, in this step, we conducted weight trimming, which bounds the computed predicted probabilities 

of follow-up survey response.36

• Step 3: We computed each survey respondent’s weight by dividing the overall survey response rate by 

the predicted probability of response, as computed in steps 1 and 2. This method ensures that higher 

weights are assigned to individuals with characteristics that were underrepresented (relative to the 

baseline survey sample) among the survey respondent sample. 

• Step 4: Finally, we computed adjusted nonresponse weights by dividing each sample member’s 

nonresponse weight (as computed in step 3) by the overall mean value of the nonresponse weights. 

Scaling the nonresponse weights by the mean ensures that the mean of the adjusted nonresponse 

weights is equal to 1 and that the sum of all adjusted nonresponse weights is equal to the sample size. 

35 This method is commonly used in applied evaluation research and is described by Hsueh et al. (2012), for example.

36 As described by Izrael, Battaglia, and Frankel (2009), weight trimming refers to increasing the value of low weights and decreasing 

the value of high weights to reduce their impact on the variance of the estimates. By trimming low and high weight values, one generally 

lowers sampling variability but may incur some bias. The mean squared error will be lower if the reduction in variance is large relative to the 

increase in bias arising from weight trimming.
58 The First-Time Homebuyer Education and Counseling Demonstration: The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

D

Appendix D: Works Cited

An, Xudong, Raphael W. Bostic, and Vincent W. Yao. 2015. Financial Literacy and Mortgage Credit: 

Evidence From the Recent Mortgage Market Crisis. Working paper. http://dx.doi.org/10.2139/

ssrn.2631948. 

Barron, John, and Michael E. Staten. 2012. Is Technology-Enhanced Credit Counseling as Effective as 

In-Person Delivery? Working Paper No. 2012-WP-05. Terre Haute, IN: Indiana State University, 

Networks Financial Institute. http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2088917.

Collins, Michael, and Collin O’Rourke. 2011. Homeownership Education and Counseling: Do We Know 

What Works? Research Institute for Housing America Special Report. Washington, DC: Mortgage 

Bankers Association, Research Institute for Housing America. http://papers.ssrn.com/sol3/papers.

cfm?abstract_id=1823429.

Cutts, Amy Crews, and William Merrill. 2008. “Interventions in Mortgage Default: Policies and Practices 

To Prevent Home Loss and Lower Costs.” In Borrowing To Live: Consumer and Mortgage Credit 

Revisited, edited by Nicolas P. Retsinas and Eric S. Belsky. Washington, DC: Brookings Institution 

Press: 203–254.

Hart Research Associates. 2014. “How Housing Matters: The Housing Crisis Continues To Loom Large 

in the Experiences and Attitudes of the American Public.” PowerPoint Presentation. Chicago: 

MacArthur Foundation. https://www.macfound.org/media/files/How_Housing_Matters_2014_FINAL_

REPORT.pdf.

Herbert, Christopher E., Daniel T. McCue, and Rocio Sanchez-Moyano. 2013. Is Homeownership Still an 

Effective Means of Building Wealth for Low-Income and Minority Households? (Was it Ever?). 

Working Paper No. HBTL-06. Cambridge, MA: Joint Center for Housing Studies of Harvard 

University.

Hsueh, JoAnn, Desiree Principe Alderson, Erika Lundquist, Charles Michalopoulos, Daniel Gubits, and David 

Fein. 2012. The Supporting Healthy Marriage Evaluation: Early Impacts on Low-Income Families: 

Technical Supplement. OPRE Report 2012-27. Washington, DC: U.S. Department of Health and 

Human Services, Office of Planning, Research and Evaluation, Administration for Children and 

Families.

Isaacs, Julia. 2012. The Ongoing Impact of Foreclosures on Children. Social Genome Project Research, No. 

36. Washington, DC: The Brookings Institution. http://www.brookings.edu/~/media/research/files/

papers/2012/4/18-foreclosures-children-isaacs/0418_foreclosures_children_isaacs.pdf.
2

59 The First-Time Homebuyer Education and Counseling Demonstration: Early Insights

DA Appendix D

Izrael, David, Michael P. Battaglia, and Martin R. Frankel. 2009. “Extreme Survey Weight Adjustment as a 

Component of Sample Balancing (a.k.a. Raking).” In Proceedings: SAS Global Forum 2009. Paper 

No. 247-2009. http://support.sas.com/resources/papers/proceedings09/247-2009.pdf.

Jefferson, Anna, Jonathan Spader, Jennifer Turnham, and Shawn Moulton. 2012. Foreclosure Counseling 

Outcome Study: Final Report. Prepared for the U.S. Department of Housing and Urban 

Development. https://www.huduser.gov/portal//publications/pdf/foreclosure_counseling_v2.pdf.

Puma, Michael J., Robert B. Olsen, Stephen H. Bell, and Cristofer Price. 2009. What To Do When Data 

Are Missing in Group Randomized Controlled Trials. NCEE 2009-0049. Washington, DC: U.S. 

Department of Education, Institute of Education Sciences, National Center for Education Evaluation 

and Regional Assistance. http://ies.ed.gov/ncee/pdf/20090049.pdf.

Rohe, William M., and Mark Lindblad. 2013. Reexamining the Social Benefits of Homeownership After the 

Housing Crisis. Working Paper No. HBTL-04. Cambridge, MA: Joint Center for Housing Studies of 

Harvard University. http://www.jchs.harvard.edu/sites/jchs.harvard.edu/files/hbtl-04.pdf. 

Smith, Marvin M., Daniel Hochberg, and William H. Greene. 2014. The Effectiveness of Pre-Purchase 

Homeownership Counseling and Financial Management Skills. Philadelphia: Federal Reserve Bank 

of Philadelphia.

Theodos, Brett, Margaret Simms, Mark Treskon, Christina Stacy, Rachel Brash, Dina Emam, Rebecca 

Daniels, and Juan Collazos. 2015. An Evaluation of the Impacts and Implementation Approaches 

of Financial Coaching Programs. Washington, DC: Urban Institute. http://www.urban.org/sites/

default/files/alfresco/publication-pdfs/2000448-An-Evaluation-of-the-Impacts-and-ImplementationApproaches-of-Financial-Coaching-Programs.pdf. 

Tsai, Alexander C. 2015. “Home Foreclosure, Health, and Mental Health: A Systematic Review of Individual, 

Aggregate, and Contextual Associations,” PLoS ONE 10 (4): e0123182. DOI: 10.1371/journal.

pone.0123182. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4388711/.

Turnham, Jennifer, Christopher E. Herbert, and Christopher N. Rodger. 2008. The State of the Housing 

Counseling Industry: 2008 Report. Prepared for the U.S. Department of Housing and Urban 

Development. https://www.huduser.gov/portal//Publications/PDF/hsg_counsel.pdf.

Turnham, Jennifer, and Anna Jefferson. 2012. Pre-Purchase Counseling Outcome Study: Research Brief.

Prepared for the U.S. Department of Housing and Urban Development. https://www.huduser.gov/

publications/pdf/pre_purchase_counseling.pdf.
U.S. Department of Housing and Urban Development

Office of Policy Development and Research

Washington, DC 20410-6000

June 2016