Social and Behavioral Sciences Team

Annual Report

Executive Office of the President
National Science and Technology Council

September 2015

SOCIAL AND BEHAVIORAL SCIENCES TEAM 2015 ANNUAL REPORTThis page intentionally left blank

IISOCIAL AND BEHAVIORAL SCIENCES TEAM 2015 ANNUAL REPORTEXECUTIVE OFFICE OF THE PRESIDENTNATIONAL SCIENCE AND TECHNOLOGY COUNCILWASHINGTON, D.C. 20502

Dear Colleagues:
The Obama Administration has made significant progress both in using and building evidence
to improve government performance and deliver better results at a lower cost for the Nation. A
strong body of evidence demonstrates that research insights about behavior, when incorporated
into the design of Federal programs and policies, have significantly improved Americans’ lives,
whether by boosting retirement savings nationwide or helping more low-income students get tocollege each year.
Building on this work, in 2014, the White House Office of Science and Technology Policy assembled the Social and Behavioral Sciences Team (SBST)—a cross-agency group of experts in applied
behavioral science that translates findings and methods from the social and behavioral sciences
into improvements in Federal policies and programs. The Social and Behavioral Sciences Team
Annual Report represents SBST’s successful efforts in its first year to apply behavioral insights
to Federal policy and to measure the impact of these applications on key policy outcomes using
rigorous evaluations.
The report covers two core areas on which SBST focused in its first year: streamlining accessto programs and improving government efficiency. As a result of SBST projects, more Servicemembers are saving for retirement, more students are going to college and better managing theirstudent loans, more Veterans are taking advantage of education and career counseling benefits,
more small farms are gaining access to credit, and more families are securing health insurance
coverage. And improvements in government program integrity and efficiency are saving taxpayer
dollars.
The publication of this report marks an important milestone for the Administration’s policy ofusing behavioral insights to strengthen the ways in which Federal programs and policies serve
the Nation—a policy that was codified in September 2015 by an Executive Order entitled “Using
Behavioral Science Insights to Better Serve the American People.” This Executive Order directs
agencies to: (i) identify policies, programs, and operations for which behavioral insights may yield
substantial improvements in social welfare and program outcomes; (ii) develop strategies for implementing these insights and rigorously testing and evaluating their impacts; (iii) recruit behavioral science experts to join the Federal Government; and (iv) seek opportunities to strengthen
agency relationships with the research community to better utilize findings from the social andbehavioral sciences.
I look forward to seeing further results from the Team’s efforts to leverage behavioral insights toimprove outcomes for the Nation. I am confident that the results in this report represent a solid
foundation on which to build future work.
Sincerely,
John P. Holdren
Assistant to the President for Science and Technology
Director, Office of Science and Technology Policy
III

SOCIAL AND BEHAVIORAL SCIENCES TEAM 2015 ANNUAL REPORT

About the National Science and Technology Council
The National Science and Technology Council (NSTC) is the principal means by which the
Executive Branch coordinates science and technology policy across the diverse entities thatmake up the Federal research and development (R&D) enterprise. One of the NSTC’s primaryobjectives is establishing clear national goals for Federal science and technology investments.
The NSTC prepares R&D packages aimed at accomplishing multiple national goals. The NSTC’s
work is organized under five committees: Environment, Natural Resources, and Sustainability;
Homeland and National Security; Science, Technology, Engineering, and Mathematics (STEM)
Education; Science; and Technology. Each of these committees oversees subcommittees andworking groups that are focused on different aspects of science and technology. More information is available at www.whitehouse.gov/ostp/nstc.
About the Office of Science and Technology Policy
The Office of Science and Technology Policy (OSTP) was established by the National Scienceand Technology Policy, Organization, and Priorities Act of 1976. OSTP’s responsibilities include
advising the President in policy formulation and budget development on questions in which science and technology are important elements; articulating the President’s science and technology
policy and programs; and fostering strong partnerships among Federal, state, and local governments, and the scientific communities in industry and academia. The Director of OSTP also
serves as Assistant to the President for Science and Technology and manages the NSTC. Moreinformation is available at www.whitehouse.gov/ostp.
About the Subcommittee on the Social and Behavioral Sciences TeamThe Subcommittee on the Social and Behavioral Sciences Team (SBST) contributes to the activities of NSTC’s Committee on Technology (CoT). SBST’s purpose is to coordinate the application of social and behavioral science research to help Federal agencies advance their policy andprogram goals and better serve the Nation. SBST works to identify opportunities for Federal
agencies to leverage social and behavioral science insights to advance the goals of their policies
and programs, demonstrate the impact of these applications, and build capacity for applicationsof social and behavioral science across Federal agencies.
About this Document
This document was developed by the Subcommittee on the Social and Behavioral SciencesTeam. The document was published by OSTP.
Copyright Information
This document is a work of the United States Government and is in the public domain (see 17
U.S.C. §105). Subject to the stipulations below, it may be distributed and copied with acknowledgement to OSTP. Copyrights to graphics included in this document are reserved by the original copyright holders or their assignees and are used here under the government’s license andby permission. Requests to use any images must be made to the provider identified in the image
credits or to OSTP if no provider is identified.
Printed in the United States of America, September 2015.
IV

SOCIAL AND BEHAVIORAL SCIENCES TEAM 2015 ANNUAL REPORT

Acknowledgements
Many individuals across the Federal Government contributed to this report and the work it describes. Above all, none of the program improvements described here would have been possible
without the creativity and diligence of the agency and program staff who collaborated on thiswork.
Particular recognition also goes to a set of individuals who at various points devoted a dedicatedportion of the their time to the efforts of the SBST, including: Kelly Bidwell, Hyunsoo Chang,Amira Choueiki, William J. Congdon, Natalie Dion, Irina Feygina, Lori Foster, Nathaniel Higgins, Daniel J. Hopkins, Zhanrui Kuang, Leah Lozier, Prashant Malaviya, Robert D. Metcalfe,
Samuel Quinney, Elana Safran, Maya Shankar, Mona Siddiqui, Will Tucker, and David Yokum.
Hannah Safford in the Office of Science and Technology Policy also provided valuable assistancewith the production of this report. Finally, SBST wishes to thank the leadership and staff of GSA
for their continued support of SBST, with special thanks due to Adam Neufeld, Christine Harada, Giancarlo Brizzi, and Kevin Donahue, the Executive Director of the Performance Improvement Council from 2012 to 2015.

V

SOCIAL AND BEHAVIORAL SCIENCES TEAM 2015 ANNUAL REPORTThis page intentionally left blank

VI

SOCIAL AND BEHAVIORAL SCIENCES TEAM 2015 ANNUAL REPORTNATIONAL SCIENCE AND TECHNOLOGY COUNCILCOMMITTEE ON TECHNOLOGYSUBCOMMITTEE ON THE SOCIAL AND BEHAVIORAL SCIENCES TEAM

National Science and Technology Council
Chair

Staff

John P. Holdren

Afua Bruce

Assistant to the President for Scienceand Technology and Director,
Office of Science and Technology Policy

Executive Director (as of June 8th, 2015)
Office of Science and Technology Policy

Jayne Morrow

Executive Director (through June 7th, 2015)
Office of Science and Technology Policy

Committee on Technology
Chair

Staff

Thomas Kalil

Terah Lyons

Deputy Director for Technology and Innovation
Office of Science and Technology Policy

Executive Secretary
Office of Science and Technology Policy

Subcommittee on the Social and Behavioral Sciences Team
Chair

Staff

Maya Shankar

Amira Choueiki

Senior Advisor for the Social and Behavioral Sciences
Office of Science and Technology PolicyVII

Executive Secretary
General Services Administration

SOCIAL AND BEHAVIORAL SCIENCES TEAM 2015 ANNUAL REPORT

Subcommittee Members
The following Federal departments and agencies are represented on the SBST andthrough it, work together to coordinate the application of social and behavioral science
research to advance policy and program goals and better serve the Nation:
Department of Agriculture
Department of Defense
Department of Education
Department of Energy
Department of Health and Human Services
Department of Housing and Urban Development
Department of JusticeDepartment of Labor
Department of the Treasury
Department of Veterans Affairs
General Services Administration (Executive Secretariat)
National Science Foundation
Social Security Administration
United States Agency for International Development
The following offices of the Executive Office of the President are also represented on the
SBST:
Council of Economic Advisers
Domestic Policy Council
National Economic Council
Office of Management and Budget
Office of Science and Technology Policy (Chair)

VIII

Social and Behavioral
Sciences Team2015 Annual Report
Executive Office of the President
National Science and Technology Council

This page intentionally left blankXExecutive Summary
The Federal Government administers a wide array of programs on behalf of the American people: financial aid toassist with college attendance, social insurance programsand tax incentives to promote retirement security, healthinsurance programs to ensure access to healthcare and financial protection for families, disclosure requirements to
help people obtain safer mortgages, and others.

American families.3 Similarly, the Pension Protection Act
of 2006, which codified the practice of automatically enrolling workers into retirement savings plans, is based on
behavioral economics research showing that switchingfrom an opt-in to an opt-out enrollment system dramatically increases participation rates.4 Since the implementation of this policy, automatic enrollment and automatic
escalation have led to billions of dollars in additional savings by Americans.5

But Americans are best served by these programs only if
the programs are easy to participate in and present options and information clearly. When programs are designed without these considerations in mind, Americans
can incur costs that go beyond lost time and frustration.
Research from behavioral science demonstrates thatseemingly small barriers to engagement—such as hardto-understand information, burdensome applications, or
poorly presented choices—can prevent programs fromworking effectively for the very people they are intended
to serve.1

In 2014, the White House Office of Science and Technology Policy (OSTP) established the Social and Behavioral
Sciences Team (SBST) to ensure that our best understanding of behavior—how people engage with, participate in,and respond to policies and programs—is integrated into
the policymaking process. SBST is a cross-agency group of
experts in applied behavioral science that translates findings and methods from the social and behavioral sciences
into improvements in Federal policies and programs forthe benefit of the American people. SBST works to identify how behavioral insights can be integrated into Federal
agency programs in order to help agencies achieve their
missions and objectives.

For example, one behavioral science study found that acomplex application process for financial aid was morethan just a hassle: it discouraged applications for aid, andactually led some students to delay or forgo going to college altogether.2 Fortunately, behavioral science not only
identifies aspects of programs that can act as barriers toengagement, but also provides policymakers with insight
into how those barriers can be removed. That same study
showed that streamlining the process of applying for financial aid—by providing families with application assistance and enabling families to automatically fill partsof the application form using information from their tax
return—increased the rates of both aid applications andcollege enrollment.

As detailed in this report, SBST efforts are already paying
dividends for Americans. Due to SBST projects, more Servicemembers are saving for retirement, more students aregoing to college and better managing their student loans,more Veterans are taking advantage of education andcareer counseling benefits, more small farms are gaining
access to credit, and more families are securing health insurance coverage. And improvements in government program integrity and efficiency are saving taxpayer dollars.
Results from SBST’s First Year

When behavioral insights—research findings from behavioral economics and psychology about how people
make decisions and act on them—are brought into policy,
the returns are significant. Because the Federal Government leveraged insights from the financial aid study andsimplified the process of applying for Federal student
aid, college is now more readily accessible to millions ofOver its first year, SBST focused on executing proof-ofconcept projects where behavioral insights could be embedded directly into programs at a low cost and lead to
immediate, quantifiable improvements in program outcomes. To generate reliable evidence about the effectiveness of integrating behavioral insights into programs, SBST

1

Daniel Kahneman, “Maps of Bounded Rationality: Psychology for Behavioral Economics,” American Economic Review 93 (2003): 1449–1475. Richard H. Thaler and Cass R. Sunstein, Nudge (Yale, 2008).
2
Eric P. Bettinger, Bridget Terry Long, Philip Oreopoulos, and Lisa Sanbonmatsu, “The Role of Application Assistance and Information in College
Decisions: Results from the H&R Block Fafsa Experiment,” Quarterly Journal of Economics 127 (2012): 1205–1242.
3
U.S. Department of Education, “Fiscal Year 2016 Budget Summary and Background Information,” (2015), p. 40.
4
Brigitte C. Madrian and Dennis F. Shea, “The Power of Suggestion: Inertia in 401(k) Participation and Savings Behavior,” Quarterly Journal of Economics 116 (2001): 1149–1187. Pension Protection Act of 2006, Public Law 109-280, U.S. Statutes at Large 120 (2006): 780–1172.
5
Shlomo Benartzi and Richard H. Thaler, “Behavioral Economics and the Retirement Savings Crisis,” Science 339 (2013): 1152–1153.

XI

designed these projects, in nearly all cases, as randomized
trials. This report presents the results of all completed
SBST projects, including projects that did not yield statistically significant improvements.

insights from this pilot across bases and installations withhigher troop concentrations.
• To assist nearly 140,000 Servicemembers who were required to re-enroll in their Roth TSP in order to continue
making contributions, SBST and DOD redesigned anemail that alerted Servicemembers of the requirement
to re-enroll. The redesigned email led 22 percent more
Servicemembers to re-enroll in TSP within a week—
3,770 more re-enrollments than among those sent a
standard message. Based on this result, DOD immediately scaled up the successful messaging in subsequent
outreach reminding Servicemembers to re-enroll in TSP.

Because SBST projects are designed to address only the
behavioral barriers that affect how people engage with
programs, project effects can be modest. Yet, because behavioral changes to program administration often require
little or no additional cost, returns on investment can belarge even when project effects are small. It is no more
expensive to send an effective version of an email than an
ineffective one.

Improving College Access and Affordability

In building an initial portfolio of work, SBST focused on
projects in two areas where behavioral science had astrong role to play and impacts could be demonstrated
relatively rapidly: (1) streamlining access to programs and(2) improving government efficiency.

• To help students enroll in college, SBST and the Department of Education’s (ED) office of Federal Student Aid(FSA) provided technical expertise to researchers andthe nonprofit uAspire on messages notifying collegeaccepted, high school graduates of required pre-matriculation tasks for their respective colleges. A seriesof eight personalized text messages to low-income students reminding them to complete these tasks led to a
5.7 percentage point increase in college enrollment,from 66.4 to 72.1 percent.

Streamlining Access to Programs
Working closely with agencies across the Federal Government, SBST used behavioral insights to expand program
reach and promote retirement security, increase college
access, connect workers and small businesses with economic opportunities, and improve health outcomes.

• To help Federal student loan borrowers stay on top of
their payments, SBST and FSA sent a reminder email toover 100,000 borrowers who had missed their first payments. The reminder email led to a 29.6 percent increase in the fraction of borrowers making a payment
in the first week after it was sent, from 2.7 to 3.5 percent.

Promoting Retirement Security• To promote participation in the Thrift Savings Plan
(TSP), a workplace savings plan for Federal employees,
SBST and the Department of Defense (DOD) launched
an email campaign that sent approximately 720,000
unenrolled Servicemembers one of nine email variants,
designed using behavioral insights, notifying recipientsof the opportunity to participate in TSP. Compared to no
message, the most effective message nearly doubled
the rate at which Servicemembers signed up for TSP.
Emails informed by behavioral insights led to roughly
4,930 new enrollments and $1.3 million in savings injust the first month after the emails were sent. DOD
is now scaling up this intervention by sending periodic
emails informed by behavioral insights to Servicemembers about the benefits of TSP.

• To increase awareness of income-driven repayment(IDR) plans among student loan borrowers, SBST and
FSA sent an informational email about IDR plans to over
800,000 borrowers who had fallen behind on their payments. The low-cost, timely message led to a fourfold
increase in applications for income-driven repayment
plans, with 4,327 applications for IDR made withintwenty days of the email being sent.
Advancing Economic Opportunity• To increase Veterans’ uptake of education and career
counseling benefits, SBST and the Department of Veterans Affairs (VA) sent notices informing Veterans of
their benefits and the steps needed to apply. Relative tosimply notifying them of their eligibility, highlighting to
Veterans that they had earned the benefits led nearly
9 percent more Veterans to access the application for
those benefits. This increase represents only 146 additional Veterans clicking through to the application form,
however, suggesting that barriers to accessing this program lie elsewhere.

• To further promote saving, SBST and DOD prompted
Servicemembers to make a “Yes” or “No” choice about
whether or not to contribute to TSP during an orientation briefing upon their arrival at a new military base.
More Servicemembers enrolled in TSP during theprompted choice pilot: 8.7 percent of non-enrolled
Servicemembers, compared to 2.9 percent on average at three comparison bases, and 4.3 percent during comparison time periods at the pilot base. Basedon the success of this pilot, DOD is committing to applying
XII• To improve economic outcomes for small-scale andspecialized farmers, SBST and the United States Department of Agriculture (USDA) collaborated on a campaign
to increase knowledge and utilization of loan options.
Farms that were sent a personalized letter with acustomized set of action steps for applying for a Microloan were 22 percent more likely to obtain a loan,
representing an increase from 0.09 to 0.11 percent.

• To reduce inappropriate prescribing of controlled substances, SBST and HHS’s Centers for Medicare andMedicaid Services (CMS) sent providers with unusuallyhigh billing patterns a letter comparing their prescribing
rates with the prescribing rates of their peers, as well as
educational information about proper prescribing practices. No measurable impact was seen on prescription
rates.

Helping Families Get Coverage and Stay Healthy

Ensuring Cost-Effective Program Operations• To assist individuals and families with obtaining healthinsurance, SBST and the Department of Health andHuman Services (HHS) sent one of eight behaviorally
designed letter variants to each of more than 700,000
individuals who had already begun the enrollment process, but had not yet completed an application. Thosesent the most effective version of the letter were 13.2
percent more likely to enroll in health insurance thanthose sent no letter, with enrollment rates of 4.56 and4.03 percent, respectively.

• To determine if letters could encourage security holders
to transfer accounts to an online platform, SBST collaborated with the Department of the Treasury’s Bureau of
the Fiscal Service to design outreach to account holders.
Letters that included a personal appointment time witha call center led to 23 percent higher call-in rates thanstandard letters, representing an increase from 10.6 to
13.0 percent. Account conversion rates remained low
for both groups, however. Based in part on this result
there are no further mailings planned.
• To increase response rates to a workplace survey that
determines Federal office space management strategies, SBST and the GSA worked together to incorporate
behavioral insights into the timing and messaging of
emails announcing the survey. Among other findings,
email click rates were highest at lunchtime, with 15.3
percent of emails sent at 11:55 a.m. resulting in recipients clicking through to the survey, compared with 13.3
percent at 8:55 a.m.

Improving Government Efficiency
Many measures of government efficiency depend on thedecisions and actions of individuals such as program officials, beneficiaries, and contractors. SBST applied behavioral insights to improve program integrity, save money on
operational expenses, and identify programs the Government could run more efficiently.
Promoting Program Integrity and Compliance• To encourage double-sided printing, SBST and the
USDA’s Economic Research Service (ERS) created a dialog box that asked employees to change their default
printer setting to double-sided after employees had initiated a single-sided print job. This prompt increased
the likelihood of double-sided printing by 5.8 percentage points, from a baseline of 46 percent. Based on thisfinding, ERS plans to change the default setting of all
printers to double-sided.

• To improve the accuracy of sales figures self-reported
by vendors selling goods and services to the Government, SBST and the General Services Administration(GSA) redesigned an online data-entry form to include
a signature box at the top of the page in which the user
had to confirm the accuracy of self-reported sales. Because vendors pay to the Government a small fee basedon those sales reports, introducing this box led to an additional $1.59 million in fees collected within a single
quarter. Based on this result, GSA is making permanentchanges to the form to incorporate a signature box.

SBST’s initial portfolio of work has led to real improvements in outcomes that matter for Americans and demonstrated that Federal agencies can successfully integrate
behavioral insights into their programs. The results of
SBST projects completed to date represent a foundation onwhich the Federal Government, can, moving forward, basedecisions about both program administration and policy
design. Based on results from SBST pilots, agencies havealready made lasting reforms to program administrationin order to better serve the American people.

• To increase debt recovery from individuals with outstanding non-tax debt, SBST worked with the Department of
the Treasury’s Debt Management Service (DMS) to redesign a collection letter. No difference in payment rates
was observed, but changes such as shortening the web
address for making an online payment led 45 percent
more individuals to pay online, representing an increase
from 1.5 to 2.2 percent. DMS has now permanently
shortened the web link in the collection letter.

XIII

This page intentionally left blank

XIVTable of Contents
I. Introduction

1

II. Streamlining Access to Programsa. Promoting Retirement Security
b. Improving College Access and Affordabilityc. Advancing Economic Opportunityd. Helping Families Get Health Coverage and Stay Healthy

6
6
9
11
12

III. Improving Government Efficiency
a. Promoting Program Integrity and Complianceb. Ensuring Cost-Effective Program Operations

14
15
17

IV. What’s Next

19

Appendices		
a. Executive Order		23b. Project Abstracts		
29

XV

This page intentionally left blank

XVI

Members of SBST brief President Obamain the Oval Office in January 2015

I. Introduction“A growing body of evidence demonstrates that behavioral science insights—research findings from fields
such as behavioral economics and psychology about
how people make decisions and act on them—can beused to design government policies to better serve theAmerican people...By improving the effectiveness andefficiency of Government, behavioral science insights
can support a range of national priorities, including
helping workers to find better jobs; enabling Americans to lead longer, healthier lives; improving accessto educational opportunities and support for success inschool; and accelerating the transition to a low carbon
economy.”The Federal Government administers a wide array of programs
on behalf of the American people: financial aid to assist withcollege attendance, social insurance programs and tax incentives to promote retirement security, health insurance programs to ensure access to healthcare and financial protection
for families, disclosure requirements to help people obtainsafer mortgages, and others.
But Americans are best served by these programs only if
the programs are easy to participate in and present options and information clearly. When programs are designed
without these considerations in mind, Americans can incur
costs that go beyond lost time and frustration. Research frombehavioral science demonstrates that seemingly small barriers to engagement—such as hard-to-understand information,burdensome applications, or poorly presented choices—canprevent programs from working effectively for the very people
they are intended to serve.1— Barack Obama, Executive Order entitled “Using Behavioral
Science Insights to Better Serve the American People”

1

Daniel Kahneman, “Maps of Bounded Rationality: Psychology for Behavioral Economics,” American Economic Review 93 (2003): 1449–1475. Richard H. Thaler and Cass R. Sunstein, Nudge (Yale, 2008).

1

Consider, as an example, financial aid for college. One behavioral science study found that a complex application process
for financial aid was more than just a hassle: it discouraged
applications for aid, and actually led some students to delay
or forgo going to college altogether.2 Fortunately, behavioral
science not only identifies aspects of programs that can act
as barriers to engagement, but also provides policymakers
with insight into how those barriers can be removed. Thatsame study showed that streamlining the process of applying
for financial aid—by providing families with application assistance and enabling families to automatically fill parts of theapplication form using information from their tax return—increased the rates of both aid applications and college enrollment.

that translates findings and methods from the social
and behavioral sciences into improvements in Federal
policies and programs for the benefit of the American
people. SBST works to identify how behavioral insights
can be integrated into Federal agency programs in orderto help agencies achieve their missions and objectives.
SBST is chaired by OSTP and has representation froma dozen member agencies across the Federal Government, as well as offices within the Executive Office of
the President (EOP). SBST also receives critical support
from the General Services Administration’s (GSA) Officeof Evaluation Sciences (OES), which recruits behavioral
science experts from within and outside of government
to work on SBST efforts.

When behavioral insights—research findings from behavioral economics and psychology about how people make decisions and act on them—are brought into policy, the returns
are significant. Because the Federal Government leveraged
insights from the financial aid study and simplified the process of applying for Federal student aid, college is now more
readily accessible to millions of American families.3 Similarly,
the Pension Protection Act of 2006, which codified the practice of automatically enrolling workers into retirement savings plans, is based on behavioral economics research showing that switching from an opt-in to an opt-out enrollment
system dramatically increases participation rates.4 Since theimplementation of this policy, automatic enrollment and automatic escalation have led to billions of dollars in additionalsavings by Americans.5

As detailed in this report, SBST efforts are already paying
dividends for Americans. Due to SBST projects, more Servicemembers are saving for retirement, more students
are going to college and better managing their student
loans, more Veterans are taking advantage of educationand career counseling benefits, more small farms are
gaining access to credit, and more families are securing
health insurance coverage. And improvements in government program integrity and efficiency are saving taxpayer
dollars.
Building on SBST’s successful first year, on September 15,
2015, President Obama signed an Executive Order entitled “Using Behavioral Science Insights to Better Serve
the American People.”

Social and Behavioral Sciences Team

Executive Order: Using Behavioral Science Insights to
Better Serve the American People

An effective and efficient government must reflect ourbest understanding of behavior—how people engage
with, participate in, and respond to policies and programs. But scientific advances do not translate into policy improvements without a coordinated effort. Policy design and program administration must actively integrate
behavioral science research findings, and promising findings must be adapted to a government context.

The Executive Order formally establishes SBST and directs Federal agencies to integrate behavioral insights
into their policies and programs. Specifically, the Orderdirects agencies to: (i) identify policies, programs, andoperations where applying behavioral science insights
may yield substantial improvements in public welfare,program outcomes, and program cost effectiveness; (ii)develop strategies for applying behavioral science insights to programs and, where possible, rigorously test
and evaluate the impact of these insights; (iii) recruit
behavioral science experts to join the Federal Govern-

To meet this need, in 2014 the White House Office of
Science and Technology Policy (OSTP) established the Social and Behavioral Sciences Team (SBST). SBST is a crossagency group of experts in applied behavioral science2

Eric P. Bettinger, Bridget Terry Long, Philip Oreopoulos, and Lisa Sanbonmatsu, “The Role of Application Assistance and Information in College
Decisions: Results from the H&R Block Fafsa Experiment,” Quarterly Journal of Economics 127 (2012): 1205–1242.
3
U.S. Department of Education, “Fiscal Year 2016 Budget Summary and Background Information,” (2015), p. 40.
4
Brigitte C. Madrian and Dennis F. Shea, “The Power of Suggestion: Inertia in 401(k) Participation and Savings Behavior,” Quarterly Journal of Economics 116 (2001): 1149–1187. Pension Protection Act of 2006, Public Law 109-280, U.S. Statutes at Large 120 (2006): 780–1172.
5
Shlomo Benartzi and Richard H. Thaler, “Behavioral Economics and the Retirement Savings Crisis,” Science 339 (2013): 1152–1153.

2

ment; and (iv) strengthen agency relationships with theresearch community to better use empirical findings
from the behavioral sciences.

ciaries, by giving greater consideration to ways in which
information format, timing, and medium can affect understanding.
• Where programs and policies offer choices, SBST works tocarefully consider how the presentation and structure ofthose choices, including default settings and the number
and arrangement of options can empower participants tomake the best choices for themselves and their families.

To implement this directive, this Executive Order charges
SBST with providing policy guidance and advice to help
agencies achieve these objectives. Pursuant to the Executive Order, SBST focuses on four areas where priorresearch and practice indicate that behavioral insights
play an especially strong role in program outcomes: 6• SBST looks for opportunities to help qualifying individuals,
families, and businesses access programs and benefits bystreamlining processes that may otherwise limit participation.

• Where policies create incentives to take specific actions,such as saving for retirement, SBST considers how thefrequency, presentation, and labeling of benefits, taxes,
subsidies, and other incentives can more effectively andefficiently promote those actions, with a particular focus
on opportunities to use nonfinancial incentives.

• SBST works to improve how the government presents information to consumers, borrowers, and program benefi-

SBST works with agencies to develop specific recommendations for modifying elements of program design or administra-

Identifying Federal Applications of Behavioral Insights

PolicyGoals

Individual
Behavior

Applying behavioral insights in the right context canlead to substantial improvements in program outcomes. Given the vast array of policies and programs
across the Federal Government, and the full rangeof their objectives, how does SBST identify where to
apply behavioral insights? SBST has found that three
key elements define the most promising opportunities to apply behavioral insights:

ProgramTouchpoints

sence of an SBST project. When Federal policy outcomes do not depend on behavior, applications of
behavioral insights will have little relevance, no matter how worthwhile the goal.
Program Touchpoints: SBST projects operate at apoint of direct interaction between the Federal Government and individuals. This point could be an application process or form, a website that offers program choices, publicly available information about
government resources, or the structure of incentives(both financial and nonfinancial). SBST explores how
behavioral insights can inform that interaction in waysthat can potentially improve outcomes. If the goal isto help borrowers manage their student debt, for example, the application process for alternative repayment plans represents a touchpoint where behavioral
insights could help borrowers choose the option thatis right for them.

Policy Goals: SBST projects support well-definedpolicy goals or already-identified program challenges—for example, improving program integrity in
Medicare to save taxpayer dollars. Successful projects require clear goals in part because SBST itself
does not set policy.
Individual Behavior: SBST projects identify a link
between a policy goal or program outcome and thedecisions or actions of individuals. This link is the es6

Raj Chetty, “Behavioral Economics and Public Policy: A Pragmatic Perspective,” American Economic Review 105 (2015): 1–33; Eldar Shafir, ed.,The Behavioral Foundations of Public Policy, (Princeton University Press, 2012); William J. Congdon, Jeffrey R. Kling, and Sendhil Mullainathan,
Policy and Choice: Public Finance through the Lens of Behavioral Economics, (Brookings Institution, 2011).
7
Brigitte C. Madrian, “Applying Insights from Behavioral Economics to Policy Design,” Annual Review of Economics 6 (2014): 663–688.

3

tion, based on findings from behavioral science research.7
Close collaboration with these agencies facilitates adaptation of these insights to the specific context of each program.

gold standard of evidence among policymakers and social
scientists because they allow for robust causal inference—the estimated impact can be said to be due to the implemented change, rather than other factors. And because
these methods identify both what works and what does
not, SBST reports the results of all of its completed projects,
including projects that did not yield statistically significant
improvements.10

For further discussion of how SBST works with agencies to
identify promising applications of behavioral insights, see“Identifying Federal Applications of Behavioral Insights.”
Results from SBST’s First YearFor additional discussion on the role of evidence in SBST’s
application of behavioral insights, see “Behavioral Insights
and the Role of Evidence.”

The report that follows presents results from all fully completed SBST projects. In building an initial portfolio of work toapply behavioral insights to improve program and policy outcomes, SBST drew on lessons from many sources—academic
research, precedents at the Federal level, and beyond.8 In
particular, SBST builds on the efforts of pioneering agencies, such as the Department of Health and Human Services(HHS), the Department of Agriculture (USDA), and the Department of Labor (DOL), which are already using behavioral
insights to assist low-income families, promote healthy foodchoices, and design better labor market policies.9

As the results detailed in this report show, SBST projects
have led to real improvements in outcomes that matterfor Americans, and have demonstrated that Federal agencies can successfully integrate behavioral insights into theirprograms. Because these projects are designed to address
only the behavioral barriers that affect how people engage
with programs, project effects can be modest. For example,
there are many reasons why people may choose not to enroll in a retirement savings program. A behavioral insights
project that sends an email notice about the program mightmake a difference for someone who had been intending toenroll, but just needed a reminder. But it would not be expected to make a difference for someone who hadn’t enrolled because, say, he or she was focused instead on savingfor other goals, like a down payment on a house. Yet, because behavioral changes to program administration often
require little or no additional cost, returns on investment
can be large even when project effects are small. It is no
more expensive to send an effective version of an email
than an ineffective one.

Over its first year, SBST focused on executing proof-of-concept projects in two areas where behavioral insights could
be embedded directly into programs at a low cost andwhere quantifiable improvements in program outcomes
could be demonstrated relatively rapidly:
1. Streamlining access to programs: Projects to streamline access to programs in retirement security, education,
economic opportunity, and health care.
2. Improving government efficiency: Projects to improve program integrity, promote efficiency among Federal employees, and help officials make cost-effective
decisions.

The results of SBST projects completed to date represent afoundation on which the Federal Government, can, moving
forward, base decisions about both program administrationand policy design. Agencies have already made substantive
reforms to program administration based on results fromSBST pilots. The results in this report contribute to establishing a compelling evidence base for why, how, and where
Federal policymakers should consider further applicationsof behavioral insights to program design.

To generate reliable evidence about the effectiveness of integrating behavioral insights into programs, SBST designed
these projects, in nearly all cases, as randomized trials (details about the methods used on each project are givenin the project abstracts collected below, “Appendix: SBST
Project Abstracts”). Randomized trials are considered the8

The idea, for example, of creating a central, Federal group focused on behavioral economics goes back to at least 2004, when a session wasorganized by Richard Thaler at the 2004 Annual Meeting of the American Economic Association entitled, “Memos to the Council of BehavioralEconomics Advisors.” SBST has also drawn inspiration from international governments, most notably the United Kingdom, which have createdsimilar teams. See: www.behaviouralinsights.co.uk/
9
For more on the HHS efforts, see the Behavioral Interventions to Advance Self-Sufficiency (BIAS) project, at: www.acf.hhs.gov/programs/opre/
research/project/behavioral-interventions-to-advance-self-sufficiency, For more on the USDA efforts, supporting research centers on behavioral
economics and nutrition, see: www.ers.usda.gov/topics/food-choices-health/food-consumption-demand/behavioral-economics, For more on the
DOL efforts, investigating Behavioral Interventions for Labor Related Programs, see: www.dol.gov/asp/evaluation/currentstudies/7.htm
10Unless otherwise noted, all impact estimates reported below are statistically significant at the 5 percent level; all abstracts also report the 95
percent confidence interval on reported impact estimates.

4

Behavioral Insights and the Role of Evidence
At its core, SBST’s role is one of translation: from academic research findings to pragmatic program solutions. But designing the best way to communicate withthe public, to structure choices and incentives, and tohelp qualifying individuals access public programs, is
not as simple as conducting a review of the scientific
literature. Decades of behavioral research reveals that
how individuals respond to the presentation of information or the structure of choices, for example, is highly context specific.1

dard of evidence among policymakers and social scientists because they allow for robust causal inference: theestimated impact can be said to be due to the implemented change, rather than any other factors.
This type of evaluation often is worthwhile because
the effects of these projects can be meaningful and thecosts of the evaluation can be low. Behavioral science
indicates that small details of program administration—
the timing of a mailing or the order of choices presented on a website—can matter more for program outcomes than might be expected.2 Randomized trials thattest the impact of behavioral insights such as these canoften be embedded directly into the administration ofa program at little cost, and impacts can be observed atlittle cost using existing administrative data. Even when
impacts are modest, they can return savings or benefits
that far outweigh the expense of either the intervention or the evaluation.

As a result, this process of translation requires constantevaluation and feedback. SBST works with agencies
to, where possible, rigorously test the impact of these
insights on program outcomes before implementing
them widely. In this way, SBST can learn about whatworks, what works best, and what does not work. To
achieve this goal, SBST often implements randomized
trials. Randomized trials are considered the gold stan1

Daniel Kahneman and Amos Tversky, “Prospect Theory: An Analysis of Decision Under Risk,” Econometrica 47 (1979): 263–291. Amos
Tversky and Itamar Simonson, “Context-Dependent Preferences,” Management Science 39 (1993): 1179–1189.
2
Richard H. Thaler and Cass R. Sunstein, Nudge (Yale, 2008).

5

II. Streamlining Access to Programs
Enrollment in Thrift Savings Plans

People can realize the benefits of Federal Government policies and programs only if they can access them. Behavioral
science research demonstrates that seemingly small barriers
to engagement, such as hard-to-understand information,burdensome applications, or poorly presented choices, canprevent programs from effectively reaching the very people
they are intended to serve.11

To help Servicemembers interested in saving for the future
participate in TSP, SBST and DOD launched an email campaign notifying non-enrolled Servicemembers of the opportunity to sign up for TSP, the potential benefits of investing,and the steps required to enroll.12 Research demonstrates
that timely, clear messages of this kind can help individuals
better understand their options, make more informed decisions, and follow through on their intentions.13 Prior to thispilot, DOD did not routinely send targeted communications
about TSP to non-enrolled Servicemembers.

SBST used behavioral insights to streamline access to programs that promote retirement security, increase college
access, connect workers and small businesses with economic opportunities, and improve health outcomes.
a. Promoting Retirement Security

Together with DOD, SBST designed eight email variants withdifferent combinations of features based on behavioral insights. These features included: providing clear steps to enrollment, framing the decision to enroll as a choice between
two options (“Yes, I want to enroll” or “No, I do not want toenroll”), displaying projected financial returns to investing in
TSP, and making salient either the short- or long-term benefits of participating in TSP.14 SBST also designed a ninth email
variant with language drawn from the TSP website.

Retiring with financial security is a widely shared goal among
workers. Federal policies support this goal by encouraging
private savings, protecting workplace pensions, and providing Social Security retirement benefits. Behavioral insights
such as automatic enrollment, where the default setting is
for workers to be enrolled in a workplace savings plan andaction is required on the part of workers only if they choose
not to participate, have already helped millions of workers in
the private sector save for their retirement. The civilian workforce of the Federal Government has benefited as well; since2010, civilian agencies have automatically enrolled new hires
in a defined contribution plan, the Thrift Savings Plan (TSP).

Figure 1: Servicemember TSP Sign-Up Rates in May 2015
2.5%

2.10%
1.91%

2.0%

Unlike their civilian counterparts, Servicemembers are notautomatically enrolled by the Federal Government in TSP.
While 87 percent of civilian Federal employees are enrolled
in TSP, just 42 percent of active duty Servicemembers areenrolled. To make TSP more accessible for interested Servicemembers, SBST worked with the Department of Defense(DOD) on a series of pilots informed by behavioral insightsthat encouraged participation in TSP and prevented alreadyenrolled Servicemembers from losing access to TSP benefits.

1.56%
1.5%

1.14%

1.0%
0.5%

Tre T
atm opent

Aver
Tre age
atm ofent

Stan
Medard
ssa
ge

N
Em o
ail

0.0%

Notes: Percentage of Servicemembers enrolling in TSP in May 2015.
Error bars display 95 percent confidence intervals. All differences are
statistically significant at the 5 percent level. n = 806,861.

11

Marianne Bertrand, Sendhil Mullainathan, and Eldar Shafir, “Behavioral Economics and Marketing in Aid of Decision Making Among the Poor,” Journalof Public Policy & Marketing 25 (2006): 8–23.
12Further details on this and other reported projects are provided in Appendix B: Project Abstracts.
13
See, e.g.: Saurabh Bhargava and Dayanand S. Manoli, “Psychological Frictions and the Incomplete Take-Up of Social Benefits: Evidence from an IRS Field
Experiment,” American Economic Review (forthcoming).
14
John Beshears, James J. Choi, David Laibson, and Brigitte C. Madrian, “Simplification and Saving,” Journal of Economic Behavior & Organization 95(2013): 130–145. Eleanor Putnam-Farr and Jason Riis, “‘Yes, I want to enroll.’: Yes/No Response Formats Increase Response Rates in Marketing Communications,” (working paper, 2015).

6

The most effective email nearly doubled the rate at
which unenrolled Servicemembers signed up for a workplace savings plan.
In late April 2015, each of the 806,861 Servicemembers notenrolled in TSP was sent either the TSP website language
email, one of the eight email variants designed using behavioral insights, or no email (the business-as-usual practice).

havioral insights to Servicemembers about the benefits ofparticipating in TSP. 15

Emails significantly increased Servicemember enrollment in
TSP. Figure 1 shows the fraction of each group that signed
up for TSP in May 2015. Servicemembers who were sent theemails that incorporated behavioral insights had an average
enrollment rate that was 0.77 percentage point higher thanthose who had not been sent an email. A portion of this effect—0.42 percentage point—is due to simply sending anemail, while the remaining effect is a result of incorporating
behavioral insights into the content of the email.

Building on the outreach effort detailed above, SBST
and DOD developed a second pilot to further increase
Servicemember participation in TSP. Behavioral science research indicates that prompting individuals to
make a decision about whether or not to take an action at a relevant point in time can help motivate action among those who are interested.16 For example,
asking employees to actively choose whether or not toparticipate in workplace savings plans has effectively increased enrollment in these plans.17

Prompting a Choice for Thrift Savings Plans

The increase in enrollments due to the emails informed bybehavioral insights, relative to no email, translates into 4,930
additional TSP enrollments and $1.3 million in additional savings by Servicemembers in a single month. The most effective email, which clarified the action steps needed to enroll
in TSP and emphasized the potential long-term benefits ofsaving even a little each month, nearly doubled the rate at
which Servicemembers signed up for TSP.

Based on this insight, SBST and DOD implemented a
project that prompted a choice regarding whether or
not to enroll in TSP at an orientation session for Servicemembers who were moving to a new military base.
For Servicemembers, the point at which they move toa new base—a transition often tied to changes in compensation and duties—is a natural moment to offer
such prompts. SBST designed a cover sheet that featured a box to check indicating a “Yes” or “No” decision
about enrolling in TSP (as shown in Figure 2), as well as
general information about TSP and the benefits of in-

Based on the success of this pilot, DOD committed at the2015 White House Conference on Aging to apply these insights at scale by sending periodic emails informed by beFigure 2: TSP Prompted Choice
COMPLETE AND SUBMIT:

YES, I choose to enroll and save through TSP or make changes to my contribution(Complete sections I, II, IV)
NO, I choose NOT to enroll and save through TSP
I am already saving through TSP and do not want to make any changes

Notes: As part of a prompted choice pilot, Servicemembers were presented with the check-box above as part of other paperwork they were alreadyrequired to complete during in-processing.

15

See: www.whitehouse.gov/the-press-office/2015/07/13/fact-sheet-white-house-conference-aging
Punam Anand Keller, Bari Harlam, George Loewenstein, and Kevin G. Volpp, “Enhanced Active Choice: A New Method to Motivate Behavior
Change,” Journal of Consumer Psychology 21 (2011): 376–383.
17
Gabriel D Carroll, James J. Choi, David Laibson, Brigitte C. Madrian, and Andrew Metrick, “Optimal Defaults and Active Decisions,” Quarterly
Journal of Economics 124 (2009): 1639–1674.
16

7

vesting. The cover sheet was circulated, along with the
TSP application form, to Servicemembers at an orientation session at a military base during the pilot period.

Based on the success of this pilot, DOD is committing to applying insights from this pilot across bases and installations
with higher troop concentrations.

As shown in Figure 3, introducing a prompted choice resulted in substantially higher TSP enrollment rates among Servicemembers. Enrollment rates were initially similar at thepilot base and three comparison bases (4.34 and an average of 2.88 percent, respectively), but the prompted choicegenerated a much higher rate at the pilot base (8.71 percent), compared to average rates at the three comparisonbases over the same period (2.91 percent on average). Taking into account differences across both the time periods
and the different bases, the prompted choice intervention
is estimated to have led to a 4.3 percentage point increasein the rate at which non-enrolled Servicemembers enrolled
in TSP.

Thrift Savings Plan Re-Enrollment
In addition to promoting enrollment in TSP among currently
non-enrolled Servicemembers, SBST also worked with DOD
on a project to help already-enrolled Servicemembers remain in their plans. In January 2015, due to a change in the
online military pay system, nearly 140,000 Servicemembers
needed to re-enroll in their Roth TSP to avoid having theircontributions suspended indefinitely.
Figure 4: Servicemember Roth TSP Re-enrollment Rates
35%
28.7%

30%

Figure 3: Impact of a Prompted Choice on Enrollment
Rates for TSP among Servicemembers

25%

23.5%

20%

10%
8.71

8%

15%
10%

6%
4%
2%
0%

5%

4.34

0%

2.88

2.91

Comparison
Period

Pilot
PeriodPilot Base

Message A

Message B

Notes: Percentage of Servicemembers re-enrolling in Roth TSP within one
week after sending an email. Error bars display 95 percent confidence
intervals. The difference is statistically significant at the 1 percent level.
n = 139,273.

Using behavioral insights, SBST redesigned DOD’s standard,
informational email announcing the opening of the re-enrollment window. On January 2, 2015, Servicemembers weresent either DOD’s standard message or the redesigned version of the message. The redesigned message emphasized
the New Year as a chance for Servicemembers to make a
fresh start with their finances, clarified the three steps need-

Comparison Bases

Notes: Enrollment rates for TSP among non-enrolled, in-processing
Servicemembers at the pilot base and comparison bases during the pilot
period and the earlier, comparison period.

Prompting a choice to enroll in a workplace savings plan
led to a 4.3 percentage point increase in the fraction ofServicemembers signing up.

8Compared to a standard message, a message
designed using behavioral insights led 22 percent
more Servicemembers to re-enroll in a workplacesavings plan.
Summer Melted to complete the re-enrollment process, and encouraged
action to avoid losing the chance to contribute savings.18 It
also included a tailored greeting (“Dear Servicemember,”)and restated the key message about re-enrollment in a postscript. 19

Every year, roughly 20 to 30 percent of college-acceptedhigh school graduates in urban districts fail to matriculate incollege in the fall, because they do not complete required
pre-matriculation tasks such as filling out course-enrollment
forms, taking placement tests, or com- Figure 5: Example Summer Meltpleting the FAFSA Text Message
form.21 This phenomenon is known as “summer melt.” Prior work
has shown that sending students low-costtext message reminders to complete thesetasks is an effective
tool for curbing summer melt.22

As shown in Figure 4, the email redesigned using behavioral
insights led to a 5.2 percentage point increase (from 23.5
percent to 28.7 percent) in re-enrollments in the first week
after sending the emails, representing 22 percent more Servicemembers re-enrolling in their Roth TSP. Based on thisresult, DOD scaled up the effective behavioral messaging infollow-up messages to all Servicemembers that encouraged
Servicemembers to act before the re-enrollment deadline.
b. Improving College Access and Affordability
Having a college education has never been more important
for the economic success of Americans.20 A college degreetranslates into higher earnings, a lower risk of unemployment, and greater economic opportunity. At the same time,
the rising costs of college have made affordability a challenge
for many students and their families. SBST collaborated with
the Department of Education’s (ED) office of Federal Student
Aid (FSA) to unlock college access for more students and help
borrowers effectively manage their student loan debt.

In a project aimed
at reducing summer
melt and boosting col- Notes: One of the text messages sent
lege-enrollment rates, by uAspire as part of the summer melt
SBST and FSA provided intervention.
technical
expertiseto the nonprofit, college-access organization uAspire and ateam of academic researchers. SBST provided feedback on
the design and content of personalized text messages that
reminded students to complete required pre-matriculation

18

Hengchen Dai, Katherine L. Milkman, and Jason Riis, “The Fresh Start Effect: Temporal Landmarks Motivate Aspirational Behavior,” Management
Science 60 (2014): 2563–2582. Daniel Kahneman and Amos Tversky, “Prospect Theory: An Analysis of Decision under Risk,” Econometrica, 47(1979): 263–291.
19
Siegfried Vögele, Handbook of Direct Mail: The Dialogue Method of Direct Written Sales Communication (Prentice Hall, 1992).
20
Claudia Goldin and Lawrence F. Katz, The Race between Education and Technology, (Harvard University Press, 2009).
21
Benjamin L. Castleman and Lindsay C. Page, Summer Melt: Supporting Low-Income Students Through the Transition to College (Cambridge, MA:
Harvard Education Press, 2014).
22
Benjamin L. Castleman and Lindsay C. Page, “Summer Nudging: Can Personalized Text Messages and Peer Mentor Outreach Increase College
Going Among Low-Income High School Graduates?,” Journal of Economic Behavior & Organization 115 (2015): 144–160.

9

Sending low-income, college-accepted high school
graduates personalized text message reminders tocomplete required pre-matriculation tasks led 8.6 percent more students to successfully enroll in college.
Missed Student Loan Payments

tasks at their intended college.23 An example of one of thetext messages is shown in Figure 5. uAspire sent a series ofeight text messages informed by behavioral insights to a random sample of students over the summer months.

Behavioral insights can also be used to help students who
have already gone to college and taken out Federal loans
stay on track with their loan repayments. In one project, SBST
collaborated with FSA to send email reminders to borrowers
who had missed their first payments. The reminder was designed to help borrowers who were new to repayment andborrowers for whom missed payments may have been inadvertent. 24

The texting intervention proved effective, boosting college
enrollment by 3.1 percentage points (from 64.9 percent to68.0 percent). The impact of the texts was particularly large
for the lowest-income students, who saw a 5.7 percentage
point increase in college enrollment (from 66.4 percent to72.1 percent; see Figure 6), amounting to 8.6 percent more
low-income students successfully enrolling in college.

The email made clear that the borrower had missed a payment, sought to clarify the borrower’s relationship with his or
her loan servicer, and included a prominent, direct link to his
or her servicer’s login page. Borrowers were also directed toa link to more information about income-driven repayment
plans if they could not afford their payment.

Figure 6: College Enrollment Rates Among Low-Income
Students
100%
80%

After one week the reminder email led to a 29.6 percent increase in the fraction of borrowers making a payment (from2.7 percent to 3.5 percent). This intervention did not simply
prompt people who would pay regardless to pay earlier; rather, more than three months later, the difference persisted,
increasing the percentage of borrowers who made a payment from 16.0 percent to 16.6 percent. While the effects
of a single email were small in absolute terms, the effects
indicate that low-cost email notices and reminders can be aneffective tool for promoting payment among some borrowers who fall behind.

72.1%
66.4%

60%
40%
20%
0%
No Texts

Texts

Notes: Percentage of low- income high school seniors enrolling in college
in the fall of 2014. Error bars display 95 percent confidence intervals.
The difference is statistically significant at the 5 percent level. n = 2,010.

23

FSA and SBST provided technical assistance on message content and complexity for these projects. The trials themselves (and the relevant data collection) were administered exclusively by the stated outside entities.
24
Dean Karlan, Margaret McConnell, Sendhil Mullainathan, and Jonathan Zinman, “Getting to the Top of Mind: How Reminders Increase Saving,” Management Science (forthcoming). Ximena Cadena and Antoinette Schoar, “Remembering to Pay? Reminders vs. Financial Incentives for Loan Payments,”(NBER Working Paper No. 17020, 2011). Peter Baird, Leigh Reardon, Dan Cullinan, Drew McDermott, and Patrick Landers, “Reminders to Pay: Using
Behavioral Economics to Increase Child Support Payments,” (OPRE Report 2015-20, 2015).

10

Sending timely, informational emails to struggling student loan borrowers led to a four-fold increase in applications for income-driven repayment plans.
Income-Driven Repayment Applications

Moreover, in response to the President Obama’s Student Aid
Bill of Rights, FSA is expanding efforts to apply behavioral insights more broadly to meet the needs of loan borrowers.26
To continue improving outcomes for borrowers, SBST and ED
are working to find the most innovative and effective ways tocommunicate with borrowers by leveraging the latest scientificresearch and identifying key factors that influence borrower
repayment.27 This effort includes a series of additional pilots
that reach out to borrowers in default, borrowers who are at
risk of withdrawing from school, and borrowers who are in an
income-driven repayment plan and need to recertify their information each year. Results from these projects are forthcoming.

For student loan borrowers who need help managing theirdebt, ED offers income-driven repayment (IDR) plans that linkborrowers’ monthly payments to their incomes. In order forthese plans to have their intended impact, borrowers need toknow IDR plans exist, understand the associated options andtradeoffs, and determine whether these plans are a good fit
for them.
FSA, in collaboration with SBST, developed an email campaign
to increase awareness of IDR and help borrowers make moreinformed decisions about loan repayment options given theircircumstances. In total, the campaign sent emails to more than
three million struggling borrowers. SBST and FSA embedded
a rigorous evaluation into the broader campaign to measure
the impact of emails on 841,442 borrowers who were 90–180
days delinquent on their loans. These emails indicated IDR eligibility criteria, listed the benefits associated with taking action
and the costs associated with inaction, and contained the relevant web links and servicer contact information.25

c. Advancing Economic Opportunity
The Federal Government supports economic opportunity
through numerous programs and services, from job search
assistance and training support, to credit programs for businesses and farmers. SBST leveraged behavioral insights to
help connect Veterans with education and career counseling benefits and farmers with credit.

To evaluate the impact of sending these emails on IDR application rates, the emails were delivered to borrowers in two
waves, roughly three weeks apart. Over that period, IDR application rates were four times higher among borrowers who
had been sent an email than among those who had not yet
been sent an email. A total of 4,327 applications for IDR weremade by the former group, compared with only 982 applications by the latter.

Education and Career Counseling Benefits for Veterans
The Department of Veterans Affairs (VA) is committed toensuring that Servicemembers successfully reintegrate into
civilian society. Through its Chapter 36 benefits program, VA
offers a variety of education, training, and job placement
services that empower Servicemembers to capitalize on
employment opportunities.

Based on this work, SBST has collaborated with FSA to support
ongoing efforts by ED to revise the IDR application form, with afocus on making it simpler and easier for student loan borrowers to complete.

SBST worked with VA to increase awareness of this program
through low-cost informational emails. Prior to this pilot, VA
did not send direct email communications to Veterans about

25

Saurabh Bhargava and Dayanand S. Manoli, “Psychological Frictions and the Incomplete Take-Up of Social Benefits: Evidence from an IRS Field Experiment,” American Economic Review (forthcoming). Raj Chetty and Emmanuel Saez, “Teaching the Tax Code: Earnings Responses to an Experiment with
EITC Recipients,” American Economic Journal: Applied Economics 5 (2013): 1–31. Jeffrey B. Liebman and Erzo F.P. Luttmer, “Would People Behave Differently If They Better Understood Social Security? Evidence From a Field Experiment,” American Economic Journal: Economic Policy 7 (2015): 275–299.
26
See: www.whitehouse.gov/blog/2015/03/10/student-aid-bill-rights-enhancing-protections-student-loan-borrowers
27
See: www.whitehouse.gov/the-press-office/2015/03/10/fact-sheet-student-aid-bill-rights-taking-action-ensure-strong-consumer-11

Highlighting to Veterans that they had earned an employment benefit, as compared to simply notifying themof their eligibility, led to a 9 percent increase in accessto the benefit application.
Microloans for Farmers

Chapter 36 benefits. Veterans were sent one of two email
variants—one highlighting Veterans’ eligibility for the benefit
and one highlighting that Veterans had earned the benefit
through their years of service—or no email (the business-asusual practice).28 The emails are shown in Figure 7.

Because farming often produces irregular income and requires large capital investments, the United States Department of Agriculture (USDA) offers loans to in-need farmers.
The loans are intended to benefit farmers who have difficulty obtaining credit from a commercial source and whose
farms require a timely injection of capital. To meet the financing needs of small, beginning, and non-traditional farmoperations, USDA’s Farm Service Agency, USDA’s Economic
Research Service, and SBST launched an informational campaign to increase awareness and uptake of one such loan
program, known as Microloans. Farmers were sent personalized letters that contained facts about Microloans, a customized set of action steps for applying, and the contact
information of the recipient’s local loan officer.

Figure 7: Veterans Career Benefits Emails
Veterans Affairs Career
counseling!

You’ve earned Veterans
Affairs Career counseling!

Dear [NAME],
Good news! You are
eligible for personalized
career counseling from the
Department of Veterans Affairs. This benefit will help
you choose a career paththat’s right for you andtake better advantage ofyour other VA benefits andresources.

Dear [NAME],
Good news! You haveearned personalized career counseling from the
Department of Veterans
Affairs. This benefit will
help you choose a career
path that’s right for you
and take better advantageof the other VA benefits
and resources you’ve
earned.

The informational campaign significantly increased participation in the loan program. As a result of the campaign, more
farmers inquired about, applied for, and were approved forloans. In areas that received the targeted outreach, the proportion of farmers who obtained a Microloan increased by
22 percent (from 0.09 percent to 0.11 percent of farmers).
USDA plans to build on this success with a second campaign
this coming year.

Notes: On the left, the text of the basic email containing information on
the benefit; on the right, the email describing the benefits as “earned.”d. Helping Families Get Health Coverage andStay Healthy

The “earned” email outperformed the “eligible” email: nearly 9 percent more Veterans who were sent the “earned”
email clicked through to the application than veterans who
were sent the “eligible” email. But overall uptake levels remained quite low—between November 2014 (when theemails were sent) and March 2015, only a total of 146 Veterans who received an email (of either type) applied. These
results suggest that barriers to enrollment in the Chapter 36
benefit program lie elsewhere.

Physical and mental health is a central element of wellbeing and carries significant social and economic implications for American society in general. A variety of Federal Government programs exist to advance the healthof the Nation, including those that prevent disease,
ensure food safety, offer health care to Servicemembers and Veterans, and make health insurance affordable. Programs such as Medicaid, Medicare, and, most28

Daniel Kahneman, Jack L. Knetsch, and Richard H. Thaler, “Experimental Tests of the Endowment Effect and the Coase Theorem,” Journal of Political
Economy 98 (1990): 1325–1348. Tanjim Hossain and John A. List, “The Behavioralist Visits the Factory: Increasing Productivity Using Simple Framing
Manipulations,” Management Science 58 (2012): 2151–2167.

12

recently, the Federal Health Insurance Marketplace, canuse behavioral insights to promote access, informed decision-making, and follow-through among beneficiaries.
SBST applied behavioral insights to help individuals andfamilies obtain health insurance and stay healthy.

SBST used behavioral insights to design eight different versions of these letters, each of which prompted individuals tofinish the application process and submit a completed application for coverage. One version, shown in Figure 8, included both a picture of the sender to grab the recipient’s attention and personalize the message, as well as what behavioral
scientists refer to as an “implementation intention prompt”:
an invitation to the recipient to write in the time and date at
which he or she planned to return to the online exchange
and complete enrollment.29 Prior research has demonstrated that people are more likely to follow through on an intended action—such as receiving a vaccination—when they
commit to taking that action at a specific moment in time.30

Federal Health Insurance Marketplace Enrollment
In one project, SBST worked with the Department of Healthand Human Services’ (HHS) Office of Communications tosend letters to individuals who had already created an account in the Federal Health Insurance Marketplace (FHIM),
but had not yet completed their application.
Figure 8: Example FHIM Letter

Figure 9: Enrollment Rates of the Eight Letter Variant
Groups, Sorted by Effectiveness4
2
3
Letter #

7
1
5
6
8
0
3

4
5
Percent Enrolled (95% CI)

6

Notes: The blue bar coincides with the 95 percent confidence interval of
the group receiving no letter. n = 744,510.

Notes: This letter included a picture and implementation intention
prompt at bottom for planning when to finish enrolling on the
FHIM. This is variant number 4 in Figure 9.

A new outreach letter led to a 22 percent increase ineligible farmers obtaining business loans.
29

Marianne Bertrand, Sendhil Mullainathan, Eldar Shafir, and Jonathan Zinman, “What’s Advertising Content Worth? Evidence from a Consumer Credit
Marketing Field Experiment,” Quarterly Journal of Economics 125 (2010): 263–306. Peter M. Gollwitzer, “Implementation Intentions: Strong Effects of
Simple Plans,” American Psychologist 54 (1999): 493–503 (1999).
30
Katherine L. Milkman, John Beshears, James J. Choi, David Laibson, and Brigitte C. Madrian, “Using Implementation Intentions Prompts to Enhance
Influenza Vaccination Rates,” Proceedings of the National Academy of Sciences 108 (2011): 10415–10420.

13

Individuals were sent one of the eight letter variants or noletter several weeks before the open enrollment deadline.
Individuals who were sent the mailer were significantly
more likely to finish enrollment in a health care plan thanthose who were not sent a letter—a 0.3 percentage point
increase. Some letters were notably more effective thanothers; the version that included both an implementation
intention prompt and a picture of the sender boosted enrollments by 13.2 percent, while a version that had neither ofthese elements increased enrollments by only 1.8 percent(Figure 9).

chlorination of water in Kenya by making it more convenient
to do so.33 To extend the application of such findings, SBST
and the United States Agency for International Development(USAID) have entered into a multi-year collaboration to use
behavioral insights to improve outcomes for populations
around the world. As part of this collaboration, SBST and
USAID are working together on a series of pilots to promote
global health. In one project, SBST is working with USAID in
Mozambique to send text reminders and educational messages to patients co-infected with tuberculoisis (TB) and thehuman immunodeficiency virus (HIV) to boost attendanceat doctors’ appointments and adherence to treatment regimens. Results of this pilot are forthcoming.

In a closely related project, SBST and HHS worked together
on an email campaign to promote FHIM enrollments. The
full results of this project are forthcoming, but early findings
are promising. One pilot, for instance, examined whether information about the probability of qualifying for certain tax
credit benefits should be presented to individuals in terms ofa percentage (“66%”) or frequency (“2 out of 3”). Although
the two statements are equivalent, prior work suggests thatpeople more easily interpret frequency statements.31 Results generated to date demonstrated that individuals sent
the frequency statement were more likely to submit an application than those sent a percentage statement.

III. Improving Government Efficiency
The Federal Government faces an obligation to serve theAmerican people as efficiently as possible. This means that
Federal programs should pay only qualifying individuals andfor qualified expenses, and branches of Government thatcollect payments from individuals and businesses should receive the full amount that is owed in taxes, debts, and fees.
Less-effective programs should have less claim to scarce budgetary dollars, and government operations should be freefrom waste.

Flu Prevention

Because these measures of government efficiency ultimately
depend on the decisions and actions of individuals such as
program officials, beneficiaries, and contractors, behavioral
insights can improve results in this domain. For example, improving program integrity is in many cases a matter of promoting compliance with program rules. Behavioral sciencesuggests that programs should be designed to make compliant actions salient and easy for the relevant decision makers
to take.34 When compliant behaviors become the obvious
path or the path of least resistance, individuals and businesses may be more likely to pay what they owe or refrainfrom claiming benefits for which they do not qualify. Just as itshould be easy for qualifying individuals to access programs,
it should also be easy for people to play by the rules.

On average, the flu results in 120,000 hospitalizations and
36,000 deaths each year in the United States, yet fewer thanhalf of Americans typically get a flu shot.32 SBST and HHS collaborated on a project to promote vaccinations for the influenza virus among Medicare beneficiaries. During the 2014
flu season, HHS sent beneficiaries low-cost mailers informedby behavioral insights that contained information on the importance and benefits of getting vaccinated. Results of thisproject are forthcoming.
Global HealthSome of the most promising translations of behavioral science to health improvements have been implemented in
the developing world—for example, successfully promoting

31

Paul Slovic, John Monahan, and Donald G. MacGregor, “Violence Risk Assessment and Risk Communication: The Effects of Using Actual Cases, Providing
Instruction, and Employing Probability Versus Frequency Formats,” Law and Human Behavior 24 (2000): 271–296.
32
William W. Thompson, David K. Shay, Eric Weintraub, Lynnette Brammer, Nancy Cox, Larry J. Anderson, and Keiji Fukuda, “Mortality Associated with
Influenza and Respiratory Syncytial Virus in the United States,” JAMA 289 (2003): 179–186. William W. Thompson, David K. Shay, Eric Weintraub, Lynnette
Brammer, Carolyn B. Bridges, Nancy J. Cox, and Keiji Fukuda, “Influenza-Associated Hospitalizations in the United States,” JAMA 292 (2004): 1333–1340.
For flu vaccination coverage rates, see: cdc.gov/flu/fluvaxview/coverage-1314estimates.htm
33
Michael Kremer, Edward Miguel, Sendhil Mullainathan, Clair Null, and Alix Peterson Zwane, “Social Engineering: Evidence from a Suite of Take-up Experiments in Kenya” (unpublished working paper, April 2011).
34
Nina Mazar and Dan Ariely, “Dishonesty in Everyday Life and Its Policy Implications,” Journal of Public Policy & Marketing 25 (2006): 117–126. Lisa L.
Shu, Nina Mazar, Francesca Gino, Dan Ariely, and Max H. Bazerman, “Signing at the Beginning Makes Ethics Salient and Decreases Dishonest Self-Reports
in Comparison to Signing at the End,” Proceedings of the National Academy of Sciences 109 (2012): 15197–15200. Michael Hallsworth, John A. List,
Robert D. Metcalfe, and Ivo Vlaev, “The Behavioralist as Tax Collector: Using Natural Field Experiments to Enhance Tax Compliance,” (NBER Working Paper
No. 20007, 2014).

14

Adding a signature box to the top of an online form,
where vendors confirmed the accuracy of selfreported sales, led to an additional $1.59 million infees collected by the Federal government within asingle quarter.
Behavioral insights can also improve government efficiency
by providing information about which programs and operations are more or less cost-effective. New findings about how
individuals do or do not respond to programs can provide
program officials with a basis for making decisions about
where, or whether, to spend program dollars.

vendors are required to do two things: first, to report thosesales to the Government; and second, to pay to the Government a small fraction of their reported sales as an administrative fee, known as the Industrial Funding Fee (IFF).36
To promote more accurate self-reporting of the sales and,
consequently, more accurate collections of IFF, SBST and
GSA introduced a required signature box at the top of an online payment form for a random sample of vendors. The signature box asked vendors to confirm the truth and accuracyof the information about to be reported, as shown in Figure 10. Research indicates that requiring users to sign theirnames to confirm the accuracy of self-reported statements
can reduce self-reporting errors if the signature is requested
at the beginning of a form.37 Interestingly, signature prompts
at the end of a form seem to have no such effect.

SBST used behavioral insights to generate additional collections through improved program integrity, save money oninternal operational expenses, and identify programs theGovernment could run more effectively.
a. Promoting Program Integrity and Compliance
Program integrity and compliance are essential componentsof government efficiency. Integrity efforts include eliminating waste, fraud, and abuse in government programs, only
paying program dollars for services or benefits according toprogram rules, and ensuring that programs collect all that
individuals or businesses owe. Prior research from behavioral economics has demonstrated that, for example,
simplifying tax collection letters and referencing the
behaviors of others can accelerate tax collections. 35
SBST built on these insights to improve collectionsof fees from government contractors, promote repayment of non-tax debts, and encourage program
compliance by medical providers.

Figure 10: Industrial Funding Fee Signature Box

Industrial Funding Fee Reports
SBST collaborated with the General Services Administration (GSA) to improve the accuracy of self-reported fees
and collections. When the Government purchases goods
and services from vendors under certain contracts, those

Notes: Screenshot of the online form where vendors self-reported sales.
Those vendors required to provide the upfront signature reported significantly more in sales.

35

Marsha Blumenthal, Charles Christian, and Joel Slemrod, “Do Normative Appeals Affect Tax Compliance? Evidence From A Controlled Experiment In
Minnesota,” National Tax Journal 54 (2001), 125–138. Michael Hallsworth, John A. List, Robert D. Metcalfe, and Ivo Vlaev, “The Behavioralist as Tax Collector: Using Natural Field Experiments to Enhance Tax Compliance,” (NBER Working Paper No. 20007, 2014).
36
For general background on the IFF see: https://72a.gsa.gov/ifffaq.cfm#01
37
Lisa L. Shu, Nina Mazar, Francesca Gino, Dan Ariely, and Max H. Bazerman, “Signing at the Beginning Makes Ethics Salient and Decreases Dishonest SelfReports in Comparison to Signing at the End,” Proceedings of the National Academy of Sciences 109 (2012): 15197–15200.

15

The signature box was effective; the Federal Government
collected an additional $1.59 million in fees within a single
quarter as a result of the box. The median self-reported sales
amount was $445 higher for vendors signing at the top of
the form compared with those vendors who were not required to make this confirmation.

SBST and DMS used behavioral insights to redesign DMS’s
standard collection letter, and worked together to test theeffect of the new letter compared with the original. The
new letter included simplified language, a shortened web
address to encourage online rather than paper payments,
a personalized salutation, a prominent reference to the total
amount owed in the letter’s opening line, and a statementthat 91 percent of Americans pay their debt on time (appealing to a social norm of timely payments).38

Based on this pilot result, GSA is permanently updating theonline form to incorporate a signature box. SBST will also explore opportunities to apply the signature box intervention
to other government contexts in which financial informationis self-reported.

The redesigned letter was sent to a random sample of individuals with outstanding debt, and no difference in payment
rates was observed between those who received the redesigned letter and those who received the standard collection
letter. This may have been due to the age and status of thisdebt: individuals who are already 180 days or more behindin their payments may face financial barriers to repayment
rather than informational barriers. Individuals sent the newletter, however, were significantly more likely to make apayment online (2.16 percent of those sent the new letter
made a payment online, versus 1.49 percent of those sent
the standard letter). This was likely due to the fact the web
address was shortened in the new letter and was changed
from a long string of meaningless characters to a shorter setof meaningful words (see Figure 11).

Delinquent Debt Repayment
In another project addressing payments, SBST worked
with the Department of the Treasury’s Debt Management
Service (DMS) to increase collections from individuals withoutstanding non-tax debt. Individuals might incur this typeof debt by, for example, failing to repay Medicare for an
overpayment they received. When individuals fall behind on
payments of these debts by 180 days, their debt is referredto DMS. DMS then commences its own collection efforts,
which include sending collection notices by mail.
Figure 11: Debt Letter Web Address

Based on this finding, this debt collection letter now uses
the shortened form of the web address.

Before:
You may also make an electronic payment via pay.gov:(https://www.pay.gov/paygov/forms/formInstance.
html?agencyFormId=16531440)

Prescriber Letters
A different set of program integrity and compliance efforts
relate to what the Government pays as opposed to what itcollects. Combating fraud, waste, and abuse is especially important in health care, where programs such as Medicare
and Medicaid combine to account for nearly a quarter of all
Federal spending. Applying behavioral insights to promotecompliance by providers in these contexts may save costs.

After:
Pay online: Visit www.pay.gov/paygov/paymydebt
Notes: The text above is taken from letters redesigned using behavioral
insights sent out to individuals with outstanding debt.

Shortening the web address for an online payment
platform on a debt collection letter led to a 45 percent
increase in use of online payments.
38

Michael Hallsworth, John A. List, Robert D. Metcalfe, and Ivo Vlaev, “The Behavioralist as Tax Collector: Using Natural Field Experiments to Enhance Tax Compliance,” (NBER Working Paper No. 20007, 2014). “91 percent of Americans…” calculated based on data available at: newyorkfed.org/
householdcredit/2013-Q1/data/xls/HHD_C_Report_2013Q1.xlsx

16Sending medical providers a letter illustrating their
unusually high drug prescribing rates relative to theirpeers had no measurable impact on prescription rates.
One healthcare program integrity challenge is inappropriate drug prescribing, which can both threaten patient healthand increase healthcare costs. The Centers for Medicare
and Medicaid Services (CMS) conducts a variety of innovative practices to combat overprescribing, such as proactively
identifying providers suspected of inappropriate activity andpursuing legal action through law enforcement channels.

identify where programs could save money by forgoing costineffective measures, as well as projects to encourage costeffective actions among government workers.
Legacy Treasury Direct Account Conversion
In one project, SBST worked with the Department of theTreasury (Treasury) on a program called Legacy Treasury
Direct (LTD). LTD previously allowed investors to purchasesecurities such as savings bonds directly from Treasury via
mail, fax, or phone. Starting in 2011, Treasury began theprocess of phasing out LTD in favor of TreasuryDirect, an
online system.

Behavioral science research reveals a potentially cost-effective way of curbing noncompliant behavior among those
who exhibit it: namely, highlighting the contrast between
the actions of the individual and the social norm (how mostpeople act). For example, households use less electricity
when they are informed that their energy consumption exceeds that of their neighbors.39 Drawing on this insight, SBST
and CMS worked together on a project that sent a letter toa subset of providers with unusually high billing patterns for
Schedule II prescriptions (e.g., opioids). The letter compared
the provider’s prescribing rates with the prescribing rates of
his or her peers and provided educational information aboutproper prescribing practices.

Figure 12: Default Appointment Intervention

What Should
I do Next?
We’ve scheduled an appointment for you withyour Treasury Retail Securities representative:

This intervention had no measurable impact. Medicare drug
coverage data collected 90 days after the letter was mailed
showed no effect of the letter on Schedule II prescribing.
SBST is currently implementing additional pilots to address
over-prescription, which will improve on the initial pilot andre-examine the design, timing, and frequency of letters.

Please call your Treasury Retail Securities representative on Tuesday, March 12, 2014, between 7
am and 11 am ET, at 111-111-1111 (extension 4).
If this appointment doesn’t work for you, call Monday through Friday 7 am to 4 pm ET to rescheduleor walk through your account transfer today.

b. Ensuring Cost-Effective Program Operations
Just as compliance and integrity are important for government efficiency, so too is cost-effectiveness. The Government should achieve desired outcomes by spending taxpayer dollars on programs and operations that yield positive
results at reasonable costs. SBST worked on projects to

Notes: As part of the LTD pilot, selected individuals received a letter indicating a default call-in time.

39

Hunt Allcott, “Social Norms and Energy Conservation,” Journal of Public Economics 95 (2011): 1082–1095.

17

Treasury and SBST worked together on a project to learn
whether low-cost notices could effectively encourage LTD account holders to transition to TreasuryDirect. Account holders
were randomly assigned to receive one of two letter variants
designed using behavioral insights. One letter was a simplified
version of letters sent in previous years. The second letter wasa simplified version of previous phase-out letters, but also gave
account holders a specific appointment time to call their Treasury representative. This addition draws on behavioral science
research showing that individuals are more likely to follow
through on plans that involve clear action steps at specific moments in time.40 Relative to the simplified letter, the “specificappointment” letter (the relevant portion of which is displayedin Figure 12) boosted call-in rates by 2.37 percentage points.
However, fewer than 5 percent of all accountholders whocalled in (across both letter types) said they would definitely
switch to the online TreasuryDirect system.

one week increased by about 1 percentage point (from 7.05
percent to 8 percent). Other analyses revealed how responses
to the email varied by the time of day they were sent, the dayof week they were sent, and across different subject lines.
Among other findings, click rates were highest at lunchtime;
15.3 percent of emails sent at 11:55 a.m. saw recipients clickthrough to the survey, compared with 13.3 percent of emails
sent at 8:55 a.m.
Double-Sided Printing
In a project focused directly on the costs of government operations, SBST collaborated with USDA’s Economic Research Service (ERS) to promote the use of double-sided printing among
ERS employees. During a pilot period, if employees initiated asingle-sided print job, a dialog box appeared that prompted individuals to change their default printer setting to double-sided, but still allowed them to continue their single-sided print
job if that was preferred.

Treasury and SBST concluded that, while it was feasible toget more investors to call in to discuss account options withrepresentatives, these calls did not lead account holders totransition to TreasuryDirect. Treasury has no further mailings
planned and is pursuing other avenues to transition analog
services to digital.

Individuals who received this prompt were more likely to print
double-sided. This prompt increased the likelihood of doublesided printing on a given job by 5.8 percentage points, from abaseline of 46.0 percent. Based on this result, ERS is changing
the default settings of all of its printers to double-sided.

Tenant Satisfaction Survey
The General Service Administration’s (GSA) Public Buildings
Service (PBS) owns or leases over 9,600 assets, including morethan 370 million square feet of space for over a million Federal
employees. Each year, PBS sends Federal employees the Tenant Satisfaction Survey (TSS), the results of which help determine Federal facilities strategy. In 2014, PBS collaborated with
SBST to simplify the survey and test different messages forlaunching it in order to increase response rates.
SBST designed a launch email that emphasized the request(“please take the survey today!”), included a clear, mobile-optimized button linking to the survey, and had a link to a calendar
invite that recipients could use to set a self-reminder to takethe survey. As a result of this email, survey response rates after

Individuals sent a letter with a specific appointment
time to call in were 23 percent more likely to do so.
40

Todd Rogers, Katherine L. Milkman, Leslie K. John, and Michael I. Norton, “Making the Best Laid Plans Better: How Plan-Making Increases FollowThrough, Behavioral Science and Policy (forthcoming).

18

IV. What’s Next
SBST’s inaugural year generated promising results. As detailed in this report, SBST efforts are already paying dividends for Americans. Due to SBST projects, more Servicemembers are saving for retirement, more students aregoing to college and better managing their student loans,more Veterans are taking advantage of education andcareer counseling benefits, more small farms are gaining
access to credit, and more families are securing healthinsurance coverage. And improvements in government
program integrity and efficiency are saving taxpayer dollars. These improvements were generated at little to noadministrative cost.

Finally, SBST results further demonstrate the potential of
behavioral insights to strengthen future policy reform efforts. If prompting enrollment at key points can boost participation in a retirement savings plan, what might thatmean for the way access to that plan should be designed?
If a single email can help individuals choose a student loan
repayment program, what does that suggest about the waythat choice set should be structured? Through small changes to program administration, the impact of SBST projects
point to broader opportunities for policymakers to use behavioral insights to achieve policy goals across the Federal
Government and better serve Americans.

Based on results from early SBST pilots, a number of agencies are already starting to make lasting changes to program
design and administration. SBST will continue to work withagencies to take the lessons from these early successes toscale, so that more Servicemembers, more students, more
farmers—and more Americans from all walks of life—canbenefit from a Government that makes full use of the insights that behavioral science can offer. To do so, SBST willcontinue to help build capacity within agencies to effectively and independently apply behavioral insights to theirprograms.

Other SBST Efforts
In addition to projects related to streamlining program access and improving government efficiency,
SBST worked to identify promising research insightsfrom the social and behavioral sciences that informed
agency policy agendas, inspired collaborations withthe private sector, and contributed to existing policy
processes.

back on how to improve the frequency and qualityof interactions with their babies. Every time parents
talk, sing, or read to their baby, it makes it easier for
their baby to learn later, as heightened language input protects the brain from the pruning of unusedneural connections.3 To help bridge the word gap,
OSTP, ED, HHS, and the Institute of Museum and Library Services worked with Too Small to Fail and the
Urban Institute on an event that showcased innovative
Federal, state, and local efforts to tackle this important problem. The event generated a series of commitments, including a technology incentive prize toinspire new behavioral tools for low-income parents,
and a national research network grant to empower
the research community to answer pressing questions
about how to most effectively bridge the word gap.4

Increasing opportunities for low-income children bybridging the word gap
During the first few years of life, a child from a lowincome family hears millions of fewer words than her
more affluent peers.1 This deficit, known as the word
gap, is associated with disparities in vocabulary size
and, critically, school readiness.2 Fortunately, social
science has shed light on one low-cost solution: providing parents with behavioral reminders and feed-

continued >>>19

Empowering students to reach their full potential
through learning and belonging mindsets

evant, and up-to-date evidence (see “Behavioral Insights and the Role of Evidence”). SBST is collaborating with DPC, the Office of Management and Budget(OMB), OSTP, and the Council of Economic Advisers(CEA) as part of a coalition to promote the more effective use of evidence and evaluation at the Federal
level. These efforts are currently in progress, and will
result in guidance to agencies on the use of evidence
to inform program decisions.

Research demonstrates that brief interventions thatteach students their minds are like muscles that cangrow with hard work and perseverance, as compared to fixed traits like eye color, can increase student achievement, with the greatest gains seen for
low-achieving students and females in STEM fields.5
Research also shows that brief social belonging interventions—in which older classmates share stories of
their own struggles and hardships with younger students, as well as a roadmap to success—improve resilience and persistence in college among these younger
students.6 OSTP, in collaboration with the Domestic
Policy Council (DPC), the Raikes Foundation, and theHewlett Foundation, convened experts and practitioners over the past year to advance basic research andpromote effective applications to policy. As a result of
these efforts, researchers at Stanford University andthe University of Texas at Austin have announced thelargest-ever study of learning mindset interventions atover 100 high schools nationwide. The College Transition Collaborative, a partnership of researchers and
17 colleges and universities, delivered and evaluatedbelonging interventions with approximately 40,000
incoming college students throughout summer 2015.7

Using behavioral insights to streamline governmentforms
To assist with the implementation of the aforementioned Executive Order, the Office of Information and Regulatory Affairs (OIRA) and OSTP are issuing new guidance that will help Federal agencies
use behavioral science insights to improve Federal
forms. A burdensome, complex form can pose
more than just an inconvenience to Americans:it can actually inhibit Americans from accessing a
program or benefit. SBST will provide behavioral
insights support to agencies on forms that: (1)reach a large number of Americans; (2) are required to access government services; and (3) currently take a long time to complete. SBST will also
help agencies on forms that present operational
concerns such as high error rates.

Promoting the use of evidence in Federal policymaking
and program administration
For the Federal Government to make responsible
and effective decisions about policy design andprogram administration, policymakers need goodevidence about what works and what does not. Findings from the behavioral sciences only reinforce theimportance of basing policy design on rigorous, rel-Betty Hart and Todd R. Risley, Meaningful Differences in the Everyday Experience of Young American Children, (Baltimore, MD: Brookes, 1995).
Jeanne Brooks-Gunn, Cecilia E. Rouse, and Sara McLanahan, “Racial and Ethnic Gaps in School Readiness,” in School Readiness and the Transition to Kindergarten in the Era of Accountability, ed. R.C. Pianta, M.J. Cox, and K.L. Snow, 283–306 (Baltimore, MD: Brookes, 2007).
3
Peter R. Huttenlocher, Neural Plasticity: The Effects of Environment on the Development of the Cerebral Cortex, (Harvard University Press,
2002).
4
See: www.whitehouse.gov/sites/default/files/microsites/ostp/federal_word_gap_fact_sheet_final.pdf
5David S. Yeager and Gregory M. Walton, “Social-Psychological Interventions in Education: They’re Not Magic,” Review of Educational Research,81 (2011): 267–301. David Paunesku, Gregory M. Walton, Carissa Romero, Eric N. Smith, David S. Yeager, and Carol S. Dweck, “Mindset
Interventions Are a Scalable Treatment for Academic Underachievement,” Psychological Science, 26 (2015): 784–793. Catherine Good, Joshua
Aronson, and Michael Inzlicht, “Improving Adolescents’ Standardized Test Performance: An Intervention to Reduce the Effects of Stereotype
Threat,” Journal of Applied Developmental Psychology, 24 (2003): 645–662.
6
Gregory M. Walton, “The New Science of Wise Psychological Interventions,” Current Directions in Psychological Science, 23 (2014): 73–82.
Gregory M. Walton and Geoffrey L. Cohen, “A Brief Social-Belonging Intervention Improves Academic and Health Outcomes Among Minority
Students,” Science 331 (2011): 1447–1451.
7
www.whitehouse.gov/the-press-office/2015/03/23/fact-sheet-president-obama-announces-over-240-million-new-stem-commitmen
1220Appendices21

This page intentionally left blank22

Appendix A: Executive Order
Using Behavioral Science Insights to Better Serve the American People

23

25

This page intentionally left blank24

Executive Order
Using Behavioral Science Insights to Better Serve the American People
A growing body of evidence demonstrates that behavioral science insights – research findings
from fields such as behavioral economics and psychology about how people make decisions andact on them – can be used to design government policies to better serve the American people.
Where Federal policies have been designed to reflect behavioral science insights, they havesubstantially improved outcomes for the individuals, families, communities, and businesses thosepolicies serve. For example, automatic enrollment and automatic escalation in retirement savings
plans have made it easier to save for the future, and have helped Americans accumulate billionsof dollars in additional retirement savings. Similarly, streamlining the application process forFederal financial aid has made college more financially accessible for millions of students.
To more fully realize the benefits of behavioral insights and deliver better results at a lower costfor the American people, the Federal Government should design its policies and programs to reflect our best understanding of how people engage with, participate in, use, and respond to thosepolicies and programs. By improving the effectiveness and efficiency of Government, behavioral
science insights can support a range of national priorities, including helping workers to findbetter jobs; enabling Americans to lead longer, healthier lives; improving access to educational
opportunities and support for success in school; and accelerating the transition to a low-carbon
economy.
NOW, THEREFORE, by the authority vested in me as President by the Constitution and the lawsof the United States, I hereby direct the following:
Section 1. Behavioral Science Insights Policy Directive.
(a) Executive departments and agencies (agencies) are encouraged to:(i) identify policies, programs, and operations where applying behavioral science insights
may yield substantial improvements in public welfare, program outcomes, and program
cost effectiveness;(ii) develop strategies for applying behavioral science insights to programs and, where
possible, rigorously test and evaluate the impact of these insights;(iii) recruit behavioral science experts to join the Federal Government as necessary toachieve the goals of this directive; and(iv) strengthen agency relationships with the research community to better use empiricalfindings from the behavioral sciences.

25

(b) In implementing the policy directives in section (a), agencies shall:(i) identify opportunities to help qualifying individuals, families, communities, and businesses access public programs and benefits by, as appropriate, streamlining processes
that may otherwise limit or delay participation—for example, removing administrative
hurdles, shortening wait times, and simplifying forms;(ii) improve how information is presented to consumers, borrowers, program beneficiaries, and other individuals, whether as directly conveyed by the agency, or in setting
standards for the presentation of information, by considering how the content, format,
timing, and medium by which information is conveyed affects comprehension and action
by individuals, as appropriate;(iii) identify programs that offer choices and carefully consider how the presentation andstructure of those choices, including the order, number, and arrangement of options, canmost effectively promote public welfare, as appropriate, giving particular consideration to
the selection and setting of default options; and(iv) review elements of their policies and programs that are designed to encourage ormake it easier for Americans to take specific actions, such as saving for retirement orcompleting education programs. In doing so, agencies shall consider how the timing, frequency, presentation, and labeling of benefits, taxes, subsidies, and other incentives canmore effectively and efficiently promote those actions, as appropriate. Particular attention
should be paid to opportunities to use nonfinancial incentives.
(c) For policies with a regulatory component, agencies are encouraged to combine this behavioral science insights policy directive with their ongoing review of existing significant regulations
to identify and reduce regulatory burdens, as appropriate and consistent with Executive Order
13563 of January 18, 2011, (Improving Regulation and Regulatory Review) and Executive Order13610 of May 10, 2012, (Identifying and Reducing Regulatory Burdens).
Sec. 2. Implementation of the Behavioral Science Insights Policy Directive.
(a) The Social and Behavioral Sciences Team (SBST), under the National Science and Technology Council (NSTC) and chaired by the Assistant to the President for Science and Technology,
shall provide agencies with advice and policy guidance to help them execute the policy objectives outlined in section 1 of this order, as appropriate.
(b)The NSTC shall release a yearly report summarizing agency implementation of section 1 ofthis order each year until 2019. Member agencies of the SBST are expected to contribute to thisreport.
(c)To help execute the policy directive set forth in section 1 of this order, the Chair of the SBST
shall, within 45 days of the date of this order and thereafter as necessary, issue guidance to assist
agencies in implementing this order.

26

Sec. 3. General Provisions.
(a) Nothing in this order shall be construed to impair or otherwise affect:(i) the authority granted by law to a department or agency, or the head thereof; or(ii) the functions of the Director of the Office of Management and Budget relating to budgetary, administrative, or legislative proposals.
(b) This order shall be implemented consistent with applicable law and subject to the availabilityof appropriations.
(c) Independent agencies are strongly encouraged to comply with the requirements of this order.
(d) This order is not intended to, and does not, create any right or benefit, substantive or procedural, enforceable at law or in equity by any party against the United States, its departments,
agencies, or entities, its officers, employees, or agents, or any other person.

THE WHITE HOUSE

27

This page intentionally left blank28

Appendix B: Project Abstracts
SBST seeks, where feasible and cost-effective, to implement applications of behavioral insights to Federal policies and programs as demonstration projects, in order to generate evidence about the effectiveness of these applications and learn whatworks, what works best, and what does not work. Often, these projects implement and evaluate the impact of behavioral
insights directly within Federal Government programs using rapid, rigorous, and low-cost methods (for more, see “Behavioral
Insights and the Role of Evidence”).
The project abstracts collected below provide additional information on the design of, and results from, projects completed by
SBST through mid-2015. Wherever possible, SBST implemented these projects as randomized trials. Where random assignment proved infeasible, SBST sought to derive credible estimates of effects using the best, most practical non-experimental
identification techniques applicable in each circumstance. Specific methods for each project are detailed below.
The abstracts below mirror the order of their presentation in the report, and include:
Servicemember TSP Enrollment
On-Base Servicemember TSP Enrollment
Servicemember Roth TSP Re-Enrollment
Summer Melt
Missed Student Loan Payments
Income-Driven Repayment
Education and Career Counseling Benefits for VeteransMicroloans for Farmers
Federal Health Insurance Marketplace Enrollment
Industrial Funding Fee Reports
Delinquent Debt Repayment
Prescriber Letters
Legacy Treasury Direct Accounts
Tenant Satisfaction Survey Response
Double-Sided Printing

29

30
31
32
33
34
35
36
37
38
39
40
41
42
43 - 45
46

Clear action steps, presenting choices, and emphasizing
benefits promote savings enrollment

2.10%
1.91%

2.0%
1.56%
1.5%

1.14%

1.0%
0.5%
0.0%

Tre T
atm opentMethods. DFAS, in collaboration with the Social and
Behavioral Sciences Team (SBST) and academic researchers, tested the impact on TSP enrollment ofsending unenrolled Servicemembers emails that highlighted the opportunity and potential benefits of TSP.43
In addition to a no-email control and a standard message drawn from TSP and DFAS web materials, eight
different messages utilized different combinations of
behavioral insights: clear action steps, fresh-start messaging, presentation of the decision to enroll in TSP
as a choice between two options, and emphasis on
short- and long-term benefits.44 The 806,861 Servicemembers who were not enrolled in TSP as of April 27
were assigned to these ten groups based on the lasttwo digits of their Social Security Number (SSN). After
DFAS sent out the emails on April 29, 2015 (May 4 forMarines), it tracked TSP enrollment by SSN.

2.5%

Av
er
Tre age
atm ofent

Background. The Federal Government operates
a workplace savings program called the Thrift Savings Plan (TSP) for all of its employees.41 While theFederal Government automatically enrolls its civilian
employees in TSP, it does not automatically enroll Servicemembers, and military enrollment rates averageroughly 42 percent.42 In order to enroll, Servicemembers need to log in to the Department of Defense’s(DOD) Defense Finance and Accounting Service (DFAS)
MyPay website and select a contribution percentage.
As TSP savings through traditional (pre-tax) or Roth(after-tax) contributions can confer both short- andlong-term benefits, many of the over 800,000 unenrolled Servicemembers might choose to enroll in TSP if
actively presented with a chance to do so.

Results. One month after the emails were sent,
14,491 Servicemembers had enrolled: 920 in the noemail group (1.14 percent), compared with 1,255 senta standard message (1.56 percent) and 12,316 acrossthose sent the eight treatment emails (1.91 percent).
The most effective email clarified the action steps
needed to enroll and emphasized the potential longterm benefits of saving even a little each month. The
increase in enrollment rates due to the standard email
was 0.42 percentage point (p < 0.01, 95% CI [0.30,
0.53]), compared with 0.77 percentage point for treatment emails (p < 0.01, 95% CI [0.69, 0.85]). This meansthat 67 percent more Servicemembers—4,930—enrolled in TSP in May 2015 as a result of being sent atreatment message.

StanMedardssa
ge

Agency Objective. Increase enrollment in Thrift Savings Plans among interested active duty Servicemembers using behaviorally designed email communications.

N
Em o
ail

P r o j e c t

A b s t r a c tServicemember TSP Enrollment

Conclusions. Informational email campaigns are effective at prompting Servicemember enrollment in
TSP, especially when designed based on behavioral
insights. Lessons about which messages were most effective at motivating Servicemember action are being
used to inform future outreach efforts that continue
to support Servicemembers, and may help to informfuture reforms to other elements of the military compensation and retirement system currently under consideration.45

For general background information on TSP, see: tsp.gov
Enrollment rates for TSP as of late 2014 are reported here: frtib.gov/pdf/minutes/MM-2014Dec-Att1.pdf
The differing enrollment procedures are described at: tsp.gov/planparticipation/eligibility/establishingAccount.shtml
43
Katherine L. Milkman collaborated with SBST on this project.
44
John Beshears, James J. Choi, David Laibson, and Brigitte C. Madrian, “Simplification and Saving,” Journal of Economic Behavior &Organization 95 (2013): 130–145. Eleanor Putnam-Farr and Jason Riis, “‘Yes, I want to enroll.’: Yes/No Response Formats Increase
Response Rates in Marketing Communications,” (working paper, 2015).
45
See: www.whitehouse.gov/the-press-office/2015/07/13/fact-sheet-white-house-conference-aging
41
42

SOCIAL AND BEHAVIORAL SCIENCES TEAM

2015 Annual Report30

A b s t r a c t
P r o j e c t

On-Base Servicemember
TSP Enrollment
Prompted choice during a reset moment promotes
savings enrollment
Agency Objective. Actively presenting Servicemembers with a choice to enroll in Thrift Savings Plans (TSP),
in order to promote enrollment.
Background. The Federal Government, including themilitary, operates a savings program for its employees
known as the Thrift Savings Plan (TSP). Roughly 58 percent of the over 1.3 million active duty Servicemembersin the Armed Forces are not currently enrolled in any
TSP plan, and only around 1 percent of non-enrolled
Servicemembers newly enroll each month. The successof “prompted choice” interventions in other workplacesavings contexts—where employees have to actively
choose whether to contribute or not—suggests thatmany Servicemembers might enroll if prompted.*
Because Permanent Change of Duty Station (PCS), ortransferring to a new location, is often tied to changes
in compensation and duties, in-processing briefings area natural reset moment during which Servicemembers
may benefit from prompts to make a financial decision.
Methods. In collaboration with the Department of Defense, the Social and Behavioral Sciences Team (SBST)
designed a cover sheet for the TSP-U-1 Thrift Savings
Plan Election Form that (a) provided information on
benefits of TSP investing, and (b) actively prompted Servicemembers to contribute—or not—to TSP. The coversheet was included in the in-processing package at Ft.
Myer in Arlington, VA, and briefed Servicemembers
were required to submit the completed cover sheet
along with other required paperwork. Briefing logistics
made random assignment infeasible; for purposes of
estimating impacts of the treatment period at Ft. Myer,
TSP enrollment data was collected on enrollment rates
at Ft. Myer over the year preceding the pilot period andfrom a comparison set of comparable forts: Fts. Belvoir,
Bragg, and Meade.

Results. TSP enrollment rates for Servicemembers inprocessed and not already enrolled at Ft. Myer during
the pilot period were 8.71 percent, compared to 2.91
percent in the other three forts (Belvoir, Bragg, andMeade) during the same period. During the same date
range one year prior to the pilot in 2014 (4/20/14–
6/12/14), and over a comparable length window 53
days prior to the pilot period (2/23/15–4/17/15), theaverage enrollment rate at Ft. Myer was 4.34 percent
compared with an average of 2.88 percent at the otherthree forts. Using a difference-in-difference estimate,
the pilot is estimated to have led to a 4.3 percentage
point increase in the rate of TSP enrollment among unenrolled Servicemembers (p < 0.05, 95% CI [0.26, 8.42]).

10%
8.71

8%
6%
4%
2%
0%

4.34
2.88

2.91

Comparison
Period

Pilot
PeriodPilot BaseComparison Bases

Conclusions. While auto-enrollment and auto-escalation savings plans have shown dramatic success atincreasing participation in savings plans by new employees, prompted choice interventions can also be
used to increase participation by existing employees
through finding reset moments. These findings can inform broader policy discussions regarding defined contribution savings participation in the armed forces andbeyond.

* Previously cited as notes 16 & 17: Punam Anand Keller, Bari Harlam, George Loewenstein, and Kevin G. Volpp, “Enhanced Active
Choice: A New Method to Motivate Behavior Change,” Journal of Consumer Psychology 21 (2011): 376–383; Gabriel D Carroll,James J. Choi, David Laibson, Brigitte C. Madrian, and Andrew Metrick, “Optimal Defaults and Active Decisions,” Quarterly Journal of
Economics 124 (2009): 1639–1674.
SOCIAL AND BEHAVIORAL SCIENCES TEAM

2015 Annual Report31

A b s t r a c t
P r o j e c tServicemember Roth TSP
Re-Enrollment
A “fresh start,” action-oriented communicationencourages Servicemember re-enrollment
Agency Objective. Increase re-enrollments in RothThrift Savings Plans among interested Servicemembers, using action-oriented, simplified email communications.
Background. Due to a change in the military pay system, 139,273 members of the Armed Forces needed to
re-enroll in their Roth Thrift Savings Plans (TSP) in January 2015 to avoid having their contributions suspended
indefinitely. To re-enroll, Servicemembers needed tolog in to the Department of Defense’s (DOD) Defense
Finance and Accounting Service (DFAS) MyPay website
and select a contribution percentage. DFAS conducteda broad media and email campaign to encourage affected Servicemembers to re-enroll in TSP.
Methods. As part of this campaign, DFAS, in collaboration with the Social and Behavioral Sciences Team(SBST), redesigned the standard email sent by DFAS
to Servicemembers announcing the opening of the reenrollment window. The redesigned, simplified email
emphasized the New Year as a fresh start, laid out thethree steps needed to complete the re-enrollment
process, and encouraged action in order to avoid losing the chance to contribute savings.46 The 139,273
Servicemembers were assigned to two groups basedon the last two digits of their Social Security Number (SSN), with those with SSNs ending in 0–49 (n =69,318) sent the standard email, and Servicememberswith SSNs ending in 50–99 (n = 69,955) sent the redesigned email. After sending the emails on January
2, 2015, DFAS tracked re-enrollment requests by SSN.
Results. One week after the emails were sent, 16,291
Servicemembers who were sent the standard email(23.5 percent) had re-enrolled, compared with 20,061
Servicemembers who were sent the redesigned email(28.7 percent). Because the last four digits of SSNs
are effectively randomly assigned, the increase in reenrollment rates of 5.2 percentage points (p < 0.01,
95% CI [4.7, 5.6]) can be attributed to the redesigned
email. This means that 22 percent more Servicemembers—3,770—re-enrolled in their Roth TSP, or at least
accelerated their re-enrollment, as a result of being
sent the redesigned email instead of the standard
email.
35%
28.7%

30%
25%

23.5%

20%
15%
10%
5%
0%
Message A

Message B

Conclusions. Because the redesigned email was
more effective at prompting re-enrollment after one
week, a modified version of the more effective message was subsequently scaled to the entire population
in follow-up messages that encouraged Servicemembers to act before the deadline. Thus, in addition todemonstrating the relative impact of using behavioral
messaging to drive action, this project also illustrates
the rapid scalability of insights identified as effective
via this type of rapid low cost pilot.

Hengchen Dai, Katherine L. Milkman, and Jason Riis, “The Fresh Start Effect: Temporal Landmarks Motivate Aspirational Behavior,”
Management Science 60 (2014): 2563–2582. Daniel Kahneman and Amos Tversky, “Prospect Theory: An Analysis of Decision underRisk,” Econometrica, 47 (1979): 263–291.
46

SOCIAL AND BEHAVIORAL SCIENCES TEAM

2015 Annual Report32

P r o j e c t

A b s t r a c t

Summer Melt
Text messages reminding high school seniors of key tasks
boost college enrollment
Agency Objective. Increase college enrollment rates
among college-accepted high school graduates by sending them text message reminders to complete required
pre-matriculation tasks over the summer.
Background. Every year 20–30 percent of collegeaccepted high school graduates in urban districts failto matriculate in college in the fall—a phenomenon
known as “summer melt.”47 In the summer of 2014, theDepartment of Education’s (ED) office of Federal Student Aid (FSA), along with the Social and Behavioral Sciences Team (SBST), provided technical expertise to thenon-profit organization uAspire and a team of researchers on a text messaging campaign to students and theirparents.48 Text messages were personalized for each
student and reminded each student of tasks she needed to complete to successfully matriculate. The text
messages also provided recipients with a connection toindividualized college and financial aid advising.
Methods. High school students from five cities participated in the pilot (n = 4,882). Students were randomly
assigned to one of three groups: (1) a control group thatwas not sent messages; (2) a group in which only students were sent messages; (3) a group in which bothstudents and their parents were sent messages. Priorwork has shown that sending students low-cost text
message reminders to complete pre-matriculation tasks
is an effective tool for curbing summer melt.49 The text
message campaign began in late June and continued
through late August 2014. The research team obtained
student-level demographic and academic achievement
data from uAspire and student-level college enrollment
information from the National Student Clearinghouse.

students and for first-generation students. Among students with an expected family contribution of $0, for example, the text messages increased college enrollment
by 5.7 percentage points (p < 0.01, 95% CI [1.4, 10.0]),
from 66.4 percent to 72.1 percent. The differences
between the students-only and students-and-parents
treatment groups were not statistically significant.
100%
80%

72.1%
66.4%

60%
40%
20%
0%
No Texts

Texts

Conclusions. These findings underscore the importance of supporting college-intending high school students during the summer months leading up to thestart of college, as well as the efficacy of personalized
text messaging as a strategy to help students successfully matriculate in college.

Results. Among all students sent text messages, 68.0
percent enrolled in college in the fall compared with
64.9 percent of those not sent messages—a differenceof 3.1 percentage points (p < 0.05, 95% CI [0.4, 5.8]).
Enrollment effects were larger for the lowest-income

Benjamin L. Castleman and Lindsay C. Page, Summer Melt: Supporting Low-Income Students Through the Transition to College,(Cambridge, MA: Harvard Education Press, 2014).
48
Benjamin L. Castleman and Lindsay C. Page were the lead researchers on this project. FSA and SBST provided technical expertiseon message content. The trials themselves (and the relevant data collection) were administered exclusively by the stated outside
entities.
49
Benjamin L. Castleman and Lindsay C. Page, “Summer Nudging: Can Personalized Text Messages and Peer Mentor Outreach
Increase College Going Among Low-Income High School Graduates?,” Journal of Economic Behavior & Organization 115 (2015):
144–160.
47

SOCIAL AND BEHAVIORAL SCIENCES TEAM

2015 Annual Report33

P r o j e c t

A b s t r a c t

Missed Student Loan Payments
Reminders help student loan borrowers get backon track with payments
Agency Objective. Help student loan borrowers
who miss payments get on track with their payments
using email reminders sent directly from Federal Student Aid.
Background. Federal student loan borrowers who
miss their initial loan payments are in danger of defaulting on their loan.50 To assist these borrowers, theDepartment of Education’s (ED) office of Federal Student Aid (FSA), in collaboration with the Social and Behavioral Sciences Team (SBST), ran an email campaign
to determine whether having FSA send reminder
emails to borrowers—in addition to the communications from loan servicers they already receive—could
help borrowers get back on track after missing theirinitial payments. A reminder email of this kind could
particularly help those borrowers who are unawarethat payments are now due or lack clarity about how
to make payments.
Methods. The email campaign took place in the last
week of February 2015, targeting borrowers who had
missed the first or first and second payment on theirmonthly loan (n = 149,115). The email made clearthat the borrower had missed a payment, sought toclarify the borrower’s relationship with their loan servicer, and included a prominent, direct link to theirservicer’s login page to help with making a payment.
Evidence from other contexts suggests that low-cost
reminders of this nature can help individuals to makepayments.51 A link to more information about incomedriven repayment (IDR) plans was also included, towhich borrowers were directed if they could not afford
their payment.

The project proceeded in two phases: In the first, FSA
identified an effective subject line by randomly sending different subject lines to four groups of 12,500 borrowers each and comparing open rates. (The line “Youmissed a payment on your Federal student loan” wasmost effective by this measure). In the second phase,emails using the winning subject line were sent to arandomly selected set of the remaining borrowers (n= 77,115). A control group (n = 22,000) did not receive
any communications from FSA, but continued to receive any ongoing communications from their loan
servicers. Payment rates for all groups were tracked
through the end of June 2015.
Results. In the seven days after the emails were sent,
3.5 percent of borrowers who were sent an email
made at least one payment compared to 2.7 percentof the control group—a difference of 0.8 percentage
points (p < 0.01, 95% CI [0.6, 1.0]). This difference persisted: more than three months after the emails weresent, the fraction of those groups having made a payment had risen to 16.6 and 16.0 percent, respectively,
a difference of 0.6 percentage point (p < 0.05, 95% CI[0.1, 1.1]). There were no significant differences in thenumber of IDR applications; however, borrowers who
received emails were less likely to access FSA’s repayment calculator (p < 0.01) or to switch to deferment orforbearance on their loans (p < 0.01).
Conclusions. Emails that reminded student loanborrowers about missed payments and clarified how
to make a payment led to significantly, but modestly,
higher payment rates for the treatment group, andthese higher rates persisted throughout the follow-up
period.

See: studentaid.ed.gov/sa/repay-loans/default
Dean Karlan, Margaret McConnell, Sendhil Mullainathan, and Jonathan Zinman, “Getting to the Top of Mind: How Reminders
Increase Saving,” Management Science (forthcoming). Ximena Cadena and Antoinette Schoar, “Remembering to Pay? Reminders
vs. Financial Incentives for Loan Payments,” (NBER Working Paper No. 17020, 2011). Peter Baird, Leigh Reardon, Dan Cullinan, Drew
McDermott, and Patrick Landers, “Reminders to Pay: Using Behavioral Economics to Increase Child Support Payments,” (OPRE Report
2015-20, 2015).
50
51

SOCIAL AND BEHAVIORAL SCIENCES TEAM

2015 Annual Report34

P r o j e c t

A b s t r a c t

Income-Driven Repayment
Notifications to borrowers about income-driven
repayment (IDR) plans increase application rates
Agency Objective. Increase applications for incomedriven repayment plans among delinquent student loanborrowers with email notices designed using behavioral
insights.
Background. As student loan balances have risen inrecent years, an increasing numbers of borrowers havestruggled to stay on track with their payments.52 At-riskand delinquent borrowers may benefit from incomedriven repayment (IDR) options, which link monthly
payment amounts to their incomes, and can make repayment more manageable.53 However, enrolling in an
IDR plan requires that borrowers learn of and apply forsuch a plan. In late 2013, the office of Federal Student
Aid (FSA) within the Department of Education (ED), in
collaboration with the Social and Behavioral Sciences
Team (SBST), conducted an email campaign to help atrisk student loan borrowers learn of and apply for thesealternative payment plans.
Methods. Borrowers who were 90 to 180 days delinquent (n = 841,442) were randomly assigned along two,
independent dimensions. First, the timing of the email
was varied so that half of the sample was sent the email
in November and half in December 2013. Second, theformat and content of the email were varied, comparing four variants designed using behavioral insights: alonger, more comprehensive email; a shorter and lessdetailed version of the email; a loss-framed email; and again-framed email.54

Results. Emails had a significant, positive impact oncompleted IDR applications. Of those borrowers who
were sent an email, 1.02 percent submitted an IDR application in the 20 days following the email, comparedwith only 0.23 percent of those not sent the email, a difference of 0.79 percentage point (p < 0.01, 95% CI [0.75,
0.82]). Across variants, the longer email was slightly
more effective at prompting IDR applications. Borrowers sent the longer email completed applications at a
rate of 1.04 percent in the 20 days after sending, compared with 0.86 percent of those sent the shorter email(p < 0.01, 95% CI [0.12, 0.24]). The loss-framed email
was slightly more effective at leading to applications
than the gain-framed email, but each was less effective
than the more comprehensive, longer email (all differences p < 0.05).
Conclusions. These findings suggest that low-cost,
timely notices can make a significant difference on IDR
enrollment rates among struggling student loan borrowers. While the effects are small in size in relative
terms, this outcome is a measure of application rates
within a twenty-day window following a single email.
Given the large population of at-risk and delinquentborrowers, the absolute effect is substantial: the results suggest that sending just a single email led roughly
6,600 additional borrowers to sign up for an IDR plan.

See: newyorkfed.org/studentloandebt/
For more on IDR plans, see: studentaid.ed.gov/sa/repay-loans/understand/plans/income-driven
54
Saurabh Bhargava and Dayanand S. Manoli, “Psychological Frictions and the Incomplete Take-Up of Social Benefits: Evidence
from an IRS Field Experiment,” American Economic Review (forthcoming). Raj Chetty and Emmanuel Saez,” Teaching the Tax Code:Earnings Responses to an Experiment with EITC Recipients,” American Economic Journal: Applied Economics 5 (2013): 1–31. Jeffrey
B. Liebman and Erzo F.P. Luttmer, “Would People Behave Differently If They Better Understood Social Security? Evidence From a
Field Experiment,” American Economic Journal: Economic Policy 7 (2015): 275–299. Daniel Kahneman and Amos Tversky, “Prospect
Theory: An Analysis of Decision Under Risk,” Econometrica 47 (1979): 263–291.
52
53

SOCIAL AND BEHAVIORAL SCIENCES TEAM

2015 Annual Report35

A b s t r a c t
P r o j e c t

Education and Career Counseling
Benefits for Veterans
Communications emphasizing that benefits are earned
through their service to encourage Veterans to apply andenroll for an education and career counseling benefit
Agency Objective. Increase the take-up of education and career counseling benefits among interested Veterans using email outreach designed using
behavioral insights.
Background. The Department of Veterans’ Affairs(VA) is committed to help military Servicemembers
fully reintegrate back into society through education, training, and career counseling programs.55 VA’s
Chapter 36 Education and Career Counseling benefit(Chapter 36) is a reintegration service provided by
VA’s Vocational Rehabilitation and Education office(VR&E). Chapter 36 services include personalized
counseling on education, career options, and accessing related VA benefits.56 The Chapter 36 application
process is paper-based and involves multiple steps,
which may create barriers to access.
Methods. VR&E, in collaboration with the Social
and Behavioral Sciences Team (SBST) and academic
researchers, identified email as a low cost tool to inform Veterans about Chapter 36 benefits, their eligibility, the application process, and what to expect after an application was submitted.57 Eligible Veteranswith a valid email address on record were randomly
assigned to one of three conditions:58 (1) Business as
usual: sent no email from the VA about the benefit(n = 21,423); (2) Basic email: sent an email that explained they were eligible for free career counselingand provided a link to download the application form(n = 21,424); and (3) Earned: sent an email otherwise
identical to the basic email, but with an emphasis on

the fact that Veterans had earned the benefit throughtheir service (n = 21,423).59 To measure the impact of
the emails, four outcomes were tracked using existing VA data: email open rates, click rates (on the linkto the application form), applications submitted, andVeterans who completed services with counseling.
Results. The Earned email outperformed the Basicemail on open (42.6 versus 40.0 percent, p < 0.01,
95% CI [1.6, 3.6]) and click rates (4.1 vs 3.6 percent,p < 0.03, 95% CI [0.11, 0.89]). The emails increased
applications and completion of services with counseling measurably, but overall uptake rates remained
low. Without the emails, rates of application remained near 0 (only one person in the Business as
usual group applied for benefits during the trial period); the Basic email increased applications to 0.37
percent (p < 0.01) and the Earned email to 0.30 percent (p < 0.01). There is no meaningful difference between the two emails (p = 0.79).
Conclusions. These results suggest that email canbe an inexpensive and effective way to share information about Chapter 36 benefits, but that they
may not lead to substantial improvements in utilization. More extensive changes may be necessary to
promote access to these benefits. However, communication about VA benefits should describe thebenefits as those that Veterans have earned throughtheir service.

See: va.gov/op3/docs/strategicplanning/va2014-2020strategicplan.pdf.
For more on Chapter 36 benefits, see: benefits.va.gov/vocrehab/edu_voc_counseling.asp
57
David W. Nickerson collaborated with SBST on this project.
58
The random assignment was stratified by branch of service, state of residence, age, years since discharge, and prior benefits used;
to improve statistical efficiency, all analysis includes a dummy variable to account for strata of randomization.
59
Daniel Kahneman, Jack L. Knetsch, and Richard H. Thaler, “Experimental Tests of the Endowment Effect and the Coase Theorem,”
Journal of Political Economy 98 (1990): 1325–1348. Tanjim Hossain and John A. List, “The Behavioralist Visits the Factory: Increasing
Productivity Using Simple Framing Manipulations,” Management Science 58 (2012): 2151–2167.
55
56

SOCIAL AND BEHAVIORAL SCIENCES TEAM

2015 Annual Report36

P r o j e c t

A b s t r a c t

Microloans for Farmers
Personalized outreach to farmers increases successful
small-business loan applications
Agency Objective. Provide information on the USDA
Microloans program to farmers.
Background. The U.S. Department of Agriculture’s,(USDA) Farm Service Agency (FSA) launched the Microloan program in January 2013 to better serve theneeds of small farms, beginning farmers, and farmers from historically socially disadvantaged groups.60
As their name suggests, these loans are smaller thanother direct operating loans, with an initial maximumof $35,000 that was raised to $50,000 in November
2014. The loans are designed to be more convenientand accessible to nontraditional producers who mightlack the business and credit history that traditional clients of the direct operating loan program have. This
includes a streamlined application process and relaxed
requirements for collateral and previous experience infarming.
Methods. In collaboration with the Farm Service
Agency (FSA), the USDA’s Economic Research Service
and SBST designed an outreach letter that provided information on the benefits of the Microloan program,personalized contact information for local loan officers, and a shortened web address for accessing more
information.* The letters were sent in late April tofarmers in a random sample of zip codes in nine states:
Alabama, Arkansas, Georgia, Kentucky, Louisiana, Mississippi, North Carolina, South Carolina, and Tennessee. Farmers who were interested in Microloans wereencouraged to follow up by calling, emailing, or visiting
their local county FSA offices to discuss options withan FSA loan officer. Letters were sent to all farms in
1,848 treatment zip codes.

Results. The letter increased the number of successful applicants for Microloans. The number of farms
that apply for and receive a Microloan is small compared to the overall population of farms—131 farms
in control zip codes, or approximately 0.09 percent,applied for and received a Microloan between late
April and July 2015. In the treatment group, thatfigure increased to 165 farms, or 0.11 percent of all
farms in treatment zip codes, a difference of 0.02
percentage point (p < 0.05, 95% CI [0.00, 0.05]).
Consistent with this result, data collected by FSA on
activity at county offices indicate that the letters increased the amount of Microloan-related activity
in FSA county offices. Approximately 1.8 percent of
the recorded activity in FSA county offices from late
April through July of 2015 had to do with Microloans.
Of those customers who gave information about
themselves when they inquired about Microloans,
2.85 percent of the office activity was generated byMicroloans in zip codes that did not receive letters,
while 4.90 percent of the office activity was generated by Microloans in zip codes that received letters,a difference of 2.05 percentage points (p < 0.01).
Conclusion. Providing actionable information to new
farmers can increase access to Microloan credit.

USDA News Release 0010.13. http://www.fsa.usda.gov/FSA/printapp?fileName=nr_20130115_rel_0010.html&newsType=newsrel* Dean Karlan collaborated with SBST on this project.
60

SOCIAL AND BEHAVIORAL SCIENCES TEAM

2015 Annual Report37

Action language, planning prompts, and personalization
promote follow-through on health insurance enrollment
Agency Objective. Assist uninsured Americans withcompleting their health insurance application in timeby sending behaviorally designed letters prior to thedeadline for open enrollment.
Background. During the Open Enrollment Period,
qualifying individuals and families can purchasehealth insurance plans through the Federal Health
Insurance Marketplace (FHIM).61 For the 2015 enrollment season, the close of open enrollment was February 15, 2015. As of early February 2015, many people had visited HealthCare.gov and started an online
account, but not yet selected a plan. The Department
of Health and Human Services (HHS), in collaboration
with the Social and Behavioral Sciences Team (SBST),
developed and sent letters to assist these individuals
with completing their insurance application in time.
Methods. From February 2 to 4, 2015, individuals
who had registered for a HealthCare.gov user account but not yet enrolled in an insurance plan (n =811,795) were randomly assigned to be sent one ofeight letter variants or no letter at all (the “hold-out”
group). The eight letters varied behavioral dynamics
including action language, an implementation intention prompt, a picture, social norm messaging, apledge, and loss aversion.62 The core content of theletter about how to enroll (information about thebenefits of enrolling, the deadline, the website link,
and a phone number) was held constant across eachof the eight letters.
Results. By the February 15 deadline, enrollments
were 4.03 percent in the hold-out group and 4.32
percent across letter variants—a 7.15 percent increase (p < 0.001, 95% CI [5.89, 8.42]) amounting to

1,924 marginal enrollments. Not all letters had equal
effects (see figure): the highest performing letter (#4,
designed with behavioral dynamics of action language, an implementation intention, and a picture)
boosted enrollments by 13.17 percent comparedwith only 1.84 percent for letter #8 (a “kitchen sink”
variant including all dynamics minus the pledge). A
social norm message (about the “millions of Americans” enrolled) in #8, 6, and 5 was ineffective.

42
3
7
Letter #A b s t r a c t
P r o j e c tFederal Health InsuranceMarketplace Enrollment

1
5
6
8
0
3

4
5
Percent Enrolled (95% CI)

6Enrollment rates of the eight letter variants, sorted by effectiveness. The light blue bar indicates the 95% confidence
interval of the group that was sent no letter.

Conclusions. Letter campaigns are effective at
prompting completion of health insurance applications for those who have demonstrated interest andinitiated the process. This is especially the case wheneffort is invested in behavioral design and randomized testing to measure relative performance.

For more information on the FHIM, including the Open Enrollment Period, see www.healthcare.gov.
Katherine L. Milkman, John Beshears, James J. Choi, David Laibson, and Brigitte C. Madrian, “Using Implementation Intentions
Prompts to Enhance Influenza Vaccination Rates,” Proceedings of the National Academy of Sciences 108 (2011): 10415–10420.
Marianne Bertrand, Sendhil Mullainathan, Eldar Shafir, and Jonathan Zinman, “What’s Advertising Content Worth? Evidence froma Consumer Credit Marketing Field Experiment,” Quarterly Journal of Economics 125 (2010): 263–306. Hunt Allcott, “Social Norms
and Energy Conservation,” Journal of Public Economics 95 (2011): 1082–1095. Daniel Kahneman and Amos Tversky, “Prospect
Theory: An Analysis of Decision Under Risk,” Econometrica 47 (1979): 263–291.
61
62

SOCIAL AND BEHAVIORAL SCIENCES TEAM

2015 Annual Report38

A confirmation prompt reduces financial
self-reporting error
Agency Objective. Reduce financial self-reporting
errors using a redesigned data-entry form.
Background. Federal vendors are required to pay afee, called the industrial funding fee (IFF), currently set
at 0.75 percent of quarterly sales on certain transactions.63 The size of the IFF payment is determined fromself-reports submitted via a website (https://72a.gsa.
gov/). In fiscal year 2013, the General Services Administration (GSA) collected approximately $269 million in
IFF across roughly 47,000 transactions.
The GSA, in collaboration with the Social and Behavioral Sciences Team (SBST), introduced a confirmation
prompt as part of the IFF data-entry form in order toreduce financial self-reporting errors. Research has
shown that inserting a confirmation prompt, where
the user signs his or her name confirming the accuracy of the self-reported statements, reduces self-report
errors if done at the beginning of a form; prompts at
the end of a form seem to have no effect.64

P r o j e c t

A b s t r a c t

Industrial Funding Fee Reports

Methods. The randomized controlled trial was fielded during the third reporting quarter of 2014, where
vendors (n = 18,477) were randomly assigned to use
either: (a) the existing reporting system (control); or(b) a modified interface (treatment), redesigned toinclude an opening signature box confirming, “I promise that the information I am providing is true and accurate.” Administrative data on paid IFF provided theprimary outcome measure.
Results. The median self-reported sales amount was$445 (p < 0.05, 95% CI [87, 803]) higher for vendors
signing at the top of the form compared with thosevendors who were not required to make this confirmation. The increase in IFF remittances in the treatmentgroup in just the third quarter of 2014 was $1.59 million.
Conclusions. Confirmation prompts at the beginningof a form are a promising approach to reducing financial self-reporting errors, especially given the near-zero marginal cost to implement.

Multiple Award Schedule contracts are subject to the IFF. For general background information on the IFF, see: https://72a.gsa.gov/
ifffaq.cfm#01
64
Lisa L. Shu, Nina Mazar, Francesca Gino, Dan Ariely, and Max H. Bazerman, “Signing at the Beginning Makes Ethics Salient and Decreases Dishonest Self-Reports in Comparison to Signing at the End,” Proceedings of the National Academy of Sciences 109 (2012):
15197–15200.
63

SOCIAL AND BEHAVIORAL SCIENCES TEAM

2015 Annual Report39

P r o j e c t

A b s t r a c t

Delinquent Debt Repayment
Simplified letters promote greater use of online payments
for delinquent debts
Agency Objective. Increase the rate of debt recovery
by simplifying notification letters.
Background. The Department of the Treasury’s Debt
Management Services (DMS) collects delinquent debts
from individuals who have incurred and fallen behindon non-tax debts to Federal agencies.65 Agencies are required to refer debts to DMS at 180 days delinquent.66
DMS collection letters are typically sent within 4 daysof receiving the debt from a referring Federal agency.
In 2013, DMS, in collaboration with the Social and Behavioral Sciences Team (SBST), modified its collection
letter using behavioral insights to increase the rate ofdebt recovery.
Methods. The new letter was simplified (e.g., multiple
mailing addresses linked to the Department of Treasury were removed from the letter and replaced witha single address; the web address for online payment
was shortened substantially), personalized (individuals
were addressed by name rather than by debt ID number), and the total debt owed was emphasized in theletter’s opening line. In addition, based on recent research from the United Kingdom showing that social
comparisons can increase tax collections, the new letter
highlighted the fact that 91 percent of Americans pay
their debt on time.67

ter during a trial conducted between October 2013 andMarch 2014. Individuals (n = 21,305) with debts from six
creditor agencies were randomly assigned to be sent either the status quo or new letter. Three outcomes were
measured at 30 and 60 days from sending: (i) receipt of
payment on debts; (ii) debtor contact via inbound calls
to DMS; and, (iii) debtor-initiated payments to pay.gov.
Results. No differences were found in overall payment
rates or inbound calls across the letter types. Individuals were significantly more likely to make a payment via
pay.gov (rather than by mail) if they were sent the newletter (2.16 versus 1.49 percent, p < 0.05, 95% CI [0.30,
1.04]). Some collection was made within 30 days on
5.89 percent of all debts included in the sample. There
were significant differences across agencies in the proportion of all debts on which payment was received(ranging from 1.79 to 12.78 percent). Debt size was also
a significant predictor of payment (smaller debts weremore likely to receive payment).
Conclusions. The evidence from the trial suggests thatthe new letter did not affect overall payment rates ondelinquent, non-tax debt. It did affect the method of
payment, however, by increasing utilization of online
payments.

To determine the effectiveness of the changes, the newletter was compared to the status quo delinquency let-

For more information, see: fiscal.treasury.gov/fsservices/gov/debtColl/dms/xservg/debt_crosserv.htm
In practice, agencies have varying policies to determine referral, so age of debt (and other factors) at first receipt by DMS is not
uniform across agencies.
67
Michael Hallsworth, John A. List, Robert D. Metcalfe, and Ivo Vlaev, “The Behavioralist as Tax Collector: Using Natural Field Experiments to Enhance Tax Compliance,” (NBER Working Paper No. 20007, 2014).
“91 percent of Americans…” calculated based on data available here: newyorkfed.org/householdcredit/2013-Q1/data/xls/HHD_C_
Report_2013Q1.xlsx
65
66

SOCIAL AND BEHAVIORAL SCIENCES TEAM

2015 Annual Report40

A b s t r a c t

Letters to providers as a means to promote program integrity in Medicare Part D

P r o j e c t

Prescriber Letters

Conclusions. The letter as designed did not exert adetectable effect. Given the low cost of letter interventions, and the fact that informative letters havebeen shown to work in other contexts, this finding
has prompted the research team to explore alternative approaches to reaching providers as well as thedesign, timing, and frequency of the letters. Additional
letter-based testing is currently underway.

Agency Objective. Reduce inappropriate prescribing of controlled substances in Medicare Part D.68
Background. Inappropriate prescribing can threaten
patient health and increase healthcare costs. A bodyof research shows evidence of some providers overprescribing certain pharmaceuticals, including controlled substances, benzodiazepines, and antipsychotics.69 Through its Center for Program Integrity (CPI),
the Centers for Medicare and Medicaid Services (CMS)
uses a variety of approaches to combat overprescribing behavior, such as proactively identifying providers
suspected of inappropriate activity and pursuing legal
action through law enforcement channels.
Studies have shown that letters, especially those highlighting social comparisons, can motivate individuals
to more carefully examine their own behavior. Simply stating that “9 out of 10 people pay their taxes ontime,” for example, has been shown to substantially
increase timely tax payments.70 Physicians are more
likely to provide vaccinations after receiving feedbackon their vaccination rates relative to peers.71 CPI, in
collaboration with the Social and Behavioral Sciences
Team (SBST) and academic researchers, developedand sent letters to providers incorporating this behavioral insight to promote program integrity in Medicare
Part D.

Methods. Potential improper prescribers of Schedule
II drugs (e.g., opioids) were identified as those who
prescribed far more than their peers in the same stateand medical specialty. These providers (n = 1,518)
were randomly assigned to be sent a letter (in September 2014) or not. The letter depicted an individual’s prescribing rates in comparison to his or her peers,and provided information about proper prescribing
practices. The letter was designed to educate providers and induce them to “self-audit” to correct potentially improper payments. The eﬀect of the letter onprescribing was tracked via Part D claims data.
Results. Using data collected over the 90 days after
the letter was mailed, comparisons failed to detect aneﬀect of the letter on Schedule II prescribing. Tests canreject that the letter reduced prescribing by more than
1.4 percent, but cannot reject effects smaller than
that—though a reduction less than 1.4 percent could
be medically and economically significant.

For a full report, see: Adam Sacarny, David Yokum, Amy Finkelstein, and Shantanu Agrawal, “Reducing Inappropriate Prescribing ofControlled Substances in Medicare Part D: Evidence from a Randomized Intervention,” (submitted, 2015).
69
Department of Health and Human Services, Office of Inspector General, “Retail Pharmacies With Questionable Part D Billing,”(OEI-02-09-00600, 2012). Mark Olfson, Marissa King, and Michael Schoenbaum, “Benzodiazepine use in the United States,” JAMA
Psychiatry 72 (2015): 136–142. Government Accountability Office, “Antipsychotic Drug Use: HHS Has Initiatives to Reduce Useamong Older Adults in Nursing Homes, but Should Expand Efforts to Other Settings,” (GAO-15-211, 2015).
70
Michael Hallsworth, John A. List, Robert D. Metcalfe, and Ivo Vlaev, “The Behavioralist as Tax Collector: Using Natural Field Experiments to Enhance Tax Compliance,” (NBER Working Paper No. 20007, 2014).
71
Catarina I. Kiefe, Jeroan J. Allison, O. Dale Williams, Sharina D. Person, Michael T. Weaver, and Norman W. Weissman, “Improving Quality Improvement Using Achievable Benchmarks for Physician Feedback: A Randomized Controlled Trial,” JAMA 285 (2001):
2871–2879.
68

SOCIAL AND BEHAVIORAL SCIENCES TEAM

2015 Annual Report41

P r o j e c t

A b s t r a c t

Legacy Treasury Direct Accounts
Default call-in appointments encourage accountholder response
Agency Objective. Determine if letters work to encourage account transfers by securities holders.
Background. The Department of the Treasury’s Legacy
Treasury Direct (LTD) program allowed investors to purchase marketable securities directly from the Treasury,
via mail, fax, and phone.72 The program was phasedout starting in 2011, before being decommissioned inlate 2014. Before and during the phase out, Treasury
sent several letters to LTD account holders encouraging
them to transition their accounts to, among other places, TreasuryDirect, an online system opened in 2002.
Methods. In an effort to move customers to the allelectronic system, the Bureau of the Fiscal Service (Fiscal Service) at the Treasury collaborated with the Social
and Behavioral Sciences Team (SBST) to develop twoletters for LTD account holders. First, a Clarified Standard letter, a simplified version of the previous years’
letters. Second, a Default Appointment letter, with similar text as the first, and which also gave recipients anappointment with their Treasury representative during
a low call volume time window at the Fiscal Service callcenter. The Default Appointment letter builds on behavioral science research finding that individuals are
more likely to follow through on plans that identify specific moments of action.73
The letters were mailed to the 33,500 remaining LTD account holders over 16 business days, with each of thetwo letters mailed to batches of 1,000 accounts by zip
code in sequential numerical order. Fiscal Service’s callcenter collected data on LTD account holder call-in actions, including indicators for actions taken by or indicated by the caller, such as whether the caller indicated aninterest in taking any account actions, and whether thecaller established an online account. Using letter identification numbers printed on each letter (e.g., 9965 or
8865 in the top right corner of the letter), the type ofletter (Clarified Standard or Default Appointment) was
linked to data on call-in actions, which allowed for observation of differences in call-in actions by letter type.
Results. Compared with the Clarified Standard letter, the Default Appointment letter resulted in a call-in
rate that was 2.37 percentage points higher (p < 0.05,
95% CI [1.69, 3.07]), an increase of 23 percent. Fewerthan 5 percent of the LTD account holders who calledin (across both treatment groups) said they would definitely switch to the online Treasury Direct system.
14%
12%

13.0%
10.6%

10%
8%
6%
4%
2%
0%

Short Businessas Usual

Default
Appointment

Conclusions. Creating a moment of action for calling inresulted in more investors calling in to discuss accountoptions with Treasury representatives. However, themailings did not result in significant customer accountchanges from LTD to TreasuryDirect. There are no further LTD mailings planned.

For more information on the LTD program, see: treasurydirect.gov/indiv/myaccount/myaccount_legacytd.htm
Todd Rogers, Katherine L. Milkman, Leslie K. John, and Michael I. Norton, “Making the Best Laid Plans Better: How Plan-Making
Increases Follow-Through, Behavioral Science and Policy (forthcoming).
72
73

SOCIAL AND BEHAVIORAL SCIENCES TEAM

2015 Annual Report42A b s t r a c t
P r o j e c t

Tenant Satisfaction Survey
Response: Calendar Invite
Simplification and calendar invitations
increase survey response
Agency Objective. Increase response rates in a workplace survey that directly informs Federal facility strategy.
Background. The General Service Administration(GSA)’s Public Buildings Service (PBS) owns or leases
over 9,600 assets, with more than 370 million square
feet of space for over a million Federal employees. Each
year, PBS sends Federal employees the Tenant Satisfaction Survey (TSS), the results of which help determine
Federal facilities strategy. In 2014, PBS collaborated
with the Social and Behavioral Sciences Team (SBST)
to simplify the survey and test different messages forlaunching the survey, with the goal of increasing response rates.
Methods. Pilot TSS announcement emails were randomly assigned to a randomly selected sample of 29,997
employees at agencies using Federally owned or leased
office space. A Business as Usual launch email was sent
to 14,997 employees, with content based on the previous year’s announcement materials, emphasizing thatthe survey was online and saved the government paper
and time. A Simplified Calendar Reminder launch email
was sent to 15,000 employees, emphasizing what theemail was requesting (“please take the survey today!”);it also had a clear, mobile-optimized button linking to thesurvey and an embedded link to a calendar invite recipients could use to set a reminder to take the survey at theofficial launch time. The two emails had distinct links to
two identical versions of the survey, allowing tracking ofsurvey views, starts, and completions by email group.

Results. One week after sending the emails, theSimplified Calendar Reminder email resulted in anearly 1 percentage point (p < 0.01, 95% CI [0.35,1.5]) higher completion rate amongst people sent
the email, a 13.5 percent higher relative completion
rate. The Simplified Calendar Reminder also promoted survey completion conditional on starting the survey—among those who viewed and started the survey, it led to completion rates that were 3 percentage
points higher (p < 0.01, CI [2.15, 4.25]).
10%
8%

7.05%

8.00%

6%
4%
2%
0%

Business
as Usual

Simplified Calendar
ReminderConclusions. A simplified, clear request, with an embedded reminder and a button to easily follow through,led more people to view, begin, and complete a workplace survey. Two days following the pilot, the more
successful version of the pilot launch message was sentout in the full TSS launch, helping to immediately scale
insights from the pilot test to help PBS reach their goalof gathering feedback from as many Federal tenants as
possible.

Background information on the Public Buildings Service is available here: gsa.gov/portal/content/104444
Todd Rogers, Katherine L. Milkman, Leslie K. John, and Michael I. Norton, “Making the Best Laid Plans Better: How Plan-Making
Increases Follow-Through, Behavioral Science and Policy (forthcoming).
77
78

SOCIAL AND BEHAVIORAL SCIENCES TEAM

2015 Annual Report43A b s t r a c t
P r o j e c t

Tenant Satisfaction Survey
Response: Time of Day
Lunchtime emails maximize click and open rates
Agency Objective. Increase open and click rates in
response to an email announcing a workplace survey.
Background. The General Service Administration(GSA)’s Public Buildings Service (PBS) owns or leases
over 9,600 assets, with more than 370 million square
feet of space for over a million Federal employees.79
Each year, PBS sends Federal employees the Tenant
Satisfaction Survey (TSS), the results of which help
determine Federal facilities strategy. In 2014, PBS
collaborated with the Social and Behavioral Sciences
Team (SBST) to simplify the survey and test different
messages for launching the survey, with the goal ofincreasing response rates.
Methods. In scaling the successful messages fromthe TSS launch pilot (see above, “TSS Response: Calendar Invite”), PBS spread out the full launch emails
among the intended 873,755 recipients over thecourse of Thursday, July 10, 2014. In order to reduceburden on the survey site contractor and ensure thesite operated, the same email was scheduled to besent to randomly selected groups of 96,000 federal
employees every hour over the course of the day (at8:55 a.m., 9:55 a.m., etc.) (all send times Eastern).
The response to these launch emails overloaded
the survey contractor’s website, and further emails
were postponed following the 1:55 p.m. email (theremaining workers were sent launch emails over thefollowing week; see below, “TSS Response: Subject
Lines and Day of Week”). Data on recipient agency
and location came from Office of Personnel Management (OPM) records. GovDelivery, the email service,provided data on email open and click rates.

79

SOCIAL AND BEHAVIORAL SCIENCES TEAM

2015 Annual Report

Results. Relative to the 8:55 a.m. send, the highest
email open and click rates came near noon, at the
11:55 a.m. send. Controlling for the effects of states
and agency of recipients, emails sent at 11:55 a.m.
had an open rate that was 2.5 percentage points
higher (p < 0.01, 95% CI [2.16, 2.86]), and a click ratethat was 1.5 percentage points higher (p < 0.01, 95%
CI [1.19, 1.82])—that is, sending emails three hours
later meant that approximately 1,500 more people
read and followed through on the email request.
30%
25%
20%
15%

13.3

15.3

14.7

14.6

9:55

10:55

14.9

12:55

1:55

10%
5%
0%

8:55

11:55

Click Rate

Conclusions. Lunchtime was a better time of day
than 9 a.m. to send workplace survey requests to awork email address to maximize open and click rates.
These lessons may be useful for other broad workplace requests. Moreover, the methods of testing
different times of sending workplace email requests
are more broadly applicable across the Federal Government.

Background information on the Public Buildings Service is available here: gsa.gov/portal/content/104444

44

14.8

A b s t r a c t
P r o j e c t

Tenant Satisfaction Survey Response:
Subject Lines and Day of Week
Maximizing response to workplace emails by varying sendday, send time, and subject lines
Agency Objective. Determine what day of week
and time of day is best to send emails to increase email
opens and clicks; determine what subject line content
and punctuation increases opens and clicks.
Background. The General Service Administration(GSA)’s Public Buildings Service (PBS) owns or leases over
9,600 assets, with more than 370 million square feet ofspace for over a million Federal employees.80 Each year,
PBS sends Federal employees the Tenant Satisfaction
Survey (TSS), the results of which help determine Federal facilities strategy. In 2014, PBS collaborated with theSocial and Behavioral Sciences Team (SBST) to simplify
the survey and test different messages for launching thesurvey, with the goal of increasing response rates. Afterscaling the successful messages from the TSS launchpilot (see above, “TSS Response: Calendar Invite”), andoverwhelming the survey contractor’s website with afull launch (see above, “TSS Response: Time of Day”),
PBS and SBST spaced out the remaining announcement
population into three batches of roughly 23,000 recipients each day for seven business days.
Methods. An announcement email was sent to two
randomly assigned groups of approximately 11,500
Federal employees three times a day for a week—at
9:00 a.m., 12:30 p.m., and 4:00 p.m. (all send times
Eastern). In addition to time-of-day and day-of-weekvariation, PBS and SBST varied aspects of the subject line, testing paired permutations of the following
against each other: “2014 Tenant Satisfaction Survey,”“2014 Tenant Satisfaction Survey – Take it Today” and“2014 Tenant Satisfaction Survey – Please Take it Today.”The emails also varied the use of an exclamation pointon each of those subject lines. Data on recipient agency
and location came from Office of Personnel Management (OPM) records. GovDelivery, the email service,provided data on email open and click rates.
Results. Controlling for state and agency effects, thehighest likelihoods of opens and clicks came on Tuesdays (1.8 percentage points higher open, 1.2 percentage points higher click) and Wednesdays (1.9 percentage points higher open, 1.0 percentage point higher
click), relative to Fridays (p < 0.01). Relative to mornings, lunchtime sends had higher clicks (0.3 percentage
point) and opens (0.3 percentage point) (p < 0.01). And
relative to the lowest performing subject line (“2014
Tenant Satisfaction Survey – Take it Today”) adding“Please” led to 1.0 percentage point more clicks and
0.5 percentage point more opens (p < 0.01); adding an
exclamation point led to 0.8 percentage point fewer
opens, and 0.5 percentage point fewer clicks (p < 0.01).
Conclusions. The findings from these TSS tests—
that sending emails near lunchtime on Tuesdays andWednesdays resulted in higher responses; and thatsaying “please” in subject lines helped while adding an
exclamation point hurt—may not hold in every government or workplace email context. However, they provide meaningful evidence on responses to workplacerequests. Perhaps more significantly, the methods andpractices of testing different approaches to workplaceemail requests are broadly applicable across government.

Background information on the Public Buildings Service is available here: gsa.gov/portal/content/104444

80

SOCIAL AND BEHAVIORAL SCIENCES TEAM

2015 Annual Report45

A b s t r a c t

One small prompt saves money and resources

P r o j e c t

Double-Sided Printing

Methods. The Department ofAgriculture’s (USDA) Economic
Research Service (ERS), in collaboration with the Social and
Behavioral Sciences Team (SBST),
tested the effectiveness of adding a small cost to printing single- rather than doublesided for its employees. Behavioral science researchfrom other contexts suggests that even relatively minor costs associated with taking one action can be sufficient to lead individuals to take a different action.76

Agency Objective. Reduce cost and total paper use
at a Federal Government agency by employing a simple, low-cost pop-up message, delivered when users
print, to discourage single-sided printing.
Background. Not surprisingly given its scale, the Federal Government uses a great deal of paper. A 2009
survey indicated that the average government employee printed about 30 pages per day.74 At 2.6 million
executive branch employees and 240 working days
per year, that amounts to over 18 billion pages printed
per year.75 Reliable figures on therate of duplex printing (doublesided printing) across the government do not exist, but in data forthis project the baseline rate of
duplex printing was 46 percent
of all print jobs, implying significant scope to reduce total paper
use and reduce costs through theincreased use of double-sided
printing.

Implementation of this cost was randomized at theprinter level within ERS. The small cost to printing
single-sided was introduced by presenting individuals, when they attempted to print a single-sided document at a network printer in the treatment group, witha pop-up box (an image of which is displayed at right).

This pop-up required a second mouse click before single-sided printing would occur. If the individual did not
click “Print” after five minutes, the print job was deleted. The text of the pop-up notified individuals that
if they changed their default settings to duplex, theywould not face the pop-up in the future.
Results. This simple prompt increased the likelihoodof duplex printing on a given job by 5.8 percentage
points (p < 0.01, 95% CI [4.2, 7.4]), from a baseline of
46.0 percent.

Conclusions. These results suggest that a simple
message, which could easily be scaled to any networked printer using existing software, can be an inexpensive and effective way to reduce the proportion of
single-sided print jobs. Based on this result, ERS is scaling up the use of the pop-up box to all its networked
printers. That individuals are responsive to this intervention suggests that stronger interventions, such aschanging printer defaults to double-sided printing,could be more powerful still in this context.

From a 2009 survey conducted by Lexmark, summarized here: govexec.com/pdfs/051209rb1.pdf
Figures on Federal employment taken from: opm.gov/policy-data-oversight/data-analysis-documentation/federal-employmentreports/historical-tables/total-government-employment-since-1962/76
Marianne Bertrand, Sendhil Mullainathan, and Eldar Shafir, “A Behavioral-EconomicsView of Poverty,” American Economic Review
94 (2004): 419–423. Kurt Lewin, Field Theory in Social Science, (New York: Harper, 1951).
74
75

SOCIAL AND BEHAVIORAL SCIENCES TEAM

2015 Annual Report46This page intentionally left blank47

Report produced by the Graphics Team at the General Services Administration