PAY FOR SUCCESS

Collaboration among 

Federal Agencies 

Would Be Helpful as 

Governments Explore 

New Financing 

Mechanisms

Report to the Chairman, Committee on 

the Budget, U.S. Senate

September 2015

GAO-15-646

United States Government Accountability Office
Highlights of GAO-15-646, a report to the 

Chairman, Committee on the Budget, U.S.

Senate 

September 2015

PAY FOR SUCCESS

Collaboration among Federal Agencies Would be 

Helpful as Governments Explore New Financing 

Mechanisms

Why GAO Did This Study

Federal, state, and local agencies play 

an important role in improving social 

outcomes for society’s most vulnerable 

populations. A small number of state, 

local, and foreign governments are 

employing PFS to fund efforts 

designed to better serve these 

vulnerable populations.

GAO was asked to provide information 

about PFS. This report examines (1) 

how selected PFS projects have been 

structured and what potential benefits 

these projects can provide; (2) how 

selected PFS contracts have been 

structured to address potential project 

risks; and (3) the potential roles for the 

federal government’s involvement in 

PFS projects.

To address these objectives, GAO 

reviewed relevant literature on PFS; 

selected 10 PFS projects that reflected 

a variety of policy areas and were in 

different stages of implementation in 

state and local governments in the 

United States and the United Kingdom, 

where PFS originated; reviewed key 

documents and interviewed

stakeholders from organizations that 

played a role in the selected projects; 

and reviewed documents and 

interviewed officials from the eight 

federal agencies that participated in an 

informal PFS working group.

What GAO Recommends

To identify and broadly disseminate 

information on leading practices and 

lessons learned, the Director of OMB 

should establish a formal means for 

federal agencies to collaborate on 

PFS. OMB concurred with this 

recommendation and is working with 

agencies to explore options for 

collaboration.

What GAO Found

Pay for Success (PFS), also known as Social Impact Bonds, is a new contracting 

mechanism to fund prevention programs, where investors provide capital to 

implement a social service— for example, to reduce recidivism by former 

prisoners. If the service provider achieves agreed upon outcomes, the 

government pays the investor, usually with a rate of return, based on savings 

from decreased use of more costly remedial services, such as incarceration.

Stakeholders from the 10 PFS projects in GAO’s study said that PFS offers 

potential benefits to all parties in the project. For example, governments can 

implement prevention programs that potentially lead to reduced spending on 

social services and transfer the risk of failing to achieve outcomes to investors.

Roles of Organizations Involved in PFS Projects

The PFS projects in GAO’s review sought to manage potential risks in the 

feasibility assessment, design, and implementation of complex PFS contracts. 

For example, without safeguards, there is a risk that paying for outcomes could 

create perverse incentives, such as focusing on individuals who are easiest to 

serve rather than those most in need. To address this risk, PFS contracts 

included various provisions, such as only including those with the greatest need 

in the evaluation that determines if the government makes payments.

To date, federal government involvement in PFS has been limited. The Office of 

Management and Budget (OMB) has encouraged agencies to explore the use of

PFS as appropriate. Potential roles federal agencies could play in PFS projects

include making outcome payments or helping build capacity. We have previously 

reported that collaborative mechanisms, such as interagency groups, can be 

used to implement programs and share information. However, a formal 

mechanism for federal agencies to collaborate on PFS does not exist. Given the 

evolving nature of PFS, a mechanism for federal agencies to collaborate on PFS 

would increase access to leading practices.

View GAO-15-646. For more information, 

contact Michelle Sager at (202) 512-6806 or 

sagerm@gao.gov.

United States Government Accountability Office
Letter 1

Background 5

PFS Projects Were Structured in a Variety of Ways and Sought to 

Achieve Potential Benefits, Such as Improved Social Outcomes 

and Cost Savings 11

Participating Organizations Sought to Manage Risks in the 

Feasibility Assessment, Design, and Implementation Phases of 

Complex PFS Contracts 33

The Federal Government Could Play a Role in Funding, Building 

Capacity for, and Managing Investor Risk in PFS, but a Formal 

Mechanism to Collaborate and Share Lessons Learned Does 

Not Exist 58

Recommendation for Executive Action 65

Agency Comments and Our Evaluation 66

Appendix I Objectives, Scope, and Methodology 67

Appendix II Illustrative Examples of Evidence-Based Programs from Selected 

Federal Agencies 71

Appendix III Overview of PFS Projects in Our Study 73

Appendix IV Example of United Kingdom Rate Card for Social Impact Bonds 77

Appendix V Comments from the Office of Management and Budget 79

Appendix VI GAO Contact and Staff Acknowledgments 80

Tables

Table 1: Federal Agencies with Funds Appropriated for PFS 

Projects 6

Contents

Page i GAO-15-646 Pay For Success
Table 2: Examples of Outcome Payment Structures from Selected 

Pay for Success Projects 45

Table 3: Illustrative Examples of Evidence-Based Programs from 

Selected Federal Agencies 71

Table 4: Overview of U.S. PFS Projects in Our Study 73

Table 5: Overview of United Kingdom (UK) PFS Projects in Our 

Study 75

Figures

Figure 1: Types of Organizations Most Commonly Involved in Pay 

For Success Projects and the Roles They Play 7

Figure 2: Phases of a PFS Project 9

Figure 3: Roles and Funding Characteristics of the Massachusetts 

Juvenile Justice Pay For Success Initiative 14

Figure 4: Roles and Funding Characteristics of the New York City 

Rikers Island Project 16

Figure 5: Roles and Funding Characteristics of the New York 

Increasing Employment and Improving Public Safety Pay 

For Success Project 18

Figure 6: Roles and Funding Characteristics of the Utah High 

Quality Preschool Project 20

Figure 7: Roles and Funding Characteristics of the Peterborough 

Prison Project 22

Figure 8: Roles and Funding Characteristics of the London 

Homelessness Project 24

Figure 9: Roles and Funding Characteristics of the It’s All About 

Me Project 26

Figure 10: Estimated Savings and Benefits and Outcome 

Payments at Various Reductions in Recidivism Rates 39

Figure 11: United Kingdom Department for Works and Pensions 

(DWP) Rate Card 78

Abbreviations

AIM4Fresno Asthma Impact Model For Fresno

ARRA American Recovery and Reinvestment Act of 2009 

CEO Center for Employment Opportunities

CNCS Corporation for National and Community Service

DLS Maryland Department of Legislative Services

Page ii GAO-15-646 Pay For Success
DOCCS New York Department of Corrections and Community

Supervision

DOJ Department of Justice

DWP UK Department for Works and Pensions

Education Department of Education

HHS Department of Health and Human Services

HUD Department of Housing and Urban Development

i3 Investing in Innovation Fund 

IAAM It’s All About Me Adoption Project

IMLS The Institute of Museum and Library Services

JRI Justice Reinvestment Initiative 

Labor U. S. Department of Labor

LEAs local educational agencies 

MIECHV Maternal, Infant, and Early Childhood Home Visiting 

Program

NY DOL New York Department of Labor

NVQ National Vocational Qualifications

OMB Office of Management and Budget

P3 Performance Partnership Pilots for Disconnected Youth

PFS Pay for Success

PROMISE Promoting the Readiness of Minors in Supplemental 

Security Income 

RCT random control trial

REA/RES Reemployment and Eligibility Assessments and 

Reemployment Services 

SIF Social Innovation Fund

SIB Social Impact Bond

SSA Social Security Administration

SSI Supplemental Security Income 

Treasury Department of the Treasury

UK United Kingdom

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.

Page iii GAO-15-646 Pay For Success
441 G St. N.W.

Washington, DC 20548

September 9, 2015

The Honorable Mike Enzi

Chairman

Committee on the Budget

United States Senate

Dear Mr. Chairman:

Federal, state, and local agencies play an important role in improving 

outcomes for our society’s most vulnerable populations, who often face 

complex social or economic problems. Agencies at all levels of 

government can improve the lives of vulnerable populations and make 

effective use of available resources by allocating them to prevention 

programs that demonstrate the promise to innovatively solve intractable 

social challenges. These challenges include reducing recidivism rates of 

recently released prisoners, helping homeless individuals find stable 

accommodation, and expanding access to quality early childhood 

education for economically disadvantaged children. Moreover, by defining 

and achieving prevention outcomes, agencies can potentially reduce 

future budgetary pressures by lessening the need for costly remediation 

programs.

A small number of foreign and U.S. state and local governments are 

employing a contracting mechanism known as a Social Impact Bond 

(SIB)1 or a Pay for Success (PFS)2 program to fund preventive efforts 

designed to more effectively serve vulnerable populations. In a PFS 

contract, private or philanthropic capital is used to fund an intervention 

that targets the achievement of specific outcomes, such as a reduction of 

the recidivism rate and an increase in the employment rate of a target 

population. After confirming that outcomes have been achieved, a 

government entity repays the other funders, usually with a rate of return. 

1

The term SIB is used in the United Kingdom (UK). According to the UK Centre for Social 

Impact Bonds, a SIB is a way of financing a Payment by Results (PBR) contract. Under a 

PBR contract, government pays a service provider if it achieves certain results.

2

Throughout this report, we refer to the mechanism as Pay for Success, as the term is 

used more widely than Social Impact Bond in the United States. Both terms are used by 

other entities to describe this contracting mechanism. 

Letter

Page 1 GAO-15-646 Pay For Success

 
Proponents of PFS believe that this contracting mechanism allows 

government entities to implement innovative, evidence-based social 

services that better serve vulnerable populations, and that it does so at a 

reduced level of risk to agency budgets because government payments 

are contingent on achieved outcomes. On the other hand, skeptics of 

PFS believe that the mechanism is unlikely to reduce the level of risk to 

agency budgets and could actually increase a government entity’s cost of 

implementing a program as a result of the higher administrative costs 

associated with the complexity of the project design.

To date, four federal agencies have allocated resources to support PFS 

projects. The President and members of Congress have made proposals 

to expand federal support for PFS projects. Since fiscal year 2014, the 

President’s budgets have proposed a $300 million appropriation for a new 

PFS Incentive Fund in the Department of the Treasury to expand the role 

of the federal government and support state and local governments and 

other intermediaries seeking to establish PFS projects. Members of 

Congress have also introduced legislation during the 113th and 114th 

Congresses on PFS to expand federal support for PFS projects.3

You asked us for information to help Congress have a better 

understanding of PFS. This report examines (1) how selected PFS 

projects have been structured and what potential benefits these projects 

can provide; (2) how selected PFS contracts have been structured to 

address potential project risks; and (3) the potential roles for the federal 

government’s involvement in PFS projects.

To address these objectives, we identified and reviewed relevant 

literature on PFS, including work produced by public sector, nonprofit, 

academic, and private sector organizations, and interviewed individuals 

with public management expertise who had examined PFS and published 

work on or related to PFS. Based on the literature review and the 

interviews, we identified jurisdictions in the United States and abroad that 

were implementing, planning to implement, or studying the potential of 

PFS projects. For illustrative purposes, we selected seven U.S. and three 

United Kingdom (UK) cases to review in depth. We selected cases from 

the UK to capture the country’s experience as the first to develop and 

3

Social Impact Partnership Act, H.R. 1336, 114th Cong. (2015); Social Impact Partnership 

Act, S. 1089, 114th Cong. (2015); Social Impact Bond Act, H.R. 4885, 113th Cong. (2014); 

Pay for Performance Act, S. 2691, 113th Cong. (2014).

Page 2 GAO-15-646 Pay For Success

 
implement PFS projects as well as the location with the longest 

experience with this financing mechanism. For the U.S. projects we 

selected the four PFS projects that were being implemented in the United 

States at the time we began our review in July 2014. Those projects were 

the Massachusetts Juvenile Justice PFS Initiative; the New York City 

Rikers Island Project; the New York PFS project: Increasing Employment 

and Improving Public Safety; and the Utah High Quality Preschool 

Program. Because those projects focused heavily on reducing recidivism 

and improving early childhood education, we selected two additional U.S. 

projects that were in development to reflect an additional policy area 

(improving health outcomes). Those projects were the South Carolina 

Maternal Health PFS Initiative and the Asthma Impact Model for Fresno 

(AIM4Fresno). Finally, we selected one case in which a state considered 

but ultimately decided not to move forward with a PFS project. That case 

was Maryland’s review of a potential PFS initiative focused on reducing 

recidivism. For the UK projects, we selected three projects that varied in 

policy area, structure, and the participating organization that conceived 

them. Those cases were the Peterborough Prison Project; the London 

Homelessness Project; and the It’s All About Me Project on reducing wait 

times for adoption.

We reviewed documents relevant to the case illustrations, including 

feasibility studies, contracts, and evaluation reports. We conducted indepth interviews with representatives of organizations participating in 

each of these projects, including representatives from government 

entities, intermediaries who help manage PFS projects, service providers, 

investors, evaluators, and validators, who validate evaluation results. We 

refer to individuals from these organizations by using particular categories 

of stakeholders, such as “investors,” when the distinction is meaningful. 

We refer to them collectively as “stakeholders” when all organization 

representatives provided a similar point of view.

To assess the PFS contract structure, we reviewed our prior reports and 

contracting literature to identify contracting practices from the federal 

experiences relevant to PFS contracts, which are complex service 

acquisitions. We compared these contracting practices to the PFS 

contracts in our study to establish their relevance to the design and 

implementation of PFS contracts.

We also examined federal agencies’ involvement in PFS by reviewing 

relevant documents, including the President’s Budget for fiscal years 

2012 through 2016. We identified eight federal agencies that were 

involved in an informal PFS working group and have played different roles 

Page 3 GAO-15-646 Pay For Success
vis-á-vis PFS. The Office of Management and Budget (OMB), working 

with the White House Office of Social Innovation and Civic Participation, 

has provided informal guidance on PFS efforts and participated in the 

informal working group. The Department of the Treasury (Treasury) 

participated in the informal working group. The Departments of Labor 

(DOL), Housing and Urban Development (HUD), and Justice (DOJ) and 

the Corporation for National and Community Service (CNCS) have 

funded or plan to fund specific PFS projects. The Departments of Health 

and Human Services (HHS) and Education (Education) have been 

exploring opportunities to pursue the PFS concept. We interviewed 

officials from each of these agencies about their involvement in PFS. Of 

the 8 agencies in the working group, we also interviewed officials and 

reviewed budget documents from 6 agencies that typically fund social 

programs—CNCS, DOJ, DOL, Education, HHS, and HUD—to identify 

illustrative examples of how these agencies have used evidence-based 

approaches in their decision making more generally.

We conducted our work from July 2014 to September 2015 in accordance 

with generally accepted government auditing standards. Those standards 

require that we plan and perform the audit to obtain sufficient, appropriate 

evidence to provide a reasonable basis for our findings and conclusions 

based on our audit objectives. We believe that the evidence obtained 

provides a reasonable basis for our findings and conclusions based on 

our audit objectives.

Page 4 GAO-15-646 Pay For Success
Since 2010, OMB has used its annual budget guidance to broadly 

encourage federal agencies to use evidence to consider the effectiveness 

of their programs and to institutionalize the use of evidence to foster 

innovation rooted in research and rigorous evaluation. In a 2013 memo, 

OMB identified areas in which agencies could improve their use of 

evidence, including: (1) strengthening evaluation capacity and proposing 

new evaluations; (2) developing high-quality, low-cost evaluations and 

rapid, iterative experimentation; (3) using innovative outcome-focused 

design; and (4) increasing agency capacity to use evidence.4 In 2015, 

OMB required agencies to include an evidence template with their fiscal 

year 2017 budget submissions. The template should provide an overview 

of evidence-building strategies and identify related priorities.5 The six 

federal agencies in our study that typically fund social programs said they 

used evidence to inform a range of program proposals or decisions. See 

appendix 2 for illustrative examples of these agencies’ evidence-based 

programs.

As part of its agenda to base management and policy decisions on 

evidence, the White House and OMB have encouraged agencies to 

explore PFS, where appropriate, as one tool to incorporate evidence into 

grant making. As of July 2015, four federal agencies have awarded PFS 

grants or are developing PFS grant proposals (see table 1).

4

Office of Management and Budget, Next Steps in the Evidence and Innovation Agenda,

OMB Memorandum M-13-17 (Washington, D.C.: July 26, 2013). 

5

Office of Management and Budget, Fiscal Year 2017 Budget Guidance, OMB 

Memorandum M-15-11 (Washington, D.C.: May 1, 2015).

Background

OMB Guidance 

Encourages Agencies to 

Use Evidence-Based 

Approaches, Including 

PFS

According to OMB’s 2013 Memo on the 

Next Steps in the Evidence and Innovation 

Agenda

...evidence includes evaluation results, 

performance measures, and other relevant 

data analytics and research studies, with a 

preference for high-quality experimental and 

quasi-experimental studies.

Source: OMB | GAO-15-646

Page 5 GAO-15-646 Pay For Success

 
Table 1: Federal Agencies with Funds Appropriated for PFS Projects

Agency Program name Amount Description

Department of Labor 

(DOL)

Workforce Innovation 

Fund

$24 million In fiscal year 2013, DOL awarded grants to support outcome payments 

for anti-recidivism PFS projects in New York and Massachusetts. The 

New York and Massachusetts PFS projects in our review were the 

recipients of these grants.

Corporation for 

National and 

Community Service 

(CNCS)

Social Innovation 

Fund

$12 million In fiscal year 2014, CNCS awarded $12 million in grants to 

intermediaries to provide technical assistance to states and agencies 

interested in pay for success projects that cover a range of issues, 

such as youth development, asthma management, homelessness 

prevention and storm water infrastructure.

Department of 

Housing and Urban 

Development (HUD) 

and Department of 

Justice (DOJ)

PFS Supportive 

Housing 

Demonstration

Up to $10 

million

In fiscal year 2014, HUD and DOJ entered into an interagency 

agreement to develop a demonstration project to test the effectiveness 

and cost-effectiveness of the permanent supportive housing concept 

funded via a PFS model. For this demonstration, the target population 

consists of people experiencing homelessness who are high users of 

corrections facilities, homeless services, health care and other crisis 

services.

Source: GAO analysis of data provided by selected agencies. | GAO-15-646

PFS is a program funding mechanism that some U.S. state and local and 

foreign governments are using to contract for prevention services to 

address entrenched social problems. The first PFS project began in the 

United Kingdom in 2010. In a PFS contract, private investors provide 

upfront capital that a service provider uses to implement an evidencebased prevention program. An intermediary organization can solicit 

funding from the investors, facilitates the flow of capital from the investors 

to the service provider, and oversees the provider’s work throughout the 

project. If the program is successful, the government entity makes 

payments to the intermediary, which in turn may repay the investors their 

capital along with interest. These payments are, in theory, based on the 

savings that result from decreased use of government services. PFS is 

intended to allow government to transfer the risk of paying for a program 

that does not achieve desired outcomes from the government to other 

participating organizations. If the program is successful, the government 

obtains improved outcomes for vulnerable populations. In addition, PFS 

can potentially capture cost savings by reducing the need to provide 

costly remediation services to the individuals in vulnerable populations. 

However, stakeholders said that PFS can also be used to contract for 

outcomes that a government deems worthwhile, even if the outcomes do 

not generate savings. Because the government makes payments based 

on the service provider meeting outcome targets, all parties to the 

contract have a vested interest in the success of the service provider.

PFS Is a Mechanism for 

Contracting for Social 

Outcomes

Page 6 GAO-15-646 Pay For Success
An individual PFS project comprises a series of contracts and other 

vehicles, such as loan and grant agreements, that defines entities’ roles 

in the project and identifies the risks they each agree to assume and the 

benefits they each stand to receive by virtue of their participation.

Although individual PFS projects may vary in terms of roles and 

responsibilities for the parties to the contract, there are five main types of 

organizations that commonly participate in a PFS project: government, 

intermediaries, investors, service providers, and evaluators (see figure 1).

Figure 1: Types of Organizations Most Commonly Involved in Pay For Success 

Projects and the Roles They Play

Page 7 GAO-15-646 Pay For Success
Government originates a contract for defined outcomes and pays on the 

basis of those outcomes being achieved. In the United States, state and 

local governments have initiated PFS projects to date, while in the UK 

both national and local-level governments have initiated them. Examples 

of government entities that have been involved in PFS projects include 

state or local programmatic agencies and central management offices, 

which often work together collaboratively on a project. As mentioned 

above, DOL and CNCS have awarded grants to support state and local 

PFS projects.

Investors provide capital to finance a program. They provide capital on 

the condition that the government will repay the investment if outcomes 

are achieved, usually with a rate of return. In U.S. projects to date, capital 

has been provided by a mix of investor types, such as banks, 

foundations, and high net worth individuals, whereas in UK projects, 

capital has been provided solely by social impact investors.6

Intermediaries enter into a government contract in a PFS project and 

usually receive a fee for the services they provide. Intermediaries can 

take on a variety of roles in a PFS project, such as providing technical 

support to governments on the feasibility of a project, identifying the 

service provider, raising operating funds from investors, negotiating the 

terms of contracts or agreements with governments, service providers, 

and investors; disbursing payments to service providers; and overseeing 

day-to-day performance management of the project. Examples of 

intermediaries that have been involved in PFS projects include nationally 

focused nonprofit organizations with public policy and financial expertise 

and a locally focused nonprofit organization with experience implementing 

programs within local communities.

Service providers enter into contracts with an intermediary or a 

government to administer the evidence-based prevention program. 

Examples of service providers that have been involved in PFS projects 

include nonprofit or charitable organizations with established track 

records of implementing programs in specific policy areas, such as job 

training or homelessness prevention.

6

Social impact investors seek investments that intentionally target specific social 

objectives along with a financial return and measure the achievement of both types of 

outcomes. G8 Social Impact Investment Taskforce, Impact Investment: The Invisible Heart 

of Markets (Sept. 15, 2014).

Page 8 GAO-15-646 Pay For Success

 
Evaluators measure a program’s outcomes or evaluate a program’s 

effect on the target population. The evaluator’s work assures the other 

participating organizations that defined outcomes were achieved and are 

likely due to the program, rather than to other factors, including chance. 

Outcome payments are based on these findings. In some cases, a 

validator, independent of the evaluator, is responsible for certifying the 

evaluator’s findings. Examples of evaluators that have been involved in 

existing PFS projects include a research office within a government 

department and an organization with experience evaluating programs 

within a specific public policy area, such as early childhood education.

The PFS projects we reviewed generally had three phases: a feasibility 

assessment, project design, and program implementation (see figure 2).

Figure 2: Phases of a PFS Project

As described previously, a PFS project is a complex set of contracts 

through which multiple organizations work together to deliver social 

service programs. PFS projects are similar to complex acquisitions. 

Complex acquisitions are defined, in part, by the difficulty in defining 

precise terms of exchange for all the characteristics of the product or 

service at the time of the contract negotiation. We have previously 

identified key factors that government entities should incorporate into 

complex acquisitions to achieve improved service acquisition outcomes.7

Key literature on complex acquisitions we reviewed reinforces these 

7

GAO, Defense Acquisitions: Tailored Approach Needed to Improve Service Acquisition 

Outcomes, GAO-07-20 (Washington, D.C.: Nov. 9, 2006).

Practices from Federal 

Experience with Complex 

Service Acquisitions Can 

Inform PFS Projects

Page 9 GAO-15-646 Pay For Success

 
factors.8 Taken together, this understanding can inform the design and 

management of PFS contracts. To address the inherent ambiguity in 

complex acquisitions, parties can include processes in the contract to 

manage decision making and to make ongoing adjustments during 

implementation. Specifically, providing clear standards in the following 

areas can facilitate PFS contracts:

• Valid and well-defined requirements. We have previously reported 

that establishing a valid need for a service, and then translating that 

need into a service acquisition requirement, is essential for obtaining 

the right outcome.9 Literature on complex contracting underscores the 

need for well-defined requirements, but states that it can be 

challenging to define all requirements upfront in a complex 

acquisition. Undefined requirements leave room for interpretation that, 

if not carefully managed, can lead to outcomes that are inconsistent 

with expectations.10

• Properly structured business arrangements. We have previously 

reported that once requirements have been defined, it becomes 

necessary to develop a business arrangement to meet the 

requirements and protect the government’s interests. At a basic level, 

properly structuring business arrangements includes defining a clear 

scope of expected contractor performance and developing an 

objective means of assessing the contractor’s performance. There 

should be clearly established relationships among the tasks the 

contractor is expected to perform, the contract terms and conditions, 

and performance evaluation factors and incentives.11 Literature on 

complex contracting underscores the importance of properly 

structured business arrangements, including governance rules, which 

should define how the government and contractor will interact to make 

decisions during the contract’s implementation.12

8

Trevor L. Brown, Matthew Potoski, and David M. Van Slyke, Complex Contracting: 

Government Purchasing in the Wake of the U.S. Coast Guard’s Deepwater Program (New 

York: Cambridge University Press, 2013).

9

GAO-07-20.

10Brown, Potoski, and Van Slyke, Complex Contracting.

11GAO-07-20.

12Brown, Potoski, and Van Slyke, Complex Contracting.

Page 10 GAO-15-646 Pay For Success

 
• Proactively managed outcomes. We have previously reported that 

managing and assessing post-award performance entails various 

activities performed by government officials to ensure that the delivery 

of services meets the terms of the contract, including adequate 

oversight resources, proper incentives, and a capable workforce for 

overseeing contractor activities. Each of these activities requires 

metrics and tools to encourage contractors to provide superior 

performance and to manage and document acceptable contractor 

performance.13 Literature on complex contracting also underscores 

the importance of rewarding successful outcomes, particularly in 

cases where it is difficult to define requirements, such as efforts to 

acquire services.14

In our selected case illustrations, governments used PFS to address 

entrenched social problems. The individuals who face these problems 

have complex, multi-faceted needs and are often poorly served by 

existing interventions or programs. The organizations participating in the 

PFS projects in our study said that these individuals can be better served 

through prevention programs focused on defined outcomes. Stakeholders 

told us that PFS is just one tool among many for addressing social 

problems and that PFS is not a panacea.

13GAO-07-20.

14Brown, Potoski, and Van Slyke, Complex Contracting. 

PFS Projects Were 

Structured in a Variety 

of Ways and Sought 

to Achieve Potential 

Benefits, Such as 

Improved Social 

Outcomes and Cost 

Savings

PFS Projects Were 

Structured to Implement 

Prevention Programs to 

Address Entrenched 

Social Problems

Page 11 GAO-15-646 Pay For Success

 
Stakeholders said PFS is most appropriate in policy areas in which 

populations face complex, entrenched problems. One stakeholder also 

noted that PFS may work best on projects that supplement, not supplant, 

existing services, such as prevention programs for which existing services 

have been insufficient in achieving desired outcomes. Projects in our 

study used PFS to implement programs intended to reduce recidivism 

among an adolescent jail population, gaps in academic achievement 

among preschoolers, homelessness for difficult-to-serve individuals, wait 

times for adoption, and to provide preventive health care to address poor 

birth outcomes and asthma.

Page 12 GAO-15-646 Pay For Success
The Massachusetts Juvenile Justice PFS Initiative, which began in 

October 2013, was designed to address high rates of recidivism among 

young men. Every year approximately 4,000 high-risk young men age out 

of the state’s juvenile justice system or are released from adult 

probation.15 Sixty-four percent of these young men are incarcerated at 

least once within 5 years of release. Since its initial design, the project 

has been expanded to serve young men connected to the adult 

corrections system who are at high risk of future adult incarceration. The

project is being used to fund an intervention provided by Roca, Inc.,16

which has been shown to reduce incarceration rates for the young men it 

serves by 33 percent.

Massachusetts will make payments on the basis of reduced incarceration, 

increased employment, and for each individual program participant who 

meets 9 or more times per quarter with a Roca staff member to discuss 

job readiness issues. The project’s target impact is a 40 percent reduction 

in participants’ days in incarceration compared to a control group during 

the 5 years following their enrollment in the program. At that level, 

Massachusetts expects to make $21 million in outcome payments and 

realize $22 million in gross savings during the approximately 6-year 

lifespan of the project. If participants’ days in incarceration are reduced by 

55 percent, Massachusetts expects to make the maximum $28 million in 

outcome payments and realize $33 million in gross savings. The U.S. 

Department of Labor (DOL) awarded Massachusetts an $11.7 million 

grant for the project. Up to $10.8 million of this amount is available for 

outcome payments associated with the first 2 years of service for up to 

535 young men.17 Massachusetts made its first payments related to the 

job readiness metric in May 2015. The state will make its first payments 

related to the incarceration and employment metrics in the fall of 2017. 

See figure 2 for an example of the roles of the organizations participating 

in the project and the size of the project’s investment and potential 

savings.

15Third Sector Capital Partners, The Massachusetts Juvenile Justice PFS Initiative: 

Project Brief (Boston, MA: 2014).

16Roca is a nonprofit organization that works with high-risk young people ages 17 to 24 in 

Massachusetts to help them break cycles of poverty, violence, and incarceration. 

17The remainder of the grant will fund the project’s evaluation and other administrative 

costs. 

Massachusetts: Reducing 

Recidivism and Increasing 

Employment among ExOffenders

Massachusetts Juvenile Justice PFS 

Initiative 

The “Roca Model,” which the project is 

funding, includes 4 basic elements: outreach 

to young men; intensive case management; 

life skills, educational, prevocational, and 

employment programming; and a connection 

to work opportunities. Project stakeholders 

believe that the Roca Model will be able to 

help young men exiting the criminal justice 

system to reduce the average 2.4 years they 

spend in prison for new convictions after they 

are released. 

Source: Commonwealth of Massachusetts, Roca, and Third 

Sector Capital | GAO-15-646

Page 13 GAO-15-646 Pay For Success

 
Figure 3: Roles and Funding Characteristics of the Massachusetts Juvenile Justice Pay For Success Initiative

a

Massachusetts contracted with Third Sector Capital Partners and Roca for the Juvenile Justice PFS 

Initiative.

b

The senior loan will be repaid before the junior loan, and receive higher returns.

c

Third Sector Capital Partners and Roca will be repaid their deferred service fees if the program 

achieves its target impact of a 40 percent reduction in days in incarceration.

Page 14 GAO-15-646 Pay For Success
Adolescents released from Rikers Island jail in New York City have a 47 

percent chance of returning to jail within 1 year.18 The New York City 

Rikers Island Project, which began in September 2012, was designed to 

address this recidivism rate. To do so, Osborne Association and Friends 

of Island Academy19 administered a form of cognitive behavioral therapy 

called Moral Reconation Therapy to adolescents at Rikers Island.20

Project participants estimated that the city could save $1.7 million in net 

long-term costs if participants’ future days in jail are reduced by 11 

percent compared to a similar group that did not receive the therapy. 

They estimated the city could save $20.5 million if participants’ future 

days in jail are reduced by 20 percent. In July 2015, the project’s 

evaluator determined that the program did not lead to reductions in 

recidivism rates for participants in its 1-year assessment of the project. 

The New York City government did not make outcome payments to 

investors and the program was discontinued as of August 31, 2015. See 

figure 3 for a description of the roles of the organizations participating in 

the project and the size of the project’s investment and potential savings.

18MDRC, Financing Promising Evidence-Based Programs: Early Lessons from the New 

York City Social Impact Bond (New York, NY: Dec. 2013).

19The Osborne Association is a nonprofit social service organization that provides 

treatment, educational, and vocational services to men and women involved in the 

criminal justice system in New York. Friends of Island Academy is a nonprofit youth 

development organization that provides prerelease and transitional services on Rikers 

Island and neighborhood-based services in education, employment and well-being for 

high-risk New York City youth involved in the justice system

20Moral Reconation Therapy is a cognitive behavioral program designed to help offenders 

reevaluate their choices and enhance their decision-making abilities.

New York City: Reducing 

Recidivism in an Adolescent 

Jail Population

The New York City Rikers Island Project 

This project grew out of the Mayor Bloomberg 

Administration’s Young Men’s Initiative (YMI), 

an effort to improve the lives of black and 

Latino men, who experience much higher 

rates of poverty, unemployment, and 

homicide than their white and Asian peers. 

Ideas for programs within YMI exceeded 

available funding, so city leaders decided to 

try to fund a program within YMI with PFS, 

which they saw as a way to fund an 

innovative approach during a period of 

constrained government funding. 

Source: MDRC | GAO-15-646

Page 15 GAO-15-646 Pay For Success

 
Figure 4: Roles and Funding Characteristics of the New York City Rikers Island Project

a

New York City contracted with MDRC for the Rikers Island Project. In July 2015, the evaluator 

determined that the program did not lead to reductions in recidivism rates for participants in its 1-year 

assessment of the project. The New York City government did not make outcome payments to 

investors and the program was discontinued as of August 31, 2015.

Page 16 GAO-15-646 Pay For Success
The New York PFS project, which began in October 2013, was designed 

to reduce the high recidivism rate and increase the low employment rate 

of high-risk individuals recently released from state prison each year. 

According to a report on the project,21 these individuals spend an average 

of 460 days in prison or jail in the 5 years following their release. The 

project is being used to fund an intervention by the Center for 

Employment Opportunities (CEO).22 A random control trial (RCT) 

evaluation showed CEO’s intervention reduced recidivism for recently 

released participants by 16 to 22 percent and days incarcerated for highrisk participants by 30 percent. New York will make payments on the 

basis of reduced recidivism, increased employment, and engagement in 

transitional jobs for recently released, high-risk individuals. New York 

estimates that it can capture savings and benefits ranging from $13 

million if recidivism is reduced by 10 percent to $37 million if recidivism is 

reduced by 40 percent over a 5-year period. The U.S. Department of 

Labor (DOL) awarded New York a $12 million grant for the project. Up to 

$11.1 million is available for outcome payments associated with the first 2 

years of the project.23 The first outcome payment would occur in 2017. 

See figure 4 for a description of the roles of the organizations participating 

in the project and the size of the project’s investment and potential 

savings and benefits.

21State of New York, Center for Employment Opportunities, and Social Finance, Investing 

in What Works: “Pay for Success” in New York State: Increasing Employment and 

Improving Public Safety (March 2014).

22The Center for Employment Opportunities is a nonprofit organization that provides 

employment services, including life skills education, short-term paid transitional 

employment, and full-time job placement, for men and women with criminal records.

23The grant will also be used for other expenses, such as funding the validator for the 

project. 

New York: Increasing 

Employment and Improving 

Public Safety

New York: Increasing Employment and 

Improving Public Safety 

New York recently established the broader 

“Work for Success” initiative, which aims to 

improve the process by which those who 

have served time in prison are trained and 

connected to businesses looking to hire. The 

state has launched the PFS project to 

complement the initiative because it focuses 

on delivering results for hard-to-serve, 

formerly incarcerated individuals, and ensures 

that resources are only expended if results 

are achieved. 

Source: State of New York, Center for Employment 

Opportunities, and Social Finance | GAO-15-646

Page 17 GAO-15-646 Pay For Success

 
Figure 5: Roles and Funding Characteristics of the New York Increasing Employment and Improving Public Safety Pay For 

Success Project

a

New York contracted with Social Finance for the Increasing Employment and Improving Public 

Safety project.

b

The first-loss guarantee is intended to protect up to $1.3 million of investor principal, or 

approximately 10 percent of the total capital raised.

Page 18 GAO-15-646 Pay For Success
In 2011, a 3-year study conducted by Utah’s Granite School District 

showed that between 25 percent and 33 percent of low-income students 

in the district would likely be placed in special education without a 

preschool intervention.24 The Utah PFS project was designed to decrease 

the use of special education and to address academic achievement gaps 

for low-income students who enter kindergarten without having attended 

high-quality preschool. The project began in school year 2013-2014, with 

outcome payments being supported by Salt Lake County and the United 

Way of Salt Lake. Beginning in school year 2014-2015, Utah became the 

outcome payor.

The project aims to expand Granite School District’s high-quality 

preschool program statewide, which evidence has shown reduces by 95 

percent the need for special education among the low-income students 

who would otherwise likely be placed in special education programs. 

Economically disadvantaged students who are eligible to receive a free or 

reduced price lunch are eligible to be enrolled in the program. Of these 

students, those with the most extreme negative scores (as reflected by 

their scoring two standard deviations below the mean) on a test taken at 

the beginning of preschool are tracked from kindergarten through sixth 

grade to verify whether they are enrolled in special education.25 Beginning 

in the summer of 2016, Utah will make annual payments to investors that 

equal 95 percent of cost savings for each year from kindergarten through 

sixth grade that each designated child is not enrolled in special education. 

Utah estimates it can save approximately $2,700 per year if a child is not 

enrolled in special education. See figure 5 for a description of roles of the 

organizations participating in the project and the size of the project’s 

investment and potential savings and benefits.

24Janis Dubno, Voices of Utah Children, A Sustainable Financing Model High Quality 

Preschool for At-Risk Children: Results from the Granite School District in Utah (Salt Lake 

City, UT: Sept. 7, 2011). 

25According to the project intermediary, the predicted use of special education is based on 

scoring at or below 70, or two standard deviations below the mean, on the Peabody 

Picture Vocabulary Test. According to the intermediary, research has supported a 

relationship between the test and later school and special education outcomes. The 

Peabody Picture Vocabulary Test measures receptive vocabulary in English.

Utah: Reducing Gaps in 

Academic Achievement among 

Preschoolers

Utah High Quality Preschool Project 

The United Way of Salt Lake (UWSL), which 

led the development of the Utah PFS project 

in partnership with Voices for Utah Children, 

Salt Lake County, and Granite School District, 

said that the project is a component of its 

focus on “collective impact,” a place-based, 

evidence-driven, collaborative approach to 

providing social services. UWSL believes the 

PFS model is a mechanism for leveraging a 

cross-sector collaborative strategy to bring 

the effective preschool program to scale to 

improve outcomes for more at-risk children. 

Source: United Way of Salt Lake | GAO-15-646

Page 19 GAO-15-646 Pay For Success

 
Figure 6: Roles and Funding Characteristics of the Utah High Quality Preschool Project

a

Utah contracted with the United Way of Salt Lake for the High Quality Preschool Project.

b

The subordinate loan funding reduces risk to the senior lender if the preschool program proves to be 

ineffective.

Page 20 GAO-15-646 Pay For Success
The Peterborough Prison Project, which began in September 2010, was 

designed to reduce the recidivism rate among prisoners released from 

Peterborough Prison after serving a sentence of less than a year by 

improving their access to supportive social services. The project is being 

used to fund a range of services for the prisoners, including housing 

assistance, drug or alcohol treatment, and mental health support. The UK 

Ministry of Justice agreed to make outcome payments on the basis of

reductions in recidivism of 10 percent for each program cohort or group of 

prisoners served during a 2-year time frame, or a 7.5 percent reduction 

for the three cohorts as a whole.26 In April 2014, the Ministry of Justice 

announced that it would close the Peterborough Project 2 years early 

because of a shift to the Transforming Rehabilitation program, which will 

provide statutory services similar to those provided in the Peterborough 

Project for all short-sentenced prisoners. See figure 6 for a description of 

roles of the organizations participating in the project and the size of the 

project’s investment and potential savings and benefits.

26Social Finance, Social Impact Bonds: The One Service. One Year On (London, United 

Kingdom: November 2011). 

United Kingdom: Improving 

Services for Short-Sentenced

Prisoners

United Kingdom: Peterborough Prison 

Project

Social Finance, the project’s intermediary, 

determined social impact bonds could be 

effective in the policy area of criminal justice. 

Specifically, it identified the challenges that 

were preventing social service organizations 

from successfully helping short-sentenced 

prisoners reduce their recidivism rates and 

worked with stakeholders to develop a project 

that would help to overcome the challenges 

and improve the prisoners’ outcomes.

Source: Social Finance | GAO-15-646

Page 21 GAO-15-646 Pay For Success

 
Figure 7: Roles and Funding Characteristics of the Peterborough Prison Project

a

The UK Ministry of Justice contracted with Social Finance (UK) for the Peterborough Prison project.

b

A special purpose vehicle is a legal entity that is created solely for a particular financial transaction or 

to fulfill specific objectives.

Page 22 GAO-15-646 Pay For Success
The London Homelessness Project, which began in November 2012, was 

designed to improve outcomes for a cohort of entrenched homeless 

individuals who do not respond to traditional models of resettlement and 

who are not targeted by other initiatives. The project funds two providers 

to deliver services tailored to the specific needs of the individuals within 

the cohort. The Greater London Authority will pay the providers for 

achieving outcomes including a reduction in the number of individuals 

sleeping on the street; an increase in individuals who sustain stable 

accommodation and gain employment; a reduction in average use of the 

healthcare system for accident or emergency episodes; and reconnecting 

individuals to a country outside of the UK if he or she is a foreign national.

Targets for each of these outcomes are based on historical data on 

homeless individuals within London. The project does not include an 

evaluation for the purposes of determining outcome payments because it 

was not possible to create a matched comparison group for the cohort. 

This is because other cities in the UK do not have as complete data on 

homeless individuals as does London and the cohort is unique within 

London. A feasibility study estimated the net present value of providing 

services across the five outcome areas to the entire cohort over 5 years 

to be approximately £30 million (approximately $49 million). If all of the 

project outcomes are achieved, the maximum outcome payments and 

administrative costs will total £5 million (approximately $8 million), thus 

resulting in substantial public sector savings.27 See figure 7 for a 

description of roles of the project’s participating organizations.

27United Kingdom Department for Communities and Local Government, Qualitative 

Evaluation of the London Homelessness Social Impact Bond: First Interim Report

(London, United Kingdom: September 2014).

United Kingdom: Improving 

Services for Homeless 

Individuals

United Kingdom: London Homelessness 

Project 

The Mayor of London in 2012 established the 

Rough Sleeping Commissioning Framework 

2011-2015, which outlined 17 different 

services for homeless individuals. The 

London Homelessness Project sits alongside 

those services and was designed to support 

homeless individuals who are unable to 

access some services, due to complex needs 

or antisocial behavior. 

Source: UK Department for Communities and Local 

Government | GAO-15-646

Page 23 GAO-15-646 Pay For Success

 
Figure 8: Roles and Funding Characteristics of the London Homelessness Project

a

The Greater London Authority contracted with Thames Reach and St. Mungo’s Broadway for the 

London Homelessness project.

Page 24 GAO-15-646 Pay For Success
The It’s All About Me (IAAM) adoption project, which began in October 

2013, was designed to find adoptive homes for children that otherwise 

would not find them. The project funds adoption agencies to recruit 

parents to adopt hard-to-place children. The agencies also provide these 

parents with support services. The program’s goal is for up to 140 of 

these children to be adopted each year. A UK local government pays a 

service provider defined amounts at different points in the adoption 

process, such as when a service provider places a child with an adoptive 

family, and after 1 and 2 years of the child remaining with the adoptive 

family. According to the project intermediary, while the maximum cost is 

greater than what a local authority usually would pay adoption agencies, it 

is also estimated to be about half of the cost of a child remaining in foster 

care for 2 years.28 Evaluations are being conducted on the health and 

psychological outcomes for the adopted children; outcomes for families 

who adopt the children; and how decision making changes within local 

governments as a result of the project. See figure 8 for a description of 

the roles of the organizations participating in the project.

28It’s All About Me, How IAAM Works, accessed May 14, 2015, 

http://iaamadoption.org/?page_id=9. 

United Kingdom: Reducing 

Wait Times for Adoption

United Kingdom: It’s All About Me Project

Many UK children wait for over a year for 

adoption while in care of the government. The 

program seeks adoptive parents for hard-toplace children and provides these parents 

with training and support during the adoption 

process. 

Source: IAAM | GAO-15-646

Page 25 GAO-15-646 Pay For Success

 
Figure 9: Roles and Funding Characteristics of the It’s All About Me Project

a

A local authority contracts with an adoption agency on a case-by-case basis.

In addition to the case illustrations identified above, we examined 3 

additional PFS initiatives that have yet to be implemented or were not 

implemented.

Stakeholders in South Carolina are developing a PFS initiative focused 

on controlling costs and improving health and other outcomes of mothers 

and newborns in South Carolina’s Medicaid program. The project is 

expected to expand the Nurse-Family Partnership program, which, 

according to stakeholders, is an evidence-based nurse home visiting 

program for first-time, low-income mothers and their children, to rural and 

South Carolina Maternal 

Health PFS Initiative

Page 26 GAO-15-646 Pay For Success
under-served parts of the state.29 Trained professionals provide services 

and support to pregnant women and families with young children, 

primarily during visits to families’ homes. According to a feasibility study 

prepared for the project, multiple trials and evaluations have 

demonstrated NFP’s effectiveness, which includes reduced emergency 

room visits and better health outcomes in later years.30

In Fresno, California, a nonprofit organization is conducting a 

demonstration project to determine whether a PFS project is feasible. The 

project, which began in March 2013, was designed to improve the health 

of low-income children with asthma and reduce the costs that result from 

emergency treatments. Fresno, according to the stakeholders, has one of 

the highest asthma rates in the United States, and approximately 20 

percent of children there have been diagnosed with the disease. The 

stakeholders are conducting an evaluation to demonstrate the social and 

financial benefits of a comprehensive in-home asthma management 

program for children at high risk of asthma emergencies.31 The program 

includes in-home education, environmental assessment, and remediation, 

such as mold removal to help manage asthma. Based on the outcomes of 

the demonstration project, stakeholders plan to pursue PFS financing to 

scale up the program to more children who can benefit.

In January 2013, the Maryland Department of Legislative Services (DLS) 

published the findings of its review of the feasibility and potential benefits 

and risks of a PFS project for financing reentry programs for state 

prisoners.32 The review considered, among other things, whether a PFS 

project could generate cost savings for the state. In its review, DLS 

assumed that the PFS project would fund a pilot program to serve 1,250 

participants over 5 years and reduce recidivism by 10 percent or 20 

29South Carolina Department of Health and Human Services, Request for Information: 

Social Impact Bond RFI #1 (Sept. 17, 2013).

30Institute for Child Success, Using Pay for Success Financing to Improve Outcomes for 

South Carolina’s Children: Results of a Feasibility Study (Greenville, S.C.: September 

2013).

31Social Finance, The California Endowment Awards Grant to Social Finance and 

Collective Health (Boston, MA: March 25, 2013).

32Maryland Department of Legislative Services, Office of Policy Analysis, Evaluating 

Social Impact Bonds as a New Reentry Financing Mechanism: A Case Study on Reentry 

Programming in Maryland (Annapolis, MD: January 2013).

Asthma Impact Model for 

Fresno (AIM4Fresno)

Maryland Review of a Potential 

PFS Initiative

Page 27 GAO-15-646 Pay For Success

 
percent. DLS concluded that even if the pilot program reduced recidivism 

by 20 percent, that result would be insufficient to close a wing of a prison 

or an entire prison facility, which DLS stated are the largest cost savings 

associated with avoided reimprisonment. Therefore, the state would be 

unable to realize cost savings through the project. Maryland did not 

pursue a PFS project to finance reentry programs after the report was 

released.

Stakeholders in PFS projects stated that governments face significant 

challenges in implementing programs that effectively address entrenched 

social problems. They also stated that government can benefit from using 

PFS as a tool to address these challenges. The benefits of PFS include 

the following:

Funding for prevention programs and potential cost savings.

Government stakeholders said constrained budgets in recent years have 

made it difficult for governments to fund new prevention programs, even 

though they would potentially lead to cost savings resulting from a 

reduced demand for remediation services. Because investors provide 

capital that a government only has to repay if the specified outcomes are 

achieved, a PFS project helps governments implement programs that 

they might not otherwise be able to fund. In addition, if successful, a PFS 

project reduces the need for remediation services and associated budget 

pressures and, therefore, frees up additional resources that government 

can reinvest in priority programs. For example, an official from the 

Massachusetts Executive Office of Administration and Finance told us 

that any savings attributable to the Massachusetts Juvenile Justice PFS 

project will likely be reinvested in the same or similar programs. Further, 

the recipients of the prevention program are expected to benefit from 

improved outcomes, such as better education outcomes resulting from 

high-quality preschool.

The ability to implement innovative programs by transferring risk.

PFS is designed so that the government only pays for a program if its 

outcomes are achieved, thus shifting the risk of failure to investors. 

Governments, Service 

Providers, and Investors 

Pursued PFS for Potential 

Performance and 

Financial Benefits

Potential Benefits for 

Governments

Page 28 GAO-15-646 Pay For Success
Stakeholders said that PFS creates incentives to both scale prevention 

programs where strong evidence exists and to innovate where it does not 

exist. For example, the Massachusetts project expanded a proven 

program to reduce recidivism rates. The service provider had extensive 

experience working with young men in Massachusetts at risk of 

recidivating and had successfully reduced their incarceration rates by 33 

percent. The Massachusetts project expanded the service to more than 

900 young men in Boston and Springfield, areas where the service 

provider had extensive experience. If successful, Massachusetts will pay 

for the program and expects to benefit from improved social outcomes 

and cost savings in its budget. If the service provider is unsuccessful, 

Massachusetts does not pay.

PFS also creates incentives to innovate and apply evidence-based 

approaches to new or different populations. In the New York City project 

the intervention was administered in a new environment, a jail, with a 

service provider with prior experience working in prisons, but without prior 

experience with the specific intervention. New York City and the project 

intermediary chose Moral Reconation Therapy for the intervention 

because of the available evidence base and its flexibility in 

implementation that could be adapted to a jail setting. The intermediary 

selected a provider to administer the service. The PFS structure allowed 

the project’s stakeholders to implement the intervention at Rikers Island 

for the first time while providing the city with assurance that investors will 

absorb financial loss if the intervention does not reduce participants’

recidivism. When the intervention did not reduce recidivism at the target 

rates, New York City did not pay for the program.33

A potential benefit of PFS is the opportunity for governments to 

implement innovative programs without an established evidence base by 

transferring risk to investors. In practice, investors told us they prefer to 

back programs that already have a rigorous evidence base because 

these programs have a known likelihood for success. In Maryland, the 

DLS concluded that PFS may limit the implementation of innovative 

programs and government’s opportunity to apply evidence-based 

approaches to new populations because investors would be most likely to 

back projects with well-established records of success in order to 

33Vera Institute of Justice, Impact Evaluation of the Adolescent Behavioral Learning 

Experience (ABLE) Program at Rikers Island: Summary of Findings (July 2015).

Page 29 GAO-15-646 Pay For Success

 
minimize their risk. Therefore, PFS may exclude new service providers 

and program types because of the perceived risk.

One of the organizations involved in the PFS projects in our review did 

not think that risks would be transferred from a government to investors. 

For example, in its review,34 the Maryland DLS concluded that in practice 

there are significant challenges to transferring risk to investors, such as 

identifying investors who are willing to invest in PFS projects and 

designing contract mechanisms to fully transfer risk. This was a factor in 

the DLS recommendation against using a PFS contract for a prisoner 

reentry program. The Maryland DLS stated that it may be difficult to 

design contract provisions to provide an enforcement mechanism to 

prevent investors from terminating the contract early. In addition, during 

the implementation of the project, the costs to administer the intervention 

may be greater than the investors’ upfront funding and it is not clear 

whether the government or the investors would be responsible for those 

additional costs.

Collaboration across government agencies. Government stakeholders 

told us that many social services are “siloed” within governments. In other 

words, individual agencies, largely on their own, design, budget for, and 

implement narrowly targeted social services, such as job training courses 

or temporary accommodation for homeless individuals. These services 

are usually not integrated with others that could collectively help 

vulnerable individuals overcome complicated issues and are potentially 

not as effective as a more integrated approach. PFS projects target 

outcomes for populations or individuals rather than the output of specific 

services, which would be the responsibility of a specific agency. PFS 

projects bring together multiple government agencies to work 

collaboratively to find solutions for difficult social problems. For example, 

the New York project targets improved employment and recidivism 

outcomes for program participants and therefore brought together 

multiple state agencies to oversee their achievement. Officials from the 

New York Department of Labor and the Department of Corrections and 

Community Supervision are members of an executive steering committee 

that provides strategic direction and represents the project across agency 

boundaries. Similarly, a stakeholder in Massachusetts said that PFS can 

34Maryland Department of Legislative Services, Office of Policy Analysis, Evaluating 

Social Impact Bonds as a New Reentry Financing Mechanism: A Case Study on Reentry 

Programming in Maryland (Annapolis, MD: January 2013). 

Page 30 GAO-15-646 Pay For Success

 
help break down silos because it can bring together multiple agencies to 

work together on an important social problem.

Sustained focus for entrenched social problems. Government 

stakeholders told us that governments find it challenging to sustain focus 

on difficult social problems over multiple years. Governments generally 

operate on annual budget cycles, making it difficult to set resource 

allocations over multiple years. In addition, changes in political leadership 

may shift policy priorities and support for levels of funding or program 

continuation. PFS contracts include upfront agreements about investor 

funding levels over multi-year time frames, providing sufficient time to 

collect, track, and analyze data, and measure outcomes that take multiple 

years to achieve.

A focus on outcomes and building an evidence base. Government 

stakeholders told us that governments sometimes are not focused on 

their programs’ outcomes because for social services they tend to 

contract for inputs instead of outcomes. As a result, governments have 

difficulty knowing whether a particular social program is having the 

desired effect on its target population. For PFS projects, governments 

contract for outcomes, and the contracts generally include a requirement 

that the program’s impact be rigorously evaluated, which allows 

governments to demonstrate the effect the program has as well as to add 

to the evidence base to inform future programs.

Stakeholders told us that PFS offers benefits to service providers, 

including the following:

Stable funding for operations. Stakeholders told us that service 

providers must often pursue multiple grant and fundraising efforts to fund 

their services and enter into contracts in which they are not reimbursed 

for the full cost of services provided. PFS provides an opportunity for 

stable funding spanning multiple years. Officials from the service 

providers in New York and Massachusetts told us that upfront, multiyear 

funding for their programs is one of the key reasons why their 

organizations were excited about participating in a PFS project. An official 

from Roca in Massachusetts said that without the consistent funding 

provided under PFS, the organization typically has had to piece funding 

together from multiple grants to fund its programs.

Stakeholders also told us that PFS can be preferable for service providers 

compared to traditional performance-based contracts. In a traditional 

performance-based contract, the service provider is not paid all or a 

Potential Benefits for Service 

Providers

Page 31 GAO-15-646 Pay For Success
portion of its fees until it meets defined outcomes. In such a model, the 

service provider assumes financial risk that could damage it if it does not 

succeed, therefore potentially excluding new or small service providers. 

For example, UK government officials told us that in a performance-based 

contract, the service provider must cover the initial costs of delivering 

services, which is difficult for many nonprofit service providers because 

they do not have the capital available to provide services in advance of 

being paid. A benefit of PFS is that investors offer service providers full 

upfront funding, or cost reimbursement, for service provision while 

assuming most or all of the financial risk.

Flexibility in implementing service delivery. Service providers 

identified two ways PFS projects can give service providers flexibility 

compared to a traditional government contract or grant. In some of our 

cases, service providers were using PFS to administer their proven 

approaches on a larger scale. For example, officials from the service 

provider in New York told us that participation in the PFS project has 

allowed the organization to implement its service delivery model, for 

which the organization believes it has a solid evidence base, on a larger 

scale rather than conform to government-determined service delivery 

requirements. PFS can also offer flexibility to service providers in their 

day-to-day operations because the government contracts for outcomes, 

rather than how the service provider spends the funding on the activities it 

undertakes. Service providers said they have more flexibility to tailor their 

approaches to meet the needs of the target population.

Stakeholders told us that PFS also offers benefits to investors, including 

the following:

Potential for return on investment. Stakeholders told us that investors 

benefit from potential financial returns. Investors conduct due diligence on 

projects and invest in projects where they believe there is potential for 

return on their investment. Investors and intermediaries noted that the 

level of potential returns for PFS projects has so far been below-market, 

compared to the level of risk.

Opportunity for social investment. Stakeholders said that certain 

investors are interested in making investments with positive social 

impacts. Officials from the bank that helped to facilitate investments in the 

New York project told us that the individuals who invested in the project 

are interested in aligning their investments with their values and creating 

positive financial and social impacts. Social investors, such as nonprofit 

philanthropic organizations in the United States and UK, said PFS is a 

Potential Benefits for Investors

Page 32 GAO-15-646 Pay For Success
tool to support social programs consistent with their missions. For 

example, an official from a philanthropic organization that invested in the 

Peterborough Project told us that the organization had previously 

provided grants to support services for short-sentenced prisoners and 

decided to invest in the project, in part, because it offered a sustainable 

solution to reducing recidivism among such prisoners. The organization 

reasoned that if the project turned out to be successful and could be 

shown to deliver cost savings to government, the UK government would 

be incentivized to directly fund services for short-sentenced prisoners in 

the future. In addition, social investors told us that because they will be 

repaid if a service provider meets outcome measures in a PFS project, 

they will have the opportunity to recycle the funds into other projects that 

align with their missions.

PFS is predicated on the idea that governments shift the risk associated 

with developing and implementing a prevention program to investors. 

Stakeholders told us that the various parties to PFS contracts face a 

range of risks that are inherent to contracts:

• Financial risk. The risk that one or more parties to the contract could 

lose money by participating in a PFS project. For example, a PFS 

project may not generate savings that are greater than its costs, which 

can include management fees for an intermediary, evaluation costs, 

and investors’ contracted outcome payments. Critics of PFS have 

argued that, given the extra costs associated with intermediaries and 

investors, using PFS to implement a program is inherently more 

expensive than providing the prevention program as a direct 

government service. Because interest rates are low for government 

Participating 

Organizations Sought 

to Manage Risks in 

the Feasibility 

Assessment, Design, 

and Implementation 

Phases of Complex 

PFS Contracts

Governments, Service 

Providers, and Investors 

Recognize that PFS 

Projects Present Risk

Page 33 GAO-15-646 Pay For Success
borrowing, it could still cost less to borrow funds to pay for the 

service.35 However, some investors and intermediaries said they 

believe the complexity and associated costs of implementing PFS 

projects may decrease over time if the projects become more 

replicable. PFS proponents in general have argued that such direct 

government funding of new prevention programs has been difficult to 

do in the current budget environment.

At the same time, investors risk losing their investment if the service 

provider fails to achieve the desired outcomes. Officials from a bank 

that has invested in multiple U.S. projects told us that the organization 

carefully considers the evidence of the intervention that the service 

provider will deliver in order to determine whether it is willing to take 

the financial risk associated with an investment and how that risk 

should be priced.

In addition, organizations participating in PFS projects told us that 

governments, service providers, and investors have dedicated a 

significant amount of time and resources to developing PFS projects, 

at times without being compensated for their work. There is a risk that 

the parties are unable to agree on a project’s complex details and fail 

to launch the project, in which case participants lose the time and 

resources that they devoted to developing the project.

Stakeholders identified other risks that could potentially result in 

increased financial risk for parties to the PFS contract:

• Perverse incentives. The risk that outcome payment structures will 

create perverse incentives for stakeholders to take actions at the 

expense of the individuals receiving treatment in the project. For 

example, creaming is the risk that service providers focus on 

individuals that are easiest to serve to achieve the defined outcomes. 

As a result of the perverse incentive to focus on those that are easiest 

to serve, those that are most in need and most difficult to serve could 

receive fewer services or a diminished focus. If creaming occurs, the 

government is potentially paying outcome payments when the target 

population is not being served.

35Elizabeth Lower-Basch, Center for Law and Social Policy (CLASP), Social Impact 

Bonds: Overview and Considerations (Washington, D.C.: Mar. 7, 2014).

Page 34 GAO-15-646 Pay For Success

 
• Termination risk. The risk that one or more parties to the contract 

decide to terminate their participation in a PFS project, requiring the 

project to end earlier than initially planned. The early termination of a 

PFS project presents a risk that the target population may be 

disadvantaged or hurt by losing access to the intervention. Officials 

from an investor in multiple U.S. projects said that the organization 

understands that the early termination of a PFS project has the 

potential to negatively affect the programs’ target populations. 

Therefore, the organization worked with other parties during contract 

negotiations to create rules that would allow for the gradual 

termination of unsuccessful projects while taking into account the 

needs of current participants.

A variant of termination risk is policy risk, or the risk that a 

government initiates a policy change that prevents a PFS project from 

operating as initially intended. A policy change could disrupt a service 

provider’s program delivery, putting the achievement of outcomes—

and the investors’ investment—at risk. For example, the UK Ministry 

of Justice terminated the Peterborough Project 2 years early. The UK 

government implemented a new initiative called Transforming 

Rehabilitation that provides a similar set of services for all shortsentenced prisoners in the UK. Officials from the UK Ministry of 

Justice said maintaining the PFS program at Peterborough until 2017 

for the third and final cohort as planned was not possible, as the 

majority of prisoners within that group would already be receiving 12 

months of supervision and rehabilitation as a result of the wider 

reforms to probation. According to officials from the intermediary, it 

would not have been possible to maintain a control group for the 

evaluation of the last cohort in the project. The Ministry of Justice and 

other project participants agreed to transition arrangements for the 

project that, among other things, established that investors could 

receive outcome payments if the project evaluation shows recidivism 

across the first two cohorts was reduced by 7.5 percent and that the 

Ministry of Justice would pay termination costs. Officials from the 

intermediary for the project said that investors are pleased that the 

needs of short-sentenced prisoners are being met under Transforming 

Rehabilitation. However, because the Peterborough Project ended 

early, information and learning about the effectiveness of the program 

was not fully realized.

• Performance risk. The risk that an intervention fails to achieve 

defined outcomes. A government does not make payments if defined 

outcomes are not achieved. However, if outcomes are not achieved, 

then the government has lost time developing the program and failed 

Page 35 GAO-15-646 Pay For Success
to meet mission-related goals and objectives. Performance risk 

includes how the prevention program is operationalized, managed, 

and implemented. This risk can be heightened if the intervention is 

poorly structured or poorly tailored to the needs of a particular 

participant population, or if service delivery is poorly executed, which 

could lead to potential harm to program recipients. Investors also risk 

losing some or all of their investment. Stakeholders from the 

government and an investor in New York City told us that they 

considered the implications of potential performance risks in the 

project during its conception and development. For example, an 

official from the New York City government told us that the parties 

involved had extensive discussions, including about program 

outcomes and timelines. In addition, officials from an organization that 

invested in the project told us that they had to decide how likely, 

based on existing evidence, the program was to achieve the 

established outcome.

• Appropriation risk. From the investors’ perspective, the risk that a 

government fails to appropriate funds necessary to make outcome 

payments. If a government does not appropriate necessary funds, 

investors could lose some or all of their investment, making them less 

likely to invest in PFS projects in the future. Investors’ perception of 

appropriations risk affects their decision on whether to participate in a 

PFS project and on how they view the risk of a project. For example, 

officials from a bank that has invested in multiple U.S. projects 

identified appropriations risk as one of the primary risks it sees for 

investors considering PFS projects. The officials told us that since 

investors assume substantial performance risk in the projects, they 

must have an assurance that, in the end, the government is going to 

make the outcome payments to which it initially committed. 

Stakeholders in our UK case illustrations did not view appropriations 

risk as a concern in the United Kingdom PFS market.

• Reputational risk. The risk that a party’s reputation will be affected 

negatively by events that occur in a PFS project. Stakeholders in New 

York City told us that they are assuming reputational risk by 

participating in the Rikers Island project. For example, a former official 

from the New York City government told us that the city took a large 

reputational risk by launching the first PFS project in the United 

States, and the city is therefore very invested in its success. In 

addition, an official from one of the service providers working to 

implement the New York City project told us that the organization has 

assumed reputational risk by working as the service provider for the 

project. If the service provider fails to achieve outcomes for the 

Page 36 GAO-15-646 Pay For Success
project, it could make it challenging to get future contracts. 

Furthermore, an official from the organization that is evaluating the 

New York City project told us that since there are individuals and 

organizations within the criminal justice sector who are skeptical of 

PFS or believe that it is a flawed concept, the organization is 

potentially risking its reputation within the criminal justice sector by 

participating in the project.

To assess project feasibility, governments and other stakeholders 

identified and evaluated potential projects in an effort to maximize cost 

savings and also to manage potential financial and performance risk. 

Stakeholders from the selected case studies stated that, in the first phase 

of PFS project development, they evaluated the feasibility of using PFS to 

address a particular social problem within their jurisdiction. We found that 

in practice PFS contracts were complex and governments relied 

extensively on outside technical assistance to assess the feasibility of 

their projects.

Attention to cost-benefit analysis. Governments estimate the potential 

effect the PFS project will have on future budgets to determine if the 

project is feasible. Using cost-benefit analysis and cost modeling,36

governments evaluate whether the prevention program is likely to 

generate cost savings, cost avoidance, or other benefits over the course 

of the project that exceed the project’s costs, which include the total 

amount of outcome payments and associated administrative costs. Based 

on this analysis, governments determine whether a project is feasible. For 

example, as noted earlier, the Maryland Department of Legislative 

Services determined that the state should not pursue a PFS project 

36According to DOL officials, a cost-modeling methodology is different than a cost-benefit 

analysis; while cost modeling accounts for various forms of benefits, including cost 

savings, it may not account for all the expenditures and costs that would be included in a 

typical cost-benefit analysis.

During Project Feasibility 

Assessments, 

Governments Reported 

Seeking to Maximize Cost 

Savings While Managing 

Financial and 

Performance Risk

Page 37 GAO-15-646 Pay For Success

 
because even relatively large reductions in recidivism among program 

participants were unlikely to result in the closure of a wing of a prison 

facility, the type of large, fixed-cost savings that it believed would be 

needed to cover the program’s costs.

Governments’ cost-benefit analyses may also include indirect costs and 

social benefits. For example, New York assessed both the direct cost 

savings of reduced prison expenses to the government as well as 

quantifiable public sector benefits of savings from reduced crime resulting 

from the project. The New York project’s outcome payments are derived 

from these estimated public sector savings and benefits. According to a 

detailed summary of the project, the state estimated and included an 

amount for the cost savings from reduced crime for victims based on a 

study that estimated the intangible victim costs. To estimate these costs, 

the state used other costs such as medical costs and lost earnings for 

crime victims. The state also used a jury-compensation approach that 

used the money awarded to victims by juries to estimate the intangible 

victim costs of crime.37 Figure 10 shows the potential savings and 

benefits, as well as the outcome payment costs that New York identified 

for its PFS project. For example, a 10 percent reduction in recidivism 

would result in potential savings and benefits that equal outcome 

payment costs. As the rate of reduction in recidivism increases, the 

state’s potential savings and benefits (after making outcome payments) 

increase.

37State of New York, Center for Employment Opportunities, and Social Finance, Investing 

in What Works: “Pay for Success” in New York State Increasing Employment and 

Improving Public Safety: A Detailed Project Summary (March 2014). 

Page 38 GAO-15-646 Pay For Success

 
Figure 10: Estimated Savings and Benefits and Outcome Payments at Various 

Reductions in Recidivism Rates

Identification of entities that capture financial benefits and budget 

savings. Stakeholders said that in order to maximize cost savings to the 

entity who is sponsoring the PFS project, governments must know which 

government budget will capture and benefit from the savings. 

Stakeholders said that a barrier to potential PFS projects occurs when 

expected savings accrue to a different government entity than the one 

that is leading the project. This is referred to as the “wrong pockets 

problem.” If the savings resulting from a successful intervention by one 

government entity accrue to a different government entity, without 

agreement from the other government entity to contribute to outcome 

payments in proportion to the savings, the project may not be costbeneficial to implement. For example, stakeholders in South Carolina 

have been developing a PFS project to fund a program designed to 

improve maternal health outcomes. Since potential savings generated by 

Page 39 GAO-15-646 Pay For Success
the program would accrue to Medicaid, those savings would be split 

between the state and the federal government.38 Stakeholders told us that 

the state would find it challenging to agree to make all of the project’s 

potential outcome payments since it would only be capturing 

approximately 30 percent of the project’s savings, with the federal 

government capturing the rest. A stakeholder in New York City said the 

city chose to pursue the Rikers Island project specifically because the city 

bears all of its own criminal justice costs and therefore would not have to 

involve the state or federal government in developing the project or 

paying for the outcomes achieved.

Access to, and capacity for, robust data collection and analytics. To 

determine project feasibility, governments must have access to complete 

and accurate cost and benefit data and must have the capacity to analyze 

it.39 However, stakeholders said that the availability of government data 

on social programs and the capacity to analyze the data is a challenge to 

the growth of PFS. Stakeholders said that governments need data 

systems in place that allow agencies to collect data and share it between 

agencies, as needed. However, in some cases, the necessary data can 

be difficult to obtain or share. For example, government stakeholders in 

Massachusetts told us that differences in the ways in which jurisdictions 

within the state maintained files on individuals on probation made it 

difficult to collect and share data on potential program participants at the 

beginning of the project’s implementation. Stakeholders said legal 

barriers can prevent governments from sharing data across multiple 

systems. For example, we previously reported that matching education 

and workforce data is challenging in states where collecting a Social 

38States and the federal government share in the financing of the Medicaid program, with 

the federal government matching most state expenditures for Medicaid services on the 

basis of a statutory formula known as the Federal Medical Assistance Percentage 

(FMAP). The FMAP is calculated using a statutory formula based on the state’s per capita 

income (PCI) in relation to the national PCI: FMAP = 1.00 – 0.45 (State PCI / U.S. PCI) 

and may range from 50 to 83 percent. The federal government pays a larger portion of 

Medicaid expenditures in states with low PCI relative to the national average, and a 

smaller portion for states with higher PCIs. For more information see GAO, Medicaid: 

Assessment of Variation among States in Per-Enrollee Spending, GAO-14-456

(Washington, D.C.: July 16, 2014).

39Challenges with government capacity apply to multiple phases of PFS projects.

Page 40 GAO-15-646 Pay For Success

 
Security number in education data is prohibited by state law or agency 

policy.40

Stakeholders also expressed concerns that many governments do not 

have sufficient data collection and analysis capacity for PFS projects. 

Similarly, we have previously reported that it is critical to define the 

relevant types of information required for service acquisitions and then 

develop the appropriate data systems to collect and provide reliable 

data.41 Governments will need staff with expertise in data analysis and 

financial modeling to be able to conduct feasibility studies, identify future 

savings, evaluate service delivery, determine outcome payments, and 

negotiate PFS contracts. Government officials from both New York and 

New York City said that their government’s robust data systems facilitated 

their cost-benefit analyses. For example, according to a lessons learned 

report from the intermediary in the New York City Rikers Island project, 

the city’s strong financial planning structure allowed the city to gain a solid 

understanding of the amount of money that it could save based on 

changes in the size of its jail population. The city used this information to 

establish its willingness to pay for reductions in recidivism attributable to 

the intervention being funded by the PFS project. The cost information, 

together with historical criminal justice data, allowed the city to identify a 

specific reduction in recidivism that would make the PFS project 

beneficial for the city.42 To address challenges with government capacity, 

all of the U.S. PFS projects in our study included provisions for technical 

assistance from experts affiliated with universities or intermediary 

organizations to analyze project data and structure financial models 

needed to determine project feasibility and outcome payments.

Sufficient evidence of an intervention’s success. In addition to costbenefit analysis and access to data, stakeholders told us that during the 

feasibility phase it is important to consider the evidence of an 

intervention’s past success to help manage the performance risk of the 

project. While some stakeholders stressed the importance of sufficient 

40GAO, Education and Workforce Data: Challenges in Matching Student and Worker 

Information Raise Concerns about Longitudinal Data Systems, GAO-15-27 (Washington, 

D.C.: Nov. 19, 2014).

41GAO-07-20.

42MDRC, Financing Promising Evidence-Based Programs: Early Lessons from the New 

York City Social Impact Bond (New York, NY: December 2013). 

Page 41 GAO-15-646 Pay For Success

 
evidence of an intervention’s success, others noted that there is variation 

in the PFS field in the level of evidence and how it relates to the individual 

project. For example, evidence supporting an intervention applied to one 

population could provide the basis for testing the intervention with a 

different population. A PFS project could also be used to implement an 

innovative intervention with limited but promising evidence. Participating 

organizations—including governments, investors, and service providers—

may differ in their tolerance of risk and uncertainty associated with the 

evidence of any intervention. Stakeholders from government, 

intermediaries, and investors said they also assessed the availability of 

service providers who could implement the intervention to determine 

whether the project would be feasible. Officials from an investor in 

multiple U.S. projects told us that the bank has looked to invest in PFS 

projects that feature a service provider that has a track record of success.

Stakeholders said the availability of evidence-based social programs and 

service providers with the capacity to implement them is a challenge to 

growing the PFS field. Stakeholders expressed concerns that there is not 

a sufficient pipeline of nonprofit service providers whose interventions 

have been rigorously evaluated. We have previously reported on 

challenges nonprofits face related to funding indirect costs, such as 

program evaluation.43 In addition, stakeholders told us that in order to be 

successful in an established PFS project, service providers need 

experience implementing the evidence-based intervention, the capacity to 

implement the intervention on a larger scale, and the ability to collect and 

report data for evaluation. The enacting legislation for the Utah PFS 

Project includes a grant program to build the capacity of preschool 

providers to implement the high-quality preschool model, which is based 

on a prior evaluation, so these providers are able to participate in 

potential future PFS projects.

Leadership support. Stakeholders said that support from top 

government leaders is essential to moving a PFS project forward to the 

design and implementation phases of the project. This support can 

provide the momentum that is necessary to get a PFS project off the 

ground and sustain the project through implementation. State officials in 

New York told us that PFS was part of Governor Andrew Cuomo’s policy 

43GAO, Nonprofit Sector: Treatment and Reimbursement of Indirect Costs Vary among 

Grants, and Depend Significantly on Federal, State, and Local Government Practices, 

GAO-10-477 (Washington, D.C.: May 18, 2010).

Page 42 GAO-15-646 Pay For Success

 
agenda during his gubernatorial campaign, as it aligned with his interest 

in taxpayer efficiency and a focus on outcomes. In addition, a report on 

the project stated that it was conceived when Governor Cuomo outlined a 

state priority to use PFS to reduce recidivism and increase employment 

among high-risk ex-prisoners.44 Furthermore, stakeholders in Utah told us 

that the long-term prospects of the project were boosted by the Board of 

the United Way of Salt Lake and the Mayor of Salt Lake County’s 

agreement, during the project’s proof of concept year, to dedicate funds 

to support the project’s outcome payments. Stakeholders also said that 

leadership from a central government agency can help maximize 

collaboration across government agencies. They said that it is important 

that a central agency coordinates across multiple departments in a 

situation where an intervention in one department will accrue future 

savings in a different department or where participation from multiple 

departments is needed to implement the project. We have previously 

reported that top leadership commitment can improve outcomes and set 

the direction, pace, and tone for implementing a significant policy 

change.45 Stakeholders also said it is critical to have a dedicated 

government official to manage the development of a PFS project. For 

example, the governments in Massachusetts, New York, and Utah had a 

full-time position in their budget offices to manage their PFS efforts.

Stakeholders said contract negotiations for PFS projects are complex, 

lengthy, and resource intensive. Stakeholders from government, service 

providers, intermediaries, and investors told us that they required 

significant legal and technical capacity to support the development of 

these complex contracts because PFS contracts were new and existing 

templates were not available to use as a guide.46 While the same types of 

parties were present, the PFS projects in our study had different 

contractual arrangements. In some cases, the PFS contract was between 

44State of New York, Center for Employment Opportunities, and Social Finance, Investing 

in What Works: “Pay for Success” in New York State: Increasing Employment and 

Improving Public Safety (March 2014).

45GAO, Highlights from a GAO Forum: Mergers and Transformation: Lessons Learned for 

a Department of Homeland Security and Other Federal Agencies, GAO-03-293SP

(Washington, D.C.: Nov. 14, 2002).

46To help share knowledge on PFS, Massachusetts, New York, and Utah made their 

contracts publicly available after they were completed. 

During Contract Design, 

Parties Reported Defining 

Requirements to Manage 

Risk

Page 43 GAO-15-646 Pay For Success

 
the government and the intermediary, which held additional contracts with 

investors and service providers. In other cases, the contract was between 

the government, intermediary, and service provider. The intermediary 

holds separate agreements with the investors in the project. While the 

contractual arrangements varied, stakeholders said that all parties were 

involved with negotiating the terms of the contracts.

Parties to the PFS contracts in our study defined the requirements for the 

service provider, in part, by using performance targets for specific 

outcomes that were rewarded with outcome payments. These 

performance targets and associated payouts make up an outcome 

payment structure. See table 2 for examples of four different types of 

outcome payment structures we observed in the PFS projects we 

reviewed. For example, in the Utah High Quality Preschool project, the 

outcome payment structure was defined as a set payment for each 

student participant who avoided being enrolled in special education each 

year from kindergarten through sixth grade. Stakeholders from 

governments and service providers said that the outcome payment 

structure aligned the incentives of all parties to be invested in the service 

provider’s successful delivery of the prevention program. Further, as 

suggested in practices for complex service acquisitions, because the 

parties to the contract agree in advance to outcome measures and 

associated payments, there is less room for interpretation once the 

contract is being implemented.47

47GAO-07-20 and Brown, Potoski, and Van Slyke, Complex Contracting. 

Defining Requirements in 

Outcome Payment Structures

Page 44 GAO-15-646 Pay For Success

 
Table 2: Examples of Outcome Payment Structures from Selected Pay for Success Projects

Performance measure Target Outcome payment

All or nothing: New York 

City Rikers Island Project 

Reduction in bed days on readmission compared to a 

historical group

8.5% reduction Below 8.5%: no payments 

8.5% to more than 20%: $4.8 - $11.712 million

Fee for outcome: Utah 

High Quality Preschool 

Project

Whether or not a student from 

the testing cohort is enrolled in 

special education services after 

preschool each year 

kindergarten through 6th grade

Student not 

enrolled in special 

education services

$2,589.70, per student, per year

Fee for outcome: UK It’s 

All About Me (IAAM) 

Adoption Project

1. Register child with IAAM 

adoption service provider

Child registered £8,000 ($13,000) per child

2. Placement with adoptive 

family

Child placed £23,000 ($37,000) per child

3. Placement with adoptive 

family for 1 year

Child placed for 1 

year

£6,800 ($11,000) per child

4. Placement with adoptive 

family for 2 years

Child placed for 2 

years

£15,800 ($26,000) per child

Mix of all or nothing and 

fee for outcome: New 

York Increasing 

Employment and Improving 

Public Safetya

1. Reduction in number of bed 

days spent in jail or prison 

compared to a control group

8% reduction in 

bed days

Below 8%: no payment

Above 8%: $85 per dayb

2. Start transitional job Payment is 

conditional on 

achieving 8% 

reduction in bed 

days 

There are 2 payment structures depending on 

the average number of hours worked:

• If average hours worked is greater than or 

equal to 111 hours: $3,120, per person 

engaged in a transitional jobc

• If average hours worked is less than 111 

hours: $20 per hour engaged in a 

transitional jobd

3. Employment a year after 

release from prison or jail 

compared to a control group

5% increase in 

employment

$6,000 per persone

Source: GAO analysis of project documents. | GAO-15-646

a

The New York project will be implemented in two phases. Each phase will serve 1,000 individuals, 

measure the impact of the intervention on those individuals, and make performance-based payments 

based on that impact. Phase 1 and 2 outcome payments are different to account for inflation. The 

outcome payments presented in the table are for phase 1.

b

Phase 2 payout is $90.10 per day.

c

Phase 2 payout is $3,307 per person engaged in a transitional job.

d

Phase 2 payout is $21.20 per hour engaged in a transitional job.

e

Phase 2 payout is $6,360 per person.

Page 45 GAO-15-646 Pay For Success
We have previously reported that a balanced set of performance goals 

and measures can help organizations address varied aspects of program 

performance.48 In some of the PFS cases in our study, the parties 

included a range of performance measures in the outcome payment 

structure to more effectively serve the target population. For example, the 

New York project included performance measures on both reducing 

recidivism and increasing employment for individuals who were recently 

released from prison. According to a detailed summary of the project, 

formerly incarcerated individuals face numerous challenges, including 

barriers to finding a job upon release.49 Stable employment can be an 

important factor in success upon release from prison.

The outcome payment schedules also create financial incentives for 

investors and service providers to improve the outcomes for the target 

population. Stakeholders told us that multiple outcome measures can be 

a tool to incentivize service providers to address various participant 

needs. For example, the London Homelessness Project includes outcome 

measures to target the different causes of homelessness of individuals in 

the cohort. One of the five measures is related to supporting individuals’

progress toward employment, and another outcome measure is related to 

moving individuals into stable housing. Outcome payment structures with 

higher payouts at the end of the project create incentives for investors 

and service providers not to terminate the contract early because they will 

receive the majority of their return on investment at the end of the 

contract period.

Stakeholders said that the rates of return offered by PFS projects likely 

will not meet market returns available on traditional investments with 

similar risk profiles, therefore reducing investor demand. Lower returns 

compared to traditional investments could make it more difficult for 

governments to attract sufficient capital for PFS projects. Investors 

assess the risks of a project to determine whether or not they will invest. 

Investors’ valuation of risk is primarily based on several factors related to 

managing their financial risk, including evidence of the intervention and 

48GAO, Agency Performance Plans: Examples of Practices that Can Improve Usefulness 

to Decisionmakers, GGD/AIMD-99-69 (Washington, D.C.: Feb. 26, 1999).

49State of New York, Center for Employment Opportunities, and Social Finance, Investing 

in What Works: “Pay for Success” in New York State Increasing Employment and 

Improving Public Safety: A Detailed Project Summary (March 2014). 

Challenge of Securing 

Investors

Page 46 GAO-15-646 Pay For Success

 
the ability of the service provider to implement it, the outcome payment 

structure, and the level of appropriations risk. For example, commercial 

investors prefer interventions backed by rigorous evidence and service 

providers who have a history of strong performance because it lowers 

their financial risk. Governments also manage their financial risk by 

determining the outcome payments and rate of return they are willing to 

pay to investors. In Utah, the legislation that authorizes the High Quality 

Preschool Project included a cap on the return for investors. A Utah 

legislator said the cap helps to ensure that the state captures more of the 

cost avoidance from reduced use of special education services. The 

legislature wanted to ensure they were being good government stewards 

because they would not be at the table negotiating PFS transactions. In 

other projects, the return on investment was negotiated on a case-bycase basis. For example, in the New York project, there is a cap on the 

total outcome payments the state can make over the life of the project. In 

the UK, government officials told us that some PFS project outcome 

payments and associated rates of return funded by the Department for 

Work and Pensions’ (DWP) Innovation Fund were not subject to 

negotiation. Instead, DWP developed a rate card, where it defined the 

maximum it is willing to pay for specific outcomes (see appendix IV for an 

example of a rate card).

In the U.S. projects in our study, parties to the contracts used financing 

structures to address challenges in attracting investors, such as a tiered 

capital structure or a loan guarantee to secure the upfront capital needed 

for the project.

Tiered capital structure. A tiered capital structure, where some investors 

take on more risk, can attract bigger pools of capital because it lowers 

risk for some investors. In three of our U.S. case illustrations a 

combination of commercial and philanthropic investors made up the 

capital structure. Stakeholders said that investors’ willingness to accept 

risk can vary and that philanthropic investors may be willing to accept 

more risk at a lower rate of return than commercial investors. For 

example, according to the intermediary in the Massachusetts Juvenile 

Justice project, to encourage investment the project capital structure was 

split between philanthropic and commercial investors. Commercial 

funding partners contributed senior debt, which will be paid out first; 

foundations contributed junior debt, which will be paid out following the 

senior debt. Philanthropic grants from foundations also served as firstloss capital, which will bear the first loss in the event that the service 

provider does not meet the outcome measures. According to the 

Page 47 GAO-15-646 Pay For Success
intermediary in the project, in the first-loss role philanthropic donors can 

combine and leverage their grant making with commercial capital.

Loan guarantee. In a different model, a philanthropic investor can 

provide a loan guarantee to the primary investor to reduce its risk by 

assuming the loan if the service provider does not meet the outcome 

targets. For example, in the New York City Rikers Island Project, the city 

agreed to a maximum return of $2.1 million which was not high enough 

for Goldman Sachs to justify the level of risk it perceived regarding 

repayment of its $9.6 million loan. In an effort to balance the risk/reward 

trade-off, Bloomberg Philanthropies provided a $7.2 million grant to 

MDRC to act as a loan guarantee that would reduce Goldman Sachs’

capital at risk in the event that outcome targets were not achieved. An 

investor in the project further explained that although the service provider 

had a track record of success, its risk of failure in the NYC Rikers Island 

project was higher because it would be implementing a new curriculum in 

a difficult environment. The combination of the performance risk and its 

status as the first PFS project in the United States were the primary 

reasons for the loan guarantee.

Governance rules that define interactions and the authority to make 

decisions after the contract has been signed can encourage cooperation 

and manage the risk of unexpected termination by any party. For the 

projects in our study, the contracts also defined how parties will interact 

during the implementation of the project. PFS contracts in our study 

included outcome payment structures, oversight structures, governance 

rules, and project evaluation mechanisms to manage risk.

Oversight structures. PFS contracts included oversight structures to 

manage performance risk during contract implementation. All of the 

projects in our study had a governance structure, which defined how 

parties would make decisions during implementation of the contract, 

including how the project would be managed and overseen. For example, 

the New York project had a three-tier governance structure made up of 

the following:

• An executive steering committee: This committee meets semiannually to provide strategic direction, monitor compliance with the 

agreement, and review reports. Members include the Deputy 

Secretary for Civil Rights; the Deputy Secretary for Public Safety; the 

Commissioner of Department of Labor (DOL); the Commissioner of 

the Department of Corrections and Community Supervision (DOCCS); 

During Project Design, 

Parties Reported Using 

PFS Contracts to 

Establish Governance 

Structures and Rules for 

Project Implementation to 

Manage Risk

Page 48 GAO-15-646 Pay For Success
CEO of the intermediary; and the Executive Director of the service 

provider.

• A management committee: The committee meets monthly to monitor 

reports and program implementation and provides updates to the 

executive steering committee. Members include the Associate 

Commissioner of DOCCS; two Deputy Commissioners of DOCCS; the 

Deputy Commissioner of DOL; the Executive Director of the service 

provider’s regional office; the Director of DOCCS Research; a 

representative from the Harvard Kennedy School Social Impact Bond 

Technical Assistance Lab;50 and a director from the intermediary.

• Working groups: Working groups meet to discuss operational and 

research issues that arise over the course of the project. They are 

composed of staff and subject-matter experts from DOCCS; DOCCS 

Research; DOL; the service provider; the Harvard Kennedy School 

Social Impact Bond Technical Assistance Lab; and the intermediary.

Contract governance rules. Complex contracting practices suggest that 

including provisions for how decisions will be made when changes are 

needed during service delivery can help address implementation 

challenges. To manage various risks and inherent uncertainties, PFS 

contracts included governance rules to define how decisions would be 

made over the course of the project. PFS contracts included rules to 

define decision making after the contract was in place as well as 

provisions to address how parties to the contract could be replaced in the 

case of underperformance. For example, the New York contract included 

governance rules that defined when and how the service provider could 

be replaced. The contract specified that any replacement service provider 

is subject to approval from the New York Department of Labor and 

Department of Corrections and Community Supervision, among others.

PFS contracts also defined the circumstances under which investors 

could decide to terminate the project based on the service provider’s 

performance. A Massachusetts official said that determining the 

provisions that would allow investors to terminate the contract was one of 

the most difficult parts of the negotiations. Because the PFS project was 

designed for the service provider’s model, Massachusetts wanted to 

50The Harvard Kennedy School Social Impact Bond Technical Assistance Lab provides 

pro bono technical assistance to state and local governments, including New York, that 

are implementing PFS contracts. 

Page 49 GAO-15-646 Pay For Success

 
support the model, which requires a number of years of participation to 

achieve outcomes. Massachusetts also wanted to ensure that program 

participants would not be left without services as a result of the program’s 

early termination. Investors did not want to continue their investment if the 

service provider was underperforming. In the contract, the parties agreed 

that the lead investor has a right to exit the project at the end of the 

second and third years if the service provider’s participant attrition rate 

exceeded defined thresholds. The contract also included provisions, such 

as the development of a remediation plan, designed to improve the 

service provider’s performance if it fell below a defined threshold. These 

provisions gave investors the opportunity to reduce their financial risk if 

the service provider was underperforming. Allowing for 2 years of 

implementation and outcome measurement before investors can exit the 

contract also helped mitigate the government’s performance risk because 

it allowed enough time for service providers to collect data to measure 

outcomes and make adjustments if necessary to improve performance. 

Termination provisions also defined what would happen if governments 

terminated the contract early. For example, in the NYC Rikers Island 

Project, the City of New York could terminate without cause. However, if 

the city terminated the contract before the entire evaluation cohort 

receives services, it would lose the pay-for-success features of the project 

and must reimburse the intermediary for program costs to date.

Stakeholders said that building flexibility into contracts to allow for 

negotiations to make operational changes without amending the contract 

could help address implementation challenges. Some changes to the 

projects required updates to the PFS contract, which were costly and time 

intensive because all parties to the contract had to negotiate and agree to 

the changes. Stakeholders said that it is important to find a balance 

between specifically defining requirements and building sufficient 

flexibility into the contract to address challenges during program 

implementation. A Massachusetts official said the contract must be 

detailed enough to ensure that the program participants are protected and 

that risk is properly transferred from the government to other parties. 

However, the contract’s terms must also offer the parties a degree of 

flexibility to make changes as problems arise, because it is very time 

consuming and difficult to amend the contract.

Government budgeting structure. Because appropriations risk was a 

key concern of investors, the U.S. cases in our study managed this risk 

by establishing different mechanisms to secure funds for outcome 

payments:

Page 50 GAO-15-646 Pay For Success
• Massachusetts enacted legislation authorizing the backing of PFS 

contract payments with the full faith and credit of the commonwealth 

and establishing a Social Innovation Financing Trust Fund into which 

annual appropriations may be made.

• New York City, if the program had been successful, would have made 

outcome payments as it would for any other contractual obligation. 

There were two evaluation points that would trigger a possible 

payment. The final payment was scheduled to occur 1 year after the 

final results, giving the city more time to realize savings.

• New York pledged to obtain appropriations to make outcome 

payments. The contract includes provisions for an orderly wind down 

if the state does not renew the appropriation.

• Utah enacted legislation that authorizes a School Readiness 

Restricted Account which consists of funds appropriated by the 

legislature, federal grant funds, and private donations. Funds in this 

account may be used for contracts with independent evaluators, 

future outcome payments, and grant awards.

Page 51 GAO-15-646 Pay For Success
Project evaluation mechanisms. The PFS case studies we examined 

used evaluations to manage the governments’ various risks and to ensure 

the defined outcomes are achieved. In each case, outcome payments 

were contingent upon the evaluator certifying that the agreed-upon 

outcomes were met. Four of the evaluations in our case illustrations were 

impact evaluations, such as a random control trial51 (RCT) or a quasiexperimental design,52 where outcomes for individuals receiving the 

intervention were compared to those from a similar control group. Three 

of the evaluations used a process to verify that outcomes were being 

achieved for the target population. For example, in the Utah project’s 

evaluation, the evaluator tests each student participating in the highquality preschool program at the beginning and end of the school year. 

The students who score two standard deviations below the mean on the 

pre-test are tracked from kindergarten through sixth grade to verify 

whether they are enrolled in special education. The evaluation was 

derived from an earlier analysis that showed a decrease in special 

education use among students attending the high-quality preschool 

program. In the London Homelessness project, service providers submit 

evidence for some of the measures to the Greater London Authority, 

which is responsible for verifying the attainment of those measures. For 

example, for the outcome measure on helping clients into stable housing, 

providers submit a form from an individual’s landlord, which the Greater 

London Authority verifies and determines whether to make the associated 

outcome payment.

Stakeholders said that evaluations can manage potential perverse 

incentives, such as creaming. Some projects in our study included a 

mechanism to address creaming in the evaluation. For example, the Utah 

project on early childhood education addressed the risk of creaming by 

only including the children who score two standard deviations below the 

mean on the pre-test in the evaluation for outcome payments. 

51Random control trials compare the outcomes for groups that were randomly assigned 

either to the treatment or to a nonparticipating control group before the intervention, in an 

effort to control for any systematic difference between the groups that could account for a 

difference in their outcomes. 

52Quasi-experimental designs resemble randomized experiments in comparing the 

outcomes for treatment and control groups, except that individuals are not assigned to 

those groups randomly. Instead, unserved members of the targeted population are 

selected to serve as a control group that resembles the treatment group as much as 

possible on variables related to the desired outcome. 

Evaluation Methods in Case Illustrations

Random Control Trial

 New York Increasing Employment and 

Improving Public Safety

 Massachusetts Juvenile Justice

Quasi-Experimental Design

 New York City Rikers Island

 Peterborough Prison

Process to Verify Outcomes

 Utah High Quality Preschool

 London Homelessness

 UK It’s All About Me Adoption

Source: GAO summary of project documents

Page 52 GAO-15-646 Pay For Success

 
Stakeholders said that evaluations can also help manage perverse 

incentives for stakeholders to take actions at the expense of the 

individuals receiving treatment in the project. For example, because 

outcome payments in the Utah project are made based on students 

avoiding special education, there is potentially an incentive to withhold 

special education services to increase outcome payments. The Utah 

project manages this risk by keeping the identity of the students in the 

evaluation cohort confidential, so the teachers who determine special 

education needs are not aware of a student’s status in the project. In 

another example, a report on the London Homelessness Project stated 

that well-defined target populations can reduce the risk of creaming 

because the service provider cannot manipulate the population.53 This 

project targets 830 entrenched homeless individuals who had been seen 

sleeping rough54 or who have stayed in a London rough sleeping hostel in 

the last 3 months, and who had been rough sleeping at least six times 

over the last 2 years.

Stakeholders said that there are challenges to implementing RCT 

evaluations for PFS projects. We have previously reported that wellconducted randomized experiments are best suited for assessing 

effectiveness when multiple causal influences create uncertainty about 

what caused results.55 Such experiments are often expensive because 

they are among the most rigorous forms of evaluation, as well as difficult 

or sometimes impossible to carry out. For example, a risk of RCTs is that 

the evaluation may be inconclusive if the control group cannot be 

maintained and because a large sample size is required to yield 

statistically significant results. One stakeholder said that a RCT can 

increase the reputational risk to service providers if it is poorly designed 

and misses the real outcomes of the program. To manage the risk of 

these challenges, stakeholders from several projects identified the need 

for an alternative methodology in case the evaluation could not be 

implemented due to unanticipated implementation challenges. For 

example, in the NYC Rikers Island Project, the initial evaluation plan was 

53United Kingdom Department for Communities and Local Government, Qualitative 

Evaluation of the London Homelessness Social Impact Bond: First Interim Report

(London, United Kingdom: September 2014). 

54Rough sleeping is a term used in the UK to refer to individuals who sleep outside due to 

homelessness

55GAO-10-30.

Page 53 GAO-15-646 Pay For Success

 
based on a random control trial, but in the jail environment it was not 

possible to keep the control and treatment groups separate. Therefore, 

project stakeholders designed an alternative methodology, a quasiexperimental design, which provided a sufficient degree of confidence 

that the outcomes achieved could be attributed to the program.56

Stakeholders also addressed potential ethical concerns associated with 

RCTs, such as withholding the service intervention from a population that 

could potentially benefit from the services in order to maintain a control 

group for an impact evaluation. While there could be ethical concerns 

associated with using a control group for a random control trial evaluation 

of a service, limited availability of funding often prevents treating the 

entire population. Therefore, stakeholders indicated they could possibly 

draw from patients on a wait list for the program to serve as a control 

group. The Massachusetts project addressed ethical concerns with RCTs 

by including a “no denial of service policy” if an individual in the 

nonparticipating control group asked to participate in the program. This 

design required a significant increase in the evaluation’s sample size to 

address potential contamination between the treatment group and the 

control group in the sample.

During the third phase of a PFS project, service providers implement the 

prevention program. To manage performance risk, participating 

organizations provide oversight, monitor performance, and evaluate the 

extent to which outcomes are achieved, based on the structures and 

governance roles established in PFS contracts.

56In the quasi-experimental evaluation for the NYC Rikers Island project, the evaluator 

compared the outcomes for a cohort of adolescents incarcerated at Rikers during calendar 

year 2013 to a historical group that did not receive the program in order to determine the 

impact of the intervention.

Ongoing PFS Contracts 

Show Benefits of DataDriven Performance 

Management, but Final 

Outcomes Are Not Yet 

Known

Page 54 GAO-15-646 Pay For Success

 
Actively managing performance to achieve outcomes during contract 

implementation is vital to achieving results. While the structures and 

membership varied, stakeholders said PFS oversight bodies regularly 

reviewed performance data during service delivery. According to 

stakeholders, intermediaries and investors can bring performance 

management expertise to service providers and provide a rigorous focus 

on performance management and accountability. Officials from the 

intermediary for the New York City project said they continued to monitor 

the implementation of the service delivery using metrics that predict 

whether the project will be successful. They said that these metrics and 

targets allowed them to troubleshoot project implementation and 

intervene early to modify elements of the program that were not working. 

For example, there is a community service element to the Moral 

Reconation Therapy program that was modified to work in the Rikers 

Island jail environment. In another example, officials from the intermediary 

for the Peterborough Project said they used data analytics and provided 

program management support to help improve service delivery. In 2014 

the UK Department of Work and Pensions evaluated its Innovation Fund 

PFS projects and found that for its first 10 PFS projects the strong focus 

on monitoring and managing project performance built capacity among 

service providers and helped to increase performance levels and improve 

efficiency of service delivery.57

Social service providers often collect and report input and output data in 

response to multiple grant requirements, but these data may not always 

be useful for monitoring the performance of a PFS project in achieving 

outcomes. Service providers said the PFS model may require new 

methods to collect and use performance information that shows the 

program is being implemented as planned and with fidelity to the model 

that was shown to be successful. Some service providers may also need 

to invest in building their capacity to collect and report outcome data to 

participate in PFS projects. An official from one service provider noted 

that her organization invested in data entry and data analyst positions and 

has a team that collects, analyzes, and processes data that it submits to 

the intermediary.

57Andrew Thomas, Rita Griffiths, and Alison Pemberton, The Innovation Fund pilots 

qualitative evaluation: Early Implementation Findings, United Kingdom Department of 

Public Works (London, UK: 2014).

Implement Prevention Program

Using Data-Driven 

Performance Management

Page 55 GAO-15-646 Pay For Success

 
Projects in our study faced unanticipated challenges in delivering services 

and stakeholders said oversight mechanisms helped to identify issues 

quickly and facilitated decision making to address problems. For example, 

stakeholders in our study reported difficulty in identifying cases for 

referrals to the service provider. Maintaining the planned level of referrals 

was important because the evaluation and outcome payments were 

dependent on the level of participation in the project. An official from the 

service provider in the New York project said that approximately 3 to 4 

months into program implementation, there was a dip in the number of 

potential participants referred to the program. He said the cause was that 

the service provider was seeking referrals from parole bureaus where it 

had traditionally drawn few participants in order to maintain the project’s 

randomized referral and evaluation plan. However, he said this strategy 

depressed the overall number of referrals. As a result, they decided to 

expand the number of parole bureaus that could serve as a source of 

referrals.

Stakeholders said that pilot periods for service delivery before the 

evaluation starts could reduce unanticipated challenges during PFS 

implementation and reduce risks that service providers will not meet 

outcome targets. According to a report on the New York City project,58 it 

may be beneficial to pilot-test a program at full scale and only evaluate it 

after it achieves a steady state of operation. On the other hand, pilot 

programs require time and money, both of which are likely to be 

constrained for PFS projects. The New York City project included a pilot 

test conducted at half scale. Stakeholders said that although the 

experience was valuable, certain issues came to light only after the 

program expanded to full scale. A pilot project can also be used to build 

evidence on an intervention to develop a future PFS project. For example, 

the Asthma Impact Model for Fresno is implementing a pilot project from 

spring 2013 through early 2016. The pilot project includes a random 

control trial evaluation on the intervention to demonstrate the outcomes of 

a home-based asthma management program. If successful, the results of 

the pilot will be used to develop a PFS strategy.

Following program implementation, the parties to a PFS project must 

evaluate whether the service provider achieved outcome measures and 

58MDRC, Financing Promising Evidence-Based Programs: Early Lessons from the New 

York City Social Impact Bond (New York, NY: 2013). 

Evaluate Outcomes and Next 

Steps

Page 56 GAO-15-646 Pay For Success

 
determine whether the government must make outcome payments based 

on the results. PFS contracts we examined spanned 3 to 10 years and 

one project—New York City—had been completed as of the time of our 

review. Results for the New York City project were reported in July 2015 

and indicated that the program did not reduce recidivism among 

adolescents at Rikers Island and therefore did not meet the 

predetermined threshold of success: a 10 percent reduction in recidivism. 

As a result, New York City did not pay for the program and the 

intervention was discontinued as of August 31, 2015. Results of the 

evaluation of the initial cohort of the Peterborough Project were reported 

in August 2014 and indicated that reoffending rates for participants were 

reduced by 8.39 percent compared to a control group. A reduction of at 

least 10 percent within the cohort was necessary to trigger an outcome 

payment to investors, so no payment was made. However, the project’s 

terms state that outcome payments will be made if reoffending rates are 

reduced for each individual cohort by 10 percent or across all cohorts by 

7.5 percent. Therefore, if the evaluation to be conducted following the 

second cohort finds that reoffending rates across both cohorts were 

reduced by at least 7.5 percent, the Ministry of Justice will make outcome 

payments to investors.

We asked participating organizations what will happen to their programs 

at the end of their PFS contracts. Some told us that governments may 

consider funding the service delivery directly. In another case, the It’s All 

About Me adoption project has been designed so that if local 

governments use the service in large numbers over 10 years, the project 

will amass sufficient reserves after repaying investors to self-finance the 

service in the future. While PFS financial models were designed based on 

future government savings, some government stakeholders indicated 

they were not planning to reduce future agency budgets if PFS outcomes 

are achieved, but will reallocate the savings to address other unmet 

needs in the same agency budget.

Page 57 GAO-15-646 Pay For Success
While the effectiveness of the PFS model remains to be seen, 

stakeholders cited three potential roles the federal government could play 

to help address challenges at the state and local levels of government 

and further develop the PFS field: providing outcome payments, building 

capacity, and providing loan guarantees.

Providing outcome payments. State and federal officials and other 

stakeholders said the federal government could provide outcome 

payments in PFS projects to address the wrong pockets problem, which 

could arise when an intervention results in savings to the federal 

government as well as the government commissioning the project. This 

issue makes it challenging for state and local governments to have 

sufficient funds available to make outcome payments. In our case studies, 

some officials said that a key consideration when assessing project 

feasibility was whether cost savings to their jurisdiction were sufficient to 

justify the cost of implementing the project when other jurisdictions, such 

as the federal government, shared in the costs and savings as well. To 

date, the federal government has not become a formal party to any PFS 

contract in which the federal government would share in the project’s 

savings. However, the federal government has provided grants to support 

outcome payments for two of the case studies we examined. To assist 

The Federal 

Government Could 

Play a Role in 

Funding, Building 

Capacity for, and 

Managing Investor 

Risk in PFS, but a 

Formal Mechanism to 

Collaborate and 

Share Lessons 

Learned Does Not 

Exist

The Federal Government 

Could Play Several Roles 

to Address PFS 

Challenges at the State 

and Local Level

Page 58 GAO-15-646 Pay For Success
New York and Massachusetts with their recidivism PFS projects, DOL 

awarded nearly $24 million in grant funding to support outcome payments 

for these projects.59 A stakeholder said that the DOL grant catalyzed the 

New York project, allowing them to fund outcome payments for the first 

cohort of 1,000 participants. Having support for outcome payments 

helped Massachusetts officials increase the number of participants in the 

project, thus allowing for a stronger evaluation of the program’s 

outcomes.

The UK government also took steps to mitigate challenges related to the 

wrong pockets issue. For example, the £20 million (approximately $31 

million) Social Outcomes Fund, managed by the Cabinet Office, was 

created to address challenges related to public sector spending silos that 

occur between agencies in central and local governments trying to 

address complex social issues. The Social Outcomes Fund provides 

outcome payments for government departments, local authorities, and 

other organizations to implement innovative projects to address social 

problems where savings could be spread among several government 

agencies. This approach addresses the difficulty faced by a single office 

attempting to justify the cost of implementing the program. The federal 

government could also commission PFS projects and provide outcome 

payments for programs where it would capture all of the savings, such as 

programs targeting Social Security Disability Insurance.

Similar to state and local governments, as an outcome payor the federal 

government could face many of the same potential benefits, risks, and 

decisions regarding project feasibility identified by stakeholders in our 

study. Playing the role of outcome payor in PFS projects could provide 

benefits from potential future savings to the federal government if 

outcomes are achieved. For example, a stakeholder said a potential area 

where the federal government could play a role as outcome payor is in 

the provision of nonmedical asthma management services including inhome education, environmental assessment, and remediation such as 

59DOL officials noted that the projects funded under DOL’s PFS solicitation were required 

to contract with an independent validator to assess whether the project met its outcome 

targets. As part of the validation methodology, grantees will credibly demonstrate that 

outcome targets are achieved due to the intervention by defining the outcome target 

relative to a well-defined comparison population or control group. DOL has also engaged 

an independent evaluator (not the independent validator associated with the partnership) 

to assess the implementation, outcomes, and benefits of projects funded under the PFS 

solicitation.

Page 59 GAO-15-646 Pay For Success

 
mold removal, which could potentially reduce the number of asthma 

attacks and emergency visits by Medicaid recipients. Reducing 

emergency room visits paid by the Medicaid program could yield savings 

to both the state and federal governments. However, as an outcome 

payor, federal agencies would need to manage financial and performance 

risks similar to state and local governments. Federal agencies 

considering entering a PFS contract as an outcome payor would need to 

assess the feasibility of individual projects, including the factors 

stakeholders in our study identified, such as evidence on the intervention, 

cost-benefit analysis of the project, and the commitment of government 

leadership.

Another potential challenge for federal agencies to consider is whether 

their existing grant program authority can accommodate outcome 

payments. PFS projects can span 3 to 10 years, while grant program 

authorities may limit the duration of a grant project. For example, DOJ 

officials told us that the reentry grant program authorities under the 

Second Chance Act limited the grant performance period to 12 months. 

This restriction would have prohibited the use of a PFS project as no 

outcomes could be achieved in such a short project period. However, this 

time limit was later eliminated for grants funded by the appropriation 

made available for DOJ’s PFS efforts for fiscal years 2014 and 2015.60

Another issue for agencies to consider is the period of availability of the 

funds and what happens to funds obligated for outcome payments if they 

are not made because the service provider does not meet the outcome 

targets. DOL officials told us that the grant funds obligated for outcome 

payments will be deobligated and unavailable for new obligations if 

outcome targets are not met in the New York or Massachusetts PFS 

projects. In another example, CNCS’s fiscal years 2014 and 2015 

appropriations include authority that allows CNCS to reobligate funds to 

other authorized projects if outcome targets are not met.61

Building capacity for PFS projects. According to stakeholders, the 

federal government could make grants for feasibility studies and project 

60Consolidated Appropriations Act, 2014, Pub. L. No. 113-76, div. B, title II, 128 Stat. 5, 63 

(Jan. 17, 2014) and Consolidated and Further Continuing Appropriations Act, 2015, Pub. 

L. No. 113-235, div. B, title II, 128 Stat. 2130, 2193-94 (Dec. 16, 2014).

61Pub. L. No. 113-76, div. H, title IV, 128 Stat. 5, 401 and Pub. L. No. 113-235, div. G, title 

IV, 128 Stat. 2130, 2506-07.

Page 60 GAO-15-646 Pay For Success

 
development to build the capacity of state and local governments, 

intermediaries, and service providers, and offset administrative costs. In 

fiscal year 2014, CNCS awarded $12 million in grants to eight 

intermediaries to help governments and nonprofits build their capacity to 

implement and structure PFS projects.62 As of July 2015, the initial eight 

intermediaries had made awards to a total of 48 state and local 

governments and nonprofit organizations to fund technical assistance on 

PFS project development. According to CNCS, the grants address the 

limited availability of funding for planning, feasibility studies, and deal 

structuring, which has constrained growth in the field. In another example, 

DOJ and HUD are jointly pursuing a PFS demonstration project on 

permanent supportive housing for the prisoner reentry population.63 The 

project will make grants to intermediary organizations that will work with 

state and local governments and service providers to implement PFS 

projects and will include an evaluation to determine if this model of 

delivering supportive housing can achieve savings by decreasing 

government spending on corrections systems, homeless services, 

Medicaid, and crisis care services, some of which are federally funded.

The UK central government addressed capacity issues by creating 

organizations to manage grant making and technical assistance for 

organizations involved in PFS projects. Through its Commissioning Better 

Outcomes Fund, the Big Lottery Fund64 provides up to £6 million 

(approximately $9 million) in grants (up to a maximum of £150,000 

(approximately $229,000) per project) for technical assistance to assess 

62The award recipients for the 2014 CNCS PFS grants are: (1) Corporation for Supportive 

Housing; (2) Green & Healthy Homes Initiative; (3) Harvard Kennedy School Social Impact 

Bond Lab; (4) Institute for Child Success, Inc.; (5) National Council on Crime and 

Delinquency; (6) Nonprofit Finance Fund; (7) Third Sector Capitol Partners, Inc.; and (8) 

University of Utah Policy Innovation Lab. Recipients are required to select subgrantees in 

a competitive process. 

63Under DOJ’s fiscal year 2014 appropriation, the department was authorized to use up to 

$5 million (of its funding available for PFS projects) for programs implementing the 

Permanent Supportive Housing Model. DOJ entered into an interagency agreement with 

HUD to carry out a permanent supportive housing demonstration project with these funds. 

In fiscal year 2015, an additional $5 million of funding was authorized for PFS projects 

implementing the Permanent Supportive Housing model.

64The UK’s Big Lottery Fund is a government sponsored organization that is responsible 

for distributing a percentage of all funds raised by the National Lottery to projects 

supporting health, education, environment, and charitable purposes, from early years 

intervention to commemorative travel funding for World War II veterans. 

Page 61 GAO-15-646 Pay For Success

 
the feasibility of PFS projects. It also provides outcome payments to 

encourage and sustain government engagement. The Better Outcomes 

Fund seeks to increase the capacity of service providers, enhance 

understanding of how to organize and implement a successful PFS 

project, and increase the knowledge base of local government officials 

working with PFS projects in the field. A stakeholder from the Centre for 

Social Impact Bonds said these grants have helped improve the technical 

and financial capacity of key players in PFS projects.

State and federal officials also said that the federal government is 

uniquely positioned to help build capacity by encouraging knowledge 

sharing among parties that are implementing PFS projects. For example, 

to share knowledge with its grantees, DOL leads a working group for the 

state and local government recipients of PFS grant awards in an effort to 

increase knowledge and share best practices. DOL officials said that the 

monthly meetings have included speakers with PFS expertise, such as 

officials from Social Finance, the Peterborough Project, and other federal 

agencies. In addition, the White House Office of Social Innovation and 

Civic Participation co-hosted, with the Arnold Foundation and the 

Nonprofit Finance Fund, a series of PFS Summits in late 2014 and early 

2015 to highlight and advance regional activity underway across the 

country, build the longer term Pay for Success infrastructure to catalyze 

future projects, and facilitate an open, transparent dialogue among public, 

private, and philanthropic stakeholders. Officials from one state 

government said that knowledge sharing by the federal agencies has 

been helpful to the state as it develops and implements PFS projects.

UK government officials also said that knowledge sharing was an 

important component of the growth of PFS. As part of the annual 

reporting requirements for PFS grants, the Big Lottery Fund requires that

grantees inform officials of plans to communicate information on their 

learning and experience as the project progresses. Where the Big Lottery 

Fund supports outcome payments, projects are also required to evaluate 

their own performance, contribute to an independent program evaluation, 

and support dissemination of learning from these evaluations. In the UK, 

officials from the Centre for Social Impact Bonds said the Centre has also 

addressed knowledge sharing issues by coordinating meetings with other 

government organizations, holding workshops, networking, and making 

connections between interested parties.

Providing loan guarantees. A challenge to PFS projects that could limit 

growth is obtaining capital from investors. Investors said that the federal 

government could provide loan guarantees or credit enhancements for 

Page 62 GAO-15-646 Pay For Success
PFS projects to increase the availability of capital by reducing risk for 

investors. Stakeholders said that having the backing of the federal 

government would provide greater incentives for investment in PFS 

projects and help establish the market. For example, Bloomberg 

Philanthropies provided a $7.2 million loan guarantee for the New York 

City Riker’s Island project to reduce some of the risk to investors of losing 

their investment if outcome targets were not achieved. However, some 

stakeholders said a government loan guarantee would negate a primary 

benefit of PFS to transfer the risk of program implementation away from 

the government. Such guarantees provided by the federal government 

would potentially leave the federal government liable for paying for a 

project that did not achieve agreed-upon outcomes.

In our prior work, we stated that collaborative mechanisms, such as 

interagency groups or collaboration technology, can be used to develop 

policies, implement programs, and share information.65 The agencies in 

our study participated in an interagency PFS working group that met 

between 2012 and 2014 to share information on agency efforts related to 

PFS. Federal officials said the working group is no longer meeting 

because of staff turnover in leadership positions. Officials who 

participated in the working group said it was beneficial and furthered their 

agencies’ efforts related to PFS. For example, a federal agency official 

said the PFS working group helped officials network with their peers and 

collaborate on specific efforts. When CNCS developed its fiscal year 2014 

PFS grant notice of funding availability, it consulted with other federal 

agency officials with PFS expertise. Though officials found these 

meetings useful, they were not maintained with a regular schedule and 

information resulting from meetings was not documented or broadly 

disseminated. In May 2015, OMB officials noted that the White House 

and federal agency officials are working to restart interagency meetings 

on PFS.

A mechanism for agencies to share knowledge and information on PFS 

could give agencies interested in potential PFS projects access to leading 

practices and current information on how federal participation is evolving. 

65GAO, Managing for Results: Implementation Approaches Used to Enhance 

Collaboration in Interagency Groups, GAO-14-220 (Washington, D.C.: Feb. 14, 2014) and

Managing for Results: Key Considerations for Implementing Interagency Collaborative 

Mechanisms, GAO-12-1022 (Washington, D.C.: Sept. 27, 2012). 

There Is No Formal 

Mechanism for Federal 

Agencies to Collaborate 

on PFS

Page 63 GAO-15-646 Pay For Success

 
In addition, a collaborative mechanism would allow agencies to share 

information on more technical aspects of PFS, such as practices on how 

to evaluate savings that accrue to federal agencies as the result of 

successful PFS efforts. Officials from one federal agency said that 

information about the various roles that federal agencies have played in 

the PFS field would be informative, given that the approach to supporting 

PFS has varied by agency. Having a central mechanism for sharing 

information on PFS would also decrease the risk of duplicative efforts in 

projects that affect multiple agencies.

The PFS model is evolving as one among many tools for policymakers to 

consider when seeking to provide solutions for some of the nation’s most 

vexing social problems. According to stakeholders, PFS allows 

governments to shift the focus of social service interventions from costly 

remediation to lower cost prevention, while simultaneously mitigating risks 

associated with scale-up by funding projects with private investment. For 

service providers, the PFS model provides a source of upfront capital for 

operations, as well as an opportunity to scale-up promising programs and 

refine their capacity for sophisticated data collection and analysis.

As organizations consider pursuing PFS contracts, they will need to think 

about how they will manage the risks inherent in PFS contracting. For 

example, a government’s costs of implementing a PFS project, which can 

include investor returns, management fees, and evaluation costs, can be 

high, so the government must decide whether potential benefits outweigh 

these costs. Flexibilities built into PFS contracts can help all parties to the 

project manage risks by allowing them to make changes as 

implementation and evaluation methodologies evolve. In addition, the 

parties to a project will need to manage the potential risks that unintended 

consequences will negatively affect the population receiving treatment. 

Managing these risks requires rigorous design of project implementation 

and evaluation methodology to minimize perverse incentives for 

organizations to focus treatment on the easiest cases while avoiding the 

more challenging ones that could negatively affect outcomes. PFS is 

most likely an appropriate tool for prevention programs that address 

entrenched social problems. It is important to consider key project 

feasibility factors, such as the availability of sufficient evidence on an 

intervention’s effectiveness and the capacity of PFS stakeholders to 

collect and analyze the data required by these complex projects.

The PFS model is evolving, and early programs are just beginning to 

evaluate results. It is too soon to tell if governments will realize the 

Conclusions

Page 64 GAO-15-646 Pay For Success
benefits that induced them to try PFS. For example, a potential benefit of 

PFS is that it allows a government to shift the risk of implementing an 

innovative program to investors. In practice, investors whose return on 

investment is contingent on positive results may prefer projects that are 

based on rigorous evidence of success and may avoid innovative 

approaches that have not been rigorously tested. If this potential flight to 

programs with a strong evidence base turns out to become reality, it may 

not make sense for governments to rely on PFS projects. Instead, they 

may consider funding these types of programs directly, through traditional 

performance-based contracts that incorporate features of PFS projects 

that reduce the government’s risk, such as independent evaluation and 

governance rules that allow for strong management and oversight. By 

undertaking a PFS project to implement a program that is known to be 

successful, a government could be taking on extraneous costs for little or 

no benefit.

As the PFS field continues to grow and as Congress and federal agencies 

consider new investments in these projects, important considerations 

remain about the appropriate role for the federal government. Similar to 

state and local governments, the federal government faces potential 

benefits and challenges stemming from participation in PFS programs. As 

federal agencies consider expanding their involvement in PFS, it 

becomes increasingly important for officials at all levels of government to 

collaborate to share knowledge and experiences.

OMB led an informal working group on PFS for a limited period. Despite 

the stated usefulness of this group, notes and findings from the meetings 

were not shared among agencies, and the meetings were eventually 

discontinued. Having a mechanism for federal collaboration on PFS 

projects as the field grows would allow agencies to leverage the 

experience of early federal actors in the PFS field and would decrease 

the potential for missteps in developing projects due to information gaps 

and failure to learn from experience with this evolving tool of government.

To identify and broadly disseminate information on leading practices and 

lessons learned, the Director of OMB should establish a formal means for 

federal agencies to collaborate on PFS. This could include creating a 

formal working group and providing collaboration technologies, such as 

shared databases or web portals.

Recommendation for 

Executive Action

Page 65 GAO-15-646 Pay For Success
We provided a draft of this report to the Director of the Office of 

Management and Budget, the Secretaries of the Treasury, Labor, 

Housing and Urban Development, Justice, Health and Human Services, 

and Education, and the Chief Operating Officer of the Corporation for 

National and Community Service. 

The Office of Management and Budget (OMB) provided written comments 

concurring with our recommendation and is working with agencies to 

explore options for continued collaboration on Pay for Success. The 

response from OMB is reproduced in appendix V. The Departments of 

Labor, Housing and Urban Development, Health and Human Services, 

Education, and the Corporation for National and Community Service 

provided technical comments that were incorporated into the draft as 

appropriate. The Departments of Treasury and Justice had no comments 

on the draft report.

We are sending copies of this report to the Director of OMB, the 

Secretaries of the Treasury, Labor, Housing and Urban Development, 

Justice, Health and Human Services, and Education, and the Chief 

Executive Officer of CNCS.

If you or your staff have any questions about this report, please contact 

me at (202) 512-6806 or sagerm@gao.gov. Contact points for our Offices 

of Congressional Relations and Public Affairs may be found on the last 

page of this report. GAO staff who made key contributions to this report 

are listed in appendix VI.

Sincerely yours,

Michelle Sager

Director

Strategic Issues

Agency Comments 

and Our Evaluation

Page 66 GAO-15-646 Pay For Success
Appendix I: Objectives, Scope, and 

Methodology

This report responds to a request that we examine Pay for Success (PFS) 

to help Congress gain a better understanding of how the mechanism has 

been defined in theory and practice and what is known about its 

implementation and results to date. We examined (1) how selected PFS 

projects have been structured and what potential benefits these projects 

can provide; (2) how selected PFS contracts have been structured to 

address potential project risks; and (3) the potential roles for the federal 

government’s involvement in PFS projects.

To address these objectives, we began by identifying and reviewing 

relevant literature on PFS, including work produced by public sector, 

nonprofit, academic, and private sector organizations. We also 

interviewed individuals with public management expertise who had 

examined PFS and published work on or related to PFS. We identified 

some of these individuals through our literature review, while others were 

referred to us by other experts. These individuals were Rob Duggar of 

ReadyNation; Mildred Warner of Cornell University; Leonard Gilroy of the 

Reason Foundation; John Roman of the Urban Institute; Jitinder Kohli of 

Deloitte Consulting LLP; and Shar Habibi of In the Public Interest. During 

these interviews we asked the interviewees, among other things, about 

the potential benefits and challenges of PFS; the advantages and 

disadvantages that PFS has compared to other program funding 

mechanisms; and the potential roles that the federal government could 

play in PFS projects.

Based on our literature review and our interviews with individuals 

knowledgeable about PFS, we identified jurisdictions in the United States 

and abroad that were implementing, planning to implement, or studying 

the potential of PFS projects. We then selected seven U.S. and three 

United Kingdom (UK) case illustrations to review in depth. We selected 

cases from the UK in order to capture the country’s experience as the first 

to develop and implement PFS projects as well as because it is the 

location with the longest experience with this financing mechanism. We 

selected four PFS projects that were being implemented in the United 

States at the time we began our engagement in July 2014. Those projects 

were the Massachusetts Juvenile Justice PFS Initiative; the New York 

City Rikers Island Project; the New York PFS project: Increasing 

Employment and Improving Public Safety; and the Utah High Quality 

Preschool Program. Because those cases focused heavily on reducing 

recidivism and improving early childhood education, we selected two 

additional U.S. projects that were in development to reflect an additional 

policy area (improving health outcomes). Those projects were the South 

Carolina Maternal Health PFS Initiative and the Asthma Impact Model for 

Appendix I: Objectives, Scope, and 

Methodology

Page 67 GAO-15-646 Pay For Success
Appendix I: Objectives, Scope, and 

Methodology

Fresno (AIM4Fresno). Finally, we selected one case in which a state 

considered but ultimately decided not to move forward with a PFS project. 

That case was Maryland’s review of a potential PFS initiative focused on 

reducing recidivism. For the UK projects, we selected three projects that 

were also in implementation and varied in issue area, structure, and the 

participating organization that conceived them. Those cases were the 

Peterborough Prison Project; the London Homelessness Project; and the 

It’s All About Me Adoption Project, on reducing adoption wait times.

To obtain information about each of these case illustrations, we 

conducted site visits to Boston, Massachusetts; New York City, New 

York; Albany, New York; Annapolis, Maryland; Salt Lake City, Utah; and 

London, United Kingdom. During these visits, we requested and reviewed 

documents relevant to the case illustrations, including feasibility studies, 

contracts, and evaluation reports. We conducted in-depth interviews with 

representatives of organizations participating in the PFS projects in these 

locations. For each case, as appropriate, we interviewed representatives 

from government entities, intermediaries, service providers, investors, 

evaluators, and validators. We conducted some of the interviews by 

telephone before or after site visits due to scheduling conflicts. During the 

interviews, we asked the interviewees, among other things, about the 

potential benefits and risks of PFS; the advantages and disadvantages 

that PFS has compared to other program funding mechanisms; the 

specific roles their organizations play in PFS projects; the PFS project 

contract negotiations, including the establishment of outcome payment 

and governance structures; and the potential roles that the U.S. federal 

government or UK national government could play in PFS projects.

The interviewees for our case illustrations were representatives from: 

Bank of America Merrill Lynch; Bates Wells Braithwaite; Big Lottery Fund; 

Big Society Capital; Bloomberg Philanthropies; The Boston Foundation; 

Bridges Ventures; Center for Employment Opportunities; Chesapeake 

Research Associates; Collective Health; Esmee Fairbairn Foundation; 

Goldman Sachs; Greater London Authority; Granite School District 

Preschool Services; Harvard University’s Social Impact Bond Technical 

Assistance Lab; Imprint Capital; Maryland Department of Legislative 

Services; Massachusetts Executive Office of Administration and Finance; 

MDRC; New Profit Inc.; New York City Mayor’s Office; New York 

Department of Corrections and Community Supervision; New York 

Division of the Budget; The Osborne Association; Public Consulting 

Group; Roca Inc.; The Robin Hood Foundation; Rockefeller Foundation; 

Salt Lake County Mayor’s Office; Sibayltics LLC; Social Finance (UK); 

Social Finance (U.S.); St. Giles Trust; Thames Reach; Third Sector 

Page 68 GAO-15-646 Pay For Success
Appendix I: Objectives, Scope, and 

Methodology

Capital Partners; UK Cabinet Office; UK Department for Communities and 

Local Government; UK Ministry of Justice; United Way of Salt Lake; Utah 

Governor’s Office of Management and Budget; Utah House of 

Representatives; Utah School Readiness Board; Utah State University’s 

Early Intervention Research Institute; Vera Institute of Justice; and Voices 

for Utah Children.

To assess the PFS contract structure, we reviewed our prior reports and 

contracting literature to identify contracting practices from the federal 

experiences relevant to PFS contracts, which are complex service 

acquisitions. We compared the contracting practices to the PFS contracts 

in our study to establish their relevance to the design and implementation 

of PFS contracts.

Because the scope of our review was to assess the potential benefits and 

risks of PFS and the structure of PFS projects, we did not assess whether 

the outcomes established for specific projects and any cost savings that 

were identified with their attainment were appropriate, ambitious, or met 

other dimensions of quality. In addition, since most of the case 

illustrations we reviewed were in early stages of implementation, we did 

not request any performance information related to them and did not 

assess the likelihood of the projects successfully achieving outcomes.

To obtain information about federal agencies’ involvement in PFS, we 

reviewed relevant documentation, including the President’s Budget for 

fiscal years 2012 through 2016. Through this review, we identified eight 

federal agencies that were involved in an informal working group and 

played different roles vis-à-vis PFS. The Office of Management and 

Budget, working with the White House Office of Social Innovation and 

Civic Participation, has provided informal guidance on PFS efforts, and 

participated in the informal working group. The Department of the 

Treasury participated in the informal working group. The Departments of 

Labor (DOL), Housing and Urban Development (HUD), and Justice 

(DOJ), and the Corporation for National and Community Service (CNCS) 

have funded or plan to fund PFS projects. The Departments of Health and 

Human Services (HHS) and Education (Education) have been exploring 

opportunities to pursue the PFS concept. We interviewed officials from 

each of these agencies about their involvement in PFS, and asked them, 

among other things, about their strategic approaches for using PFS; the 

advantages and disadvantages that PFS presents compared to other 

financing tools; the amount of funding they allocated to PFS; interagency 

collaboration on PFS; and early lessons learned from their involvement in 

PFS.

Page 69 GAO-15-646 Pay For Success
Appendix I: Objectives, Scope, and 

Methodology

Furthermore, of the eight agencies involved in the informal working group 

on PFS, we interviewed officials and reviewed budget documents from six 

agencies that typically fund social programs—CNCS, DOJ, DOL, 

Education, HHS, and HUD—to identify illustrative examples of how these 

agencies have used evidence-based approaches in their decision making 

more generally.

We conducted our work from July 2014 to September 2015 in accordance 

with generally accepted government auditing standards. Those standards 

require that we plan and perform the audit to obtain sufficient, appropriate 

evidence to provide a reasonable basis for our findings and conclusions 

based on our audit objectives. We believe that the evidence obtained 

provides a reasonable basis for our findings and conclusions based on 

our audit objectives.

Page 70 GAO-15-646 Pay For Success
Appendix II: Illustrative Examples of EvidenceBased Programs from Selected Federal 

Agencies

To provide examples of how federal agencies are using evidence to 

inform decision making, we asked the agencies in our study to provide us 

with examples of existing programs or program proposals that were 

consistent with Office of Management and Budget (OMB) guidance on 

areas in which agencies could improve their use of evidence.

Table 3: Illustrative Examples of Evidence-Based Programs from Selected Federal Agencies

Example Agency Description 

Social Innovation Fund 

(SIF) 

Corporation for 

National and 

Community Service 

(CNCS)

Since fiscal year 2010, the SIF has combined public and private resources to grow the 

impact of innovative, community-based solutions that have compelling evidence of 

improving the lives of people in low-income communities throughout the United States. 

The SIF integrates rigorous evaluation into its program model. 

Justice Reinvestment 

Initiative (JRI)

Department of 

Justice (DOJ)

Since fiscal year 2010, JRI has provided targeted technical assistance and grant 

funding to help state, local, and tribal governments. The goals of the initiative are to 

use evidence to 

• analyze data on criminal justice systems to identify what factors are driving prison 

and jail population growth;

• develop strategies to control costs, improve public safety, reduce unnecessary 

confinement, and improve reentry programs; and 

• provide implementation grants to encourage adoption of significant policy and 

legislative changes resulting from JRI-supported efforts.

The JRI also supports the work of the blue ribbon Charles Colson Task Force on 

Federal Corrections, which is charged with finding practical, data-driven approaches to 

addressing overcrowding in federal prison

Reemployment and 

Eligibility Assessments 

and Reemployment 

Services (REA/RES)

-Demonstration and 

Evaluation

-Expanded REA/RES in 

all states

Department of 

Labor (DOL)

The REA/RES program seeks to reduce improper unemployment insurance payments 

and provide comprehensive services that assist participants with reemployment. Under 

the program, participants are provided referrals to services that facilitate job searches, 

including resume writing, interview techniques, and other job placement activities. In 

fiscal year 2015, DOL expanded the reemployment services model nationwide based 

on strong evidence of combined reemployment services and eligibility assessments for 

the Unemployment Insurance claimants. DOL meanwhile is further testing specific 

strategies in demonstration replications with rigorous evaluation.

Investing in Innovation 

Fund (i3) 

Department of 

Education 

(Education)

The Investing in Innovation Fund, established by the American Recovery and 

Reinvestment Act of 2009, provides funding to support (1) local educational agencies 

(LEA) or (2) nonprofit organizations in partnership with one or more LEAs or a 

consortium of schools. The purpose of this program is to support innovative and 

proven approaches to address persistent education challenges while also building 

knowledge of what works in key areas such as implementing college- and careerready standards, using data to inform instruction and personalize learning, and 

improving low-performing schools. The i3 program uses a tiered-evidence framework 

to direct larger awards to projects with the strongest evidence base and support 

promising projects that are willing to undergo rigorous evaluation.

Officials stated that the funding requested for 2016 would allow the department to 

continue fostering educational innovation while developing an evidence base in areas 

of high need and generating private-sector investment to complement the federal 

investment.

Appendix II: Illustrative Examples of 

Evidence-Based Programs from Selected 

Federal Agencies

Page 71 GAO-15-646 Pay For Success
Appendix II: Illustrative Examples of EvidenceBased Programs from Selected Federal 

Agencies

Example Agency Description 

Promoting Readiness of 

Minors in Supplemental 

Security Income 

(PROMISE)

Education, Social 

Security 

Administration 

(SSA), DOL, 

Department of 

Health and Human 

Services (HHS)

The PROMISE program is a crosscutting joint initiative funded by Education and SSA. 

In fiscal year 2013, Education awarded 5-year demonstration projects under the 

PROMISE program to develop, test and evaluate state-level interventions to improve 

outcomes for children with disabilities and their families and to reduce long-term 

reliance on the Supplemental Security Income (SSI) program and other public 

assistance. The projects are charged with establishing partnerships with state and 

local agencies and other entities to improve interagency collaboration in carrying out 

interventions and in developing innovative methods of providing services and 

supports. Awards were made to five states and one consortium of six states.

Performance 

Partnership Pilots 

Disconnected Youth 

(P3)

Education, DOJ, 

DOL, 

HHS, Institute of 

Museum and 

Library Services 

(IMLS), CNCS

P3 enables state, local, and tribal pilot sites to test innovative, outcome-focused 

strategies to achieve significant improvements in educational, employment, and other 

key outcomes for disconnected youth. Pilots must participate in a federally sponsored 

national evaluation of P3, which will consist of the analysis of participant 

characteristics and outcomes, an implementation analysis at all sites, and rigorous 

impact evaluations of promising interventions in selected sites.

Maternal, Infant, and 

Early Childhood Home 

Visiting (MIECHV) 

Program

HHS In fiscal years 2010-2015, the programs provided funding to support evidence-based 

models such as curricula designed to be administered to adolescents in school 

settings and youth-based leadership activities) to prevent teen pregnancy, sexuallytransmitted infections, and associated sexual risk behavior. This initiative 

encompasses programs administered by the Administration for Children and Families, 

the Centers for Disease Control and Prevention, and the Office of the Assistant 

Secretary for Health. 

Teen Pregnancy 

Prevention

HHS In fiscal years 2012-2015, the program provided funding to support evidence-based 

models to prevent teenage pregnancy, such as curricula designed to be administered 

to adolescents in school settings and youth-based leadership activities. This initiative 

encompasses programs administered by the Administration for Children and Families, 

the Centers for Disease Control and Prevention, and the Office of the Assistant 

Secretary for Health.

Source: GAO analysis of information provided by selected agencies. | GAO-15-646

Page 72 GAO-15-646 Pay For Success
Appendix III: Overview of PFS Projects in Our 

Study

Table 4: Overview of U.S. PFS Projects in Our Study 

New York City (NYC) Rikers 

Island Project

Massachusetts (MA)

Juvenile Justice PFS 

Initiative

New York (NY): Increasing 

Employment and 

Improving Public Safety 

Utah High Quality 

Preschool Program

Policy area Recidivism Recidivism and employment Recidivism and employment Early childhood 

education

Project time 

frame

Beginning in September and 

running 3, 4, or 6 yearsa Oct. 2013-Dec. 2019 Oct. 2013- March 2019 School year 2013-2014 – School year 2017-

2018

Investment $9.6 million loan from 

Goldman Sachs. 75 percent 

of the loan is guaranteed by 

Bloomberg Philanthropies.

$16.11 million, including a $8 

million loan from Goldman 

Sachs and $8.11 million in 

funding from various 

philanthropic organizations. 

$13.5 million equity 

investment from 40+ 

investors and foundations. 

10 percent of the 

investment is guaranteed by 

the Rockefeller Foundation. 

$7 million, including a 

$4.6 million loan from 

Goldman Sachs and a 

$2.4 million subordinate 

loan from J.B. Pritzker

Foundation. 

Size of target 

population 

Approximately 3,000 

individuals per year

929 individuals 2,000 individuals Approximately 3,500 

individuals

Stakeholders • 2 government agencies

(NYC Mayor’s Office and 

Department of 

Corrections) 

• Intermediary (MDRC) 

• 2 service providers

(Osborne Association 

and Friends of Island 

Academy) 

• 1 private-sector 

investor (Goldman 

Sachs) and 1 

philanthropic loan 

guarantor (Bloomberg 

Philanthropies)

• Evaluator (Vera Institute 

of Justice)

• 12 government 

agencies; (U.S. 

Department of Labor, MA 

Executive Office of 

Administration and 

Finance, MA Office of the 

Commissioner of 

Probation, and MA 

Department of Youth 

Services, MA Department 

of Corrections; the MA

Parole Board; the Suffolk 

County Sheriff’s 

Department; the

Middlesex County Sheriff’s 

Department; the Hampden 

County Sheriff’s 

Department; and the 

Essex County Sheriff’s 

Department. Also, data 

for evaluation and

validation come from the 

Executive Office for Labor 

and Workforce 

Development (wage data) 

and the Department of 

Criminal Justice 

Information Services 

(criminal records) 

• 5 government offices

(U.S. Department of 

Labor, NY Governor’s 

Office, the NY Division 

of the Budget, NY 

Department of Labor, 

and NY Department of 

Corrections and 

Community 

Supervision); 

• Intermediary (Social 

Finance)

• Service provider

(Center for 

Employment 

Opportunities) 

• Over 40 private 

investors) and 3 

philanthropic 

investors (Robin Hood 

Foundation, 

Rockefeller 

Foundation, and Laura 

and John Arnold 

Foundation)

• 2 government 

offices (Salt Lake 

County and the 

Utah School 

Readiness Board) 

• Intermediary

(United Way of Salt 

Lake)

• 6 service 

providers 

• 1 private-sector 

investor (Goldman 

Sachs) and 1 

philanthropic 

investor (J.B. 

Pritzker

Foundation)

• Evaluator (Utah 

State University)

Appendix III: Overview of PFS Projects in 

Our Study

Page 73 GAO-15-646 Pay For Success
Appendix III: Overview of PFS Projects in Our 

Study

New York City (NYC) Rikers 

Island Project

Massachusetts (MA)

Juvenile Justice PFS 

Initiative

New York (NY): Increasing 

Employment and 

Improving Public Safety 

Utah High Quality 

Preschool Program

• Intermediary (Third 

Sector Capital Partners) 

• Service provider (Roca) 

• 1 private-sector investor

(Goldman Sachs) and 5 

philanthropic investors 

• Fiscal services providerb

(New Profit) 

• Evaluator (Urban 

Institute) 

• Validator (Public 

Consulting Group)

• Evaluator (NY 

Department of Labor 

Research and NY 

Department of 

Corrections and 

Community 

Supervision Research)

• Validator (Chesapeake 

Research Associates)

Number of 

outcome 

metrics

1 3 3 1

Evaluation 

method

Quasi-experimental 

evaluation

Random control trial evaluation Random control trial 

evaluation

Review of evidence for 

the achievement of 

specific outcomes

Range of 

outcome 

payments

$0-$11.7 million $0-$28 million $0-$21.5 million Dependent on 

numerous variables

First 

outcome 

payment due

Summer 2015 Job readiness payment: May 

2015

Recidivism and employment 

payments: Fall 2017

Early 2017 Fall 2015

Government

budget 

structure 

No new budget structure for 

outcome payments.

The state requests an annual 

appropriation for outcome 

payments and backs future 

payments with its full faith and 

credit.

The state is renewing a 2-

year appropriation every 

year.

Legislation created a 

special account into 

which an appropriation 

to cover outcome 

payments was placed.

Oversight 

structure

Intermediary was responsible 

for project management, 

including making adjustments 

to service delivery and 

coordinating with and 

reporting to other program 

partners.

Provided by operating and 

oversight committees, 

composed of the project’s 

stakeholders.

Provided by the executive 

steering committee, which 

provides strategic guidance, 

the management 

committee, which provides 

operational management, 

and working groups, which 

provide implementation 

support.

Provided by the Utah 

School Readiness 

Board, which receives 

and reviews regular 

operational reporting 

from the intermediary.

Source: GAO analysis of project documents. | GAO-15-646

a

In July 2015, the evaluator determined that the program did not lead to reductions in recidivism rates 

for participants in its 1-year assessment of the project. The New York City government did not make 

outcome payments to investors and the program was discontinued as of August 31, 2015.

b

The fiscal services provider in this project is responsible for processing and overseeing payments to 

the intermediary and service provider.

Page 74 GAO-15-646 Pay For Success
Appendix III: Overview of PFS Projects in Our 

Study

Table 5: Overview of United Kingdom (UK) PFS Projects in Our Study

Peterborough Prison Project London Homelessness Project It’s All About Me Project

Policy area Recidivism Homelessness Adoption

Project time frame September 2010-August 2017a Begun in November 2012 and 

running for three years

Beginning in October 2013 and 

continuing for 10 years

Investment £5million ($8 million) Information not publicly available £3 million ($5 million)

Project size 3,000 individuals 830 individuals 140 children per year 

Stakeholders • 2 national-level outcome 

payors (Ministry of Justice and 

Big Lottery Fund)

• Intermediary (Social Finance)

• Consortium of service 

providers (One Service)

• Multiple private and 

philanthropic investors

• Assessors (Qinetiq and 

University of Leicester and 

Greenwich)

• 1 national-level government

organization (Department for 

Communities and Local 

Government)

• 1 local-level government 

organization (Greater London 

Authority)

• 2 service providers (Thames 

Reach and St. Mungo’s 

Broadway)

• Intermediary (for St. Mungo’s 

Broadway: Triodos Bank)

• Multiple investors

• Multiple local-level 

governments (UK Local 

Authorities)

• 1 contract management 

organization (It’s All About Me 

Service Company)

• Multiple service providers

• 2 Investor organizations

(Bridges Ventures and Big 

Society Capital)

Number of 

outcome metrics

1 5 4

Method for 

determining 

outcome 

achievement

Quasi-experimental evaluation –

propensity score matching

Greater London Authority reviews 

evidence for the achievement of 

specific outcomes

Review of government documents 

that show achievement of specific 

outcomes

Range of outcome 

payments

£0-£8 million ($0-$12 million) £3-£5 million ($5 million-$8 million) Up to £53,600 per child ($0 -

$87,000)

First outcome 

payment due

August 2014b Payments made quarterly 

throughout the project

On registration of the child in the 

project

Budgeting 

structure for 

outcome payments

Funding for outcome payments was 

provided by the Ministry of Justice 

and the Big Lottery Fund.

The UK Department for 

Communities and Local 

Government provided grant funding 

of up to £5 million ($8 million) to the 

Greater London Authority to 

commission the project and make 

outcome payments.

Local authorities enter into a 

contract with a voluntary adoption 

agency

Oversight structure Intermediary has responsibility for 

meeting with and reporting to the 

other stakeholders. 

Two groups composed of various 

stakeholders provide strategic and 

operational management. There is 

also regular government monitoring 

of service providers. 

The Board of the IAAM Service 

Company, composed of 

representatives of the developers, 

service providers, and investors, 

manages oversight.

Source: GAO analysis of project documents. | GAO-15-646

a

The project was intended to include three cohorts of prisoners, each closing after 2 years or 1,000 

prisoners had been discharged. As a result, the project was expected to include six years of prisoner 

intake, with service provision continuing for seven years. The period for data analysis and verification 

would take the entire length of the contract to eight years. However, the UK Ministry of Justice 

Page 75 GAO-15-646 Pay For Success
Appendix III: Overview of PFS Projects in Our 

Study

decided to terminate services for the third cohort after initiating a new criminal justice policy change, 

shortening the length of the project. 

b

The UK Ministry of Justice and Big Lottery Fund were due to make outcome payments if the 

evaluation of outcomes for the first cohort of prisoners showed a 10 percent or larger decrease in 

recidivism. However, in August 2014, the assessors found that recidivism among prisoners was 

reduced by 8.39 percent, and therefore no outcome payment was made.

Page 76 GAO-15-646 Pay For Success
Appendix IV: Example of United Kingdom Rate 

Card for Social Impact Bonds

Government officials from the United Kingdom (UK) told us that the rate 

of return for Social Impact Bonds funded with the Department for Work 

and Pensions’ (DWP) Innovation Fund are not subject to individual 

negotiations. The Innovation Fund is a pilot initiative that has 

commissioned Social Impact Bonds to help young people who are 

disadvantaged participate and succeed in education or training, therefore 

improving their employability and reducing their dependency on benefits. 

The Innovation Fund is 100 percent outcome based. Investors or 

intermediaries work with service providers to bid on Social Impact Bond 

projects based on the predetermined outcome payments published in 

DWP’s rate card (see figure 11). For example, if a service provider meets 

an outcome on an individual’s sustained employment, DWP pays £2,000 

($3,000). DWP makes awards for Social Impact Bond projects through a 

competitive bidding process. DWP will pay the investors or intermediaries 

solely on the basis of outcomes achieved. The service providers receive 

funding from the investors or intermediaries to cover their delivery costs, 

so they do not have to take the risk of depending on outcome payments.

Appendix IV: Example of United Kingdom 

Rate Card for Social Impact Bonds

Page 77 GAO-15-646 Pay For Success
Appendix IV: Example of United Kingdom Rate 

Card for Social Impact Bonds

Figure 11: United Kingdom Department for Works and Pensions (DWP) Rate Card

Note: National Vocational Qualifications (NVQ) are occupational certifications in the UK. The 

standards are statements of performance that describe what competent people in a particular 

occupation are expected to be able to do and cover all the main aspects of an occupation, including 

current best practices, the ability to adapt to future requirements, and the knowledge and 

understanding that underpin competent performance.

Page 78 GAO-15-646 Pay For Success
Appendix V: Comments from the Office of Appendix V: Comments from the Office of Management and Budget

Management and Budget

Page 79 GAO-15-646 Pay For Success
Appendix VI: GAO Contact and Staff 

Acknowledgments

Michelle Sager, (202) 512-6806 or sagerm@gao.gov

In addition to the above contact, Elizabeth Curda (Acting Director) and 

Barbara Lancaster supervised this review and the development of the 

resulting report. Virginia Chanley, Karin Fangman, Sherrice Kerns, Donna 

Miller, and Daniel Webb made significant contributions to this report. In 

addition, Joy Booth, Alicia Cackley, Jeffrey DeMarco, Deirdre Duffy, Gale 

Harris, Katherine Iritani, Shelby Kain, Adam Miles, Susan Offutt, Erin 

Saunders-Rath, Paul Schmidt, and William Woods made key 

contributions.

Appendix VI: GAO Contact and Staff 

Acknowledgments

GAO Contact

Staff 

Acknowledgments

(451107) Page 80 GAO-15-646 Pay For Success
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates. 

Listen to our Podcasts and read The Watchblog.

Visit GAO on the web at www.gao.gov.

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.