U.S. DEPARTMENT OF THE TREASURY

Using Evidence with State and Local Fiscal 

Recovery Funds
2

More Information

For More Information on Treasury COVID-19 

Economic Relief & Recovery Programs:

https://home.treasury.gov/policyissues/coronavirus

To Request State & Local Fiscal Recovery Funds: 

Please visit www.treasury.gov/SLFRP

For General Inquiries on State & Local Fiscal 

Recovery Funds:

Please email SLFRP@treasury.gov
3

The American Rescue Plan Act of 2021 

The American Rescue Plan Act (ARPA) is providing fast and direct economic assistance 

for American workers, families, small businesses, and industries and it: 

• Continues programs started by the CARES Act in 2020 and Consolidated 

Appropriations Act in 2021.

• Adds new phases, allocations, and guidance to address issues related to the 

continuation of the COVID-19 pandemic. 

• Creates new programs to address continuing pandemic-related crises, and fund 

recovery efforts as the United States begins to emerge from the COVID-19 

pandemic.

• Was passed by Congress on March 10, 2021, and signed into law on March 11, 

2021.
4

Coronavirus State and Local Fiscal Recovery 

Funds Overview

ARPA created the Coronavirus State and Local Fiscal Recovery Funds (SLFRF) program 

to deliver $350 billion for state, territories, municipalities, counties, and Tribal 

governments much needed aid.

The key objectives for SLFRF are to: 

• Support the urgent COVID-19 response efforts to continue to decrease spread of 

the virus and bring the pandemic under control.

• Replace lost revenue for eligible recipients to strengthen support for vital public 

services and help retain jobs.

• Support an equitable recovery through immediate economic stabilization for 

households and businesses 

• Address systemic public health and economic challenges that have contributed to 

the inequal impact of the pandemic on certain populations.
5 State and Local Fiscal Recovery: 

Reporting Guidance

The Reporting Guidance addresses priority areas to ensure a speedy and equitable recovery. 

Detailed information on reporting requirements and 

deadlines is available at: www.treasury.gov/SLFRP

Equity Focused Community 

Empowering

Results and 

evidence 

focused

WorkerCentered
6

Annual Recovery Plan: 

Evidenced-Based Interventions

Identify amount of project funds allocated to evidence-based interventions OR if 

undergoing rigorous program evaluation. 

Applies to most Expenditure Categories in:

• Public Health (EC 1)

• Negative Economic Impacts (EC 2) 

• Services to Disproportionately Impacted Communities (EC 3) 

Encouraged to use Learning Agendas and relevant evidence Clearinghouses to assess 

the level of evidence for your interventions. 

Evidence defined in Appendix 2
Building and Using Evidence
Building and Using 

Evidence 

Diana Epstein

Evidence Team Lead

U.S. Office of Management and Budget
Federal Focus on Evidence

• Foundations for Evidence-Based Policymaking Act of 2018 (“Evidence 

Act”)

• New statutory framework for agencies to strategically plan and organize 

evidence-building, data management, and data access

• Learning Agenda  Strategic evidence building plan that highlights priority 

questions and focuses attention on building the evidence needed to solve big 

problems

• Presidential Memorandum on Restoring Trust in Government 

Through Scientific Integrity and Evidence-Based Policymaking 

(January 2021)

• “It is the policy of my Administration to make evidence-based decisions 

guided by the best available science and data.” 

• SLFRF requires recipients to report if (and how much) they use funds 

for evidence-based interventions and/or rigorous program 

evaluations

• Investing in data capacity and program evaluations are allowable uses of 

funds

9
What is an Evidence-Based Intervention?

• SLFRF requires that prior studies of the intervention have demonstrated it 

has a strong or moderate evidence base

• Strong evidence: can support causal conclusions for the specific program 

proposed with highest level of confidence

• One or more well-designed and well-implemented experimental studies 

conducted on the proposed program with positive findings on one or more 

intended outcomes

• Moderate evidence: there is a reasonably developed evidence base that can 

support causal conclusions 

• One or more quasi-experimental studies with positive findings on one or more 

intended outcomes OR 

• Two or more nonexperimental studies with positive findings on one or more 

intended outcomes

10
Importance of Evaluation 

• Evaluation is an assessment using systematic data collection and analysis 

intended to assess program/policy/organization’s effectiveness and efficiency

• Certain questions can only be answered through evaluation

• What is the impact of a program, i.e. did the intervention itself cause the 

observed change?

• Which version of a program causes better outcomes? 

• Need to plan for evaluation at the outset to ensure necessary data are 

collected, comparison group formed, etc.

• Evaluation promotes learning and improvement; not simply a tool for 

accountability, and not an up or down vote on the program’s value

11
Evidence and the SLFRF

• Investing SLFRF funds in evidence-based policies helps ensure 

that dollars are used efficiently and effectively

• Maximize value of investments by using data and evidence to 

inform budgetary and programmatic decision making

• Encourages state and local leaders to invest SLFRF funds in most 

effective programs and policies, leading to improved outcomes 

for communities served

• Opportunity to test new strategies, or test strategies in new 

conditions, and conduct rigorous program evaluations to build 

evidence on what works, for whom, and how 

• Use this opportunity to build a culture of evidence-based 

policymaking and learning and improvement that can far outlast 

SLFRF funds 

12
Using and Building Evidence to Improve Decisions

• My goal is to address educational disparities

• I could use funds to invest in evidence-based educational services 

and practices (and track outcomes)

And/or

• I could conduct an evaluation of an evidence-based educational service 

or practice with a new population 

And/or

• I could conduct an evaluation of an innovative educational service or 

practice designed with my community’s unique needs in mind 

• Results can tell me if disparities are being addressed and how 

we might serve students even better

13
Resources

Evidence@omb.eop.gov

• OMB Evidence and Evaluation: 

https://www.whitehouse.gov/omb/information-for-agencies/evidence-andevaluation/

• Evaluation.gov Website (Forthcoming) 

• Learning Agendas and Building a Culture of Evidence: 

https://www.whitehouse.gov/wp-content/uploads/2021/06/M-21-27.pdf

• Evidence Act Foundational Guidance, including Learning Agendas: 

https://www.whitehouse.gov/wp-content/uploads/2019/07/M-19-23.pdf

• Evaluation Standards and Practices: https://www.whitehouse.gov/wpcontent/uploads/2020/03/M-20-12.pdf

14
Panel Discussion
The Why and What of 

Evaluation

Rohit Naimpally

Senior Research & Policy Manager

J-PAL North America
J-PAL | THE WHY AND WHAT OF EVALUATION 17
I. Why Evaluate?

II. What is Evaluation?

III. How to Evaluate?
J-PAL | THE WHY AND WHAT OF EVALUATION

Compliance and Reporting Guidance

19
Problem  Proposed solution

Problem: High rates of violence in some American cities

• Homicide and violent crime rates are far higher for 

youth being failed by the system, especially young 

men of color, than their peers.

Proposed solution: Cognitive Behavioral Therapy (CBT)

• Becoming a Man (BAM) program encourages 

individuals to examine their automatic thought 

processes and responses

J-PAL | THE WHY AND WHAT OF EVALUATION 20
J-PAL | THE WHY AND WHAT OF EVALUATION

Clear, credible results 

During the program year...

44% reduction in violent crime arrests 

– 18 arrests per 100 youth  8 arrests per 100 youth

31% reduction in overall arrests

– 52 arrests per 100 youth  36 arrests per 100 youth

Why? Follow-up experiment suggests the program helps students 

to slow automatic responses 

21
Policy attention 

Chicago Mayor Rahm Emmanuel increased funding for 

BAM across the city after the first study.

J-PAL | THE WHY AND WHAT OF EVALUATION Investments by Maxim Kulikov from the Noun Project

Former President Barack Obama made federal funding 

available to expand BAM and mentioned BAM at the 

launch of the “My Brother’s Keeper” initiative.

22
I. Why Evaluate?

II. What is Evaluation? 

III. How to Evaluate?
Program evaluation

Evaluation

Program 

Evaluation

Impact 

Evaluation

J-PAL | THE WHY AND WHAT OF EVALUATION 24

What is the causal effect of the program on outcomes?
How to measure impact?

J-PAL | THE WHY AND WHAT OF EVALUATION

Impact is defined as a comparison between:

What actually happened and

What would have happened, had the program 

not been introduced (i.e., the “counterfactual”)

25
J-PAL | THE WHY AND WHAT OF EVALUATION

Selecting the comparison group

Idea: Select a group that is exactly like the group of 

participants in all ways except one—their exposure to the 

program being evaluated

Goal: To be able to attribute differences in outcomes to the 

program (and not to other factors)

26
J-PAL | THE WHY AND WHAT OF EVALUATION

Methods as tools 

Pre-post

Simple 

Difference

DifferenceinDifference Regressions Randomized 

evaluation

Wrench by Daniel Garrett Hickey from the Noun Project

Screwdriver by Chameleon Design from the Noun Project

27
1. Identify eligible 

participants

2. Random lottery 3. Measure 

outcomes

Intervention

(2,740)

(1,473)

Randomized Evaluation of Becoming a 

Man (BAM)

J-PAL | THE WHY AND WHAT OF EVALUATION 28

No 

Intervention
I. Why Evaluate?

II. What is Evaluation? 

III. How to Evaluate?
Who?

• Internal evaluation 

groups:

– E.g. Minnesota 

Management and Budget, 

The Lab @ DC

• Local universities:

– E.g. California Policy Lab

• Evaluation firms:

– E.g. MDRC, Mathematica

J-PAL | THE WHY AND WHAT OF EVALUATION 30
J-PAL | THE WHY AND WHAT OF EVALUATION 31

Evaluation Resources

Groups like J-PAL offer 

several resources:

• Trainings and workshops

• Toolkits and guides

• Technical assistance
J-PAL | THE WHY AND WHAT OF EVALUATION 32

Evidence Resources
INSERT PEW COVER SLIDE HERE

The Pew Charitable Trusts

Results First Initiative

Sara Dube, Project Director

sdube@pewtrusts.org

SLFRF Evidence Webinar

August 11, 2021
Results First Clearinghouse Database
Evidence-Based Policymaking Resource Center

A collection of resources and promising state and county 

examples organized by key components
Incorporating evidence into state programs
A current example in Minnesota

LifeSkills Training:

• Delivering an evidence-based social-emotional learning program to 14,000 

students

• Conducting fidelity monitoring

• Evaluating impact through an experimental design study
Using evidence when allocating resources

Include sections on 

evidence and evaluation 

in budget proposal 

forms

Highlight evidence-based items for 

decision-makers
1. Legislature 

passes 

authorizing 

statute

2. Agency 

develops and 

publishes 

request for 

proposals (RFP)

3. Applicants 

submit proposals

4. Agency 

reviews 

proposals

5. Agency and 

applicants 

negotiate and 

sign contract

6. Grantees 

implement 

proposed 

activities

39

Using evidence in new or existing grant programs
Sample language

• “Evidence-based practices that could be supported with this funding include...”

• “Applications that employ licensed mental health professionals certified in any of 

the following early childhood evidenced-based practices will be given 

preference: Attachment Biobehavioral Catch-up (ABC) Child-Parent 

Psychotherapy (CPP) Parent-Child Interaction Therapy (PCIT).”

• “Responders must propose to use grant funds to cover staff time for mental 

health providers to attend training and become certified in Managing and 

Adapting Practice (MAP), Trauma-focused Cognitive Behavior Therapy (TF-CBT) 

and Cognitive Behavior Intervention for Trauma in Schools (CBITS).”
Effective Program Evaluations

August 2021

Grace Simrall

Office of Civic Innovation and Technology

Office of Performance Improvement

Louisville Metro Government

@Greendrv

@LouisvilleCIT
Establish Policy

Our commitment:

Louisville Metro values evaluation as a tool for program improvement 

and accountability, and therefore seeks to evaluate programs wherever 

possible. Evaluations, whether conducted internally or externally, 

should adhere to the following standards.

Standards:

1. Continuous Improvement

2. Accuracy

3. Equity and Ethics

4. Transparency
Evaluation in Action: F2ACT

Program Description:

F2ACT was created in July 2015 to meet a need for the 

Department of Corrections – there were inmates who would 

leave and often return to Corrections facilities. The program has 

evolved over time, but essentially assigns inmates who are 

considered at high risk for returning a case manager who helps 

plan their discharge through several avenues: ensuring they have 

housing, transportation, medication and insurance, bus tickets, 

an ID, and even a backpack with clothes and hygiene items.
Evaluation in Action: F2ACT

Purpose of the Evaluation:

To determine whether the F2ACT Program is having the intended 

impact of reducing recidivism

Recidivism is defined as having a greater number of bookings 

after release from the F2ACT Program

Success is defined as having fewer or equal number of bookings 

after release from the F2ACT Program

Complete Success is defined as having zero bookings after 

release from the F2ACT Program.
Project Name Here F2ACT

Engage stakeholders: Stakeholder Matrix

Stakeholder Known questions/interest Involvement in 

evaluation When Use of evaluation results

Corrections leadership Is the program effective? Setting questions, 

deadline

Initial stakeholder 

meeting 8/23; 

update with first 

draft of final 

evaluation

Present at a conference in 

April 2020

F2ACT Program staff How does the program affect 

subsequent bookings? What role does 

housing play? What role do other 

program components play?

Consulting throughout Initial stakeholder 

meeting 8/23; adhoc as needed for 

data questions 

and feedback

Present at a conference in 

April 2020 ; use to make 

program changes

Corrections data 

analyst

What data exists? Are there gaps? Pulling data; assisting 

with analysis

Initial stakeholder 

meeting 8/23; 

Understand for 

implementation of next 

evaluation

F2ACT participants n/a n/a Program improved to be 

more effective

Nationally accredited 

correctional facilities

Interest in program and results None April 2020 Evaluation results will be 

presented nationally at a 

conference in April 2020
Inputs Activities Outputs Outcomes Impact

F2ACT Project Name Here

Reduced 

recidivism

Describe program: Logic Model

Reduced 

costs to 

Corrections

Community 

partners/referral 

agencies

# released 

with insurance

# released 

with 

IDs/backpack 

if needed

F2ACT 

program 

staff

Stability allows 

participants to avoid 

crime and new 

arrests

Participants have 

somewhere to go 

when they reenter and their 

needs are met for 

30 days postrelease

Case management in 

the following areas:

• Housing

• Medication

• Insurance

• Transportation

• ID

• Bus tickets

• Backpack 

(clothes, 

hygiene, etc.)

• Guardianship

# released 

with housing

# released 

with 

medication if 

needed

# released 

with 

transportation 

to housing

Better 

outcomes for 

inmates
Our Data

F2ACT program participant 

data

Source: program data in an Excel file

Data was pulled from July 1, 2015 to 

August 31, 2017

Booking records for the F2ACT 

participants

Source: XJail database

Data was pulled from July 1, 2013 to 

August 31, 2019

Data Sources
Creating 

one 

dataset

Get unique number of 

participants

Join program data to 

booking data (two years 

before and after 

participation)

Calculate new variables 

and run descriptive and 

inferential statistics

Methodology
One-time participants

• 648 one-time participants

• Average age: 36.8

Repeat participants

• 68 repeat participants

– 60 participated 2 times

– 8 participated 3 or more times

• Average age: 38.7

Category Count Percent

Male 426 65.7%

Female 222 34.3%

White 462 71.3%

Black 180 27.8%

Asian 3 0.5%

Hispanic 2 0.3%

Other 1 0.1%

Category Count Percent

Male 53 77.9%

Female 15 22.1%

White 44 64.7%

Black 22 32.4%

Asian 1 1.5%

Hispanic 1 1.5%

Program Demographics
Program Services – Housing

One-time participants Repeat participants

0

20

40

60

80

00

20

1

1

1

1

1

1

1

Top 8 Housing Placements 

0

2

4

6

8

0

2

4

6

8

20

Top 7 Housing Placements
Program Services – Other Services

One-time participants Repeat participants

Category Count Percent

Medication 15 22.1%

Backpack 10 14.7%

Insurance 2 2.9%

Identification card 1 1.5%

Bus tickets 1 1.5%

Category Count Percent

Backpack 125 19.3%

Medication 109 16.8%

Insurance 53 8.2%

Identification card 21 3.2%

Bus tickets 19 2.9%
Category Count Percent

Complete success: 

Zero Bookings since Release 161 22.5%

Success: 

Fewer Number of Bookings since Release 315 44%

Success: 

Equal Number of Bookings since Release 83 11.6%

Recidivism: 

Greater Number of Bookings since Release 157 21.9%

78% 

success

Results
Average days 

booked

Total days 

booked

Before:

177.98

Before:

127,437

After:

121.53

After:

87,012

40,425 fewer days

Savings of $2,906,961.75 

Results
Effective Program Evaluations

August 2021

Grace Simrall

Office of Civic Innovation and Technology

Office of Performance Improvement

Louisville Metro Government

@Greendrv

@LouisvilleCIT
Q&A
56

More Information

For More Information on Treasury COVID-19 

Economic Relief & Recovery Programs:

https://home.treasury.gov/policyissues/coronavirus

To Request State & Local Fiscal Recovery Funds: 

Please visit www.treasury.gov/SLFRP

For General Inquiries on State & Local Fiscal 

Recovery Funds:

Please email SLFRP@treasury.gov
U.S. DEPARTMENT OF THE TREASURY

Using Evidence with State and Local Fiscal 

Recovery Funds