FY 2024 

Annual Evaluation Plan

U.S. Department of Agriculture
U.S. Department of Agriculture – FY 2024 Evaluation Plan

2

Table of Contents

Introduction ....................................................................................................................... 3

Stakeholder Engagement.................................................................................................... 3

Significant Evaluations ........................................................................................................ 4

Food and Nutrition Service (FNS)........................................................................................ 5

Child Nutrition School Meals Operations Study, School Year 2023-2024............................ 5

Child Nutrition Farm-to-School Census 2023 ...................................................................... 7

Food Distribution Programs on Indian Reservations........................................................... 9

Assessment of Mobile Technologies for Using Supplemental Nutrition Assistance Program 

(SNAP) Benefits................................................................................................................. 12

Assessment of Serving SNAP Applicants and Participants with Limited English Proficiency

.......................................................................................................................................... 13

Understanding the Relationship Between Poverty, Well-Being, and Food Security.......... 14

Rural Development (RD)................................................................................................... 16

Evaluation of the Broadband Programs ............................................................................ 16

Evaluation of the Community Facilities Program .............................................................. 18

Evaluation of the Single-Family Housing Program............................................................. 20

Evaluation of the RD Programs and Climate Impacts........................................................ 22

Acronyms and Abbreviations............................................................................................ 23
U.S. Department of Agriculture – FY 2024 Evaluation Plan

3

Introduction

The United States Department of Agriculture (USDA or the Department) serves all 

Americans by providing effective, innovative, science-based public policy leadership in 

agriculture, food and nutrition, natural resource protection and management, rural 

development, and related issues with a commitment to deliver equitable and climate-smart 

opportunities that inspire and help Americans thrive. USDA achieves its mission via 18 

individual Agencies organized under eight Mission Areas, Departmental Administration, and 

multiple Staff Offices. USDA’s vision is an equitable and climate-smart food and agriculture 

economy that protects and improves the health, nutrition, and quality of life of all 

Americans; yields healthy land, forests, and clean water; helps rural America thrive; and 

feeds the world.

The Office of Budget and Program Analysis (OBPA or the Office) oversees and facilitates the 

performance and strategic management activities of the Department, including 

development of the USDA Strategic Plan, Agency Priority Goals, deliverables required under 

the Foundations for Evidence-Based Policymaking Act of 2018 (Evidence Act), and Enterprise 

Risk Management. The Office is led by the USDA Chief Evaluation Officer. OBPA also chairs 

the USDA Performance, Evaluation, and Evidence Committee and the Enterprise Risk 

Management Committee, both of which are comprised of representation from Mission 

Areas and Departmental Administration, as well as key partners, including the Chief Data 

Officer and the Statistical Official. These individuals bring diverse perspectives and broad 

technical expertise to inform OBPA’s initiatives and facilitate buy-in among stakeholders 

across the Department. Partnership with the Chief Data Officer and Statistical Official 

provides valued insights and advisement on data access, data quality, and statistical 

methods.

Under the Evidence Act, USDA is required to produce an Evaluation Plan on an annual basis 

describing the evaluation activities planned for the coming fiscal year (FY). The following 

Evaluation Plan includes the significant evaluations planned for FY 2024, including the key 

questions to be answered, required data collections, technical approach and methodologies, 

and use and dissemination of evaluation findings.

Stakeholder Engagement

Throughout FY 2021 and 2022, OBPA conducted a number of stakeholder engagement 

activities to inform the development of the FY 2024 Evaluation Plan. USDA’s internal 

stakeholders are organized into the following standard hierarchy to ensure that all levels are 

appropriately engaged and to provide clarity around roles and responsibilities:

• Strategic Direction: Responsible for setting the Department’s strategic direction 

(Secretary, Deputy Secretary, and Sub-Cabinet officials);

• Departmental Operations: Responsible for supporting strategic implementation 

efforts in alignment with the Department’s strategic goals and objectives (staff 

offices such as OBPA, Human Resources, Office of the Chief Information Officer, 

etc.); and
U.S. Department of Agriculture – FY 2024 Evaluation Plan

4

• Mission Delivery and Performance: Responsible for aligning Agency strategic 

directions and resources with the Department-wide strategic directions and for 

delivering statutorily required missions, functions, programs, projects, etc. (all USDA 

Mission Areas and Agencies).

To inform Evidence Act deliverables, 

OBPA first stood up a short-term, crossfunctional team known as the Evidence 

Act Working Group. This working group 

merged with an established group of 

technical experts in the Performance 

Improvement Community, to form the 

Performance, Evidence, and Evaluation 

Committee (PEEC). The PEEC includes

representation from across USDA 

Mission Areas and Agencies and engages 

in dynamic dialogue of planning, 

performance, evidence, and data, 

including identifying “significant” planned evaluations. In addition to building a strong 

community of practice, the PEEC serves as a conduit between OBPA and Mission Area and 

Agency leadership. 

Significant Evaluations

For the FY 2024 Evaluation Plan, USDA chose to focus on evaluations being conducted by 

Agencies within the Food, Nutrition, and Customer Service, Food Safety, and the Rural 

Development Mission Areas. These Mission Areas are advanced in their maturity in 

evidence-building, particularly with respect to program evaluation. USDA used the following 

criteria to identify significant evaluations for inclusion in this plan:

• Supports answering a USDA Learning Agenda priority question;

• Required by statute or congressional mandate; and

• Supports a USDA leadership priority.
U.S. Department of Agriculture – FY 2024 Evaluation Plan

5 

Food and Nutrition Service (FNS)

Child Nutrition School Meals Operations Study, School Year 2023-

2024

Primary Question

How effective are FNS’s strategies in eliminating barriers to access to nutrition assistance 

programs?

Alignment with USDA Strategic Goal(s)

This evaluation aligns to Goal 4: Make Safe, Nutritious Food Available to All Americans in the 

USDA FY 2022 – 2026 Strategic Plan.

Background and Rationale

The National School Lunch Program (NSLP) is a federally assisted meal program operating in 

public and nonprofit private schools and residential childcare institutions. It provides 

nutritionally balanced, low-cost or free lunches to children each school day. The School 

Breakfast Program (SBP) provides reimbursement to states to operate nonprofit breakfast 

programs in schools and residential childcare institutions. The School Meal Operations Study 

is a continuation of a series of annual surveys of State agencies and/or school food 

authorities to obtain timely information about changes in program operations and 

characteristics at the state and local level. During the COVID-19 pandemic, this study has 

also been used to collect statutorily required reports on waiver usage in the Child Nutrition 

programs.

Timeline

Expected completion in FY 2024

Technical Approach and Methodologies

The study will collect administrative data from state agencies and maximize the use of these 

data to address research questions. School Food Authorities (SFAs) report these 

administrative data to their state agencies for the FNS-10, Report of School Program 

Operations, and the team expects that state agencies will be able to provide these data 

disaggregated either by SFA or by school. 

In addition, streamlined web surveys will be administered to state agencies and SFAs 

focused on annually identified priority research questions that cannot be answered using 

the FNS-10 administrative data. In addition to a standard series of questions about program 

operations, specific topics of interest from previous survey years include universal free 

meals, supply chain issues, Buy American, scratch cooking and other meal preparation 

issues, and nutrition education.

Data Sources

Data will include quantitative and qualitative information from surveys of state agencies and 

SFAs as well as extant quantitative administrative data. 
U.S. Department of Agriculture – FY 2024 Evaluation Plan

6 

Challenges and Mitigation Strategies There may be pandemic-related delays and challenges. 

USDA will explore contractor options to allow the study to continue. USDA will also work 

with stakeholders to ensure the study’s objectives can be met on a revised timeline and 

under waiver conditions.

Use and Dissemination

The final report will be posted publicly on the USDA website. FNS will use the data and 

surveys to inform guidance, technical assistance, and regulatory action as well as share the 

findings with state agencies to inform their internal operations. In addition, the 

administrative data will inform secondary analyses that examine more specific impacts of 

operational differences at the state and local levels.
U.S. Department of Agriculture – FY 2024 Evaluation Plan

7 

Child Nutrition Farm-to-School Census 2023

Primary Question

How effective are FNS’s programs in improving nutrition security through access to 

nutritious foods?

Alignment with USDA Strategic Goal(s)

This evaluation aligns to Goal 4: Make Safe, Nutritious Food Available to All Americans in the 

USDA FY 2022 – 2026 Strategic Plan.

Background and Rationale

The Farm to School Program helps child nutrition program operators incorporate local foods 

in the National School Lunch Program, the Summer Food Service Program and Child and 

Adult Care Food Program and all associated programs. In addition, Farm to School Program 

staff work with tribal communities by promoting food sovereignty and the integration of 

traditional food ways into tribal meal programs. The Farm to School Census and grant 

evaluation is conducted every five to seven years (last in 2019) to inform FNS of progress in 

incorporating local foods into Child Nutrition programs, as well as examine best practices.

Timeline

Expected completion in FY 2024

Technical Approach and Methodologies

This census and data analysis will provide information on the utilization of local food 

products through procurement and other farm to school activities during School Year (SY) SY 

2022-2023. The survey will also collect data on benefits of using local food products and 

barriers that prevent greater access to local foods, including challenges around processing 

and infrastructure. Data collection will take place in SY 2023-2024. 2023 will be the fourth 

Farm to School Census. Due to substantive changes made to the Census survey in 2019, it 

was not possible to compare data to prior efforts in 2013 and 2015. The 2023 survey should 

allow for comparisons to 2019 results and allow for analysis of changes in patterns of 

activity and spending among SFAs.

Data Sources

The primary data collection will be from all school food authorities enrolled in the National 

School Lunch Program. The farm to school census is a survey of all 18,000+ school food 

authorities participating in the NSLP. While FNS does have farm to school grants, the 

number of respondents for the Census is considerably larger than just the farm to school 

grantees. 

Challenges and Mitigation Strategies

There may be delays associated with acquisition and Paperwork Reduction Act (PRA) 

clearance. USDA developed and submitted the PRA Information Collection Request prior to 

securing a contractor to expedite the timeline. The Farm to School Census survey is 

disbursed to all School Food Authorities (SFAs) participating in the National School Lunch 

Program (18,000+). The 2019 Farm to School Census achieved a response rate of 67%. FNS 
U.S. Department of Agriculture – FY 2024 Evaluation Plan

8 

seeks to improve this response rate in 2023 through a robust and proactive outreach 

strategy for States and SFA personnel, building off successful strategies from prior efforts.

Use and Dissemination

The Farm to School Census (F2S Census) is a one-of-a-kind dataset used extensively by 

USDA, stakeholder partners, and the school nutrition research community. The F2S Census 

is the only national-level survey of all participating SFAs intended to capture the variety and 

depth of engagement with locally food sources and promotion of healthy local food 

products. FNS uses this information to identify both State and regional trends in local 

purchasing activity as well as for planning technical assistance development for SFAs. Other 

USDA agencies, including AMS, use this data to identify growing school purchasers for U.S. 

farmers seeking new markets for products. The final report will be posted publicly on the 

USDA website.
U.S. Department of Agriculture – FY 2024 Evaluation Plan

9 

Food Distribution Programs on Indian Reservations

Primary Question

To what extent is racial equity incorporated into internal FNS operations and the design and 

administration of Federal nutrition assistance programs?

Alignment with USDA Strategic Goal(s)

This evaluation aligns to Goal 4: Make Safe, Nutritious Food Available to All Americans in the 

USDA FY 2022 – 2026 Strategic Plan.

Background and Rationale

The Food Distribution Program on Indian Reservations (FDPIR) provides USDA Foods to 

income-eligible households living on Indian reservations and to Native American households 

residing in designated areas near reservations or in Oklahoma. USDA distributes both food 

and administrative funds to participating Indian Tribal Organizations (ITO) and state 

agencies to operate FDPIR. There is a steady decline in FDPIR participation. Understanding 

the challenges/ constraints, and identifying current strategies used to serve difficult-toreach elderly and home bound populations may help identify potential ways to improve 

program implementation.

Timeline

Expected completion in FY 2025

Technical Approach and Methodologies

This study will be a quick-response survey of all Indian Tribal Organizations and State 

Agencies administering FDPIR to determine strategies and flexibilities used to distribute 

food to underserved individuals who are eligible for FDPIR, specifically homebound and 

older individuals.

Data Sources

Data will include information from surveys with Indian Tribal Organizations and State 

agencies.

Challenges and Mitigation Strategies

There may be data collection challenges associated with PRA clearance. USDA will leverage a 

generic PRA clearance to address this challenge. FNS may encounter some challenges 

recruiting ITOs. For this reason, we plan to engage them early in the study process and meet 

with them to ensure they understand why we are doing the study and that we are able to 

address concerns they raise about the study.

Use and Dissemination

The findings from this study will help identify strategies to overcome the barriers and 

improve access. This information will be shared with the program and also the ITO's so the 

strategies can be applied as appropriate. The final report will be posted publicly on the 

USDA website.
U.S. Department of Agriculture – FY 2024 Evaluation Plan

10

Special Supplemental Nutrition Program for Women, Infants, and 

Children (WIC) Breastfeeding Policy Inventory

Primary Question

How effective are FNS’s programs in improving nutrition security through access to 

nutritious foods?

Alignment with USDA Strategic Goal(s)

This evaluation aligns to Goal 4: Make Safe, Nutritious Food Available to All Americans in the 

USDA FY 2022 – 2026 Strategic Plan.

Background and Rationale

The Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) 

provides federal grants to states for supplemental foods, health care referrals, and nutrition 

education for low-income pregnant, breastfeeding, and non-breastfeeding postpartum 

women, and to infants and children up to age 5 at nutritional risk.

The WIC program emphasizes the importance of breastfeeding as the best source of 

nutrition for young infants. The program provides breastfeeding consultation and has an 

extensive breastfeeding peer counseling program to help women who want to breastfeed 

succeed in their breastfeeding goals. 

This study helps the program understand the landscape of breastfeeding policies and 

practices at state and local WIC agencies across the country. Comparing this study’s findings 

with the last study will help the program see how the program approach has changed over 

time. This information can inform program and policy decisions regarding technical 

assistance and guidance to the states on breastfeeding practices and, when compared with 

findings from other studies, including the WIC Participant Characteristics study, can look for 

associations between breastfeeding rates and program breastfeeding policies which can 

ultimately shape guidance meant to help states increase breastfeeding rates over time.

Timeline

Expected completion in FY 2024

Technical Approach and Methodologies

This overall project will provide an update to the first WIC Breastfeeding Policy Inventory, 

published in 2015, yielding a comprehensive description of breastfeeding statistics, policies, 

procedures, and practices at the local and State WIC agency levels, including 

implementation of peer counseling programs, staff training on breastfeeding, utilization of 

the national breastfeeding campaign, and best practices to improve breastfeeding initiation 

and duration rates and reduce disparities. The study will also examine equity in the 

availability of breastfeeding supports and explore methods for routine collection of 

information on the number of WIC breastfeeding peer counselors. The study is largely 

descriptive and based on a national survey coupled with qualitative interviews of WIC staff. 

Findings help the WIC program examine the landscape of breastfeeding supports and how it 

varies across states and local WIC agencies. When compared with WIC Participant 
U.S. Department of Agriculture – FY 2024 Evaluation Plan

11

Characteristics data and breastfeeding rates, the program will be able to consider 

correlations between breastfeeding policies and rates, which could inform program 

guidance or future policies.

Data Sources

Data will include information from state and local WIC agency data collections.

Challenges and Mitigation Strategies

This is the second time FNS has collected this data. We had very high response rates and 

limited challenges in the first study; we anticipate a similar experience with this round of 

data collection. Potential challenges include participant response burden in that FNS has a 

number of large-scale studies in the field and is doing an extensive amount of work based 

on the ARPA funds distributed related to Modernization and Innovation. The most 

significant challenge is generally around delays associated with Paperwork Reduction Act 

clearance, which adds up to a year of clearance time, which causes significant delays in our 

ability to provide timely data to program partners. There may be contracts and acquisitionrelated delays. USDA will collaborate with stakeholders to communicate and adhere to 

study timelines.

Use and Dissemination

Study dissemination materials will provide a comprehensive description of breastfeeding 

statistics, policies, procedures, and practices at the local and state WIC agency levels, 

including implementation of peer counseling programs, staff training on breastfeeding, 

utilization of the national breastfeeding campaign, and equity as it relates to the availability 

of breastfeeding resources. Study results may be used to inform future decisions around 

WIC breastfeeding programs. The final report will be posted publicly on the USDA website. 
U.S. Department of Agriculture – FY 2024 Evaluation Plan

12

Assessment of Mobile Technologies for Using Supplemental 

Nutrition Assistance Program (SNAP) Benefits

Primary Question

How effective are FNS’ strategies in eliminating barriers to access to nutrition assistance 

programs?

Alignment with USDA Strategic Goal(s)

This evaluation aligns to Goal 4: Make Safe, Nutritious Food Available to All Americans in the 

USDA FY 2022 – 2026 Strategic Plan.

Background and Rationale

The Agricultural Act of 2018 authorizes the use of mobile technologies for the purpose of 

accessing SNAP benefits. This allows SNAP participants to input their Electronic Benefit 

Transfer (EBT) cards into a mobile technology, such as Apple Pay or Google Pay, and make 

SNAP purchases at the point-of-sale without the presence of the EBT card. The Act requires 

that the FNS approve no more than five projects to pilot the use of this technology and 

subsequently determine if mobile technology should be authorized nationwide. This study 

will assess the pilots in the areas of participant access, ease of use, and program integrity to 

facilitate the decision-making around the broad authorization of the use of mobile 

technologies.

Timeline

Expected completion in FY 2026

Technical Approach and Methodologies

This study will assess the pilot projects in the areas of participant access, ease of use, and 

program integrity to facilitate the determination of whether to broadly authorize the use of 

mobile technologies. Process and outcome evaluations will be conducted in up to five States 

that participate in the pilot program.

Data Sources

Data collection will include interviews with stakeholders and document review from each 

pilot project. Analysts will also utilize SNAP administrative data from State agencies and 

retailer characteristics data.

Challenges and Mitigation Strategies

Obtaining the needed data from EBT processors and SNAP State agencies poses a challenge 

as well as delays in implementation of mobile technologies. FNS will work closely with the 

pilot operations to assist with integration of mobile technology.

Use and Dissemination 

FNS will use the findings of the evaluation to determine the successes of the Mobile 

Payment Pilot and inform future regulations and policy related to the use of mobile 

technology for EBT payments The final report will be posted publicly on the USDA website.
U.S. Department of Agriculture – FY 2024 Evaluation Plan

13

Assessment of Serving SNAP Applicants and Participants with 

Limited English Proficiency

Primary Question

To what extent is racial equity incorporated into internal FNS operations and the design and 

administration of Federal nutrition assistance programs?

Alignment with USDA Strategic Goal(s)

This evaluation aligns to Goal 4: Make Safe, Nutritious Food Available to All Americans in the 

USDA FY 2022 – 2026 Strategic Plan.

Background and Rationale

SNAP provides nutrition benefits to supplement the food budget of families in need so they 

can purchase healthy food and move towards self-sufficiency. This study seeks to assess 

how States, Territories, and local areas implement language access requirements through 

policies, guidance, and operations. This study will also gather information on States’ and 

Territories’ perceptions of resources available to properly implement these requirements. 

The study will assist FNS in supporting SNAP and NAP agencies efforts to serve individuals in 

need of language access services.

Timeline

Expected completion in FY 2025

Technical Approach and Methodologies

This study will survey all 53 SNAP state agencies to catalog which languages state agencies 

offer for which types of documents and whether they offer limited English proficiency (LEP) 

services through their SNAP Employment and Training (E&T) program. The study will ask 

states how they determine which languages should be offered and how they manage the 

translation process, and if they offer any SNAP E&T LEP services and what services they offer 

if they do.

Data Sources

Data collection will include SNAP administrative data from state agencies in addition to the 

new survey data.

Challenges and Mitigation Strategies

There may be data collection challenges associated with PRA clearance as well as contracts 

and acquisition-related delays. USDA will explore ways to build the competency and 

capacity of acquisition professionals to address the complexity of research contracting 

needs. 

Use and Dissemination

FNS also intends to create a data visualization focused on language access policies and 

operations, as well as the language landscape. This visualization will support efforts to 

improve understanding of variation across States and Territories and allow FNS to target 

technical assistance accordingly. The final report will be posted publicly on the USDA 

website. 
U.S. Department of Agriculture – FY 2024 Evaluation Plan

14

Understanding the Relationship Between Poverty, Well-Being, and 

Food Security

Primary Question

How effective are FNS’ programs in improving nutrition security through access to nutritious 

foods? 

Alignment with USDA Strategic Goal(s)

This evaluation aligns to Goal 4: Make Safe, Nutritious Food Available to All Americans in the 

USDA FY 2022 – 2026 Strategic Plan.

Background and Rationale

Research has shown that the economic and demographic circumstances of households are 

closely correlated with food security status. However, not all households with similar 

circumstances experience the same food security status. This evaluation will identify 

measures of poverty and well-being associated with household food security status among 

SNAP-eligible participants and non-participants in persistent-poverty counties, defined as 

counties where 20% or more of their population lived in poverty in the last four decennial 

Censuses. Moving beyond household income, many dimensions of well-being and material 

deprivation may affect food security status and SNAP participation, such as mental health, 

depression, health-related quality of life, disablement, medical expenditures, alcohol or 

opioid addiction, place of residence, and within-household sharing of resources. Collecting 

representative, qualitative and quantitative data at the county level will enable improved 

estimates of county-level food security status, as official data at this granular level are 

unavailable. These data are unavailable because the USDA has a partnership with Census to

develop one nationwide food security rate once per year, as such, there is no official federal 

rate or data collection at anything lower than that national rate.

Timeline

Expected completion in FY 2026

Technical Approach and Methodologies

Analysts will conduct county-level representative surveys of household food security, wellbeing, and material hardship measures in at least six persistent-poverty counties to help 

identify factors other than income that impact food security status. The analysts will also 

conduct in-depth interviews with a subsample of households in these counties to provide 

additional context for the survey findings. Counties will represent a variety of spatial types 

(urban, suburban, and rural) and other policy-relevant characteristics. 

Data Sources

Data sources will consist of county-level representative surveys of household food security, 

well-being, and material hardship measures in at least six persistent-poverty counties, and 

in-depth interviews with a subsample of households in six counties.

Challenges and Mitigation Strategies

Delays associated with acquisition and the PRA clearance will pose a challenge. USDA will 

explore contracting options and work with stakeholders to ensure study objectives can be 
U.S. Department of Agriculture – FY 2024 Evaluation Plan

15

met on revised timelines and under waiver conditions. Sampling of SNAP non-participants is 

difficult given there is no predetermined list of non-participants. This contrasts with SNAP 

participants which will be sampled from State administrative data. To mitigate this, the 

study will use address-based sampling that will cover all primary sampling units in each 

county, providing a representative sample of households in each county. Furthermore, 

collecting survey data among most population groups is difficult in an era of declining 

response rates. To mitigate this, the study will collect both survey data via the internet and 

telephone and follow-up with a subsample of non-respondents via infield locating (e.g., 

knocking on doors).

Use and Dissemination

Examining food insecurity and poverty in these persistent-poverty counties will help FNS 

better understand the association between SNAP, other USDA-administered programs, and 

community-based assistance with well-being and the food environment. This information is 

crucial for designing interventions that address long-standing disparities in food insecurity 

and poverty not only in these counties, but the hundreds of other persistent-poverty 

counties across the nation. The final report will be posted publicly on the USDA website.
U.S. Department of Agriculture – FY 2024 Evaluation Plan

16

Rural Development (RD)

Evaluation of the Broadband Programs

Primary Question

What are the impacts of RD broadband programs on broadband availability and use, as well 

as on economic (property values, household income, and employment) and social 

(population growth, healthcare access and availability, and telemedicine) outcomes?

Alignment with USDA Strategic Goal(s)

This evaluation aligns to Goal 5: Expand Opportunities for Economic Development and 

Improve Quality of Life in Rural and Tribal Communities in the USDA FY 2022-2026 Strategic 

Plan.

Background and Rationale

This evaluation covers the following programs: the Rural Broadband Access Loan and Loan 

Guarantee Program, Community Connect Grants, Distance Learning and Telemedicine 

Grants and Loans, Broadband Initiatives Program, and ReConnect Program. These programs 

are designed to promote access to broadband in rural areas. E-connectivity is fundamental 

to economic development, innovation, technological advancement, workforce readiness, 

and the improvement of quality of life in rural and Tribal communities.

Timeline

Expected completion in FY 2026

Technical Approach and Methodologies

The in-depth evaluation of these programs will utilize rigorous quasi-experimental methods. 

These methods include matching, matching with regression, difference-in-difference 

analysis, synthetic matching, instrumental variable analysis, and regression discontinuity. 

The in-depth evaluation will also use the input-output analysis toolkit to estimate the shortrun impacts of building out broadband infrastructure in rural areas. In using this framework, 

the project analysts will use detailed data as inputs to simulation models to obtain estimates 

on employment and output for a specific regional economy. The IMPLAN software and 

database will serve as the modeling platform for this impact analysis.

These evaluation projects will be conducted by six trained economists and social scientists 

at the Innovation Center who have experience in both ex-ante and ex-post evaluation 

approaches. Some of the projects will be conducted in collaboration with the USDA 

Economic Research Service (ERS). The Innovation Center is working collaboratively with 

program areas to obtain administrative data.

Data Sources

Administrative data available from the USDA Rural Utilities Service (RUS) will be used in 

combination with other external data available from the Bureau of the Census, Bureau of 

Economic Analysis, Bureau of Labor Statistics, ERS, and Federal Communications 

Commission. Proprietary datasets such as the National Establishment Time Series and 

Zillow's Assessor and Real Estate Database (ZTRAX) will also be used. The various datasets 
U.S. Department of Agriculture – FY 2024 Evaluation Plan

17

will be combined using program service area information, such as service area boundaries 

available from program shapefiles, census blocks, census tracts, and Federal Information 

Processing Standard (FIPS) county and sub-county codes. 

For the input-output analysis of the ReConnect Program, data from published engineering 

cost and economic impact studies will be used in addition to detailed program data. Project 

cost data housed in the RUS represent a new type of data that are not normally stored in 

the National Rural Development database but are essential for developing model simulation 

scenarios. 

Challenges and Mitigation Strategies

RD has several programs that fund broadband access and the data available for analysis 

under these programs varies. For new programs, e.g., ReConnect, service area shapefiles are 

available for more precise analysis and attribution of impacts. In addition, linking program 

data to external data sources and the lack of publicly available data to measure certain 

outcomes can be a challenge that limits analysis and evaluation.

Use and Dissemination

The findings will be shared throughout the RD Mission Area, the wider USDA community, 

and the Office and Management and Budget through the dissemination of reports and 

presentations. RD is currently working on the creation of a public-facing website to 

disseminate findings from evaluations and other analyses. Additionally, projects are 

published in peer reviewed journals and presented across various conferences.
U.S. Department of Agriculture – FY 2024 Evaluation Plan

18

Evaluation of the Community Facilities Program

Primary Questions

1. What is the impact of Community Facilities funding on rural health (e.g., rural 

hospital closings and longer impacts of COVID outcomes)?

2. What are the impacts of Community Facilities funding on education (e.g., school 

quality)? 

Alignment with USDA Strategic Goal(s)

This evaluation aligns to Goal 5: Expand Opportunities for Economic Development and 

Improve Quality of Life in Rural and Tribal Communities in the USDA FY 2022-2026 Strategic 

Plan.

Background and Rationale

Essential community infrastructure is key in ensuring that rural areas enjoy the same basic 

quality of life and services enjoyed by those in urban areas. The Community Facilities (CF) 

Programs offer direct loans, loan guarantees, and grants to develop or improve essential 

public services and facilities in communities across rural America. These amenities help 

increase the competitiveness of rural communities in attracting and retaining businesses 

that provide employment and services for their residents.

Public bodies, non-profit organizations, and Federally recognized Tribes can use the funds to 

construct, expand, or improve facilities that provide health care, education, public safety, 

and public services. Projects include fire and rescue stations, village and town halls, health 

care clinics, hospitals, adult and childcare centers, assisted living facilities, rehabilitation 

centers, public buildings, schools, libraries, and many other community-based initiatives. 

Financing may also cover the costs of land acquisition, professional fees, and purchase of 

equipment. These facilities not only improve quality of life, but also assist in the 

development and sustainability of rural America.

The purpose of this evaluation is to focus on the two largest areas of investment in the CF 

investment portfolio: health care and education. The evaluation will focus on the impact of 

CF on health and education outcomes in recipient facilities and communities. 

Timeline

Expected completion in FY 2026

Technical Approach and Methodologies

The evaluation will utilize rigorous quasi-experimental methods. These methods include 

difference-in-difference analysis, instrumental variable analysis, and the regression 

discontinuity method. 

These evaluation projects will be conducted by six trained economists and social scientists 

at the Rural Development Innovation Center who have experience in both ex-ante and expost evaluation approaches. The Innovation Center is working collaboratively with program 

areas to obtain administrative data. 

Data Sources
U.S. Department of Agriculture – FY 2024 Evaluation Plan

19

Administrative data available from the Rural Housing Service CF Programs will be used in 

combination with other external data available from the Bureau of the Census, Bureau of 

Economic Analysis, Bureau of Labor Statistics, and National Center for Education Statistics. 

In addition, proprietary datasets on business registers, such as National Establishment Time 

Series, will be used. The various datasets will be combined using program service area 

information, such as recipient facility Data Universal Numbering System numbers, names 

and addresses, census blocks, census tracts, and FIPS county and sub-county codes.

Challenges and Mitigation Strategies

At the macro level, the lack of shapefiles for program data to delineate service areas forces 

the analysis to be at the county level or city and town level. Linking program data to 

external data sources and the lack of publicly available data to measure certain outcomes 

can also be challenges.

Use and Dissemination

The findings will be shared throughout the RD Mission Area and the wider USDA community 

through the dissemination of reports and presentations. The findings will be delivered in a 

manner that is most appropriate for the audience.

In addition to sharing the findings internally, the evaluation studies will be used to generate 

peer-reviewed and outreach publications and conference presentations.
U.S. Department of Agriculture – FY 2024 Evaluation Plan

20

Evaluation of the Single-Family Housing Program

Primary Questions

1. What is the impact of Single-Family Housing Section 502 Direct Loans on measures of 

housing affordability and stability in recipient tracts or counties?

2. What is the impact of Single-Family Housing Section 502 Direct Loans on recipient 

home values?

Alignment with USDA Strategic Goal(s) 

This evaluation aligns to Goal 5: Expand Opportunities for Economic Development and 

Improve Quality of Life in Rural and Tribal Communities in the USDA FY 2022 -2026 Strategic 

Plan.

Background and Rationale

The Single-Family Housing Program gives families and individuals the opportunity to buy,

build, repair, or own safe and affordable homes in rural America. Low interest, fixed-rate 

homeownership loans are provided to qualified persons directly by Rural Development. 

Financing is also offered at fixed rates and terms through a loan from a private financial 

institution and guaranteed by Rural Development for qualified persons. Neither of these 

home loan programs require a down payment.

The Section 502 Direct Loan Program assists low- and very-low-income applicants in 

obtaining decent, safe, and sanitary housing in eligible rural areas by providing payment 

assistance to increase an applicant’s repayment ability. Payment assistance is a type of 

subsidy that reduces the mortgage payment for a short time. The amount of assistance is 

determined by the adjusted family income. Loan funds may be used to help low-income 

individuals or households purchase homes in rural areas. Funds can be used to build, repair, 

renovate, or relocate a home, or to purchase and prepare sites, including providing water 

and sewage facilities.

Timeline

Expected completion in FY 2026

Technical Approach and Methodologies

The evaluations will utilize rigorous quasi-experimental methods; namely, difference-indifference analysis, instrumental variable analysis, and the regression discontinuity method. 

These evaluation projects will be conducted by six trained economists and social scientists 

at the Rural Development Innovation Center who have background in both ex-ante and expost evaluation approaches. The Innovation Center is working collaboratively with program 

areas to obtain administrative data.

Data Sources

Administrative data on the Single-Family Housing program will be used in combination with 

other external data available from proprietary datasets on property values such as ZTRAX. 

Publicly available data from the Bureau of the Census, Bureau of Economic Analysis, Bureau 

of Labor Statistics, and ERS will also be used. The various datasets will be combined using 
U.S. Department of Agriculture – FY 2024 Evaluation Plan

21

program information such as property addresses, location coordinates of properties, census 

tracts, and FIPS county and sub-county codes. 

Challenges and Mitigation Strategies

Linking program data to external data sources and the lack of publicly available data to 

measure certain outcomes can also be challenges.

Use and Dissemination

The findings will be shared throughout Rural Development Mission Area and wider USDA 

community through the dissemination of reports and presentations. The findings will be 

delivered in a manner that is most appropriate for the audience.

In addition to sharing the findings internally the evaluation studies will be used to generate 

peer-reviewed and outreach publications and conference presentations.
U.S. Department of Agriculture – FY 2024 Evaluation Plan

22

Evaluation of the RD Programs and Climate Impacts

Primary Questions

1. How do RD programs reduce the risks of climate impacts for rural communities? 

2. How do RD programs increase the resilience of rural communities to climate 

impacts?

Alignment with USDA Strategic Goal(s)

This evaluation aligns to Goal 1: Combat Climate Change to Support America’s Working 

Lands, Natural Resources and Communities in the USDA FY 2022 – 2026 Strategic Plan.

Background and Rationale

Rural and Tribal communities are disproportionately impacted by the effects of climate 

change. RD is committed to increasing rural community resilience to climate change, 

securing environmental justice, and spurring economic opportunity for disadvantaged 

communities that have been historically marginalized, overburdened by pollution, and have 

experienced underinvestment in essential services. 

Timeline

Expected completion in FY 2026

Technical Approach and Methodologies

These evaluations will focus on the collective benefits of RD programs and will assess 

mitigation and resilience of rural communities against climate change. These evaluations 

will utilize multiple regression combined with instrumental variable analysis.

Data Sources

Administrative data on major RD programs will be used in combination with other external 

data available from the Environmental Protection Agency, Climate and Economic Justice 

Screening Tool, FEMA National Risk Index for Natural Hazards, Bureau of the Census, Bureau 

of Economic Analysis, Bureau of Labor Statistics, and ERS. The various datasets will be 

combined by community geography.

Challenges and Mitigation Strategies

The biggest challenge will be to identify all outcomes variables and associated covariates at 

the community geography level without disclosure issues. Ensuring climate impact metrics 

accurately represent the environmental hazards experienced by rural communities and the 

resulting barriers to equity and environmental justice.

Use and Dissemination

The findings will be shared throughout the RD Mission Area and wider USDA community 

through the dissemination of reports and presentations. The findings will be delivered in a 

manner that is most appropriate for the audience.

In addition to sharing the findings internally, the evaluation studies will be used to generate 

peer-reviewed and outreach publications and conference presentations.
U.S. Department of Agriculture – FY 2024 Evaluation Plan

23

Acronyms and Abbreviations

CF Community Facilities

EBT Electronic Benefit Transfer

ERS Economic Research Service

Evidence Act Foundations for Evidence-Based Policymaking Act of 2018

FDPIR Food Distribution Program on Indian Reservations

FIPS Federal Information Processing Standard

FNS Food and Nutrition Service

FSIS Food Safety and Inspection Service

FY Fiscal Year

OBPA

PEEC

 Office of Budget and Program Analysis

 Performance, Evidence, and Evaluation Committee

PRA Paperwork Reduction Act

RD Rural Development

RUS Rural Utilities Service

SFA School Food Authority 

SNAP Supplemental Nutrition Assistance Program

USDA U.S. Department of Agriculture

ZTRAX Zillow’s Assessor and Real Estate Database