U.S. Department of Labor

EVALUATION PLAN 

2O23 

2O24 

FISCAL YEARS 

U.S. Department of Labor 
2 EVALUATION PLA TION PLAN

Summary 

The U.S. Department of Labor (DOL) is 

committed to making evidence-informed 

decisions to support America’s workers.

To bolster these efforts, on an annual 

basis DOL’s Chief Evaluation Office (CEO)

develops an evaluation plan to target 

evaluation resources on areas of strategic 

importance. The DOL Evaluation Plan

for Fiscal Years 2023–2024 describes 

significant evaluation activities that CEO

will undertake before the end of fiscal

year 2024. It does not include all of DOL’s

evaluation efforts, and DOL may identify 

additional significant evaluations during this

time period. 

The evaluation and research activities 

outlined in this document apply the 

most rigorous methods possible and will

generate evidence to inform policymaking 

decisions across DOL and among other 

federal decision-makers. Planned research

reflects the Department’s Strategic Plan

priorities, statutory requirements for 

evaluations, and priorities identified by DOL

sub-agencies in their learning agendas 

that are best addressed through program 

evaluation. This plan also fulfills legislative

mandates outlined in the Consolidated

Appropriations Act of 2022 and in Title I 

of the Foundations for Evidence-Based

Policymaking Act of 2018 (Evidence Act).

SUMMARY 
3 EVALUATION PLA TION PLAN

I.Introduction Introduction

As outlined in DOL’s Evaluation Policy, the importance of DOL’s mission demands 

continual innovation and improvement. Evaluating the effectiveness of our programs and 

activities is essential to this goal. Evaluations build high-quality evidence to help make 

DOL’s efforts as effective as possible.

On behalf of the Department, CEO sponsors evaluation research related to the programs, 

policies, and enforcement activities that DOL sub-agencies oversee across the United 

States and internationally. CEO directly funds and oversees research activities and 

collaborates with DOL sub-agencies to design and conduct evaluations that they sponsor. 

For all evaluations, CEO leverages protocols and practices to adhere to DOL Evaluation 

Policy principles of rigor, relevance, transparency, independence, and ethics.

The DOL Evaluation Plan includes studies related to two legislative requirements:

• As permitted by the Consolidated Appropriations Act of 2022, the plan includes 

studies that are directly sponsored by CEO using fiscal year (FY) 2022 set-aside

funding that is available for evaluations of programs administered by specific DOL

subagencies. These studies will be initiated by CEO prior to September 30, 2023, and

DOL will notify Congress of their intent and the purpose of transferring set-aside funds

to CEO at least 15 days prior the transfer; and

• As required by Title I of the Foundations for Evidence-Based Policymaking Act of

2018 (Evidence Act) and Office of Management and Budget (OMB) Memorandums

M-19-23 and M-21-27, the plan includes significant evaluations that the Department

plans to initiate through FY 2024. Since the DOL Evaluation Plan is issued annually 

and lists projects to be launched in the next two fiscal years, it should be noted that

some projects included in this plan for FY 2023–2024 were also listed in the previous

year’s plan covering FY 2022–2023.

The DOL Evaluation Plan communicates planned significant studies that may be funded 

through many sources, such as setting aside a portion of DOL funds for evaluations. The 

studies in this plan are deemed ‘significant’ based on their alignment with the priorities of 

the agency and administration, including those outlined in DOL’s FY 2022–2026 Strategic 

Plan, and not on the basis of their dollar value or timeline to completion. Not all evaluation 

efforts conducted at DOL are included in the DOL Evaluation Plan, and some studies 

proposed may be dependent on future appropriations. 

1

Division H, Title I, Section 107 of Public Law 117-103, (pg. 437) of the Consolidated Appropriations Act of 2022, authorizes the Secretary of

Labor to reserve not more than 0.75 percent from specific budget accounts for transfer to and use by the Office of the Chief Evaluation Officer for

departmental program evaluation. It also requires Congressional notification of the intent to transfer funds and the planned projects.

I. INTRODUCTION 
4 EVALUATION PLA TION PLAN

A Collaborative Development Process

All studies included in the DOL Evaluation Plan are developed collaboratively with subagency leaders and program experts, departmental leadership, and DOL’s Evidence

Officials, including the Chief Evaluation Officer, the Chief Data Officer, the Statistical

Official, and the Director of the Performance Management Center.

As the studies outlined in this document move forward, CEO will continue to collaborate 

with individuals and groups within and outside of DOL that have an interest in or may 

be impacted by CEO’s studies, such as DOL program offices, DOL grantees, program 

participants, members of affected communities, researchers and academics with expertise 

in the topic of study, industry groups, or other groups as appropriate to the study. CEO 

will continue to follow its Evaluation Policy to ensure that evaluations adhere to the most 

rigorous methods possible and produce independent and credible findings that are relevant 

for decision-makers. 

DOL sub-agencies with projects included in this plan include: 

• Bureau of International Labor Affairs (ILAB)

• Employee Benefits Security Administration (EBSA)

• Employment and Training Administration (ETA)

• Mine Safety and Health Administration (MSHA)

• Occupational Safety and Health Administration (OSHA) 

• Office of Disability Employment Policy (ODEP)

• Office of Federal Contract Compliance Programs (OFCCP)

• Office of the Assistant Secretary for Administration and Management (OASAM)

• Office of the Chief Financial Officer (OCFO)

• Office of Workers’ Compensation Programs (OWCP)

• Veterans’ Employment and Training Service (VETS)

• Wage and Hour Division (WHD)

• Women’s Bureau (WB)

I. INTRODUCTION 
Dissemination

CEO works to continuously improve its communication products and platforms to inform 

diverse audiences interested in labor-related research, including policymakers, program 

practitioners and decision makers, researchers, and the public. CEO’s website and 

quarterly newsletter provide various stakeholders with access to timely, relevant information 

to explore labor topics and the growing evidence base. CEO is also working to translate 

and promote findings using research summaries to make research accessible for technical 

and non-technical users. In the spirit of transparency, DOL posts completed evaluations 

and research online. To view more details about ongoing and completed studies, please 

visit the CEO website, or to receive timely updates, please subscribe to our quarterly 

newsletter by emailing us at chiefevaluationoffice@dol.gov.

Challenges

In addition to specific evaluation information below, the Office of Management and

Budget’s (OMB) guidance requires information about the anticipated challenges to

implementing these studies. Challenges for DOL evaluation research vary widely by

design and methods, but often include issues with accessing and linking federal and

state administrative datasets, recruitment of sufficient sample sizes to allow for rigorous

research designs, identification of suitable counterfactual conditions and non-program

participants, and attrition of participants or beneficiaries from intended DOL program

investments. CEO works closely with independent federal contractors, DOL agency staff,

state and local partners, and other federal agencies, including the U.S. Department of 

Health and Human Services (HHS) and U.S. Department of Education (ED), to forecast

and overcome challenges to individual studies. 

5 EVALUATION PLA TION PLAN I. INTRODUCTION 
EVALUATION PLA TION PLAN II. RESEARCH AND EVALUATION PROJECTS

 

 

6

II. Research and Evaluation Projects

This section describes the research and evaluation projects CEO intends to initiate during

FY 2023 and FY 2024. 

Table 1. Research and Evaluation Projects to be Initiated in FY 2023 and FY 2024

Note: Projects are listed alphabetically by DOL agency, then project title.

* Denotes a project that will be funded, at least in part, with FY 2022 dollars transferred to CEO through the set 

aside process and initiated through FY 2023. 

Project 1* Additional Information

Title: Identifying Impediments to Effective Access and

Enforcement of Employee Retirement Income Security

Act (ERISA) Rights of Underserved Communities

[EBSA]. EBSA would like to identify obstacles to

underserved groups accessing employee benefits, 

including: jobs that do not qualify for benefits due to

hourly thresholds; individuals not vesting into benefit

plans because of insufficient tenure and awareness of

vesting rules; individuals not accessing vested terminated

benefits because employers were unable to keep accurate

records once an employee separated; participants

and beneficiaries failing to access benefits due to 

disparate treatment for conditions impacting underserved 

communities, particularly those with substance use

and mental health conditions; and employees failing to

understand their rights, such as electronic disclosures, 

due to language barriers and lack of access. 

Priority Research Question(s):

What issues are preventing underserved populations of 

workers from participating and receiving ERISA benefits 

and protections? To what extent does lack of awareness 

of their rights prevent vulnerable populations from gaining 

ERISA benefits and protections? What mechanisms could 

best address these issues? How can EBSA leverage its 

existing programs to address these issues? 

Est. Start & Duration: FY23 (2 years) 

Data Sources: Household data and program data identifying industries and/ 

or geographic areas with high populations of underserved communities; EBSA 

program and enforcement data; Other TBD data. 

Methods/Approach: Analysis of participation and vesting requirements of benefit 

plans to identify requirements that may disproportionately affect underserved 

communities; identification of conditions with multiple treatment plans and look 

at variation by demographic groups for treatment and outcomes; analysis of 

plans that have been identified as not keeping accurate, updated records on the 

location of separated workers to see if their insufficient record-keeping methods 

disparately impact underserved communities. 

Anticipated Challenges & Solutions

Currently, program data on benefits by demographic group is unavailable and

therefore relies on survey data or proxies, which can make identifying and addressing

barriers for these groups difficult. 

6
EVALUATION PLA TION PLAN II. RESEARCH AND EVALUATION PROJECTS

 

 

7

Project 2* Additional Information

Title: Advancing Equity in Apprenticeship Programs

[ETA]. ETA seeks to build evidence on identifying effective 

pre-apprenticeship and apprenticeship strategies that help 

increase the participation of underrepresented populations 

and promote diversity, equity, inclusion, and accessibility 

(DEIA).

Priority Research Question(s):

What strategies have been effective in increasing the 

enrollment, retention, and completion of underrepresented 

populations, in registered apprenticeship and preapprenticeship programs? What pre-apprenticeship 

strategies have been effective in increasing articulation into 

a registered apprenticeship, as part of a career pathway? 

What are promising practices that merit bringing to scale?

Est. Start & Duration: FY23 (3 years) 

Data Sources: ETA administrative data; site visit interviews; possibly wage data, 

unemployment data, and surveys 

Methods/Approach: Synthesis; meta-analysis; descriptive study, impact 

evaluability assessment 

Anticipated Challenges & Solutions

None identified.

Project 3* Additional Information

Title: Sectoral Strategies and Career Pathways

Portfolio [ETA]. ETA is interested in developing studies

that will increase understanding of how the public workforce

system can conduct effective employer engagement and 

build evidence to support the features of robust sector 

strategies. 

Priority Research Question(s):

What are effective employer engagement strategies in 

the public workforce system? What is the prevalence 

and the contribution of employer-provided training to 

create positive outcomes for public workforce systems 

participants, especially for entry-level, lower-wage 

workers? How are employer engagement strategies 

incentivized, organized and implemented through the 

workforce system and sector partnerships, which include 

employers, labor organizations, educational institutions,

and other community partners? To what extent are 

effective sector partnerships implemented through 

the workforce system? How can these strategies be 

replicated and brought to scale? What are the impacts 

of sector-based training on participant employment and 

earnings impacts? In particular, how does participation 

in sector-based training affect labor market outcomes for 

underserved groups? 

Est. Start & Duration: FY23 (5 years)

Data Sources: Meta-analysis, possible surveys

Methods/Approach: Scan, synthesis, adding research questions and 

standardized data collection across existing or new studies, impact evaluability 

assessment, impact study 

Anticipated Challenges & Solutions

It will take time to identify common strategies being implemented across 

locations. It will also be important to determine if oversubscription or course 

waiting lists may exist and, if not, to explore alternative approaches to determining 

impacts, such as including sites within the same community college system as 

control groups. One possible approach to examining impacts would be to stagger 

the implementation so that some sites would wait 1–2 years to implement, which 

would require that we give them some incentive to do so. The study will need to 

account for similar employer partnerships not included in the grant funding. 

7 
 

 

 

 

Project 4*

Title: Unemployment Insurance (UI) Labor Research

[ETA]. Through a competitive research grant to a public 

academic institution(s), DOL will launch UI-related research

project(s). This project’s mission is to build rigorous UI

research with policy relevance. The project will engage in

data-sharing partnerships with at least one state UI agency,

but will also explore the potential for additional state

partnerships. 

Est. Start & Duration: FY23 (4 years) 

Data Sources: TBD depending on research awards, administrative data

Methods/Approach: Academic grants 

Anticipated Challenges & Solutions

Priority Research Question(s):

What are the indicators of access and use of UI benefits, 

especially for underserved populations? How can 

the long-term impact of UI benefits on workers, local 

communities, and the economy be measured? What 

measures could be used to assess the impact on 

workers and communities of policy differences in UI 

systems across states and/or over time, particularly for 

underserved populations? 

Establishing data-sharing agreements between the researchers and state(s) may 

be complex; DOL will facilitate discussions to support the partnerships and data 

sharing. 

Project 5 Additional Information

Title: Examining Case Management and Youth

Services [ETA]. ETA seeks to understand the landscape

of professional development opportunities for youth-serving 

staff and examine the key elements of effective case 

management/career coaching for youth, which can form the

basis of professional development opportunities. 

Est. Start & Duration: FY24 (4 years) 

Data Sources: Interviews, site visits, literature scan, administrative data

Methods/Approach: Data analytics; implementation study; impact evaluability 

assessment. Potential impact study

Priority Research Question(s):

What are the key competencies, skills, and most effective 

practices required to conduct strong case management/ 

career coaching for youth service delivery? How can 

ETA best support the professional development of youth 

service providers to help youth meet their educational/ 

employment goals? 

Anticipated Challenges & Solutions

It may be difficult to determine the effectiveness of specific service strategies 

or program components. We would first need to identify a model or strategy to 

study, which might be difficult since sites are implementing different models. 

It may be hard to conduct a rigorous impact study since it would require some 

sites to withhold their implementation. We may want to consider studying the 

offering of professional development or rigorous curriculum to case managers/ 

career coaches to determine if we can identify a strong professional development 

program. 

8 EVALUATION PLA TION PLAN II. RESEARCH AND EVALUATION PROJECTS

Additional Information
Title: Partners for Reentry Opportunities in Workforce

Development (PROWD) Grants Evaluation [ETA]. ETA

is collaborating with the Federal Bureau of Prisons (BOP)

to deploy an evidence-informed workforce development

initiative to advance First Step Act (FSA) implementation 

and reduce recidivism risk in the reentering federal 

prisoner population. This initiative will deliver workforce

development services to prisoners, continuing with an

intensive transition model and ending with a transition to

local service providers as prisoners are released. 

Priority Research Question(s):

To what extent are prisoner outcomes (recidivism, 

employment, quality jobs, earnings/wages, etc.) 

impacted by the type, intensity, and duration of workforce 

development service provision? 

Est. Start & Duration: FY22 (5 years)

Data Sources: Interviews, site visits, administrative data

Methods/Approach: Implementation study; impact study

There are challenges to conducting an impact study in this setting and with this 

population. One challenge is making sure grantees are implementing a similar 

model across the various sites. One potential solution is to develop a menu of 

strategies that grantees would be required to implement in subsequent rounds of 

this grant program. Another challenge is getting Department of Justice (DOJ)/ 

Federal Bereau of Investigations (FBI) criminal justice data, which is a long and 

tedious process. DOL is working to build relationships with the DOJ/BOP and has 

contractor expertise in getting this data. 

Title: Job Corps Study [ETA]. ETA is interested in

understanding the implementation of Job Corps services

as well as outcomes for Job Corps participants. The study

will explore partnerships, models, and components of Job

Corps services, such as career pathways with businesses,

local workforce boards, and community colleges; and

associated student outcomes such as employment, 

credential attainment, earnings, and other outcomes of 

interest. This study will result in an evaluation design for

estimating impacts of the Job Corps program. 

Priority Research Question(s):

What are the target populations for which the Job Corps 

program—a residential training program with a high 

dosage of services—is the most effective strategy to 

improve educational and employment outcomes? What 

recruitment strategies could be tested to increase uptake 

of the program for these groups? What partnerships show 

promise in promoting and advancing equity in enrollment, 

training, and employment? What effective strategies and 

support methods with employers and local partners exist 

to increase job placement and the employment retention 

rate of Job Corps participants? What are participants’ 

employment, earnings, and education outcomes? How do 

these results compare to the cost-benefit analysis or cost 

per participant of other high dosage training programs with 

a similar mix of participants, and what may account for any 

differences? 

Est. Start & Duration: FY 2024 (4 years) 

Data Sources: Interviews, site visits, administrative data, and survey data

Methods/Approach: Implementation study, data analytics, impact study 

feasibility analysis and impact evaluation design planning 

None identified.

9 EVALUATION PLA TION PLAN II. RESEARCH AND EVALUATION PROJECTS

 

 

Project 6

Anticipated Challenges & Solutions

Project 7 Additional Information

Anticipated Challenges & Solutions

Additional Information
 

Project 8*

Title: Reemployment Services and Eligibility

Assessments (RESEA) Profiling and Participant

Selection Models [ETA]. DOL continues to support 

RESEA program evaluations and the provision of technical

assistance (TA) to help states meet RESEA program

requirements and to better understand effective models 

and program components that work, under what conditions

they work, and for whom they work. For example, DOL is

building capacity within states for high quality evaluation

activities with technical assistance to states who are

conducting impact evaluations. 

Est. Start & Duration: FY23 (5 years)

Data Sources: Interviews, focus groups, administrative data

Methods/Approach: Data analytics; evaluation TA to state agencies conducting 

impact studies; impact evaluability assessment. Potential impact study, paired 

with an implementation study. Evidence review with additional approaches to be 

developed for dissemination of promising strategies with states and practitioners. 

Additionally, models will be developed that estimate the likelihood of exhaustion 

and/or related outcomes account for market conditions, demographics, and 

subgroups. 

Priority Research Question(s):

Is the delivery of RESEA services using remote methods 

as effective as delivering RESEA services in-person? 

Does the provision of more intensive RESEA services 

improve outcomes? Since some participants in the 

RESEA program are selected for participation based on 

models used by states to determine which workers are 

most likely to exhaust unemployment insurance benefits, 

like the Worker Profiling and Reemployment Services 

(WPRS) model, what are the evidence-based and most 

promising models that states can use for selecting 

participants? Which variables and data structures best 

identify participants that benefit from participation in 

these programs? For state-led evaluations, are these 

implemented based on widely accepted evaluation 

standards? How can findings from one state’s evaluation 

be reviewed for rigor and quality, and shared with other 

states for broader applicability? 

Anticipated Challenges & Solutions

For WPRS model specifications, local market information may or may not have 

explanatory value. The model(s) will need to appropriately account for local 

market conditions and correlations with other covariates.

10 EVALUATION PLA TION PLAN II. RESEARCH AND EVALUATION PROJECTS

Additional Information
EVALUATION PLA TION PLAN II. RESEARCH AND EVALUATION PROJECTS

 

 

Project 9* Additional Information

Title: Strengthening America’s Safety Net for Workers:

Lessons Learned from Unemployment Insurance (UI)

Programs During the Pandemic [ETA]. One of ETA’s

priorities is to strengthen and protect unemployment 

insurance benefits available to jobless workers. This

study will focus on lessons learned from the Pandemic

Unemployment Assistance (PUA) program, which provided

temporary income support to individuals not covered by the 

regular UI program, such as self-employed or gig economy 

workers or individuals in covered employment who had not

achieved sufficient earnings to qualify for regular UI.

Priority Research Question(s):

What is the feasibility and what are the considerations

associated with establishing a permanent program that

provides temporary income support to self-employed 

and other workers who do not qualify for coverage under

the regular UI program? What are the lessons learned

and outcomes of temporary UI programs, especially the 

Pandemic Unemployment Assistance and Federal Pandemic

Unemployment Compensation programs, which were

created in response to the COVID-19 pandemic? What

changes are needed to the Extended Benefits program to

ensure an effective and timely response to downturns in the

economic cycle? 

Est. Start & Duration: FY23 (3 years) 

Data Sources: Administrative data, secondary research, other data collection 

TBD 

Methods/Approach: Feasibility assessment, scan, descriptive analysis; 

quantitative analysis 

Anticipated Challenges & Solutions

None identified.

Project 10* Additional Information

Title: Strengthening Community College Grant

Program Evaluation [ETA]. ETA is interested in rigorously

evaluating the Strengthening Community College (SCC)

grants to identify capacity-building and/or systems change 

strategies that help reduce equity gaps in employment 

outcomes. ETA also seeks to build evidence on identifying

effective strategies with employers and within workforce

programs that have effectively reduced barriers to 

employment and advancement for underrepresented 

populations. 

Est. Start & Duration: FY23 (5 years)

Data Sources: ETA administrative data, third-party evaluator (TPE) data; site visit 

interviews; grantee surveys; secondary data

Methods/Approach: Synthesis/meta-analysis, implementation, impact study 

feasibility assessment, evaluability of an impact study examining the impact 

of SCC grants on training participant outcomes (and in particular on reducing 

equity gaps in workforce outcomes), impact study (if feasible), descriptive/journey 

mapping, customer experience, synthesis of 3rd party evaluations, evaluation TA 

Priority Research Questions:

What employer partnership strategies and other practices

are proving effective for community colleges to identify shortterm credentials or short-term training curriculum that lead to 

job placement? What short-term credentials or training for

in-demand jobs are being offered as a result of SCC funding,

and what programs are not eligible to be covered with Pell

grants? How and to what extent are community colleges

developing or changing curriculum to meet the needs of 

employers? What community college capacity-building or

systems-change strategies show promise in reducing equity

gaps in workforce outcomes for students? Specifically, what

strategies from the SCC grants show promise in reducing

equity gaps? What strategies might be effective? What

impact do SCC components or strategies have on participant

outcomes related to employment, education, and other 

outcomes? 

Anticipated Challenges & Solutions

Working with TPEs for access to their evaluation data may be difficult. An 

impact study may require studying investments made by DOL in FY24, which is 

contingent on future appropriations. 

11
EVALUATION PLA TION PLAN II. RESEARCH AND EVALUATION PROJECTS

 

 

 

Project 11* Additional Information

Title: Strengthening Employer Partnerships

and Reducing Barriers to Employment for

Underrepresented Populations [ETA]. ETA seeks to

build evidence to identify effective strategies with employers

and within workforce programs that have effectively

reduced barriers to employment and advancement for 

underrepresented populations. Similarly, ETA is interested

in understanding state and local strategies for addressing 

the priority of service requirements for adult participants in 

the Workforce Innovation and Opportunity Act.

Priority Research Question(s):

What strategies are being used by workforce programs 

to reduce barriers to employment and advancement for 

those from underrepresented populations and populations 

with significant barriers to employment? In particular, 

what specific supportive services are most promising 

and/or impactful in reducing barriers to employment and 

advancement for those from underrepresented populations 

and populations with significant barriers to employment? 

What strategies are being used in partnership with 

employers to address barriers to employment, training, 

and advancement? How are states and local areas 

implementing the Workforce Innovation and Opportunity 

Act (WIOA) priority of service requirements to provide 

services to those most in need, including individuals with 

barriers to employment? Are there promising policies or 

practices in this area? 

Est. Start & Duration: FY23 (3 years) 

Data Sources: ETA administrative data; site visit interviews

Methods/Approach: Synthesis/meta-analysis, roundtables, qualitative data 

collection 

Anticipated Challenges & Solutions

It may be difficult to obtain employer and partner engagement in the data 

collection and to identify causal impacts. 

Project 12* Additional Information

Title: Studying Longer Term Outcomes in H-1B-Funded

and Other Employment and Training Initiatives [ETA].

ETA is interested in building the evidence base on the

longer-term employment outcomes of participants in ETA

initiatives. 

Priority Research Question(s):

What are the long-term labor market outcomes of 

apprentices that have received services through 

the H-1B-funded initiatives, including the American 

Apprenticeship Initiative (AAI) and participants from the 

Trade Adjustment Assistance Community College and 

Career Training (TAACCCT) program? How do earnings 

outcomes and trajectories vary by occupation and 

participant characteristics? How, if at all, do the longterm labor market outcomes differ for those who complete 

apprenticeships and those who do not?

Est. Start & Duration: FY23 (5 years)

Data Sources: Administrative program data, wage data, and unemployment data 

to estimate employment outcomes 

Methods/Approach: Outcomes analysis 

Anticipated Challenges & Solutions

It may be difficult to conduct extensive sub-group analysis.

12
EVALUATION PLA TION PLAN II. RESEARCH AND EVALUATION PROJECTS

 

Project 13*

Title: Improving Diversity, Equity, Inclusion, and

Accessibility (DEIA) within the Federal Workforce

[federal initiative with the Office of Personnel

Mangagement (OPM)]. In partnership with OPM, DOL

will support a rigorous evaluation of strategies aimed at

improving DEIA within the Federal workforce. DOL will lead

this evaluation in close collaboration with OPM and possibly

other agencies. 

Est. Start & Duration: FY23 (3 years) 

Data Sources: Federal administrative records, potentially original data collection 

Methods/Approach: Qualitative and quantitative methods 

Anticipated Challenges & Solutions

Priority Research Question(s):

What strategies can be used to better recruit, retain,

and engage a diverse federal workforce, drawing from

existing efforts as well as relevant models from other

sectors? How can DOL, and potentially other federal

agencies, best assess the impact of these strategies on 

workers and agencies? 

Funding for an expanded study covering possibly multiple federal agencies 

was not included in FY23 appropriations, so DOL may not be able to pursue an 

expanded study. 

Project 14 Additional Information

Title: Analysis of Changing Conditions for Vulnerable

Populations [multiple DOL agencies]. This is a feasibility 

study/foundational fact-finding effort which examines

how data and statistical analysis might be used to better

understand the changing conditions for different worker

populations that vary based on demographics, geography, 

and wages. Initial analysis has been illustrative but not

comprehensive. This work will analyze a broader set of

vulnerable populations and characteristics for specific

agencies or programs. 

Est. Start & Duration: FY23 (2 years) 

Data Sources: American Community Survey data, other administrative data

Methods/Approach: Descriptive statistics 

Anticipated Challenges & Solutions

None identified.

Priority Research Question(s):

What types of datasets are readily available to better 

understand the changing conditions for vulnerable/at-risk 

populations? What types of considerations should be 

made when identifying datasets (such as whether survey 

design issues might affect interpretation of results for 

small subgroups)? What types of trend analyses might be 

most useful for specific agencies and/or programs?

13

Additional Information
Title: Worker Protection Research Center [multiple

DOL agencies]. Through a research grant to one or more 

academic institutions, DOL will launch a worker protection

research center, which will produce data and research that

can support common themes and goals across different 

DOL worker protection agencies in addition to research

focused on individual agencies. The research center 

will foster more rigorous scientific analyses, training, and

partnerships with community organizations and state and

local governments in support of DOL’s worker protection

agencies. 

Priority Research Question(s):

Potential research questions include: What strategies and 

approaches can be used to ensure workers understand 

their rights and can assert them with less fear of 

retaliation? What strategies and approaches can be 

used to identify, investigate, and refer potential workplace 

violations to appropriate partner agencies, including 

state agencies, and ensure that employers are aware of 

the law and equipped with tools for compliance? What 

are emerging economic and workforce trends that affect 

worker protection issues? How do we effectively measure 

these trends, and what are emerging strategies to address 

them? What is the relative effectiveness of different 

compliance assistance and enforcement approaches? 

Est. Start & Duration: FY23 (4 years) 

Data Sources: Existing administrative data from DOL worker protection agencies 

and data collected in potential prospective studies. 

Methods/Approach: Mixed methods, including original descriptive statistics and 

causal analysis. 

Resistance to data sharing, which can be addressed by offering secure data 

sharing environments through CEO’s restricted use program. 

Title: Advancing Equity in Hiring and Retaining Federal

Staff [OASAM]. The Department is engaging in a hiring 

surge. This is an opportunity to study how to advance

equity in hiring, accommodating, and retaining federal 

staff and apply these practices moving forward. OASAM

is working with CEO to analyze its data to identify what

barriers the Department has to retaining underrepresented 

groups. 

Priority Research Question(s):

Which types (by skill set, GS grade, demographics, job 

series) of employees are exiting DOL? Are certain types 

exiting at different rates? 

Est. Start & Duration: FY23 (1 years)

Data Sources: Administrative HR Connect data

Methods/Approach: Survival analysis (quantitative methods) for exploratory data 

analysis 

None identified.

14 EVALUATION PLA TION PLAN II. RESEARCH AND EVALUATION PROJECTS

Project 15* Additional Information

Anticipated Challenges & Solutions

Anticipated Challenges & Solutions

Project 16 Additional Information
Project 17 Additional Information

Title: Identifying Areas for Financial Data to Assist

Agencies in Evidence Building [OCFO]. OCFO is

interested in exploring how to use financial data to assist

evidence building around the department. OCFO is

interested in learning about how other CFO Act agencies

use of financial data and non-financial data interact

to optimize operations as well as how they can assist

agencies around DOL better utilize their data systems.

Est. Start & Duration: FY24 (2 years) 

Data Sources: OCFO administrative data

Methods/Approach: Linking OCFO administrative data with other DOL

agency data 

Anticipated Challenges & Solutions

Priority Research Question(s):

How can DOL agencies use financial data to make 

better decisions? Can predictive models be established 

to identify areas at risk and notify staff when action is 

needed? How do other CFO Act agencies manage and 

monitor their financial processes?

OCFO and other agencies use different formats, names, etc. Data will have to be 

processed and merged. 

Project 18

Title: Evaluation of the Equitable Transition Models

(ETM) [ODEP]. The Equitable Transition Models (ETM)

will target low-income youth with disabilities who face

challenges transitioning to employment and economic 

security. They are more likely to be unemployed or 

have low earnings as adults, be people of color, live

in underserved areas, be disconnected from services 

and supports, and be involved in the justice system. To 

address this challenge, ETM will provide comprehensive,

coordinated services to help low-income youth with

disabilities transition to employment. If funds become 

available to conduct the demonstration, ODEP seeks to

partner with CEO to procure and implement a rigorous

evaluation of the demonstration. 

Est. Start & Duration: FY23 (5 years)

Data Sources: ETM administrative data and survey data

Methods/Approach: Feasibility assessment, implementation or formative 

evaluation, outcome evaluation 

Anticipated Challenges & Solutions

Data quality of administrative data may be a challenge, which can be monitored 

closely and remedied. Starting the evaluation in FY23 will be contingent on 

securing funds for the demonstration through FY23 appropriations. 

Priority Research Question(s):

What are the different ETM models and how are they

operationalized? Are there variations in the model across

sites or target populations? What types of youth are served,

what services do they receive, and for how long? What

outcomes occur and at what point in time? What potentially

promising or innovative services or management tools are 

developed by the service providers? 

15 EVALUATION PLA TION PLAN II. RESEARCH AND EVALUATION PROJECTS

Additional Information
EVALUATION PLA TION PLAN II. RESEARCH AND EVALUATION PROJECTS

Project 19* Additional Information

Title: Assessing Technology-Driven Employment

Systems [OFCCP]. Technology is transforming how

employers hire and make employment decisions. 

Employers are increasingly reliant upon machine learning

and algorithms to recruit, hire, and assess workers. 

OFCCP is well positioned to promote safeguards when

technology is used in hiring and in other employment 

decisions. Workers are often unable to file a complaint

about these technologies due to substantial asymmetries in 

workers’ ability to access information on how employment

decisions are made. 

Est. Start & Duration: FY23 (3 years) 

Data Sources: Existing frameworks within the federal government (such as the 

National Institute of Standards and Technologies (NIST) AI Risk Management 

Framework, existing tools and models in the private sector or academia.

Methods/Approach: This will be an exploratory or descriptive study, ideally 

resulting in the development of an assessment tool. 

Anticipated Challenges & Solutions

Priority Research Question(s):

How can government contractors promote equal opportunity

when they use algorithmic selection and decision-making

systems? How can DOL ensure meaningful supervision and

accountability to promote non-discrimination in the use of 

these systems? What types of technology-driven/artificial

intelligence (AI) employment systems are different employers 

and industries using and for what positions? What

information related to artificial intelligence systems should

employers retain and have available for inspection? To what

extent do existing AI models manage bias, and to what extent

are they explainable, interpretable, and safe? 

It may be difficult to obtain training data for the design, development and testing of 

a high-quality assessment tool tailored to meet OFCCP’s needs.

Project 20 Additional Information

Title: National Construction Safety & Health

Achievement Recognition Program (SHARP) Pilot

Program (NCSP) Evaluation [OSHA]. SHARP is an

achievement recognition program administered by OSHA 

On-Site Consultation Program. SHARP recognizes the

employers that operate exemplary safety and health 

practices and programs. Historically, participation in 

SHARP was only available to single, fixed worksites. NCSP

allows the On-Site Consultation Program to work with small

businesses in the construction industry that were previously

not eligible for SHARP participation. OSHA will use NCSP

data to determine if the pilot program should be included in 

the general criteria for SHARP participation.

Est. Start & Duration: FY23 (2 years) 

Data Sources: OSHA Information System (OIS) and On-Site Consultation 

Program

Methods/Approach: Data analysis 

Anticipated Challenges & Solutions

The limited number of NCSP participants and availability of appropriate data. 

Priority Research Question(s):

How effective is NCSP in promoting safe and healthful 

work practices by small businesses in the construction 

industry, resulting in lower incident rates compared to the 

national and state averages? 

16
EVALUATION PLA TION PLAN

Project 21 Additional Information

Title: VETS Data Analytical Framework [VETS].

The goal of this effort is to advance racial equity and 

additional supports for underserved veteran communities 

by maximizing the use of internal and external data. This

study will strengthen the VETS’ data analytical framework

by identifying data-related needs, which is intended

to derive more sophisticated data analysis. It will also

examine existing data processes and data quality to identify 

areas for improvement. The resulting data analyses may 

also inform partnerships with Historically Black Colleges

and Universities (HBCU) and other institutions serving

underserved communities when possible.

Est. Start & Duration: FY24 (3 years) 

Data Sources: VETS data, DOL data, external federal and private sector data

Methods/Approach: Environmental and literature scan, data linking and data 

analysis 

Anticipated Challenges & Solutions

None identified.

Priority Research Question(s):

What are the available internal and external data sources

that can be used to answer priority DEIA questions to

improve veterans’ outcomes? What are the tools and training

needed to process and maximize use of that data? Do

existing forms/data collection (such as e-forms) capture data 

consistent with non-Transition Assistance Program services

administered by VETS? What types of data can be linked

to better understand the TSMs experience and access to

services? 

Project 22* Additional Information

Title: Study on Veteran Participation in American Job

Centers (AJC) [VETS]. The goal of this study is to improve 

VETS programs by collecting and using more customer

experience data and employment outcomes from veterans 

who access services from AJC.

Est. Start & Duration: FY23 (5 years)

Data Sources: Administrative data, qualitative data, potentially survey data 

Methods/Approach: Customer experience, qualitative and quantitative methods 

Priority Research Question(s):

What are the participation rates of veteran jobseekers in

workforce programs offered by AJCs, and how do they

compare with other subpopulations? Do some AJCs

have higher rates of veteran participation than other 

AJCs that are located in areas of similar demographics?

If so, how can these differences be explained? Do other

employment outcomes differ across different demographic 

characteristics? Are there ways to improve services to

target populations by updating policies or standard operating 

procedures (SOP), or applying behavioral science to existing

forms, communications, or procedures? 

Anticipated Challenges & Solutions

None identified.

17 II. RESEARCH AND EVALUATION PROJECTS
EVALUATION PLA TION PLAN II. RESEARCH AND EVALUATION PROJECTS

Project 23 Additional Information

Title: Transition Assistance Program (TAP) Impact

Study [VETS]. This impact study will examine the

longer-term outcomes of transitioning service members 

(TSMs) after TAP completion and military separation.

It will also provide additional exploratory analysis of

employment trends by TSM characteristics, which may

include underserved populations throughout VETS studies

wherever possible, including but not limited to Black,

Hispanic, and justice-involved veterans. The resulting data 

analyses can also inform partnerships with HBCUs and

other institutions serving underserved communities when

possible. 

Priority Research Question(s):

What are the longer-term impacts of TAP on transitioning

service members’ earnings and employment outcomes? 

What demographic and military characteristics are

associated with better reintegration and employment

outcomes for veterans? What are the earnings and labor

force participation patterns of military veterans over time? 

What are the unemployment receipt patterns of military

veterans over time? 

Est. Start & Duration: FY23 (5 years)

Data Sources: DOD/Army, wage data, and unemployment data to estimate 

earnings and employment outcomes 

Methods/Approach: Quasi-experimental analysis; exploratory analysis. 

Longitudinal data analysis to follow TSMs after TAP/separation. 

Anticipated Challenges & Solutions

Will explore the feasibility and challenges of using other data sources, such as 

Longitudinal Employer-Household Data, with the study.

Project 24 Additional Information

Title: Childcare Prices and Maternal Employment

in Local Areas [WB]. This study will use the

National Database of Childcare Prices (NDCP), a new,

comprehensive federal source of childcare prices at the 

county level in the United States, to evaluate how countylevel childcare prices are associated with county-level

maternal employment rates. Initially, the study will rely

on publicly available American Community Survey (ACS)

data. A potential second phase of the study could explore 

the relationship between childcare prices and mothers’

employment using individual-level employment records. 

Est. Start & Duration: FY23 (1 years)

Data Sources: National Database of Childcare Prices (NDCP) and American 

Community Survey (ACS) data

Methods/Approach: The study will use descriptive statistics and regression 

modeling. Regression models will control for county-level variables associated 

with women’s employment and childcare prices, including median income, men’s 

employment, occupational distribution, and county population size.

Anticipated Challenges & Solutions

Priority Research Question(s):

What is the relationship between childcare prices in local

areas and mothers’ employment?

At this time, we only have access to publicly available ACS data. Access 

to restricted county-level microdata is required to analyze individual-level 

employment records. This would entail securing access from the U.S. Census 

Bureau through an interagency agreement, which could take several years. 

Additionally, the NDCP has not yet been published so this could result in 

additional delays to the study’s completion.

18
EVALUATION PLA TION PLAN II. RESEARCH AND EVALUATION PROJECTS

Project 25* Additional Information

Title: Federal Employee Paid Leave Act (FEPLA)

Evaluation [WB]. FEPLA provides 12 weeks of paid

parental leave to certain federal employees in connection 

with the birth of an employee’s child or placement of a

child with an employee for adoption or foster care. The law

went into effect on October 1, 2020. Anticipated benefits

of the law include increased time for bonding between

parent and child, improved health of the birth mother and 

child, preservation of annual and sick leave balances for 

future family needs, increased retention and engagement 

of the federal workforce, and a positive impact to the U.S.

economy by addressing women’s declining labor force

participation. The Congressional Budget Office estimated

that implementing FEPLA would cost $3.3 billion from

2021 to 2024. OPM and DOL will collaborate to answer

questions surrounding the effects of FEPLA. For FY 2023,

the questions will focus on short-term uptake and effects of

the law and will be dependent on data availability. Future

evaluation questions may be proposed in the ongoing 

partnership. 

Priority Research Question(s):

What are FEPLA leave taking patterns, and do those vary

by employee demographics? Are leave taking patterns 

equitable across demographic groups? What is the effect

of FEPLA on the number of days of leave taken, by leave

type (e.g., FEPLA, Family and Medical Leave Act, sick leave,

annual leave)? What is the effect of FEPLA on short-term

retention of employees following paid parental leave? What

is the effect of FEPLA on employees’ career advancement,

including promotions, salary, and performance ratings? How

have needs/access to paid leave changed since the start 

of the COVID-19 pandemic (since our data is pre-COVID)?

How can we ensure that paid leave policies take into account

income, race, and gender disparities? 

Est. Start & Duration: FY22, with additional funds to be added in FY23 (2 years)

Data Sources: OPM and DOL will leverage existing data, including payroll data 

(use of paid and unpaid leaves), employee personnel status data (e.g., retention 

status, grade and step level, and pay rate), employee performance data (e.g., 

award hours or pay and performance ratings), and employee demographic data 

(e.g., gender, race and ethnicity, age, work schedule, agency).

Methods/Approach: OPM and DOL will analyze secondary administrative 

personnel data that OPM maintains in its Enterprise Human Resources 

Integration (EHRI) and Electronic Official Personnel Folder (eOPF) databases, 

conducting descriptive analyses and comparisons of outcomes prior to and 

following implementation of FEPLA.

Anticipated Challenges & Solutions

(1.) Data quality and completeness. At the time that FEPLA was implemented, 

there were no existing payroll codes for those using this leave; this data 

only started being reported with routine codes more than 6 months after 

implementation. There are also data quality issues within the payroll data, 

as agencies use differing payroll systems and approaches and do not have a 

consistent data cleaning procedure. To address this, we are proactively reviewing 

and cleaning FEPLA and leave data, while also working to improve the overall 

quality of payroll data. (2.) Selecting the appropriate time period for comparison. 

For comparisons of before and after implementation of FEPLA, there are 

complicating factors regarding time periods for comparison, including the lack 

of payroll codes and the COVID-19 pandemic, which is anticipated to influence 

childbearing and leave taking patterns. As such, we will carefully consider the 

appropriate time periods for both before and after comparison and will interpret 

findings with caution.

19
Additional Information

Title: Effective Targeting Strategies [WHD]. WHD

seeks support in carrying out a study that will capture the

range of approaches to identifying employers with certain

types of violations, the results of those cases, and how they

were developed. The study would provide a systematic

assessment of promising practices and challenges and 

establish an understanding of the variety of available 

approaches. Depending on the timeframe for the study, 

investigations could collect additional data on employer and 

worker characteristics, which would strengthen the analysis.

The study could support improved strategies within

initiatives, the development of training, and field guidance. 

Priority Research Question(s):

What business characteristics suggest a likelihood of

egregious violations requiring WHD intervention? What role

do employee and/or employer demographics play in the 

prevalence or egregiousness of violations? What are the

root causes of violations? For example, are violations largely 

caused by a lack of knowledge of what is required by the

law or by greater labor market concerns? Which practices

appear most effective at detecting certain violations? Under 

what conditions or circumstances do they work best? Do the

answers to these questions affect the type of interventions

that the agency should leverage in future efforts? 

Est. Start & Duration: FY24 (3-5 years)

Data Sources: Potential employer survey, potential new data collection by WHD 

enforcement staff during investigations, potential external/third party data sources 

Methods/Approach: Begin with literature review to identify existing data sources; 

Collect additional data during WHD investigations if necessary.

Unknown or nonexistent sources for this type of data. Potential need to collect 

data ourselves (resource constraints). 

20 EVALUATION PLA TION PLAN II. RESEARCH AND EVALUATION PROJECTS

Project 26

Anticipated Challenges & Solutions
III. Additional Data Collection

and Scoping Activities

This section describes additional data collection and research scoping activities CEO will 

conduct during FY 2023 and FY 2024, which may result in the identification of study topics 

to be included in future evaluation plans. 

Improving Customer Experiences

In FY 2023 and FY 2024, CEO and OWCP will continue collaborating on Customer 

Experience (CX) as a focus area that has a high potential for improving operations and 

perceptions of OWCP among claimants, their authorized representatives, medical providers, 

and others. The goal is to: (1) improve how the divisions communicate with external 

individuals and groups, and assist in gathering, analyzing, and then operationalizing 

information and data from external individuals and groups to assure that their customer 

experience is as effective as possible and (2) support OWCP’s efforts to advance equity 

and better engage traditionally disenfranchised communities, including those in poverty 

and in particular among communities of color. This activity began in FY22 and over the 

next two years, DOL will continue to scope out and examine data from claimant information, 

inbound correspondence, and survey responses to answer the following types of research 

questions: What data can we cull from existing sources to better inform the division about 

the experience of external individuals and groups? What new data sources and tools can 

we develop to maximize the value of input from external individuals and groups? Once 

we have actionable data, what tools and methods can, and should we employ to best 

analyze and use that data for program improvement? What systems, tools, or changes in 

process should we implement once we have completed our data collection and analysis 

activities? How can we use the responses/feedback to engage traditionally disenfranchised 

communities? 

Current Population Survey (CPS) Disability Supplement

In FY 2023 and FY 2024, CEO, ODEP, and BLS will continue collaborating to revise the 

CPS Disability Supplement for fielding in July 2024. This supplement has previously been 

fielded in FY 2012, FY 2019, FY2021. The process of revising the supplement began in FY 

2022 with DOL-convened listening sessions and working groups with internal and external 

parties interested in disability measurement issues. Over the next two years, DOL will 

continue finalizing the survey questions to more accurately capture the characteristics and 

labor market challenges of people with disabilities. 

21 EVALUATION PLA TION PLAN III. ADDITIONAL DATA COLLECTION AND SCOPING ACTIVITIES
Additional Pre-evaluation Activities

Throughout the year, CEO helps DOL agencies identify knowledge gaps and refine their 

research questions, with the goal of gathering sufficient initial information to eventually 

propose a discrete research study that will contribute to the labor evidence base. These 

pre-evaluation activities explore emerging research topics and departmental priorities as 

they arise. Pre-evaluation activities can include commissioning white papers, literature 

reviews, or convening events that bring together independent, technical and subject matter 

experts from outside DOL to share the most up-to-date research and resources on key 

topics of interest. 

During FY 2023 and FY 2024, CEO (alongside the respective partnering agencies listed 

below in brackets) intends to conduct pre-evaluation activities to explore the following topics: 

Measuring and Assessing Equity in Grant-Making and the Delivery of Program 

Services [ETA and ILAB]

DOL agencies are interested in understanding how to assess equity in their grant-making 

and service-delivery processes, including the extent that grant programs are reaching 

underserved populations and communities. Key questions include:

• How can DOL measure the extent that its grant-making and program-service delivery

processes effectively reach underserved populations and communities? What metrics

should DOL consider using to assess equity in grant-making and delivery of program 

services, and how can DOL minimize downside risks to collecting potentially sensitive

personal data, particularly in international contexts? 

• What are effective tools and approaches for assessing equity, and how can these be

leveraged in the diverse contexts in which DOL operates, including internationally?

Approaches to Measuring Worker Skills and Competencies [ETA]

ETA is interested in learning from expert researchers about methods and approaches to 

measuring skills and competency changes over time, including the benefits and drawbacks 

of each approach/method. Key questions include: 

• How can DOL best identify changes in labor market skill and competency demand

over time, particularly due to the effects of automation and artificial intelligence, or

economic shocks? What approaches can be used to estimate the available supply of

workers with in-demand skills?

• What are promising methods or approaches to identify changes in skill and

competency demand over time—particularly due to the impacts of automation and 

artificial intelligence, or other economic shocks?

22 EVALUATION PLA TION PLAN III. ADDITIONAL DATA COLLECTION AND SCOPING ACTIVITIES
Opioid Use in West Virginia and New Mexico Mining Communities [MSHA]

Miners are at high risk of experiencing a prescription opioid-related overdose and are more 

likely to receive an opioid prescription for pain medication than workers in other industries. 

MSHA would like to explore methods to reduce opioid use and identify workplace strategies 

that promote and support awareness efforts. The objective is to identify gaps in existing 

research on the impacts of workplace hazards on workplace injury and opioid use in mining. 

Key questions include: 

• What strategies can be used to implement opioid prevention and awareness training

in the mining industry? How can DOL evaluate the effectiveness of opioid prevention

and awareness training in the mining industry? What are the leading practices to

transfer research knowledge to the mining industry?

• What are workplace strategies that promote and support awareness efforts? What are

the underlying epidemiology problems and unique challenges regarding opioids use 

disorder (OUD) in rural and underserved communities? 

Topics in Whistleblower Research [OSHA] 

Not all workers covered by section 11(c) of the Occupational Safety and Health Act know 

about their protected rights under this section. OSHA seeks to convene various interested 

groups, including those representing vulnerable populations, to learn more about the 

barriers workers face when accessing and learning about their rights. Key questions 

include: 

• How can OSHA expand the number of workers that understand their rights under

section 11(c) of the Occupational Safety and Health Act?

• What barriers prevent workers from accessing their rights under section 11(c), and

how might they be addressed?

• Do workers from underserved communities face additional or different barriers?

23 EVALUATION PLA TION PLAN III. ADDITIONAL DATA COLLECTION AND SCOPING ACTIVITIES
U.S. Department of Labor