WORKFORCE 

INVESTMENT ACT

DOL Should Do More 

to Improve the Quality 

of Participant Data 

Report to Congressional Requesters

December 2013

GAO-14-4

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-14-4, a report to 

congressional requesters 

December 2013 

WORKFORCE INVESTMENT ACT

DOL Should Do More to Improve the Quality of 

Participant Data 

Why GAO Did This Study

Having reliable program data is 

important in effectively managing a 

program. However, there have been 

longstanding concerns about the 

quality of data on job seekers enrolled 

in the WIA Adult and Dislocated 

Worker Programs, which rely on states 

and local areas to track participants 

and the services they receive. Given

these concerns and WIA’s anticipated 

reauthorization, GAO was asked to 

examine the data on these WIA 

participants. This report addresses: (1) 

the factors that have affected the ability 

to report consistent and complete data

on participants in the WIA Adult and 

Dislocated Worker Programs, and (2) 

actions that DOL has taken to improve 

the quality of these data. To conduct 

this work, GAO reviewed relevant 

federal laws, regulations, guidance, 

and documentation from DOL. GAO 

interviewed officials from DOL’s

national and regional offices and state 

and local workforce officials from a 

nongeneralizable sample of eight 

states. GAO also analyzed WIA data 

from program year 2011 to determine 

the number of, characteristics of, and 

services provided to WIA participants.

What GAO Recommends

GAO recommends that DOL take steps 

to improve the consistency and 

completeness of data reported across 

states and to promote a continuous 

process for improving the data’s

quality. DOL officials did not agree or 

disagree with GAO’s overall

recommendations and detailed how 

data quality is being addressed 

primarily through existing efforts. 

However, GAO believes that the

recommendations remain valid as 

discussed in the report.

What GAO Found

Flexibility in the Department of Labor’s (DOL) data reporting guidance and 

limitations in some state information systems continue to impair the quality of the 

data on participants in the Workforce Investment Act (WIA) Adult and Dislocated 

Worker Programs. The flexibility in the guidance stems from the inherent nature 

of WIA, which allows states and local areas to tailor program design and service 

delivery to their needs. As a result, DOL’s guidance on collecting and reporting 

the data allows variation in how some WIA data elements are defined, collected, 

and reported. Specifically, an American Job Center—formerly known as a onestop center—can choose to provide certain basic services exclusively through 

WIA programs, exclusively through a partner program, or through a blend of both 

WIA and partner programs. However, this flexibility involves variations in data 

reporting that have contributed to inconsistencies among states regarding when 

job seekers are counted as WIA participants. Moreover, some aspects of DOL’s 

guidance are open to interpretation, leaving it to states to define variables such 

as type of training service received, further contributing to data inconsistencies.

In addition, some state information systems used to collect and report WIA 

participant data have limitations that hamper the affected states’ ability to report 

uniform and complete data. For example, data are incomplete to the extent that 

states may not have information systems that can track participants who access 

services online without significant staff assistance. Having inconsistent and 

incomplete data makes it difficult for DOL to compare data on program 

participants across states or to aggregate the data at a national level. 

DOL engages in various oversight activities designed to ensure the accuracy of 

states’ data on participants in the WIA Adult and Dislocated Worker Programs 

and has taken steps to improve data consistency across states. However, DOL 

does not consistently use the results of its oversight to identify and resolve 

systemic data issues nor has it evaluated the effect of oversight on the quality of 

WIA participant data. Specifically, DOL requires states to validate the data they 

collect and report on participants in the WIA Adult and Dislocated Worker 

Programs on an annual basis, but it does not strategically use the findings from 

this effort to identify systemic data issues or improve the quality of the data. 

Similarly, although DOL’s regional offices review a sample of each state’s WIA 

participant files every few years to assess states’ compliance with data reporting 

and validation requirements, DOL officials said they have not analyzed the 

findings from the most recent reviews to identify nation-wide reporting issues. 

DOL has taken steps to improve the consistency of the data by providing general 

technical assistance to states and local areas and through standardizing the way 

DOL collects WIA data. For example, since 2007, two states have been piloting a 

unified reporting system developed by DOL that uses standardized data 

definitions and is integrated across certain American Job Center programs 

administered by DOL. However, DOL officials said they have no plans to 

evaluate the system before expanding it to other states. Without an evaluation, 

DOL does not know what impact the pilot has had on the quality of WIA 

participant data.

View GAO-14-4. For more information, contact 

Revae E. Moran at (202) 512-7215 or 

moranr@gao.gov. 
Page i GAO-14-4 Numbers Served in WIA Programs

Letter 1 

Background 4

DOL’s Guidance and Limitations in State Information Systems 

Have Resulted in Inconsistent and Incomplete National Data on 

WIA Participants 10

DOL Has Increased Its Oversight Efforts and Taken Steps to 

Improve Data Consistency, but Data Quality Issues Have Not 

Been Resolved 20

Conclusions 33

Recommendations for Executive Action 34

Agency Comments and Our Evaluation 35

Appendix I Scope and Methodology 39

Appendix II Data on Participants in the WIA Adult and Dislocated Worker 

Programs for Program Year 2011 44

Appendix III Comments from the Department of Labor 49

Appendix IV GAO Contact and Staff Acknowledgments 54

Related GAO Products 55

Tables

Table 1: American Job Center Network Mandatory Partner 

Programs and Related Federal Agencies 5

Table 2: Most Prevalent Issues Identified During DOL’s Regional 

Reviews of States’ Case Files, Program Years 2006-2011a 28

Table 3: Selected Site Visit States and Local Areas 40

Contents 
Page ii GAO-14-4 Numbers Served in WIA Programs

Figures

Figure 1: Sequence of Services for the WIA Adult and Dislocated 

Worker Programs 7

Figure 2: Flow of WIA Data from Local Area to State to DOL 9

Figure 3: The Impact of Two States’ Service Delivery Models on 

When They Typically Count Job Seekers As WIA Adult 

Program Participants 13

Figure 4: Total Participants Enrolled in the Adult and Dislocated 

Worker Programs, Program Year 2011 44

Figure 5: Highest Level of Service Received by Adult and 

Dislocated Worker Program Participants, Program Year 

2011 45

Figure 6: Duration of Training for Adult and Dislocated Worker 

Program Participants Who Completed or Withdrew from 

Training, Program Year 2011 46

Figure 7: Type of Training Services Provided to Adult and 

Dislocated Worker Program Participants, Program Year 

2011 47

Figure 8: Percentage of Adult and Dislocated Worker Program 

Participants Registered in Selected Partner Programs, 

Program Year 2011 47

Figure 9: Adult and Dislocated Worker Program Participant 

Demographics, Program Year 2011 48

Abbreviations

DOL The Department of Labor

ETA Employment and Training Administration

SPRA Social Policy Research Associates 

WIA The Workforce Investment Act of 1998 

WIASRD Workforce Investment Act Standardized Record Data 

WISPR Workforce Investment Streamlined Performance Reporting

 System

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-14-4 Numbers Served in WIA Programs

441 G St. N.W.

Washington, DC 20548

December 2, 2013

The Honorable John Kline

Chairman

Committee on Education and the Workforce

House of Representatives

The Honorable Virginia Foxx

Chairwoman

Subcommittee on Higher Education and Workforce Training 

Committee on Education and the Workforce

House of Representatives

The Honorable Joe Heck

House of Representatives

The Honorable Howard P. McKeon

House of Representatives

Having reliable program data is an important factor in being able to 

effectively manage and evaluate a program. Our previous work on the 

Workforce Investment Act of 1998 (WIA), however, has raised questions 

about the quality of data that states report to the U.S. Department of 

Labor (DOL) on the number of individuals they serve.1 WIA, the 

centerpiece of the nation’s employment and training system, established 

three separately funded programs—Adult, Dislocated Worker, and 

Youth—and created a comprehensive one-stop system, now known as 

the American Job Center network, for the delivery of many federally

funded employment and training program services.2

 1

Pub. L. No. 105-220, 112 Stat. 936. GAO, Workforce Investment Act: Additional Actions 

Would Further Improve the Workforce System, 

At this writing, the 

GAO-07-1051T (Washington, D.C.: June 

28, 2007); Workforce Investment Act: Labor and States Have Taken Actions to Improve 

Data Quality, but Additional Steps Are Needed, GAO-06-82 (Washington, D.C.: Nov. 14, 

2005).

2

Training and Employment Guidance Letter No. 36-11, Announcement of American Job 

Center Network (Washington, D.C.: June 14, 2012) strongly encouraged states and local 

areas to refer to the one-stop system as the American Job Center network and to onestop career centers as American Job Centers, in order to increase job seeker and 

employer awareness of available workforce development resources. 

 
 

Page 2 GAO-14-4 Numbers Served in WIA Programs

Congress is considering proposals to reauthorize WIA, which has been 

due for reauthorization since 2003.

3

In our previous work, we found that the federal government’s efforts to 

collect and report accurate, consistent WIA program performance data 

from states were affected by (1) flexibility in federal guidance on collecting 

and reporting the data, (2) major changes to states’ information systems 

that resulted in lost data and other issues, and (3) limited monitoring by 

the Department of Labor (DOL). We also found that DOL’s WIA data do 

not include information on all participants in the Adult and Dislocated 

Worker Programs and that the data were not comparable across states 

and local areas.4 Similarly, DOL’s Office of Inspector General found that 

states were reporting inconsistent data on WIA participants in the Adult 

and Dislocated Worker Programs because DOL’s guidance did not clarify 

how certain participants should be counted.5

1. What factors have affected the ability to report consistent and 

complete data on participants in the WIA Adult and Dislocated Worker 

Programs, and

Given the longstanding 

concerns about the quality of the data and in anticipation of WIA’s 

reauthorization, you asked us to examine the data on WIA participants. 

This report addresses the following questions:

2. What actions has DOL taken to improve the quality of participant 

data?

To address our objectives, we reviewed relevant federal laws and 

regulations, as well as the guidance DOL provides to states for collecting 

and reporting data on participants in the WIA Adult and Dislocated 

Worker Programs.6

 

3

These proposals include: Supporting Knowledge and Investing in Lifelong Skills Act 

(SKILLS Act), H.R. 803, 113th Cong. (2013); Workforce Investment Act of 2013, H.R. 798, 

113th Cong. (2013); Workforce Investment Act of 2013, S. 1356,113th Cong. (2013); and 

Careers through Responsive, Efficient and Effective Retraining Act (CAREER Act) S.804, 

113th Cong. (2013).

We also interviewed officials from DOL’s Office of 

4

GAO-07-1051T; GAO-06-82.

5

U.S. Department of Labor, Office of Inspector General. Audit of Workforce Investment 

Act Data Validation for the Adult and Dislocated Worker Programs, 03-09-003-03-390 

(Washington, D.C.: Sept. 30, 2009).

6

The scope of the request only covered two of the three WIA programs—the Adult and 

Dislocated Worker Programs; it did not cover the WIA Youth Program.
 

Page 3 GAO-14-4 Numbers Served in WIA Programs

Inspector General and representatives of two national workforce-related 

organizations. To better understand the challenges faced by states in 

reporting data on these WIA programs, we visited or telephoned DOL’s 

Employment and Training Administration’s (ETA) six regional offices and 

a nongeneralizable sample of eight states. Within each state, we visited 

or contacted at least one comprehensive American Job Center—formerly 

known as a one-stop center—or a local workforce board. We selected the 

states to provide diversity on the basis of: (1) geographic location, (2) 

total federal spending on the Adult and Dislocated Worker Programs in 

program year 2010, (3) the extent of data issues identified by DOL’s data 

contractor in the fourth quarter of program year 2010, (4) whether the 

state reported participants who received only core “self-service,” and (5) 

the number of local areas within the state.7

To determine the actions DOL has taken to improve the quality of 

participant data for the WIA Adult and Dislocated Worker Programs, we 

interviewed officials from ETA’s national office and reviewed 

documentation and reports related to DOL’s various initiatives. We also 

assessed the reliability of program year 2011 data on participants in the 

Adult or Dislocated Worker Program from DOL’s Workforce Investment 

Act Standardized Record Data (WIASRD) database by testing the data 

electronically and interviewing knowledgeable agency officials and DOL’s 

data contractor about the actions they take to validate the data.8

 

7

We used DOL data from program year 2010 to identify total federal spending on the Adult 

and Dislocated Worker Programs and the extent of data issues because they were the 

most recent data available when we selected our sample of states. Program year 2010 

ran from July 1, 2010 through June 30, 2011. Core “self-services” are services that can be 

provided without significant staff assistance. We conducted in-person interviews with DOL 

officials in Regions 1 (Boston), 3 (Atlanta), and 5 (Chicago); state and local workforce 

officials in Massachusetts, Washington, Illinois, Georgia, and California; and American Job 

Center officials in Maryland. We conducted telephone interviews with DOL officials in 

Regions 2 (Philadelphia), 4 (Dallas), and 6 (Sacramento), and with state workforce 

officials in Maryland, Utah, and South Dakota. Because, in both South Dakota and Utah, 

the entire state is considered to be a single service delivery area, we did not interview 

American Job Center staff in either state. Our selection criteria included “core self-service” 

based on past findings by GAO and DOL’s Office of Inspector General that this variable is 

often underreported, and thus likely to affect the number of reported participants in the 

WIA programs. 

We 

found the data to be sufficiently reliable for the purposes of providing 

8

We analyzed data for program year 2011 because they were the most recent full year of 

WIASRD data available when we conducted our review. Program year 2011 ran from July 

1 2011 through June 30 2012. 
 

Page 4 GAO-14-4 Numbers Served in WIA Programs

estimates for the number of, characteristics of, and services provided to 

participants whose information is recorded by DOL as having been 

served by the WIA Adult or Dislocated Worker Program.9

We conducted this performance audit from August 2012 through 

November 2013 in accordance with generally accepted auditing 

standards. Those standards require that we plan and perform the audit 

work to obtain sufficient, appropriate evidence to provide a reasonable 

basis for our findings and conclusions based on our audit objectives. We 

believe that the evidence obtained provides a reasonable basis for our 

findings and conclusions based on our audit objectives.

However, we 

did not find those data to be reliable for other purposes such as making 

state-to-state comparisons because of variations in how states collect and 

report participant data for the WIA Adult and Dislocated Worker 

Programs. See appendix I for additional information on our scope and 

methodology.

WIA requires states and local areas to bring together a number of 

federally funded employment and training programs into a comprehensive 

workforce investment system, the American Job Center network. These 

programs—including the Adult and Dislocated Worker Programs—are 

known as mandatory partners, and must provide services through this 

network (see tab. 1).

 

9

For this study, we define “WIA program participants” as individuals who have a valid Date 

of Program Participation for the WIA Adult or Dislocated Worker Programs and who have 

been counted and reported to DOL as WIA Adult or Dislocated Worker Program 

participants. Program year 2011 ran from July 1, 2011 through June 30, 2012. WIASRD is 

a national database of individual records containing information on the characteristics, 

activities, and outcomes of all enrolled participants who received services or benefits 

under WIA. Participant data are typically collected by local staff at American Job Centers 

and entered into a state or local data system. After the state receives data from local 

areas, it compiles and formats the information and submits it electronically to DOL. 

Background
 

Page 5 GAO-14-4 Numbers Served in WIA Programs

Table 1: American Job Center Network Mandatory Partner Programs and Related 

Federal Agenciesa 

Federal Agency Mandatory Partner Programs

Department of Labor WIA Adult

WIA Dislocated Worker

WIA Youth

Employment Service (Wagner-Peyser) 

Trade Adjustment Assistance program

Veterans’ Employment and Training programs

Unemployment Insurance

Job Corps

Senior Community Service Employment Program

Employment and Training for Migrant and Seasonal Farm 

Workers 

Training for Native Americans

YouthBuild

Department of Education Postsecondary Vocational Rehabilitation Program

Adult Education and Literacy

Vocational Education (Perkins Act)

Department of Health and 

Human Services

Community Services Block Grant

Source: Pub. L. No. 105-220 and U.S. Department of Labor.

a

WIA specified 18 required partner programs administered by four federal agencies. One of these, the 

Welfare to Work grant program, administered by the Department of Labor, was discontinued in 2004. 

Employment and training activities administered by the Department of Housing and Urban 

Development (HUD) were also specified as a required partner but HUD does not currently administer 

any training and employment programs. Furthermore, in 2006, the YouthBuild program was 

transferred to the Department of Labor from the Department of Housing and Urban Development. 

The WIA Adult and Dislocated Worker Programs are designed to provide 

quality employment and training services to assist eligible individuals to 

find and qualify for employment and to help employers find the skilled 

workers they need. The Adult Program provides services to individuals 

over the age of 18 who are job seekers, although states and local areas 

must give priority of service to low-income individuals if funds are 

determined to be limited.10

 

1029 U.S.C. § 2864(d)(4)(E).

The Dislocated Worker Program provides 

services to workers who have been or will be terminated or laid off from 
 

Page 6 GAO-14-4 Numbers Served in WIA Programs

employment.11 For fiscal year 2013, Congress appropriated over $1.9 

billion for the Adult and Dislocated Worker Programs: $730 million for the 

Adult Program and $1.2 billion for the Dislocated Worker Program.12

DOL’s Employment and Training Administration administers the WIA 

Adult and Dislocated Worker Programs and oversees their 

implementation, which is carried out by states and local areas. Each state 

must have one or more designated local workforce investment area, and 

each local area must have at least one comprehensive American Job 

Center where job seekers can receive core services and access other 

programs and activities offered by the mandatory partners.

13

WIA provides three tiers, or levels, of service for adult and dislocated 

workers: (1) core, (2) intensive, and (3) training.

Although 

each local area must have one comprehensive center, under WIA, the 

mandatory partners have flexibility in the way they provide services 

through the American Job Center network, and can co-locate services on 

site or make referrals to external service providers or training, including to 

local community colleges.

14

• Core services include basic services, such as job search or résumébuilding assistance, and may be accessed with or without staff 

assistance.

• Intensive services include activities such as staff-assisted 

comprehensive assessment of a participant’s skill levels and case 

management.

 

11A dislocated worker is an individual who: (1) has been terminated or laid off, or has 

received a notice of termination or layoff from employment, and is eligible for 

unemployment compensation, has exhausted unemployment compensation, or is not 

eligible for unemployment compensation due to insufficient earnings or having worked for 

a non-covered employer but has demonstrated an appropriate attachment to the 

workforce and is unlikely to return to a previous industry or occupation; (2) has been 

terminated or laid off or received notification of termination or layoff from employment as a 

result of a permanent closure or substantial layoff; (3) is employed at a facility where the 

employer has made the general announcement that the facility will close within 180 days; 

(4) was self employed but is unemployed as a result of general economic conditions in the 

community or because of a natural disaster; or (5) is a displaced homemaker. See 20 

U.S.C. § 2801(9).

12Appropriations are tracked by fiscal year but participants are tracked by program year. 

1329 U.S.C. §§ 2831 and 2864(c)(2). 

1429 U.S.C. § 2864(d)(2), (3) and (4).
 

Page 7 GAO-14-4 Numbers Served in WIA Programs

• Training services include activities such as occupational skills or onthe-job training (see fig. 1).

Service at one level and a determination that a participant is unable to 

obtain employment through that service are prerequisites for service at 

the next level, although WIA does not specify the amount of time an 

individual must spend or the number of attempts that must be made to 

gain employment before moving to the next level.15 Job seekers who 

receive only core services that are self-service and informational in nature 

are not counted in the programs’ performance measures, but DOL 

requires that states count such individuals as program participants.16

Figure 1: Sequence of Services for the WIA Adult and Dislocated Worker Programs

 Selfservice and informational activities can be accessed either at an 

American Job Center or remotely, such as when a job seeker searches 

for employment over an internet connection to an American Job Center 

from a home computer.

 

15According to DOL, the determination of need for training can itself be a core and/or 

intensive service, such as an assessment or development of an Individual Employment 

Plan. Training and Employment Guidance Letter No. 14-08, Guidance for Implementation 

of the Workforce Investment Act and Wagner-Peyser Act Funding in the American 

Recovery and Reinvestment Act of 2009 and State Planning Requirements for Program 

Year 2009 (Washington D.C.: Mar. 18, 2009).

1629 U.S.C. § 2871(b)(2)(A)(i). Training and Employment Guidance Letter No. 17-09, 

Quarterly Submission of Workforce Investment Act Standardized Record Data (WIASRD)

(Washington, D.C.: Mar. 10, 2010).
 

Page 8 GAO-14-4 Numbers Served in WIA Programs

As part of its oversight, DOL collects program data from states, which are 

used to assess how well the Adult and Dislocated Worker Programs are 

working. States must submit quarterly and annual performance reports to 

DOL, in addition to uploading individual records on a quarterly basis to 

DOL’s national WIASRD database. Specifically, states must submit 

quarterly and supplemental monthly performance reports, a validated 

annual performance report at the end of each year, and quarterly 

WIASRD files for each reporting quarter for a program year.17 The 

WIASRD files include demographic and characteristic information for 

participants in the WIA Adult and Dislocated Worker Programs and 

information about services received through these WIA programs, as well 

as through some other partner programs.18

 

17DOL provides guidance to states in the form of Training and Employment Guidance 

Letters and Notices. Training and Employment Guidance Letter No. 27-10, Program Year 

2010/Fiscal Year 2011 Performance Reporting and Data Validation Timelines

(Washington, D.C.: May. 11, 2011). Program years run from July 1 through June 30. 

The process of collecting and 

reporting WIA data involves all three levels of government. More 

specifically, participant data are typically collected by staff at American 

Job Centers and entered into a state or local information system. In some 

states, local staff may enter data directly into a statewide information 

system; in other states, local areas may use their own individualized 

information systems to enter data from which they then must extract and 

compile for submission to the state. After the data is submitted to the 

state agency, it is compiled and formatted for the various submissions to 

DOL, including for quarterly WIASRD record layout submissions. (see fig. 

2). 

18WIASRD also captures data on participants in the WIA Youth Program, but this is 

outside of the scope of our review.
 

Page 9 GAO-14-4 Numbers Served in WIA Programs

Figure 2: Flow of WIA Data from Local Area to State to DOL
 

Page 10 GAO-14-4 Numbers Served in WIA Programs

DOL’s guidance to states, in the form of Training and Employment 

Guidance Letters and Training and Employment Notices, details how 

states should collect and report data on participants in the WIA Adult and 

Dislocated Worker Programs. However, the flexibility in the guidance DOL 

provides to states makes it difficult for DOL to provide consistent national 

data on participants in these programs. The flexibility in the guidance 

stems from the flexibility inherent in WIA, which allows states and local 

areas to tailor service delivery to their needs.19

 

19In this section, we are referring specifically to DOL’s data collection and reporting 

guidance for WIASRD. 

Consequently, DOL’s 

guidance is designed to accommodate the different ways that states and 

local areas can deliver services through the American Job Center 

network, including decisions about whether to enroll participants in 

partner programs. This flexibility, however, results in variations in how 

states and local areas report participant data, which makes it challenging 

for DOL to aggregate WIA data at the national level. In particular, it has 

created inconsistencies among states in when a job seeker is counted as 

a participant in the WIA Adult or Dislocated Worker Program. In addition, 

DOL’s guidance is open to interpretation, allowing states to define and 

report some variables differently, further contributing to inconsistencies in 

the data that states report to DOL on these programs. For example, while 

DOL requires states to report the type of training service provided to WIA 

participants who receive such services, the agency’s guidance only lists 

the six broad categories of training services states must report without 

defining or describing them in detail. In accordance with government 

internal control standards, management is responsible for developing 

DOL’s Guidance and 

Limitations in State 

Information Systems 

Have Resulted in 

Inconsistent and 

Incomplete National 

Data on WIA 

Participants

Flexibility in DOL’s 

Guidance Continues to 

Affect the Consistency of 

Data
 

Page 11 GAO-14-4 Numbers Served in WIA Programs

policies and procedures to achieve program objectives and clearly 

communicating these policies and procedures to facilitate understanding 

and consistent implementation.20

The flexibility in DOL’s guidance on when states and local areas should 

count individuals as participants in the WIA Adult and Dislocated Worker 

Programs has resulted in inconsistencies in the data reported on program 

participants. WIA allows for flexibility in the extent to which Adult and 

Dislocated Worker Program services are integrated with those from other 

programs.21 DOL’s guidance encourages, but does not require, states to 

integrate WIA services with those from other partner programs so that job 

seekers have access to a coordinated system of employment and training 

services.22

This variability in when states count job seekers under WIA has resulted 

in a lack of consistency and considerable differences in whether states 

report job seekers at American Job Centers as participants in the WIA 

Adult Program, the WIA Dislocated Worker Program, or both. More 

specifically, some states with relatively high general populations are 

counting fewer job seekers in these programs than states with much 

 As a result, states’ service delivery models differ in the extent 

of integration between WIA services and other programs. For example, 

state officials that we interviewed told us that they funded core services at 

their American Job Centers exclusively through WIA, exclusively through 

a partner program, or through a blend of both WIA and partner program 

funds. According to officials from DOL’s national office, states and local 

areas are best positioned to determine the mix of services that will meet 

the needs of their job seekers. However, depending on which service 

delivery model a state or local area selects, job seekers receiving the 

same types of services may be counted as WIA participants at different 

points in time, or they may never be counted under the WIA Adult or 

Dislocated Worker Program.

 

20GAO, Internal Control Management and Evaluation Tools, GAO-01-1008G (Washington, 

D.C.: August 2001), and Standards for Internal Control in the Federal Government, 

GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999). 

21Used in this context, integration can refer to the co-location of services at an American 

Job Center, or to integrated funding to provide any given service. 

22Training and Employment Guidance Letter No. 17-05, Common Measures Policy for the 

Employment and Training Administration’s (ETA) Performance Accountability System and 

Related Performance Issues (Washington, D.C.: Feb. 17, 2006).

Flexibility in When to Count 

Participants under WIA
 

Page 12 GAO-14-4 Numbers Served in WIA Programs

lower populations.23 For example, according to the state officials we 

interviewed, Utah’s American Job Centers integrate WIA Adult Program 

funding with funding for the Wagner-Peyser Program to provide core 

services. As a result, every job seeker aged 18 years or older who 

receives core services in Utah is counted as a participant in both 

programs, as permitted under DOL’s guidance.

24

As a result of these differences, Utah ranked 3rd out of 53 states in the 

total number of participants it served in the WIA Adult Program in 

program year 2011, even though it ranked 36th in overall population.

In comparison, state 

officials in California told us that the state typically funds core services 

exclusively through the Wagner-Peyser Program and therefore counts all 

individuals accessing core services from an American Job Center as 

participants in the Wagner-Peyser Program, but not as participants in the 

WIA Adult Program or the Dislocated Worker Program. Therefore, only 

job seekers who meet the eligibility criteria for the WIA Adult or Dislocated 

Worker Program and receive intensive or training services funded by 

those programs are counted as WIA participants. Because of this 

variability, the total counts of WIA Adult Program participants and WIA 

Dislocated Worker Program participants represent different populations of 

job seekers in different states, depending on the service delivery model 

the state uses.

25

 

23These trends are independent of the amounts of federal funding to states for the Adult 

and Dislocated Worker Programs. 

 At 

the same time, California ranked 27th in the total number of participants 

served in the WIA Adult Program, even though it was the most populous 

state (see fig. 3).

24The Wagner-Peyser Program is one of the mandatory American Job Center partner 

programs. The Wagner-Peyser Act of 1933 established a nationwide system of public 

employment offices known as the Employment Service. The act was amended by the 

Workforce Investment Act of 1998 to require that employment services be delivered 

through the one-stop services delivery system. The Employment Service focuses on 

providing a variety of core employment related labor exchange services including but not 

limited to job search assistance, job referral, and placement assistance for job seekers.

25Data from the program year 2011 WIASRD was used to estimate the total number of 

participants in the WIA Adult Program. Estimates from the U.S. Census Bureau of 

population 18 years of age and over for 2011 were used to rank each state and territory 

based on population. Our use of the word “states” refers to the 50 states plus Puerto Rico, 

the District of Columbia, and the U.S. Virgin Islands. 
 

Page 13 GAO-14-4 Numbers Served in WIA Programs

Figure 3: The Impact of Two States’ Service Delivery Models on When They 

Typically Count Job Seekers As WIA Adult Program Participants 

The extent to which states and local areas comply with DOL’s reporting 

guidelines also affects the consistency of data on program participants. 

We visited three American Job Centers that provided core services to job 

seekers funded by WIA without counting these job seekers as WIA 

participants despite DOL guidance that these individuals be counted 

under WIA. For example, at two American Job Centers in Georgia, 

officials stated that they were providing core services funded by WIA to 

job seekers without enrolling these individuals in any WIA program. 

Officials at one American Job Center in Georgia added that these 

individuals are not counted, in part, because it is burdensome to collect 

participant data on jobseekers who receive only core services. We noted 

a similar compliance issue at an American Job Center in Illinois, and 

DOL’s case file reviews indicate that other local areas may be providing 

WIA-funded services to individuals without recording them as 
 

Page 14 GAO-14-4 Numbers Served in WIA Programs

participants.26

Participation Date: Flexibility in DOL’s guidance contributes to 

differences in how states collect and report WIA data on services 

provided by partner programs. For example, DOL’s guidance allows 

states and local areas flexibility in what date to report as the participation 

date for WIA programs.

As a result, cross-state comparisons of the number of 

participants in WIA programs are potentially misleading and DOL is 

limited in the extent to which it can use national WIA participant data to 

manage these programs. In addition, without accurate information on 

program participants, DOL and lawmakers run the risk of making 

decisions about WIA policy and funding that are not based on a true 

picture of the number of participants and the program’s outcomes.

27

Service Dates: Similarly, DOL’s guidance allows states and local areas 

flexibility in how they collect and report data on the services WIA 

participants receive from partner programs. This makes it difficult to make 

state-to-state comparisons of the types of services participants in the WIA 

Adult or Dislocated Worker Programs receive, when they receive them, 

and whether they are funded by WIA or a partner program. For example, 

when a staff-assisted core service is provided by WIA, DOL’s guidance 

More specifically, for WIA participants who 

receive services funded by a program other than WIA, states and local 

areas can record the participation date in WIASRD as either the first date 

an individual received WIA services or the first date an individual received 

services from any other partner program. If, for example, a participant 

received services from the Wagner-Peyser Program on April 1st and then 

received services from WIA on December 15th, states can record the 

date of WIA program participation as either April 1st or December 15th. 

Among the state officials we interviewed in the eight states we selected, 

five of them said their states recorded the participation date as the first 

date a WIA service was provided to a job seeker and three of them said 

their states recorded the participation date as the first date a partner 

program’s service was provided to a job seeker. This flexibility results in 

inconsistent reporting of the participation date, which makes it difficult for 

DOL to compare the length of WIA program participation across states.

 

26Regional office staff informed us that some states are still not reporting recipients of 

WIA-funded self-services as program participants. 

27DOL’s Training and Employment Guidance Letter No. 17-05 defines the date of program 

participation, which refers to WIASRD variable 302: Date of Program Participation. 

Flexibility in Reporting on 

Services Provided by Partner 

Programs
 

Page 15 GAO-14-4 Numbers Served in WIA Programs

instructs states and local areas to report the date a participant received 

the service for the first time. However, when another partner program 

provides the service, officials from DOL’s national office said that they 

encourage but do not require states to report the first date that staffassisted core services were provided.28 As a result, some states used 

WIASRD to report core services funded by partner programs, while others 

did not. Our analysis of program year 2011 WIASRD data showed that 11 

states had missing data for “Date of First Staff-Assisted Core Service” for 

over half of the participants who received intensive or training services 

from either the WIA Adult Program or the WIA Dislocated Worker 

Program.29

For these states with missing data, DOL’s contractor uses the 

participation date in WIA to estimate when a participant received his/her 

first staff-assisted core service.30

DOL’s guidance to states for some WIASRD variables is open to 

interpretation, resulting in differences in how states define or report them, 

further contributing to the challenges DOL faces in reporting consistent 

national data on participants in these programs. According to officials 

from DOL’s national office, the guidance on how to report select variables 

in WIASRD is intentionally broad to accommodate the different ways 

states deliver services. Some of the state and local officials we 

interviewed told us that the definitions for the following variables were 

relatively broad and open to interpretation:

However, to the extent that states 

purposely leave the “Date of First Staff-Assisted Core Service” blank 

because the service was provided by a partner program, this revision 

artificially inflates the number of WIA participants who are counted as 

having received these services under WIA. This also makes it difficult for 

DOL to know whether the services reported in WIASRD were funded by 

WIA or a partner program.

 

28We are referring to WIASRD variables 332: Date of First Staff-assisted Core Service.

29As noted in our background section, core services are a prerequisite for intensive and 

training services. 

30Specifically, officials from DOL’s national office said that, when its contractor for 

WIASRD prepares the publicly available data files, for all participants without a date of first 

staff-assisted core service but with a date of first intensive service or training, the 

contractor will estimate the date of that core service by replacing the missing date with the 

Date of Program Participation.

Broad Definitions for Selected 

Variables 
 

Page 16 GAO-14-4 Numbers Served in WIA Programs

“Type of training”: While DOL requires states to report the type of 

training service provided to all WIA participants who receive such 

services, the agency’s guidance only lists the six broad categories of 

training states must report.31

“Core Services”: According to DOL’s guidance, core services that 

require “significant staff involvement” are considered to be staff-assisted 

core services in contrast to self-services and informational activities 

defined as “core and informational services that do not require significant 

staff involvement.”

Officials from one regional office and one 

state said that interpretations of these categories can vary across states 

and local areas, which can define the categories differently, particularly 

the one for “Other Occupational Skills Training.” According to our analysis 

of program year 2011 WIASRD data, most states reported that at least 75 

percent of their training recipients in both the WIA Adult Program and the 

Dislocated Worker Program received “Other Occupational Skills Training,” 

a broad category with no clear definition or description. While this is a 

legitimate and appropriate type of training, such as for job seekers with 

Individual Training Accounts allowed under WIA, it is unclear what 

comprises this category of training, which is attributed to over two-thirds 

of participants in training, because states are not required to elaborate on 

it. Without clearly defined categories, policymakers and program 

managers do not have relevant information to use to compare or evaluate 

the effectiveness of the different types of training services provided to 

WIA participants. Officials from DOL’s national office acknowledged that 

the categories were not clearly defined but noted the trade-off between 

providing flexibility to states and local areas and being too prescriptive.

32

 

31Training and Employment Guidance Letter No. 17-09. We are referring to WIASRD 

variable 341: Type of Training Service #1. DOL’s Training and Employment Guidance 

Letter No. 17-09 lists six types of training but does not provide detailed definitions for 

them. These categories include on-the-job training, skill upgrading and retraining, 

entrepreneurial training, ABE or ESL in combination with training, customized training, or 

other occupational skills training. Although “Customized training” and “on-the job training” 

are defined in statute ((29 U.S.C. §§ 2801(8) and 2801(31)), they are not defined in any of 

DOL’s guidance.

While DOL has provided detailed written guidance to 

states to distinguish staff-assisted services from self-services, the 

32Some core self-service and informational activities can be conducted remotely through 

internet connections to an American Job Center, satellite center, or partner agency. 

Participants who only receive core self-services are excluded from WIA performance 

outcome measures. 29 U.S.C. 2871(b)(2). Training and Employment No. 8-10, Workforce 

Investment Act Self-Service Participant Reporting What, Where, and How (Washington, 

D.C.: Aug. 26, 2010).
 

Page 17 GAO-14-4 Numbers Served in WIA Programs

guidance remains open to interpretation and requires local area staff to 

make subtle and subjective distinctions concerning the level of staff 

assistance provided. Officials from three states and one region told us 

that it was difficult to differentiate staff-assisted services from selfservices. As a result, states’ definitions of staff-assisted core services 

may vary, which could affect how many participants are included in WIA’s 

performance measures.

Officials from three states also told us that some of DOL’s reporting 

guidance is open to interpretation. This can create challenges for states 

to communicate a uniform interpretation to local areas. For example, 

officials from Massachusetts said that, because each American Job 

Center has some degree of flexibility in the services they offer and how 

they are delivered, interpretation of DOL guidance may vary by local area.

According to officials from five regional offices, state information systems 

vary in quality, and some lack the capability to collect and report all of the 

required data, which has affected the comparability and completeness of 

some of the data on participants in the WIA Adult and Dislocated Worker 

Programs. WIA requires each state to build and manage its own 

information system for collecting and reporting data on WIA participants.33

Some states have developed their own unique information systems while 

others, including three of the eight states whose officials we interviewed, 

contract with companies to develop and manage their WIA data 

systems.34

For example, according to officials from DOL’s national office and from 

three of its regional offices, some state systems cannot account for 

However, some WIASRD variables may not be reported due 

to limitations in some state information systems. As a result, some of the 

data on participants are inconsistent and incomplete, which makes it 

difficult for DOL to compare data on program participants across states or 

to aggregate the data at a national level. In addition, without having 

accurate data on the number and types of participants, it is difficult for 

DOL and lawmakers to make sound policy and funding decisions about 

these programs.

 

3320 U.S.C. § 2935(c)(2). 

34Two of these states, Georgia and California, were in the process of transitioning their 

state information system to a contractor when we interviewed state officials.

Limitations in State 

Information Systems 

Hamper DOL’s Ability to 

Collect and Report 

Uniform and Complete 

National Data
 

Page 18 GAO-14-4 Numbers Served in WIA Programs

participants in the WIA Adult Program who receive only core selfservices, which affects the completeness of the data.35 In addition, DOL’s 

guidance notes that the use of a voluntary online registration process to 

collect information on these self-service-only participants is the best 

approach for improving consistency in the reporting of these 

participants.36 Nevertheless, some state information systems are not 

capable of reporting data on participants who access services online, 

according to officials from one regional office. In 2010, DOL’s reporting 

guidance acknowledged the variability in reporting on self-service-only 

participants with some states reporting few or no WIA self-service-only 

participants and others reporting many hundreds of thousands of such 

participants.37 Our analysis of program year 2011 WIASRD data showed 

that this variability persists. Nine states reported that none of their 

participants were self-service-only participants, while 27 states reported 

that over 90 percent of their participants were self-service onlyparticipants.38

In addition, DOL’s guidance requires states to report an occupational 

skills training code for all participants who have completed training, but 

not all state information systems have the capability of capturing this 

code.39

 

35Training and Employment Guidance Letter No. 17-09. We are referring to WIASRD 

variable 331: Received Core Self-Services and Informational Activities. 

Our analysis of program year 2011 WIASRD data showed that 

states did not report occupational skills training codes for about 18 

percent of participants in the Adult and Dislocated Worker Programs who 

completed training. As a result of these missing data, limited information 

is available to program managers and DOL about the types of 

occupations for which WIA participants were trained. According to officials 

from DOL’s national office, occupational skills training codes are 

underreported because state information systems are either programmed 

with outdated codes or lack the capability to report valid codes. 

36Training and Employment Notice No. 8-10.

37Training and Employment Notice No. 8-10.

38Our analysis did not identify the reason that each of these states did not report data on 

self-service-only recipients. Some states may not be recording this data for reasons other 

than limitations in their information systems. 

39U.S. Department of Labor, Workforce Investment Act Standardized Record Data File 

Layout (Washington, D.C.). 
 

Page 19 GAO-14-4 Numbers Served in WIA Programs

According to officials from two regional offices, limitations in information 

systems may be the result of states’ limited funding and technical 

expertise to upgrade and maintain their information systems. For 

example, state officials we interviewed in five of the eight states reported 

that it is difficult to quickly modify their information systems to 

accommodate new data reporting guidance from DOL, which may be due 

in part to resource constraints. Officials from two states and one regional 

office added that the recent reduction of WIA funding for state set-asides, 

from 15 percent to 5 percent of WIA funds, has had a negative effect on 

states’ investment in data reporting, including their information systems 

and technical expertise.40

As a result of variations in how states collect and report participant data 

for the WIA Adult and Dislocated Worker Programs and limitations in their 

information systems, the actual number of participants in these programs 

is unknown. However, we were able to estimate the number of, 

characteristics of, and services provided to participants whose information 

is recorded by DOL as having been served by the WIA Adult and 

Dislocated Worker Programs using WIASRD data from program year 

2011. (See additional information on our review of the WIASRD data for 

program year 2011 in app. II).

 

40WIA required the Governor of each state to reserve a portion of the funding for the WIA 

Adult, Dislocated Worker, and Youth Programs for certain statewide employment and 

training activities, which DOL officials told us can include investment in information 

systems. 29 U.S.C. § 2853(a)(1). For 2011, 2012, and 2013, Congress reduced the 

maximum percentage states could set aside for these activities from 15 percent to 5 

percent. 
 

Page 20 GAO-14-4 Numbers Served in WIA Programs

To improve the quality of data on participants in the WIA Adult or 

Dislocated Worker Programs, DOL has enhanced its oversight efforts and 

introduced new initiatives including having its data contractor produce 

quarterly reports on data issues, requiring states to validate their WIA 

data on an annual basis, and engaging its regional offices in periodic 

reviews of case files from states. However, the agency has not 

established a process to review the results of oversight to identify and 

resolve systemic issues with the quality of participant data from the WIA 

Adult and Dislocated Worker Programs. Our past work has shown that 

the benefit of collecting performance information is only fully realized 

when this information is actually used by managers to make decisions 

oriented toward improving results.41

DOL’s contractor, Social Policy Research Associates (SPRA), analyzes 

the WIASRD data that states submit to DOL each quarter and produces 

quarterly reports identifying potential errors in the data reported by states 

and local areas. These reports could also provide useful assessments of 

inaccuracies in the data and potential systemic reporting errors.42

 

41

However, because not all states are aware of or receive copies of these 

reports from DOL, their value as a tool to improve the quality of 

participant data for the WIA Adult and Dislocated Worker Programs is 

limited. In addition, officials from some of the states said that some of the 

GAO-05-927, Managing for Results: Enhancing Agency Use Of Performance 

Information for Management Decision Making (Washington, D.C.: Sept. 9, 2005).

42DOL has a contract with SPRA to correct and analyze WIASRD data and provide data 

files and reports on the accuracy of the data reported by states that are made available to 

the public via DOL’s website.

DOL Has Increased Its 

Oversight Efforts and 

Taken Steps to 

Improve Data 

Consistency, but Data 

Quality Issues Have 

Not Been Resolved

DOL Performed Various 

Oversight Activities, but It 

Did Not Consistently Use 

the Results to Improve the 

Data

Social Policy Research 

Associates’ Quarterly Reports
 

Page 21 GAO-14-4 Numbers Served in WIA Programs

corrections SPRA makes to the state WIASRD data files may not be 

accurate, and may result in incorrect data for a state.

The publicly available data files that SPRA produces are released 

quarterly and include “data issues reports,” which identify issues or 

anomalies in the WIA data submitted by each state to DOL. Officials from 

DOL’s national office stated that the original intent of SPRA’s analysis 

was to create the publicly available data files and not to report on data 

anomalies or errors, even though SPRA has always identified issues with 

the data while creating the publicly available files. However, since about 

2010, when states started submitting quarterly WIASRD files instead of 

annual ones, reviewing the quality of the data and issuing the quarterly 

“data issues and anomalies report” has become standard practice under 

SPRA’s contract, according to officials from DOL’s national office. The 

agency’s regional offices are supposed to provide the states with the 

published quarterly error reports and ask them to update any errors prior 

to their next quarterly WIASRD submission.

DOL publishes SPRA’s data files and reports on its website, and its 

regional offices are expected to share these reports with their states so 

that the states can correct any data issues in their subsequent quarterly 

submissions. However, officials from DOL’s Region 5 said that although 

they receive these reports from DOL’s national office, they have not had 

the chance to review them or comment on any of the errors for states in 

their region. In addition, officials from DOL’s Region 1 said that although 

these reports have always been available on DOL’s website, they only 

recently began receiving the reports in a user friendly format and that they 

only recently began providing feedback to SPRA on issues identified for 

the states in their region. Our analysis of SPRA’s data issues reports 

suggests that some states may not be using SPRA’s reports to improve 

the quality of their data on WIA participants, which could be due to a lack 

of awareness of these reports. For example, SPRA’s data issue reports 

for the fourth quarters of both program years 2010 and 2011 identified 

some of the same issues, such as errors in the dates of service reported 

by the states.43

 

43Social Policy Research Associates, Workforce Investment Act Program Year 2010 

Fourth-Quarter Data Quality Revisions (Oakland, C.A.: Nov, 7, 2011); Program Year 2011 

Fourth-Quarter Data Quality Revisions (Oakland, C.A.: Nov. 28, 2012). 

This suggests that the states that made these errors in 

2010 may not have reviewed these reports and used them to correct the 

data reported in the following year. In addition, while officials from four 
 

Page 22 GAO-14-4 Numbers Served in WIA Programs

states were aware of the SPRA reports, officials from four other states 

told us they either were not aware of these reports or that they began 

receiving these reports from the regions only recently, beginning with the 

final report for program year 2011. When asked, officials from DOL’s 

national office stated that they do not have a plan to systematically 

identify or address recurring errors noted in SPRA’s reports. Government 

internal control standards note that, for oversight and monitoring to be 

effective, information should be recorded and communicated to 

management and others within the entity and external stakeholders, and 

this should be done within a time frame that enables them to carry out 

their internal control and other responsibilities.44

Officials added that they do not conduct formal oversight reviews or 

audits of SPRA’s data analyses because they consider SPRA to be the 

“data experts” and, therefore, do not know what kind of oversight they 

could provide. According to government standards for internal controls, 

agencies should ensure that ongoing monitoring occurs in the course of 

normal operations, which would include monitoring and oversight of 

contractors through regular management and supervisory activities.45

However, according to officials we interviewed in three of the eight states 

we selected, some of the corrections SPRA makes to the state WIASRD 

data files may not be accurate, and may result in incorrect data for a 

state. When SPRA receives the WIASRD data files from DOL, it corrects 

the data after reviewing them for apparent inconsistencies or issues 

stemming from the way states enter the data on individual case files. 

According to SPRA officials, after they receive the data, they analyze it for 

inconsistencies that look odd from a state perspective and that would not 

have been picked up by DOL’s internal edit checks. For example, SPRA 

might determine that a missing value for a certain variable is equivalent to 

a “no” response and enter “no” or “zero” instead of leaving it blank. 

However, officials from three state workforce agencies told us that they 

did not always agree with the changes SPRA makes during its process of 

These officials also said they believe any major data issues or obstacles 

would be uncovered by their internal data edit checks, which are run on 

all WIASARD data submitted by the states prior to SPRA’s review of the 

data.

 

44GAO/AIMD-00-21.3.1. 

45GAO/AIMD-00-21.3.1. 
 

Page 23 GAO-14-4 Numbers Served in WIA Programs

correcting the data. For example, officials from California explained that 

SPRA changed some of the numbers entered by local areas for WIASRD 

variable 325—Employment and Training Programs Related to Food 

Stamps—to zeros because the numbers entered seemed too high. 

According to the state officials, the numbers for that variable entered by 

the local areas were correct. States, however, are generally not provided 

an opportunity to review and verify SPRA’s changes before they are 

made, as they only receive copies of the data issues reports after they 

are published. Officials from both DOL’s national office and from some of 

the state workforce agencies noted that not all issues identified by SPRA 

represent actual errors and that some outliers on certain variables are 

acceptable.

DOL requires each state to validate the data it collects and report on 

participants in the WIA Adult and Dislocated Worker Programs on an 

annual basis, but the findings from these validation efforts have not been 

strategically used to identify systemic issues with or to improve the quality 

of the data on WIA participants.46 Government standards for internal 

controls state that for oversight and monitoring to be effective, information 

should be communicated to management and others, along with the use 

of this information for program assessment, so that it can be used to carry 

out their internal control and other responsibilities.47

Each year, states are required to review a sample of WIA participant 

records to determine whether the source documents match the 

information in the electronic records states use to collect and report data 

to DOL in WIASRD.48

 

46Training and Employment Guidance Letter No. 28-11.Program Year 2011/Fiscal Year 

2012 Performance Reporting and Data Validation Timelines (Washington D.C.: May 9, 

2012).

DOL requires states to validate the accuracy of the 

data they submit annually to ensure that decisions about WIA policy and 

47GAO/AIMD-00-21.3.1. 

48Data validation also includes report validation—assessing whether states’ software 

accurately calculated required WIA performance measures. In response to a prior GAO 

recommendation, DOL established a 2 percent error rate threshold for report validation, 

which is tied to financial incentives and sanctions. Officials from one of DOL’s regional 

offices stated that they noticed significant decreases in report validation error rates after 

DOL began using them to calculate financial awards. For more information on the 

recommendations, see: GAO-06-82; Training and Employment Guidance Letter No. 09-

07, Revised Incentive and Sanction Policy for Workforce Investment Act Title IB Programs

Washington, D.C.: Oct. 10, 2007).

States’ Annual Data Validation
 

Page 24 GAO-14-4 Numbers Served in WIA Programs

funding are made based on a true picture of the number of participants 

and program outcomes. Although DOL has established a provisional 5 

percent error rate threshold for states’ validation of the variables, the 

agency does not have plans to tie the results of these validation efforts to 

DOL’s financial awards or penalties because, according to officials from 

DOL’s national office, the results of states’ validation efforts were never 

intended to be used for enforcement purposes.49

DOL, however, does not know what effect state data validation efforts 

have had on the quality of participant data for the WIA Adult and 

Dislocated Worker Programs. In addition, our interviews with regional and 

state officials suggest that DOL’s regions and the states are not always 

using the results of these data validation efforts to improve data quality or 

target technical assistance. During our interviews with regional and state 

officials, only those from Region 3 and Massachusetts described specific 

efforts to use the results of the state data validation efforts to improve 

data quality and direct technical assistance. Officials from DOL Region 3 

noted that the region recently began requiring states to respond to 

findings from states’ annual data validation efforts and to track error rates 

found in each quarterly submission. Similarly, officials from 

Massachusetts said that they use the results of their data validation 

efforts to direct the technical assistance the state provides to local areas 

and to improve the quality of the state’s WIA data. In contrast, officials 

from one state said that, although they receive information about errors 

associated with specific case files as they enter data from each file into 

DOL’s data validation system, they do not know how to retrieve their 

state-wide results from DOL’s database and they have not received any 

reports from DOL documenting the nationwide results of the data 

validation efforts. Officials from DOL’s national office said that they were 

surprised by this, and that data element validation results are available to 

states through DOL’s reporting system.

Officials from DOL’s 

national office explained that, in their opinion, DOL’s regional offices 

should be using the states’ validation efforts as a management tool to 

improve the quality of their data by identifying inaccurate or confusing 

variables to target the technical assistance they provide to states and 

local areas.

 

49In accordance with WIA, DOL provides financial awards to states for performance that 

exceeds negotiated levels and assessed penalties for performance that does not meet 

negotiated performance levels. 20 U.S.C. § 9273 and 29 U.S.C. §2871(g). 
 

Page 25 GAO-14-4 Numbers Served in WIA Programs

Moreover, our analyses of the results of DOL’s efforts to validate 

WIASRD variables for program years 2010 and 2011 suggest that DOL’s 

data validation efforts have not prevented high error rates on certain data 

elements for select variables—nationally, error rates for certain variables 

have remained well above the 5 percent threshold over both program 

years. For example, in program year 2010, across all states, about 16 

percent of the files for the Adult Program for which the “date of program 

exit” variable was reviewed had errors, compared to around 14 percent in 

program year 2011.50 Similarly, the nationwide error rate for “date of first 

staff-assisted core service” was above 7 percent in both program years 

for both the Adult and the Dislocated Worker Programs.51

DOL’s required annual data validation efforts are resource-intensive and 

time-consuming both for DOL regions and states, according to officials 

from DOL’s national office, two DOL regional offices, and five states. 

However, DOL has not yet evaluated the process or determined its effect 

on data quality. Specifically, an official from one state workforce agency 

When asked, 

officials from DOL’s national office and two regional offices explained that 

variables containing dates frequently have high error rates due to 

discrepancies between the date reported and the date in the source 

documentation. If a date, such as date of dislocation—the date a worker 

lost his or her job—in the hard copy document differs from the date in the 

electronic record, even by one day, the variable for that record “fails” the 

validation check. Moreover, officials from DOL’s national office and three 

regional offices stated that high error rates resulting from such 

discrepancies do not necessarily reflect any serious issues with the 

reported data—a participant would still be a dislocated worker whether, 

for example, the date of dislocation was June 20th or June 21st. 

Nonetheless, DOL requires these dates to match precisely and errors 

noted in program year 2011 for these variables were still prevalent 

although similar errors were noted the previous year.

 

50We are referring to WIASRD variable 303: Date of Exit. 

51We are referring to WIASRD variable 332: Date of First Staff Assisted Core Service. The 

error rates presented are “reported data error rates.” Data element validation produces an 

overall estimate of the error rate for each data element that has been selected for 

validation. Because certain data elements may not be present in every sampled record, 

data validation also produces a reported data error rate, which includes in the 

denominator only those records for which the particular data element was validated. In this 

situation, the error rate equals the number of records in error divided by the total number 

of records for which the particular data element was validated, weighted to account for the 

over- and under-sampling of particular records.
 

Page 26 GAO-14-4 Numbers Served in WIA Programs

estimated the cost of its annual data validation efforts, including staff time, 

travel, and other expenses, to be about $200,000. Officials in another 

state explained that they would like DOL to reduce the required sample 

size for the required validation of the data elements in WIASRD in order 

to reduce the administrative burden on states. In addition, in 2011 one 

DOL regional office convened a workgroup of representatives from four 

states that analyzed the data validation procedures and provided 

recommendations to DOL’s national office for improvements to reduce the 

administrative burden on states. These recommendations included 

considering using alternative sampling methods, revisiting the frequency 

and precision requirements of data validation, and issuing guidance to 

share “best practices” across states and local areas. However, as of 

September 2013, DOL had not implemented the workgroup’s 

recommendations, and the data validation process remains unchanged. 

Officials from DOL’s national office acknowledged the trade-off between 

monitoring data quality and minimizing the administrative burden on 

states. They said that revising WIASRD’s edit checks to allow states more 

flexibility may result in making the data validation process more efficient 

by, for example, permitting states to report a range of dates, if 

appropriate, for certain variables.

DOL’s regional offices review a sample of case files from states as part of 

their oversight of the quality of data for the WIA Adult and Dislocated 

Worker Programs, but they have not used the results of these reviews to 

identify systemic issues with the quality of the data on WIA participants. 

According to officials from DOL’s national office, its regional offices are 

responsible for providing feedback to states based on these reviews, and 

it is not the national office’s role to conduct any type of systemic review to 

identify cross-state data issues. Government standards for internal 

controls state that for oversight and monitoring to be effective, information 

should be communicated to management and others, along with the use 

of this information for program assessment, so that it can be used to carry 

out their internal control and other responsibilities.52

In response to a recommendation from a prior GAO report, DOL began to 

require its regional offices to review a sample of case files to monitor 

states’ annual data validation procedures.53

 

52

In addition, over the past few 

GAO/AIMD-00-21.3.1. 

53GAO-06-82. According to DOL officials, these reviews occur every 2 to 3 years.

Reviews by Regional Offices 
 

Page 27 GAO-14-4 Numbers Served in WIA Programs

years, to address concerns about the nationwide consistency of 

monitoring activities, DOL has issued additional guidance to its regional 

offices on the process that should be used in reviewing the case files.54

Officials from DOL’s national office said that, while they discuss the 

results of these reviews with regional officials and provide state-specific 

technical assistance as needed, they do not have a regular, formal 

process for analyzing the findings from these reviews by their regional 

offices, including determining whether similar findings and areas of 

concern were identified across states. DOL officials explained that, 

because the reviews are part of the regional offices’ oversight of the 

states, they believe that the national office should not be involved in 

Officials from all six of DOL’s regional offices reported using these 

materials when they design and conduct their reviews of the case files. 

The review process begins when officials from DOL’s regions review a 

state’s most recent data validation report and identify a subsample of 

participant case files from the most recent review by the state. In addition 

to checking the data in the electronic records by comparing them to the 

source documentation, DOL’s regional staff assess whether the state 

followed the proper procedures in conducting its annual data validation 

efforts, according to officials from DOL’s national office. The process 

concludes with a report outlining the DOL regional office’s findings, 

including non-compliance with statutory or regulatory requirements for 

collecting and reporting data on WIA participants. Many of the reports 

also identify areas of concern, such as when states do not share their 

annual data validation results with local area staff from American Job 

Center partner programs, and identify promising practices observed 

during the reviews. State officials have 45 days to respond to findings in 

the region’s report and are also encouraged but are not required to 

respond to areas of concern detailed in the report. In addition, officials 

from DOL’s national office noted that states with high error rates on select 

variables are encouraged to inform the regional offices of how they plan 

to reduce their error rates in the future.

 

54U.S. Department of Labor, Office of Inspector General, 03-09-003-03-390; U.S. 

Department of Labor, Core Monitoring Guide (Washington, D.C.: April 2005); U.S. 

Department of Labor, Core Monitoring Guide: Financial Supplement (Washington, D.C.: 

October 2007); U.S. Department of Labor, Core Monitoring Guide: Data Validation 

Supplement (Washington, D.C.: March 2011). U.S. Department of Labor, Core Monitoring 

Guide with the Formula Grant Supplement (Washington, D.C.: April 2012); U.S. 

Department of Labor, Data Reporting and Validation System (DRVS): User Handbook for 

DRVS 7.2 (Washington, D.C.: April 2009). 
 

Page 28 GAO-14-4 Numbers Served in WIA Programs

monitoring the results of the reviews or the way in which they are 

conducted. As a result, DOL does not have a systematic means of 

determining the importance of the findings, their prevalence, or their likely 

effect on the quality of the national data on participants in the WIA Adult 

and Dislocated Worker Programs. This limits DOL’s ability to respond to 

data issues that are systemic or widespread. Table 2 summarizes our 

analysis of the most prevalent issues identified during the most recent 

reviews for each of the 53 states and territories.

Table 2: Most Prevalent Issues Identified During DOL’s Regional Reviews of States’ 

Case Files, Program Years 2006-2011a

Findings or Areas of Concern by Category

Number of Jurisdictions 

with Finding or Area 

of Concern (53 total)

Issues relating to inadequate, outdated or otherwise 

limited state information systems 

26

Data reporting 47

Failure to report required information or retain 

documentation

36

Issues with reported dates of participation,

service or exit

30

Incorrect policies or practices regarding 

participant information, such as eligibility

20

Issues relating to sequence of services 9 

Data element validation process 49

Incorrect data validation procedures 39

High error rates on one or more data elements 37

Insufficient documentation 23

Failure to use results to make improvements in 

data quality

9 

Source: GAO analysis of DOL’s case file review reports, program years 2006-2011.

a

While the most recent review for Arkansas looked at data from program year 2006, most of the 

reviews looked at data from program years 2008, 2009, 2010, or 2011. In addition, most reviews 

covered either 1 or 2 years of data. 

Over the past few years, DOL has issued additional guidance and 

provided technical assistance to states and local areas, including training 

and webinars, to clarify and explain the requirements for collecting and 

reporting data for the WIA Adult and Dislocated Worker Programs. 

However, some state officials said that DOL’s technical assistance is not 

always timely, and that DOL could do more to facilitate the collection or 

Technical Assistance 
 

Page 29 GAO-14-4 Numbers Served in WIA Programs

sharing of promising data collection and reporting practices across 

regions and states.

For example, DOL has provided general technical assistance on data 

reporting for the WIA Adult and Dislocated Worker Programs to states, 

and officials from three of the eight states said that the assistance 

provided by DOL’s regional offices was useful in helping them address 

some of the challenges related to data reporting. In particular, officials 

from DOL’s national office said that some regional offices issue quarterly 

performance letters to states that include program year performance data 

and any related analysis, in addition to hosting quarterly phone 

conferences with state performance specialists to discuss performance 

issues. DOL also sponsors the Workforce3one website, which contains a 

variety of training and background materials related to data collection and 

reporting for WIASRD. National and regional DOL data specialists also 

said they hold biweekly meetings to discuss data issues. In addition, DOL 

officials from each region described regular communication they have 

with state officials to provide technical assistance in response to specific 

data reporting issues, and officials from five of the six regional offices 

described conference calls that they have hosted as opportunities for 

states to discuss challenges related to the quality of their data on WIA 

participants. Furthermore, DOL’s national and regional offices have 

access to an internal data system, Infospace, which allows them to 

retrieve and review publicly available WIASRD data by state and local 

area.

Over the past few years, DOL has also issued a number of Training and 

Employment Guidance Letters and Training and Employment Notices 

related to data collection and reporting for WIA. In 2011, DOL developed 

and hosted a series of webinars for states and local areas, including one 

on data validation and data quality issues.55

 

55U.S. Department of Labor Employment and Training Administration. Fiscal and 

Performance Training Forum 2011: Data Validation—Looking Back and Moving Forward

(Washington D.C.: June 17, 2011), 122.

The materials from this 

session describe the approach DOL uses to monitor the data and explore 

issues and findings across states. They also highlight how data validation 

can be used to improve data systems and provide information on the 

guidance regional offices use to review states’ case files. While the 

webinar materials present information on some of the consistent findings 

across states—such as incorrect source documentation and exit dates—
 

Page 30 GAO-14-4 Numbers Served in WIA Programs

DOL officials said that they have not provided any additional formal 

technical assistance to address these issues on a national level because 

of resource limitations. Moreover, they said it would be difficult for DOL to 

provide such assistance without having first reviewed the results of its 

own monitoring efforts. DOL’s webinar also noted that DOL would 

establish a working group in the summer of 2011 to look into and possibly 

revise the source documentation requirements for the annual data 

validation process. However, as of the summer of 2013, DOL officials 

said that this working group had not been established due to competing 

priorities and resource constraints.

Officials from two states noted that DOL’s updates to its guidance for 

collecting and reporting the data are often not provided far enough in

advance to be implemented by the time changes take effect. For 

example, officials from California said that they often do not have enough 

lead time to properly implement new data elements or guidance when it is 

issued by DOL. According to state officials, the state generally has 1 

month or less to change its automated system to meet deadlines, which is 

not enough time. In addition, officials from Georgia said the data 

validation reports they currently receive from DOL are not timely since 

they are at least 3 months old by the time the state receives the reports. 

Some state officials also told us that they would benefit from learning how 

their peers are addressing challenges in reporting data on the WIA 

programs. Officials from DOL’s national office, however, told us they do 

not currently facilitate the collection or sharing of promising data collection 

and reporting practices across regions or states, and have no plans to do 

so—because of competing resources, the agency’s main focus is on 

service delivery rather than data collection. Best practices state that highperforming organizations continually assess performance and efforts to 

improve performance. In particular, managers can use performance 

information to identify and share more effective processes and

approaches.56

 

56GAO-05-927. 
 

Page 31 GAO-14-4 Numbers Served in WIA Programs

In addition to its regular monitoring activities, DOL has taken specific 

steps to improve the consistency of the WIA data collected by states and 

local areas. For example, the agency has developed an integrated data 

reporting system, which is being piloted in two states.57

To standardize and streamline reporting across several of DOL’s 

workforce programs—the Wagner-Peyser Program, the WIA Adult, 

Dislocated Worker and Youth Programs, Veterans Employment and 

Training Service, National Emergency Grants, and Trade Adjustment 

Assistance Programs—the agency has developed an integrated data 

reporting system, WISPR. Two states, Pennsylvania and Texas, have 

been piloting WISPR since 2007 to collect and report data on the WIA 

Adult and Dislocated Worker Programs, and a third state—Utah—plans to 

start piloting WISPR in the fall of 2013. Officials from DOL’s national 

office and from Utah explained that one of WISPR’s key advantages over 

the current separate reporting systems for each program is that WISPR 

has standardized variables that include all the required variables for each 

program.

However, DOL 

has not evaluated the results of the pilot program to determine whether it 

has had a positive effect on the quality of participant data for the WIA 

Adult and Dislocated Worker Programs and, despite not having evaluated 

its effectiveness, plans to expand the program to additional states.

58

To facilitate a future transition to WISPR in the remaining states, DOL has 

obtained approval from the Office of Management and Budget to modify 

Utah officials stated that they believe WISPR will be an 

improvement over their current system because future changes in DOL’s 

guidance for any of the workforce programs it administers—including the 

WIA Adult and Dislocated Worker Programs—would be incorporated into 

a single system, facilitating implementation of these changes.

 

57These programs include the Wagner-Peyser, WIA Adult, Dislocated Worker, and Youth, 

Jobs for Veterans State Grants, National Emergency Grants, and Trade Adjustment 

Assistance Programs. 

58Although DOL’s guidance for reporting data in WIASRD encourages states and local 

areas to provide integrated services through multiple programs, each program has its own 

reporting requirements, according to officials from DOL’s national office. As a result, it is 

not possible to track individual job seekers who receive services from multiple programs 

across the workforce system, or to determine the proportion of resources provided by 

each program for a particular service, or to attribute participant outcomes to those 

programs.

DOL Has Taken Steps to 

Improve Data Consistency, 

but It Has Not Evaluated 

These Efforts
 

Page 32 GAO-14-4 Numbers Served in WIA Programs

the current record layout of WIASRD to match that of WISPR.59 DOL 

officials told us they expect to implement the revised WIASRD record 

layout in the fall of 2013. However, as of August 2013, they said it is not 

clear whether or when nationwide implementation of WISPR will occur 

because this depends on the resources available to upgrade both federal 

and state information systems and the associated programming costs. 

While WISPR appears to offer advantages over the current reporting 

system that might make it a promising step forward, DOL does not 

currently have plans to evaluate the results of the pilot program to 

determine whether it has had a positive effect on the completeness and 

consistency of participant data for the WIA Adult and Dislocated Worker 

Programs before expanding the program to other states. DOL officials 

cited the agency’s limited resources as the reason for not planning an 

evaluation of the WISPR pilot program before expanding it to other states. 

However, best practices note that evaluation can play a key role in 

program planning, management, and oversight by providing feedback to 

program managers, legislative and executive branch policy officials, and 

the public. Further, when pilot programs are designed to produce 

change—such as by allowing for more streamlined data collection and 

reporting—assessing the impact is essential for knowing if the pilot is 

meeting its goals.60 Without an evaluation of WISPR, DOL will not know if 

this data system has resulted in the collection of more accurate WIA 

participant data when compared to WIASRD.61

Finally, DOL administers two grant programs that states can use to 

improve their WIA information systems: the Workforce Innovation Fund 

 

59This would involve adding variables to WIASRD that would allow it to capture all the 

data required by any of the workforce programs included in WISPR. The modified 

WIASRD layout would utilize variable definitions identical to those of WISPR and is 

designed to facilitate full scale implementation of WISPR.

60GAO/PEMD-95-1, Program Evaluation: Improving the Flow of Information to the 

Congress (Washington D.C.: Jan. 30, 1995). 

61GAO-13-570, Program Evaluation: Strategies to Facilitation Agencies’ Use of Evaluation 

in Program Management and Policy Making (Washington D.C.: June 16, 2013).
 

Page 33 GAO-14-4 Numbers Served in WIA Programs

and the Workforce Data Quality Initiative.

62 DOL’s Workforce Innovation 

Fund is a competitive grant program that supports innovative approaches 

to the design and delivery of employment and training services. Although 

it is not targeted specifically at information systems, at least 3 of the 26 

grants awarded by DOL have been used for local initiatives to integrate 

their workforce data systems. For example, one local area we visited in 

Chicago received a Workforce Innovation Fund grant that they plan to use 

to integrate the different data systems used by the workforce programs in 

their local area. Officials said that they hope the improvements will result 

in more data-driven decisions about service delivery, and that all 

employment and training programs in their local area will be able to share 

information electronically. Another grant program, DOL’s Workforce Data 

Quality Initiative, is also not specifically aimed at WIA data reporting but 

may have positive incidental effects on WIA data quality, according to 

DOL officials. The purpose of this initiative is to create a longitudinal 

database to chart individuals’ progress through the education system and 

beyond to the labor market. This effort would entail upgrades to state 

information systems that may resolve some data reporting issues 

currently attributed to limited technological capacities. At this time, it is too 

early to know whether these grants will have a positive effect on the 

quality of WIA participant data.

Collecting and reporting consistent and complete data is important for 

program oversight and management and to evaluate the effectiveness of 

program activities and services, but it can be difficult when federal 

programs are carried out in partnership with states and local areas. DOL 

has taken steps to improve the quality of the data on WIA’s Adult and 

Dislocated Worker Programs. However, the flexibilities in DOL’s 

guidance, reflective of those inherent in the programs’ authorizing statute, 

which allows states flexibility in program design, along with limitations in 

state information systems, present challenges to DOL in collecting and 

reporting consistent and complete data on a unique count nationwide of 

participants in the WIA Adult and Dislocated Worker Programs. Without 

 

62In June 2012, DOL awarded nearly $147 million in Workforce Innovation Fund grants to 

states to develop and expand innovative strategies to help Americans return to work by 

delivering services more efficiently and facilitating cooperation across programs and 

funding streams. Grant awards ranged from about $1.4 million to $12 million. In June 

2013, DOL awarded $6.4 million in state grants for the third round of the Workforce Data 

Quality Initiative designed to improve the quality and availability of workforce data. Grant 

awards for that program ranged from about $700,000 to about $1.2 million.

Conclusions
 

Page 34 GAO-14-4 Numbers Served in WIA Programs

such data, policymakers, program officials, and other stakeholders have 

an incomplete picture of the number of adults and dislocated workers 

served, their characteristics, and the type and level of services received.

In addition, while DOL engages in several types of oversight activities 

designed to ensure the accuracy of states’ data on participants in the WIA 

Adult and Dislocated Worker Programs, it does not consistently share the 

results of its oversight activities with states and local areas. As a result, 

states and local areas are not always aware of potential data quality 

issues, and may miss opportunities to improve their data collection and 

reporting. Moreover, since 2007, two states have been piloting a new 

information system that tracks program participants across several of 

DOL’s employment and training programs, but DOL does not plan to 

evaluate its effects on the quality of the data collected on participants in 

the WIA Adult or Dislocated Worker or other programs before it expands 

the system to other states. Similarly, DOL does not regularly collect and 

disseminate promising practices to states and local areas, which could 

facilitate the adoption of steps other states and local areas have taken to 

improve their data collection and reporting efforts. While it may not be 

possible to achieve 100 percent precision and accuracy in the data 

reported on participants in a large, complex system like the workforce 

investment system, by not appropriately targeting their available 

resources and facilitating sharing of promising practices among states to 

continuously try to improve the quality of the data, DOL misses an 

opportunity to identify and address longstanding, systemic issues.

1. To improve the consistency and completeness of national data on 

participants in the WIA Adult and Dislocated Worker Programs, we 

recommend that the Secretary of Labor take additional steps to 

improve the uniformity of participant data reported by states. These 

steps could include the following:

a. providing additional guidance to states on data reporting, such as 

how core and intensive services should be recorded for WIA 

participants who receive these services through partner programs; 

and

b. conducting an evaluation or review of WISPR to determine if it has 

resulted in more complete and consistent data collection and 

reporting for participants in the WIA Adult and Dislocated Worker 

Programs and placing a high priority on the implementation of 

Recommendations for 

Executive Action
 

Page 35 GAO-14-4 Numbers Served in WIA Programs

WISPR if it is shown to improve data consistency and 

completeness.

2. We also recommend that the Secretary of Labor promote a formal,

continuous process for improving the quality of data on participants in 

the WIA Adult and Dislocated Worker Programs through such 

measures as the following:

a. consistently sharing the results of all oversight activities with 

states and local areas, including findings from validation of 

participant data;

b. reviewing the methods used for data validation, such as its scope 

and error rate threshold, to identify opportunities to increase 

efficiencies and accountability in the process. This could include 

implementing, if appropriate, recommendations from the Regions’ 

review of data validation procedures;

c. evaluating data validation efforts to determine their effects on data 

quality, particularly on systemic errors, and providing targeted 

guidance and assistance to states and local areas to address 

such errors;

d. regularly monitoring Social Policy Research Associates’ 

corrections and analyses of state WIA participant data, sharing 

this information with states, and coordinating with states to ensure 

that any corrections are appropriate and accurate; and

e. collecting and disseminating promising practices to states and 

local areas on data collection and reporting on a regular basis. 

We provided a draft of this report to officials at DOL for their review and 

comment. We received written comments from DOL, which are 

reproduced in their entirety in appendix III. DOL officials did not state 

whether they agreed or disagreed with our recommendations. These 

officials acknowledged the importance of having reliable data to

effectively manage and evaluate the WIA Adult and Dislocated Worker 

Programs; however, they commented that data reliability should be 

balanced with the flexibility WIA gives to states and with DOL’s 

responsibility to prioritize use of its limited resources. They stated that 

WIA provides states and local areas with the flexibility to serve their

customers in the way that best suits their particular needs. DOL officials 

also stated that the agency has invested significant resources in its 

workforce performance accountability system, especially for WIA 

programs. According to officials, the agency has a robust system in place 

Agency Comments 

and Our Evaluation
 

Page 36 GAO-14-4 Numbers Served in WIA Programs

to ensure data quality and reliability and has recently made several 

enhancements to the reporting system. In their comments, DOL officials 

detailed various efforts they plan to take to address our 

recommendations. Nonetheless, we believe that these efforts will not 

sufficiently address the specific data quality issues we identified and 

encourage DOL to take more targeted steps as outlined in our 

recommendations. 

In response to our first recommendation, DOL officials said that they 

believe the agency’s current guidance is clear but that they will continue 

to work with states to develop additional guidance, as necessary, such as 

forthcoming guidance on how to avoid duplication of services when coenrolling participants across multiple partner programs. However, it is 

important that any additional guidance also specify when to count job 

seekers as WIA participants if they also receive services funded by 

partner programs. We also encourage DOL to develop additional 

guidance for the WIASRD variables noted in our report that are open to 

interpretation, such as “type of training,” to facilitate consistent reporting 

on participants in these programs. DOL officials also noted that an 

evaluation of WISPR is subject to the agency’s resource constraints, 

adding that the purpose of WISPR was never explicitly to improve data 

quality. As we stated in our report, however, WISPR seeks, in part, to 

improve the consistency of WIA data by standardizing reporting across 

the workforce system. As such, it has the potential to improve data 

quality. Therefore, we encourage DOL to evaluate the system in order to 

make an informed decision on how best to allocate finite agency 

resources going forward.

In response to our second recommendation, DOL officials stated that they 

consistently share the results of the agency’s oversight activities with 

states but acknowledged that more could be done to analyze the results 

of its activities to identify and share similar findings and areas of concern 

across multiple states. DOL officials added that they will work with the 

regional offices towards this goal. With regard to the agency’s data 

validation methods, officials said they regularly review these methods and 

solicit input from states on how to improve them. Specifically, DOL 

pointed out that, as required by the Paperwork Reduction Act, the Office 

of Management and Budget reviews DOL’s data validation process every 

3 years and solicits public comment before approving the methodology 

and authorizing data collection, and that their 2014 submission will reflect 

state input. However, given the time-consuming nature of data validation, 

we believe the agency should take additional actions to review its current 
 

Page 37 GAO-14-4 Numbers Served in WIA Programs

methods specifically with an eye toward making them more efficient and 

holding states accountable for their data validation results. 

With regard to evaluating the effectiveness of its data validation efforts, 

DOL officials said that the agency plans to consider the regional data 

validation workgroup’s findings and recommendations from 2011, explore 

ways to streamline the process, and examine the effect of data validation 

on error rates. We commend DOL’s plans, but to adequately address the 

persistently high data error rates we found in our analysis, we believe it is 

necessary to go beyond evaluating the effectiveness of its data validation 

efforts and pinpoint the underlying cause of the errors so that they can be 

addressed. 

Officials also pointed out that the agency already monitors and shares the 

analyses of state data conducted by its contractor, Social Policy 

Research Associates (SPRA). They stated that SPRA’s corrections of 

states’ data have been publicly available with the data set from the 

inception of WIA. They also noted that since program year 2011 DOL has 

provided SPRA’s analyses and corrections to the states (through ETA 

Regional Offices) on a quarterly basis for states to either correct or 

dispute. They noted that this is a formal and recurring process, and that 

ETA Regional Offices have begun to analyze state date WIASRD data on 

a regular basis as part of their annual review cycles. However, we found 

that not all states are aware of or receive copies of SPRA’s reports, and 

that some of the corrections SPRA makes to the state WIASRD data files 

may not be accurate. Furthermore, as noted in our report, DOL officials 

told us that they do not conduct formal oversight reviews or audits of 

SPRA’s data analyses. 

In addition, DOL officials reiterated that data collection and reporting are 

topics that are included in workforce3one, a point we noted in our report. 

However, we maintain that DOL could do more to facilitate the sharing of 

information across states, such as creating a forum through which states 

could learn how their peers are addressing challenges in data reporting 

for participants in the WIA programs. Finally, DOL provided technical 

comments, which we incorporated, as appropriate. 

As agreed with your offices, unless you publicly announce the contents of 

this report earlier, we plan no further distribution until 30 days from the 

report date. At that time, we will send copies to the appropriate 

congressional committees, the Secretary of Labor, and other interested 
 

Page 38 GAO-14-4 Numbers Served in WIA Programs

parties. In addition, the report will be available at no charge on the GAO 

website at http://www.gao.gov. 

If you or your staff have any questions concerning this report, please 

contact me at (202) 512-7215 or moranr@gao.gov. Contact points for our 

Offices of Congressional Relations and Public Affairs may be found on 

the last page of this report. Key contributors to this report are listed in 

appendix IV.

Revae Moran, Director 

Education, Workforce, and Income Security Issues
Appendix I: Scope and Methodology

Page 39 GAO-14-4 Numbers Served in WIA Programs

Our objectives were to determine: (1) what factors have affected the 

ability to report consistent and complete data on participants in the 

Workforce Investment Act (WIA) Adult and Dislocated Worker Programs, 

and (2) what actions has the Department of Labor (DOL) taken to improve 

the quality of participant data. To address our objectives, we reviewed 

applicable laws and regulations, as well as DOL’s guidance to states for

collecting and reporting data on participants in the WIA Adult and 

Dislocated Worker Programs.1 We also interviewed officials from DOL’s 

Employment and Training Administration, its Office of Inspector General, 

and its six regional offices. In addition, we visited or telephoned a 

nongeneralizable sample of eight states. Within each state, we visited or 

contacted at least one American Job Center—formerly known as a onestop center—or a local workforce board. We also assessed the reliability 

of program year 2011 data from the Workforce Investment Act 

Standardized Record Data (WIASRD) system by testing the data 

electronically and interviewing knowledgeable agency officials and DOL’s 

data contractor.2

To better understand the data reporting challenges faced by states and 

the actions DOL has taken to improve the quality of participant data, we 

interviewed officials from a nongeneralizable sample of eight states.

We found the data in appendix II to be sufficiently 

reliable for the purposes of providing estimates of the number of, 

characteristics of, and services provided to participants whose information 

is recorded by DOL as having received services from either the WIA Adult 

Program or the WIA Dislocated Worker Program. The data are not 

reliable for other purposes, such as making state-to-state comparisons, 

because of variations in how states collect and report data on participants 

in the WIA Adult and Dislocated Worker Programs.

3

 

1

For this study, we define “WIA program participants” as individuals who have a valid Date 

of Program Participation for the WIA Adult or Dislocated Worker Programs and who have 

been counted and reported to DOL as WIA Adult and Dislocated Worker Program 

participants. Participant data are typically collected by local staff at American Job Centers 

and entered into a state or local data system. After the state receives data from local 

areas, it compiles and formats the information and submits it electronically to DOL.

Within each state, we visited or contacted at least one American Job 

Center or the local workforce entity. We conducted in-person interviews 

2

Program year 2011 runs from July 1, 2011 through June 30, 2012. WIASRD is a national 

database of individual records containing information on the characteristics, activities, and 

outcomes for all enrolled participants who received services or benefits under WIA. 

3

We conducted these interviews between September 2012 and June 2013. 

Appendix I: Scope and Methodology

Selected States 
Appendix I: Scope and Methodology

Page 40 GAO-14-4 Numbers Served in WIA Programs

with DOL officials in Regions 1 (Boston), 3 (Atlanta), and 5 (Chicago); 

state and local workforce officials in California, Georgia, Illinois, 

Massachusetts, and Washington; and American Job Center officials in 

Maryland. We conducted telephone interviews with DOL officials in 

Regions 2 (Philadelphia), 4 (Dallas), and 6 (Sacramento), and with state 

workforce officials in Maryland, South Dakota, and Utah.4 We 

nonstatistically selected these states to provide diversity on the basis of: 

(1) geographic location, (2) total federal spending on the Adult and 

Dislocated Worker Programs in program year 2010, (3) the extent of data 

issues identified in the fourth quarter of program year 2010, (4) whether 

the state reported participants who only received core self-services, and 

(5) the number of local areas within the state.5

Table 3: Selected Site Visit States and Local Areas 

State Local Area City

California Sacramento City/County Consortium Sacramento 

 San Joaquin County Stockton 

Georgia Cobb County Marietta

 Fulton County Sandy Springs

Illinois Cook County Chicago

Maryland Prince George’s County Largo

Massachusetts Western Massachusetts Springfield

South Dakota

Utah 

Washington King County Seattle

Source: GAO state and local area interviews

In each state, we obtained general information about the state’s and the 

local area’s implementation of the WIA Adult and Dislocated Worker 

Programs and on any challenges they may have encountered in 

 

4

We did not interview local officials in Utah and South Dakota because, for both states, 

the entire state is considered to be a single service delivery area and there are no 

designated local areas. 

5

We used DOL data from program year 2010 to identify total federal spending on the Adult 

and Dislocated Worker Programs and the extent of data issues because they were the 

most recent data available when we selected our sample of states. Program year 2010 

ran from July 1, 2010 through June 30, 2011. Core “self-services” include services that 

can be provided without significant staff assistance.
Appendix I: Scope and Methodology

Page 41 GAO-14-4 Numbers Served in WIA Programs

collecting and reporting data on program participants. We also asked 

about actions DOL has taken to improve the quality of the data on 

participants in the WIA Adult and Dislocated Worker Programs. We used 

semi-structured interviews for our regional, state, and local interviews. 

Because we interviewed officials from a nongeneralizable sample of eight 

states and selected local areas, we cannot generalize our findings 

beyond the data collected on those states and local areas.

To assess the reliability of DOL’s data in the WIASRD database for 

participants in the WIA Adult and Dislocated Worker Programs, we (1) 

reviewed existing documentation related to the data sources, including 

reports issued by DOL’s Office of Inspector General; (2) electronically 

tested the WIASRD data to identify potential problems with consistency, 

completeness, or accuracy; and (3) interviewed DOL’s data contractor 

and knowledgeable agency officials to obtain information about the data.

Our electronic testing consisted of identifying inconsistencies, outliers, 

missing values, and other errors. More specifically, the electronic testing 

included assessing the reliability of data collected on the characteristics 

and the services participants in the WIA Adult and Dislocated Worker 

Programs received in program year 2011.6

 

6

Our analysis included all participants in the WIA Adult and Dislocated Worker Programs 

whose participation date was before July 1, 2012 and who had either not exited the 

programs or exited after July 1, 2011. We analyzed program year 2011 data because they 

were the most recent full year of WIASRD data available when we conducted our review. 

Program year 2011 ran from July 1, 2011 through June 30, 2012. 

Prior to testing the data, we 

combined 160 records that were overlapping or duplicative into 80 unique 

records and removed 19 records that had missing or erroneous 

participation dates. A few variables, including data on a participant’s 

dislocation date—the date a worker lost his or her job, and the 

occupational codes for participants who completed training, were found to 

not be sufficiently reliable for our purposes and were not included in our 

report. In addition, we analyzed the publicly available WIASRD data file 

for program year 2011, which was produced for DOL by its data 

contractor, Social Policy Research Associates. As part of our analysis, we 

reviewed the steps the data contractor took to correct the data and, to the 

extent possible, compared our data to the publicly available file. In light of 

variations in how states collect and report participant data for the WIA 

Adult and Dislocated Worker Programs and limitations in their information 

systems, the actual number of participants in these programs is unknown. 

Analysis of DOL’s Participant 

Data
Appendix I: Scope and Methodology

Page 42 GAO-14-4 Numbers Served in WIA Programs

However, we were able to estimate the number of, characteristics of, and 

services provided to participants whose information is recorded by DOL 

as having been served by the WIA Adult and Dislocated Worker 

Programs using WIASRD data from program year 2011.

To describe and assess DOL’s oversight and monitoring efforts, we 

reviewed technical assistance guides and material posted to 

Workforce3One, including DOL’s Core Monitoring guides and Data 

Validation Reporting System guidance. We also interviewed officials from 

DOL’s Employment and Training Administration national office and from 

all of DOL’s six regional offices. In addition, we obtained and reviewed 

copies of DOL’s monitoring reports, including the results of DOL’s 

program year 2010 and 2011 annual data validation efforts and the most 

recent case file review for each state and territory.7 To analyze the results 

of the annual data validation efforts, we calculated the average reported 

error rate for each variable across states.8 We included in our analysis all 

variables on characteristics and services.9

We conducted this performance audit from August 2012 through 

November 2013 in accordance with generally accepted government 

auditing standards. Those standards require that we plan and perform the

audit work to obtain sufficient, appropriate evidence to provide a 

To analyze trends in the 

results of the case file reviews, we reviewed the findings and areas of 

concern identified in each review and categorized them to identify 

common issues present in multiple states.

 

7

These reviews typically occur every 3 to 4 years. The date of the reviews we analyzed 

ranged from program year 2006 to program year 2011.

8

The error rates presented are “reported data error rates.” Data element validation 

produces an overall estimate of the error rate for each data element that has been 

selected for validation. Because certain data elements may not be present in every 

sampled record, data validation also produces a reported data error rate, which includes in 

the denominator only those records for which the particular data element was validated. In 

this situation, the error rate equals the number of records in error divided by the total 

number of records for which the particular data element was validated, weighted to 

account for the over- and under-sampling of particular records.

9

We reviewed the results of 11 data elements from the Adult Program and 12 from the 

Dislocated Worker Program. Data elements included in both program are 

ProgramParticipationDate, FirstCoreServiceDate, FirstIntensiveService, 

NeedyFamilyStatus, DateEnterTraining, DateExitTraining, TrainingService1, 

VeteranStatus, OtherExitReasons, and ProgramExitDate. The Adult Program also 

included LowIncomeStatus and the Dislocated Worker Program also included 

DislocationDate and DisplacedHomemakerIndicator.

Analysis of DOL’s Oversight 

and Monitoring Efforts
Appendix I: Scope and Methodology

Page 43 GAO-14-4 Numbers Served in WIA Programs

reasonable basis for our findings and conclusions based on our audit 

objectives. We believe that the evidence obtained provides a reasonable 

basis for our findings and conclusions based on our audit objectives.
Appendix II: Data on Participants in the WIA 

Adult and Dislocated Worker Programs for 

Program Year 2011

Page 44 GAO-14-4 Numbers Served in WIA Programs

We reviewed the data collected by the Department of Labor (DOL) in the 

Workforce Investment Act Standardized Record Data (WIASRD) system 

on the number of, characteristics of, and services provided to participants 

in the WIA Adult and Dislocated Worker Programs in program year 2011.1

Figure 4: Total Participants Enrolled in the Adult and Dislocated Worker Programs, 

Program Year 2011

We found the data to be sufficiently reliable for the purposes of providing 

estimates of the number of, characteristics of, and services provided to, 

participants whose information is recorded by DOL in WIASRD. These 

estimates are presented in figures 4 through 9.

 

1

WIASRD is a national database maintained by DOL of individual records containing 

information on the characteristics, activities, and outcomes for all participants enrolled in 

WIA programs who received services or benefits under WIA. We analyzed program year 

2011 data because they were the most recent full year of WIASRD data available when 

we conducted our review. Program year 2011 ran from July 1, 2011 through June 30, 

2012.

Appendix II: Data on Participants in the WIA 

Adult and Dislocated Worker Programs for 

Program Year 2011
Appendix II: Data on Participants in the WIA 

Adult and Dislocated Worker Programs for 

Program Year 2011

Page 45 GAO-14-4 Numbers Served in WIA Programs

Figure 5: Highest Level of Service Received by Adult and Dislocated Worker 

Program Participants, Program Year 2011a 

a

Numbers may not add to 100 percent due to rounding. Nine states reported that none of their Adult 

Program participants received only core self-services and informational activities and two states 

reported that no participants in their Adult or Dislocated Worker Program received staff assisted core 

services.
Appendix II: Data on Participants in the WIA 

Adult and Dislocated Worker Programs for 

Program Year 2011

Page 46 GAO-14-4 Numbers Served in WIA Programs

Figure 6: Duration of Training for Adult and Dislocated Worker Program 

Participants Who Completed or Withdrew from Training, Program Year 2011a 

a

Our analysis was restricted to participants who completed or withdrew from training. 
Appendix II: Data on Participants in the WIA 

Adult and Dislocated Worker Programs for 

Program Year 2011

Page 47 GAO-14-4 Numbers Served in WIA Programs

Figure 7: Type of Training Services Provided to Adult and Dislocated Worker 

Program Participants, Program Year 2011a 

a

Our analysis was restricted to participants who received training. 

Figure 8: Percentage of Adult and Dislocated Worker Program Participants 

Registered in Selected Partner Programs, Program Year 2011a 
Appendix II: Data on Participants in the WIA 

Adult and Dislocated Worker Programs for 

Program Year 2011

Page 48 GAO-14-4 Numbers Served in WIA Programs

a

Other partner programs include the Adult Education, Job Corps, National Farmworker Jobs, Indian 

and Native American, Veterans, Vocational Education, Vocational Rehabilitation, Youthbuild, Title V 

Older Worker, and Employment and Training Services Related to Food Stamps Programs. States are 

not required to report whether participants in the WIA Adult or Dislocated Worker Program received 

services from these partner program. Seventeen states reported that virtually no WIA Adult Program 

participants received services from any of these programs. 

Figure 9: Adult and Dislocated Worker Program Participant Demographics, Program Year 2011a 

a

Participants in the WIA Adult and Dislocated Worker Programs who did not disclose their gender 

were not included in the percentages listed above for gender. About 11 percent of participants in the 

Adult Program and less than 1 percent of participants in the Dislocated Worker Program did not 

report their gender. Percentages of ethnicity and race do not add up to 100 because individuals can 

identify in more than one category. 
Appendix III: Comments from the Department 

of Labor

Page 49 GAO-14-4 Numbers Served in WIA Programs

Appendix III: Comments from the 

Department of Labor
Appendix III: Comments from the Department 

of Labor

Page 50 GAO-14-4 Numbers Served in WIA Programs
Appendix III: Comments from the Department 

of Labor

Page 51 GAO-14-4 Numbers Served in WIA Programs
Appendix III: Comments from the Department 

of Labor

Page 52 GAO-14-4 Numbers Served in WIA Programs
Appendix III: Comments from the Department 

of Labor

Page 53 GAO-14-4 Numbers Served in WIA Programs
Appendix IV: GAO Contact and Staff 

Acknowledgments

Page 54 GAO-14-4 Numbers Served in WIA Programs

Revae E. Moran, (202) 512-7215 or moranr@gao.gov

In addition to the contact named above, Meeta Engle, Assistant Director; 

Theodore Alexander; Jenn McDonald; and Brian Schwartz made key 

contributions to this report. Also contributing to this report were Jessica 

Botsford, David Chrisinger, Kathy Leslie, Mimi Nguyen, Carol Patey, 

Rhiannon Patterson, Catherine Roark, Jerry Sandau, Walter Vance, and 

Charles Youman.

Appendix IV: GAO Contact and Staff 

Acknowledgments 

GAO Contact

Staff 

Acknowledgments
Related GAO Products

Page 55 GAO-14-4 Numbers Served in WIA Programs

Workforce Investment Act: Local Areas Face Challenges Helping 

Employers Fill Some Types of Skilled Jobs. GAO-14-19. Washington 

D.C.: December, 2013. 

Workforce Investment Act: Additional Actions Would Further Improve the 

Workforce System. GAO-07-105IT. Washington, D.C.: June 28, 2007.

Workforce Investment Act: Employers Found One-Stop Centers Useful in 

Hiring Low-Skilled Workers; Performance Information Could Help Gauge 

Employer Involvement. GAO-07-167. Washington, D.C.: December 22, 

2006.

Workforce Investment Act: Labor and States Have Taken Actions to 

Improve Data Quality, but Additional Steps Are Needed, GAO-06-82. 

Washington, D.C.: Nov. 14, 2005.

Workforce Investment Act: Labor Should Consider Alternative 

Approaches to Implement New Performance and Reporting 

Requirements. GAO-05-539. Washington, D.C.: May 27, 2005.

Workforce Investment Act: Substantial Funds Are Used for Training, but 

Little Is Known Nationally about Training Outcomes. GAO-05-650. 

Washington, D.C.: June 29, 2005.

Workforce Investment Act: Labor Actions Can Help States Improve 

Quality of Performance Outcome Data and Delivery of Youth Services. 

GAO-04-308. Washington, D.C.: February 23, 2004.

Workforce Investment Act: States and Local Areas Have Developed 

Strategies to Assess Performance, but Labor Could Do More to Help. 

GAO-04-657. Washington, D.C.: June 1, 2004.

Workforce Investment Act: Improvements Needed in Performance 

Measures to Provide a More Accurate Picture of WIA’s Effectiveness. 

GAO-02-275. Washington, D.C.: February 1, 2002.

Related GAO Products

(131203)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.