WORKFORCE 

INVESTMENT ACT

Strategies Needed 

to Improve Certain 

Training Outcome 

Data 

Report to Congressional Requesters

January 2014

GAO-14-137

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-14-137, a report to 

congressional requesters

January 2014

WORKFORCE INVESTMENT ACT

Strategies Needed to Improve Certain Training 

Outcome Data

Why GAO Did This Study

As the economy recovers, some 

employers continue to face difficulty 

finding qualified workers. The WIA 

Adult and Dislocated Worker Programs

provide services, including job training,

which aims to help participants acquire 

skills and credentials employers need.

Under WIA, states are required to 

report data on training participants who 

obtain credentials and on those who

enter employment related to the 

training they receive. Given that a WIA 

reauthorization proposal would 

establish both of these outcomes as

performance measures, GAO was 

asked to examine the capacity of 

states to report on these outcomes. 

This report addresses: 1) the extent to 

which training participants obtained 

credentials and training-related 

employment, 2) challenges states face 

in reporting data on credentials and 

what steps, if any, they and DOL are

taking to address them, and 3) 

challenges states face in reporting data 

on training-related employment and

what steps, if any, they and DOL are 

taking to address them. GAO

interviewed DOL officials, workforce 

experts, and state and local officials 

and employer organizations from a 

nongeneralizeable sample of six states 

selected in part on the basis of 

geographic diversity. GAO also 

analyzed data on credential attainment

and training-related employment for 

program years 2010 and 2011.

What GAO Recommends

GAO recommends that DOL identify 

and share with states strategies that 

may ease collection and improve the 

quality of training-related employment 

data. DOL agreed with GAO’s

recommendation.

What GAO Found

Of the more than two million total participants in the Workforce Investment Act’s 

(WIA) Adult and Dislocated Worker Programs, about 11 percent and 16 percent, 

respectively, received training in program year 2011, and about two-thirds of the 

training participants in each program attained a credential. Little is known,

however, about how many participants got jobs related to their training. From 

program year 2006 through program year 2011, the percentages of training 

participants who earned a credential declined from about 74 percent to 58 

percent for the Adult Program and from about 75 percent to 63 percent for the 

Dislocated Worker Program, according to data from the Department of Labor 

(DOL). Of those training participants who attained a credential in program year 

2011, about 65 percent earned occupational credentials, such as a welding 

certificate, followed by lower percentages who earned occupational skill licenses 

and associate’s degrees, among others. In contrast, GAO found training-related 

employment data unreliable primarily because a significant portion of the data 

was missing. 

Workforce officials in four of six selected states cited some obstacles in reporting 

data on credential attainment, and both DOL and states are taking steps to

address challenges. Officials in four states GAO contacted said reporting such 

data can be resource-intensive, largely because case managers must manually 

track this information from various sources, including participants, training 

providers, and third-party organizations. To improve credential attainment and 

reporting, DOL clarified which credentials should be reported and began 

measuring credential attainment through an agency-wide goal in 2010. Officials 

in five states have taken similar steps, such as setting goals and tracking 

credential attainment, and enhancing data exchange with training providers. 

Officials in most of the six states GAO contacted noted some obstacles to

obtaining such data. For example, officials from several states cited student 

privacy laws as a barrier in verifying credentials with training providers. Officials 

in three states told us that they ask participants to sign consent forms allowing 

training providers to give credential information to local officials.

Workforce officials in most of the selected states identified even greater 

challenges reporting data on training-related employment, including the high 

degree of resources required and the subjective nature of determining whether 

employment is linked to training. DOL has taken only limited steps to address 

these challenges. To report such data, case managers seek participants’ 

employment information from participants, employers, and wage records. Then 

they must piece it together to determine whether participants’ employment is 

“substantially related” to their training. Officials in most of the six states described 

this process as resource-intensive and noted that making such determinations 

are subjective since one case manager’s interpretation of “substantially related”

may differ from another’s. Given these challenges, officials in all six states have 

taken some steps to increase access to employment information or make 

decisions less subjective. DOL has recently stressed the importance of reporting 

data on training-related employment and shared a few practices with states to 

increase reporting rates; however, it has not identified and disseminated 

strategies to address the ongoing challenges states face regarding resource 

intensiveness and subjectivity, which could improve the quality of such data.

View GAO-14-137. For more information, 

contact Andrew Sherrill at (202) 512-7215 or 

sherrilla@gao.gov.
Page i GAO-14-137 WIA Training Outcomes

Letter 1

Background 4

DOL’s Data Show That Most Training Participants Attained 

Credentials, but Data on Training-Related Employment Are 

Unreliable 11

Selected States Cited Some Obstacles to Collecting Data on 

Credential Attainment and DOL and States Have Taken Steps 

to Address Challenges and Improve These Data 19

States Face Continuing Challenges Reporting Data on TrainingRelated Employment, Despite Efforts by States and DOL to 

Improve Reporting 27

Conclusions 35

Recommendation for Executive Action 36

Agency Comments and Our Evaluation 36

Appendix I Objectives, Scope, and Methodology 38

Appendix II Comments from the Department of Labor 42

Appendix III GAO Contact and Staff Acknowledgments 44

Related GAO Products 45

Tables

Table 1: Comparison of Performance Measures for the WIA Adult 

and Dislocated Worker Programs and Common Measures 8

Table 2: Variation among State-Reported Percentages of Training 

Participants Who Obtained Training-Related Employment 

in Program Year 2011 18

Table 3: Key DOL Initiatives Related to Credential Attainment 23

Table 4: Establishing a Performance Measure on Credential 

Attainment: Challenges, Mitigation Strategies, and 

Considerations 27

Contents
Page ii GAO-14-137 WIA Training Outcomes

Table 5: Establishing a Performance Measure on Training-Related 

Employment: Challenges, Potential Mitigation Strategies, 

and Considerations 35

Table 6: Selected Site Visit States and Local Areas 40

Figures

Figure 1: Overall Number of Participants in the Adult and 

Dislocated Worker Programs and Those Who Received 

Training, Program Years 2006 through 2011 12

Figure 2: Percentages of Participants in the Adult and Dislocated 

Worker Programs Who Received Training, Program

Years 2006 through 2011 13

Figure 3: Percentage of Training Participants in the Adult and 

Dislocated Worker Programs Who Earned Credentials, 

Program Years 2006 through 2011 15

Figure 4: Percentage of Credential Types Attained by Training 

Participants in Program Year 2011 17

Figure 5: General Process for Collecting Participants’ Employment 

Information and Determining Whether It Relates to their 

Training 29

Abbreviations

DOL Department of Labor

FERPA Family Educational Rights and Privacy Act

OJT On-the-Job Training 

O*NET Occupational Information Network

UI Unemployment Insurance 

WIA Workforce Investment Act

WIASRD Workforce Investment Act Standardized

Record Data

WIB Workforce Investment Board

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-14-137 WIA Training Outcomes

441 G St. N.W.

Washington, DC 20548

January 31, 2014

The Honorable John Kline

Chairman

Committee on Education and the Workforce

House of Representatives

The Honorable Virginia Foxx

Chairwoman

Subcommittee on Higher Education and Workforce Training 

Committee on Education and the Workforce

House of Representatives

The Honorable Joe Heck

House of Representatives

The Honorable Howard P. McKeon

House of Representatives

During this economic recovery, some jobs go unfilled because employers 

continue to face difficulty finding qualified workers. The Department of 

Labor’s (DOL) Workforce Investment Act (WIA) Adult and Dislocated 

Worker Programs provide a variety of employment services, including job 

training—which aims to help participants acquire skills and credentials 

required by employers. To assess the performance of these programs, 

WIA requires that states report on outcome measures in key areas

including employment, retention, earnings, and credential attainment. 

WIA also requires states to negotiate expected performance levels for 

each of these measures with DOL and allows DOL to hold states 

accountable for achieving their performance goals through financial 

incentives and sanctions. In 2005, DOL began allowing states to request 

waivers to replace the WIA performance measures with a set of common 

measures to assess multiple employment and training programs. These 

measures focus on employment, retention, and earnings but do not 

include credential attainment. DOL still requires states to report data on 

training participants who received credentials. In addition, while it is not a 

WIA performance measure, WIA requires states to report the number of 
Page 2 GAO-14-137 WIA Training Outcomes

training participants who obtain employment in their field of training.1

1. To what extent do WIA training participants obtain credentials and 

secure training-related employment?

However, DOL’s Office of Inspector General has raised questions about 

the quality of such data. Pending WIA reauthorization legislation includes 

these two training outcomes as performance measures. You requested 

that we examine the capacity of states and local areas to report accurate 

and consistent data on such outcomes. This report addresses:

2. What challenges do states face in reporting data on credential 

attainment and what steps, if any, are they and DOL taking to address 

them?

3. What challenges do states face in reporting data on training-related 

employment and what steps, if any, are they and DOL taking to 

address them?

To address our objectives, we reviewed relevant federal laws, 

regulations, and DOL’s guidance for the Adult and Dislocated Worker 

Programs. We also interviewed DOL officials and national experts on 

workforce issues representing six organizations. We identified these 

experts by reviewing relevant literature and using an iterative process to 

solicit recommendations from DOL officials and experts. The results of 

these interviews are not generalizeable, but provided insights about the 

overall challenges states face in reporting on these outcomes and any 

efforts by states to overcome them. To more specifically identify the 

challenges states face in reporting data on credential attainment and 

employment related to training, as well as to learn about any steps they 

are taking to address these challenges, we interviewed state and local 

workforce officials from a nongeneralizeable sample of six states: 

Alabama, Illinois, Kansas, Rhode Island, Texas, and Washington. We 

selected the states to reflect a mix of those that had leading edge data 

practices (as identified by experts) and those with either incomplete data 

or relatively high rates of reporting errors (as identified by WIA data 

 1 According to DOL, data on training-related employment are extremely important for 

program analysis and evaluation of the value of investments in WIA training. U.S. 

Department of Labor, Training and Employment Notice No. 5-13: Reporting TrainingRelated Employment under the Workforce Investment Act (WIA) Programs (September 4, 

2013).
Page 3 GAO-14-137 WIA Training Outcomes

quality reports on credential attainment).2 We also factored geographic 

diversity into state selection. In each state, we interviewed state 

workforce officials and also local workforce officials from at least one 

workforce investment board (WIB),3 and at least one American Job 

Center.4

To assess the reliability of the data DOL collects on credential attainment 

and training-related employment for participants in the WIA Adult and 

Dislocated Worker Programs, we analyzed data from DOL’s Workforce 

Investment Act Standardized Record Data system for 2010 and 2011, the 

most recent program years for which data were available. We reviewed 

existing documentation related to reporting these data, tested the data 

electronically, assessed it for factors such as completeness and 

consistency, and interviewed knowledgeable agency officials. We found 

the data on training and credential attainment to be sufficiently reliable for 

reporting overall estimates of participants who entered training, the 

minimum number of training participants who earned credentials, and the 

types of credentials they attained. However, we determined that the data 

on employment related to training were not reliable for the purposes of 

our report, as discussed in more detail later in this report. See appendix I 

for more details concerning the objectives, scope, and methodology used 

in this report.

In addition, we interviewed at least one employer organization in 

each state. In selecting these entities, we considered states’ input on 

organizations that could provide us with an employer perspective on the 

value of credentials and certain types of training for various industries. 

These employer organizations included statewide business associations, 

regional business associations, individual employers, and industryspecific representatives.

 2 Error rates were not available for training-related employment data so we did not 

consider this factor in state selection.

3 WIBs govern local workforce areas. Their responsibilities include setting policy for the 

local workforce area, coordinating workforce investment activities with economic 

development strategies, and developing relationships with employers.

4 Training and Employment Guidance Letter No. 36-11, Announcement of American Job 

Center Network (Washington, D.C.: June 14, 2012), strongly encouraged states and local 

areas to refer to the one-stop career centers as American Job Centers, in order to 

increase jobseeker and employer awareness of available workforce development 

resources. 
Page 4 GAO-14-137 WIA Training Outcomes

We conducted this performance audit from October 2012 to January 2014 

in accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit work to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives.

The WIA Adult and Dislocated Worker Programs provide employment 

services to a wide range of participants. The Adult Program serves all 

individuals age 18 and older, and the Dislocated Worker Program serves 

individuals who have been or will be terminated or laid off from 

employment, among others.5 The Adult Program prioritizes certain 

services for recipients of public assistance and other low-income 

individuals when program funds are limited.6 To enable individuals to 

participate, both programs may offer supportive services such as 

transportation, childcare, housing, and needs-related payments under 

certain circumstances.7

 5 A dislocated worker is an individual who: (1) has been terminated or laid off, or has 

received a notice of termination or layoff from employment; (2) is eligible for or has 

exhausted unemployment insurance; or (3) has demonstrated an appropriate attachment 

to the workforce, but is not eligible for unemployment insurance and is unlikely to return to 

a previous industry or occupation; (4) has been terminated or laid off or received 

notification of termination or layoff from employment as a result of a permanent closure or 

substantial layoff; (5) is employed at a facility where the employer has made the general 

announcement that the facility will close within 180 days; (6) was self employed but is 

unemployed as a result of general economic conditions in the community or because of a 

natural disaster; or (7) is a displaced homemaker. See 29 U.S.C. § 2801(9).

WIA requires that the Adult and Dislocated 

Worker Programs and other federally-funded employment and training 

programs provide services through one-stop centers—now called 

American Job Centers—so that jobseekers and employers can find 

assistance at a single location.

6 29 U.S.C. § 2864(d)(4)(E).

7 29 U.S.C. §§ 2864(e)(2) and 2864(e)(3). Supportive services are defined at 29 U.S.C. § 

2801(46).

Background

WIA Programs and 

Administrative Structure
Page 5 GAO-14-137 WIA Training Outcomes

DOL’s Employment and Training Administration administers the Adult and 

Dislocated Worker Programs and oversees their implementation, which is 

carried out by states and local areas. At the state level, the WIA Adult and 

Dislocated Worker Programs are administered by state workforce 

agencies. Each state has one or more local workforce investment areas, 

each governed by a WIB.8 WIBs select the entities that will operate 

American Job Centers, which provide most WIA services, and oversee 

the American Job Center network.9 WIA provides substantial flexibility to 

states and WIBs to determine how services are provided.

WIA represented a fundamental shift from its predecessor program, the 

Job Training Partnership Act, by decreasing the focus on job training as 

the primary means to help adults and dislocated workers get a job. The 

Adult and Dislocated Worker Programs provide participants three types of 

services:

• Core services include basic services such as job searches and labor 

market information, and may be accessed with or without staff 

assistance.

• Intensive services include such activities as comprehensive 

assessment and case management, which require greater staff 

involvement. Intensive services are available to participants who are 

unable to obtain or retain employment after receiving at least one core 

service.

• Training services include such activities as occupational skills or onthe-job training. To be eligible for training services, participants must: 

(1) be unable to obtain or retain employment after receiving at least 

one intensive service, (2) be in need of training, and (3) have the skills 

and qualifications to successfully complete the training program, 

among other requirements.10

 8 29 U.S.C. §§ 2831 and 2832.

9 29 U.S.C. § 2832(d).

10 Participants must also: 1) select a training program that is directly linked to employment 

opportunities in the local area or in another area to which the individual is willing to 

relocate, and 2) be unable to obtain grant assistance from other sources to pay training 

costs. 20 C.F.R. § 663.310.

Employment and

Training Services 

Provided under WIA
Page 6 GAO-14-137 WIA Training Outcomes

To assess participants’ skills and determine whether they need training, 

WIBs may require them to complete certain activities. We previously 

found that most WIBs required participants to complete skills 

assessments or gather information about the occupation for which they 

wanted training before entering a training program.11

DOL requires that participants eligible for training select approved training 

providers in consultation with case managers, but participants ultimately 

choose the training programs in which they participate.12 WIA requires 

that training be directly linked to in-demand occupations, which DOL 

interprets to include both currently-available jobs and occupations that are 

projected to grow in the future.13

In fiscal year 2013, Congress appropriated a total of about $1.9 billion for 

the Adult and Dislocated Worker Programs, roughly 24 percent less than 

the $2.5 billion appropriated in fiscal year 2000.14 In order to ensure that 

they receive their full funding allocations, states must meet agreed upon 

levels of performance. WIA requires states to use performance measures 

that gauge program results for job seekers in the areas of employment, 

employment retention, earnings, and credential attainment.15 WIA also 

requires measures of customer satisfaction for jobseekers and for 

employers.16 WIA requires states to negotiate performance levels for

each of these measures annually with DOL17

 11 See GAO, Workforce Investment Act: Substantial Funds Are Used for Training, but 

Little Is Known Nationally About Training Outcomes. 

and provides that DOL may 

hold states accountable for achieving these levels by tying them to 

GAO-05-650 (Washington, D.C.: 

June 29, 2005).

12 20 C.F.R. § 663.440(c).

13 29 U.S.C. § 2864(d)(4)(G)(iii). 

14 Fiscal year 2000 appropriations are presented in nominal dollars and are not adjusted 

for inflation. Thus, in real (inflation-adjusted) dollars, appropriated funds for WIA have 

declined by more than 24 percent. 

15 29 U.S.C. § 2871(b)(2)(A). 

16 29 U.S.C. § 2871(b)(2)(B).

17 29 U.S.C. § 2871(b)(3)(A).

Funding, Performance 

Measures, and 

Data Quality
Page 7 GAO-14-137 WIA Training Outcomes

financial sanctions and incentive funding.18 In addition, WIA requires 

states to use Unemployment Insurance (UI) wage records to track 

employment-related outcomes.19

In 2005, DOL began allowing states to request waivers to replace the 

WIA performance measures with a smaller set of common measures that 

focus on employment, retention, and earnings across multiple programs 

(see table 1). These measures do not include credential attainment and 

are calculated differently than the WIA measures.20

 18 In accordance with WIA, DOL may provide financial awards to states for performance 

that exceeds negotiated levels and assess penalties for performance that does not meet 

negotiated performance levels. 20 U.S.C. § 9273 and 29 U.S.C. §2871(g). For all states—

those with waivers and those without waivers—incentive or sanction determinations are 

based only on the following performance measures: entered employment, employment 

retention, and earnings. See Training and Employment Guidance Letter No. 9-07, Revised 

Incentive and Sanction Policy for Workforce Investment Act Title IB Programs

(Washington, D.C.: October 10, 2007).

19 29 U.S.C. §2871(f)(2). Each state maintains UI wage records to support the process of 

providing unemployment compensation to unemployed workers. The UI wage records 

contain basic wage information for about 94 percent of workers. Certain employment 

categories are excluded such as self-employed persons, independent contractors, federal 

employees, and military personnel. If wage information for a program participant is not 

found in the UI wage records, states may use supplemental data sources, such as followup with the participant and employer.

20 Common performance measures were developed under an Office of Management and 

Budget initiative which sought to unify definitions for performance across programs with 

similar goals. 
Page 8 GAO-14-137 WIA Training Outcomes

Table 1: Comparison of Performance Measures for the WIA Adult and Dislocated Worker Programs and Common Measures

WIA program WIA performance measures Common measures a

Adult

b

• Entered employment rate 

• Employment retention rate at 6 months

c

• Average earnings change in 6 months

• Entered employment and credential rate

• Entered employment

d

• Employment retention

c

• Average earnings

Dislocated Worker • Entered employment rate

• Employment retention rate at 6 months

c

• Earnings replacement rate at 6 months

• Entered employment and credential rated

Source: U.S. Department of Labor. 

a

While not a performance measure, states are required to report the number of training participants 

who obtained employment related to their training.

b

While they are not performance measures, states are required to report both the number of training 

participants who earned credentials and the number of training participants who obtained 

employment related to their training.

c

“Entered employment” is defined as the percentage of participants who got jobs by the end of the 

first quarter after program exit. For the Adult Program, this measure excludes participants who are 

employed at the time of registration. For the Dislocated Worker Program, it includes those employed 

at the time of registration.

d

The Department of Labor’s guidance defines the “entered employment and credential” measure as 

the percentage of trained participants who were employed in the first quarter after exit from the 

program and received a credential by the end of the third quarter after exit.

As of December 2013, a total of 48 states and territories, plus the District 

of Columbia, had obtained this waiver and used the common measures 

for the Adult and Dislocated Worker Programs.21 Although states and 

territories are no longer subject to financial sanctions or incentive funding 

for the credential attainment measure, DOL still requires them to report 

the number of training participants who earn credentials.22

 21 In addition to the 50 states and the District of Columbia, there are eight United States 

territories. As of December 2013, five of these eight territories had obtained a common 

measures waiver.

In general, 

credential attainment improves workers’ labor market experience through 

22 States must report data on credentials training participants attained either during WIA 

participation or by the end of the third quarter after the quarter of exit from WIA services. 

States with common measures waivers report these data, rather than data on training 

participants who entered employment and attained a credential, as was required under the 

WIA performance measures. U.S. Department of Labor, Training and Employment 

Guidance Letter 17-09, Quarterly Submission of Workforce Investment Act Standardized 

Record Data (March 10, 2010).
Page 9 GAO-14-137 WIA Training Outcomes

higher earnings, greater mobility, and enhanced job security, according to 

DOL and research studies.23

We have previously raised concerns about the accuracy and 

comparability of DOL’s data on credential attainment because DOL’s 

guidance allowed states and local areas considerable flexibility in defining 

what constitutes a credential.24 In February 2002, we recommended that 

DOL more clearly define this term. Since that time, DOL issued guidance 

to clarify its definition of credential attainment.25 In addition, to verify that 

the data states report on credential attainment are accurate, DOL 

requires WIA programs to collect and retain documentation on 

participants’ credential attainment.26

In addition to the performance measures, WIA requires states to report a 

wide array of data that includes whether training participants find 

 23 U.S. Department of Labor, Training and Employment Guidance Letter 15-10, Increasing 

Credential, Degree, and Certificate Attainment by Participants of the Public Workforce 

System (December 15, 2010); Debbie Reed et al., An Effectiveness Assessment and

Cost-Benefit Analysis of Registered Apprenticeship in 10 States, Mathematica Policy 

Research (Oakland, CA: July 25, 2012); W. Norton Grubb, “Learning and Earning in the 

Middle, Part I: National Studies of Pre-baccalaureate Education,” Economics of Education 

Review (2002).

24 See GAO, Workforce Investment Act: Improvements Needed in Performance Measures 

to Provide a More Accurate Picture of WIA’s Effectiveness, GAO-02-275 (Washington, 

D.C.: February 1, 2002).

25 DOL defined credential attainment in guidance issued in 2006. In 2010, DOL issued 

additional guidance stating that the term “credential” refers to an attestation of qualification 

or competence issued to an individual by a third party, such as an educational institution, 

with the relevant authority or assumed competence to issue such a credential. U.S. 

Department of Labor, Training and Employment Guidance Letter No. 17-05, Common 

Measures Policy for the Employment and Training Administration’s Performance 

Accountability System and Related Performance Issues (February 17, 2006); and U.S. 

Department of Labor, Training and Employment Guidance Letter No. 15-10, Increasing 

Credential, Degree, and Certificate Attainment by Participants of the Public Workforce 

System, (December 15, 2010).

26 U.S. Department of Labor, Employment and Training Administration, Workforce 

Investment Act Data Reporting and Validation System (DRVS) User Handbook for DRVS 

7.2 (April 2009).
Page 10 GAO-14-137 WIA Training Outcomes

employment that relates to their training.27 A WIA participant’s 

employment is considered to be related to the training received if in the 

new job the participant uses “a substantial portion of the skills taught in 

the training.”28 However, DOL does not require that states collect and 

retain documentation for employment related to training to verify the 

accuracy of data they report. In a September 2011 report, DOL’s Office of 

Inspector General raised questions about the quality of DOL’s data on 

employment related to training. Specifically, the Office of Inspector 

General found that these data were “incomplete and unreliable” based on 

its review of the data DOL maintains in its WIA database.29

Federal agencies that work in partnership with states and local areas to 

administer programs such as WIA must continually balance the 

competing objectives of collecting uniform performance data and giving 

program managers flexibility to meet local needs. Our prior work has 

found that federal agencies have considered key attributes of data quality 

for performance data, including:

The Office of 

Inspector General recommended, among other things, that DOL provide 

guidance on the best methodology for reporting such data as well as 

provide oversight to ensure that states develop or identify best practices 

for increasing the rate of training-related employment. DOL agreed with 

these recommendations.

• Completeness—the extent to which enough of the required data 

elements are collected from a sufficient portion of the target 

population or sample.

• Accuracy—the extent to which the data are free from significant 

errors.

 27 29 U.S.C. §2871(d)(2)(A). DOL requires states to report training-related employment 

only on participants that received training and entered employment in the first quarter after 

the quarter of exit from WIA services. U.S. Department of Labor, Training and 

Employment Guidance Letter No. 17-09: Quarterly Submission of Workforce Investment 

Act Standardized Record Data (WIASRD) (March 10, 2010).

28 U.S. Department of Labor, Training and Employment Guidance Letter No. 17-09, 

Quarterly Submission of Workforce Investment Act Standardized Record Data 

(WIASRD)(March 10, 2010).

29 U.S. Department of Labor Office of Inspector General Audit Report No. 03-11-003-03-

390, Additional Information Needed to Measure the Effectiveness and Return on 

Investment of Training Services Funded Under the Adult and Dislocated Worker Programs

(September 30, 2011). 
Page 11 GAO-14-137 WIA Training Outcomes

• Consistency—the extent to which data are collected using the same 

procedures and definitions across collectors and times.

• Ease of use—how readily intended users can access data, aided by 

clear definitions, user-friendly software, and easy-to-use access 

procedures.30

Congress is currently considering legislation to reauthorize the Workforce 

Investment Act of 1998, which has been due for reauthorization since the 

end of fiscal year 2003. The Supporting Knowledge and Investing in 

Lifelong Skills (SKILLS) Act (H.R. 803), passed by the House, would 

establish both credential attainment and training-related employment as 

performance measures. The Workforce Investment Act of 2013 (S. 1356), 

introduced in the Senate and reported out by the Committee on Health, 

Education, Labor, and Pensions, would establish credential attainment as 

a performance measure, but not training-related employment.

During the time-period from program year 2006 through program year 

2011, the total number of participants in WIA’s Adult and Dislocated 

Worker Programs increased significantly.31

 30 GAO, Performance Plans: Selected Approaches for Verification and Validation of 

Agency Performance Information,

Specifically, in the Adult 

Program, the number of participants increased from about 625,000 to 

GAO/GGD-99-139 (Washington, D.C.: July 30, 1999).

31 U.S. Department of Labor, Office of Performance and Technology, Employment and 

Training Administration, PY 2008 WIA Trends Over Time (January 19, 2010); and PY 

2011 WIA Trends Over Time (December 13, 2012).

Congressional Proposals 

to Reauthorize WIA 

Programs

DOL’s Data Show 

That Most Training 

Participants Attained 

Credentials, but Data 

on Training-Related 

Employment Are 

Unreliable

Most Training Participants 

Attained Credentials
Page 12 GAO-14-137 WIA Training Outcomes

about 1.25 million, and in the Dislocated Worker Program, the number of 

participants increased from about 272,000 to about 761,000. During this 

same time-period, the number of participants who received training 

services also increased, but not as dramatically as the number of overall 

participants (see fig. 1).32

Figure 1: Overall Number of Participants in the Adult and Dislocated Worker Programs and Those Who Received Training, 

Program Years 2006 through 2011

For example, the number of Dislocated Worker 

participants who received training increased from about 76,000 in 

program year 2006 to about 120,000 in program year 2011.

Note: Data are based on participants who exited the Adult and Dislocated Worker Programs. Program 

Year 2011 reflects data for the time-period from April 1, 2011 through March 31, 2012, not the actual 

program year, which ran from July 1, 2011 to June 30, 2012.

Since the number of participants who received training services did not 

increase at the same rate as the number of participants who entered into 

WIA’s Adult and Dislocated Worker Programs, the percentage of 

 32 Throughout the report, “received training” is used to describe participants who entered 

training, but may or may not have completed it. 
Page 13 GAO-14-137 WIA Training Outcomes

participants who received training generally declined, according to DOL 

reports (see fig. 2).

Figure 2: Percentages of Participants in the Adult and Dislocated Worker Programs 

Who Received Training, Program Years 2006 through 2011

Note: Data are based on participants who exited the Adult and Dislocated Worker Programs. Program 

year 2011 reflects data for the time period from April 1, 2011 through March 31, 2012, not the actual 

program year, which ran from July 1, 2011 to June 30, 2012.

There are several reasons that may have contributed to the declining 

percentage of participants who received training. Beginning in December 

2007, the significant increase in the overall number of participants is likely 

attributed to the downturn in the economy that led to a dramatic rise in 

unemployment33

 33 A general slowdown in economic activity, a downturn in the business cycle, and a 

reduction in the amount of goods and services produced and sold are all characteristics of 

a recession. The most recent recession began in December 2007 and ended in June 

2009. U.S. Bureau of Labor Statistics, The Recession of 2007–2009 (February 2012). 

and the subsequent infusion of additional funds to WIA 
Page 14 GAO-14-137 WIA Training Outcomes

programs from the American Recovery and Reinvestment Act of 2009 

(Recovery Act).34 The Recovery Act provided $500 million for grants for 

the Adult Program and $1.25 billion for grants for the Dislocated Worker 

Program. By spring of 2009, DOL began allocating these funds to states 

to supplement existing WIA funds. DOL encouraged states to use 

Recovery Act funds to increase training in an effort to help Americans 

acquire new skills and return to work.35

Similarly, the percentage of training participants who earned credentials 

has also generally declined from program year 2006 through program 

year 2011 (see fig. 3). For example, DOL’s data show the percentage of 

those who earned credentials in the Adult Program dropped from about 

74 percent in program year 2006 to about 58 percent in program year 

2011. Likewise, the percentage of those in the Dislocated Worker 

Program who earned credentials dropped from about 75 percent in 

program year 2006 to about 63 percent in program year 2011. Despite 

the decline in the percentage of training participants who earned 

credentials, DOL reported that the total number of participants attaining 

credentials increased during this time period. For example, the number of 

training participants in the Dislocated Worker Program who attained 

credentials increased from about 48,000 in program year 2006 to about 

79,000 in program year 2011.

DOL officials stated that the 

increase in demand for training services exceeded the increase in supply 

provided for by the Recovery Act, which may have led to shortages in 

training capacity that contributed to a relative decline in training. Another 

factor, according to DOL officials, is that some program participants had 

limited access to needs-based financial assistance and other supportive 

services, such as child care, which may have prevented them from 

entering WIA training services. Further, DOL noted that the preference of 

many program participants is for immediate employment rather than job 

training.

 34 Pub. L. No. 111-5, 123 Stat, 115, 172-173.

35 For WIA Adult and Dislocated Worker Programs, states report aggregate counts of all 

participants, including those whose services were funded with regular WIA formula funds 

and Recovery Act funds. 
Page 15 GAO-14-137 WIA Training Outcomes

Figure 3: Percentage of Training Participants in the Adult and Dislocated Worker 

Programs Who Earned Credentials, Program Years 2006 through 2011

Note: Data are based on training participants who exited the Adult and Dislocated Worker Programs. 

DOL requires that states report on credentials attained by training participants either during WIA 

participation or by the end of the third quarter after the quarter of exit from services.

According to DOL officials, the percentage of training participants who 

earned credentials may have declined in part as a result of changes in the 

performance measures that states negotiated. In program year 2005, 

states began requesting waivers to replace the WIA performance 

measures with a set of common performance measures that do not 

include the employment and credential attainment measure for the Adult 

and Dislocated Worker Programs.

36

 36 DOL defined the WIA performance measure on credential attainment to assess training

participants who secured employment and received a credential. U.S. Department of 

Labor, Training and Employment Guidance Letter No. 7-99, Core and Customer 

Satisfaction Performance Measures for the Workforce Investment System (March 3, 

2000).

Officials in three of the six states we 

reviewed said that after this request was approved, reporting data on 

credential attainment became a lower priority for them. Officials in one of 

these states also said they stopped collecting and reporting these data 
Page 16 GAO-14-137 WIA Training Outcomes

until DOL issued clarifying guidance in 2010 emphasizing the importance 

of credential attainment as a pathway to employment.

DOL’s data on credential attainment also show that participants in the 

Dislocated Worker Program typically have higher credential attainment 

rates than participants in the Adult Program. DOL officials explained that 

the Dislocated Worker Program has a higher funding level that supports 

more training, and that participants in this program generally have longer 

work histories and more advanced education and so are more likely to 

enter into training programs that lead to credentials. In contrast,

participants in the Adult Program are more likely to require training 

focused on remedial education and job readiness which are less likely to 

result in credentials as defined by DOL.

Of the training participants who attained credentials during program year 

2011—approximately 89,000 in the Adult Program and 84,000 in the 

Dislocated Worker Program— about two-thirds in each program earned 

an occupational credential, such as a welding certificate or clinical 

medical assistant certificate (see fig. 4). The next two most common 

types of credentials attained by training participants were an occupational 

skills license, such as a license in nursing37, and an associate’s degree. 

The fact that training participants attained occupational credentials at 

higher rates than longer-term academic degrees is consistent with DOL’s 

recommendation that states shorten training duration in an effort to 

increase credential attainment. In our December 2013 report, we found 

that in program year 2011, of those in the Adult Program who entered 

training, 75 percent spent 1 year or less receiving training services, while 

25 percent spent more time. Similarly, for the Dislocated Worker 

Program, 65 percent of training participants spent 1 year or less receiving 

training services.38

 37 Occupational licenses are required by government entities, typically state regulatory 

bodies, before an individual is allowed to be employed in and practice a trade, profession, 

or other occupation.

According to officials from one local workforce 

investment board we contacted, all training programs offered through 

their training providers must lead to a credential and must be completed 

in 12 months or less.

38 GAO, Workforce Investment Act: DOL Should Do More to Improve the Quality of its 

Participant Data. GAO-14-4 (Washington, D.C.: December 2, 2013).
Page 17 GAO-14-137 WIA Training Outcomes

Figure 4: Percentage of Credential Types Attained by Training Participants in 

Program Year 2011

Note: Data are based on training participants who exited the Adult and Dislocated Worker Programs.

We found DOL’s data on training-related employment unreliable for our 

purposes based on our analysis of the data, an Office of Inspector 

General report, and data quality reports. We were not able to determine 

how many participants in the Adult or Dislocated Worker Programs 

obtained employment related to their training in program year 2011. For 

the Adult Program, we found that states reported data on 48 percent of 

training participants, but had missing data for the remaining 52 percent. 

For the Dislocated Worker Program, states reported data on 74 percent of 

training participants, but had missing data for the remaining 26 percent. 

Further, our analysis of the reported data showed wide variation among 

states regarding the percentage of participants who obtained trainingLittle Is Known about 

the Extent to Which 

Participants Obtained 

Training-Related 

Employment
Page 18 GAO-14-137 WIA Training Outcomes

related employment, raising questions about the data’s reliability (see 

table 2). Our findings are consistent with a September 2011 Office of 

Inspector General report, which found that DOL’s data on training-related 

employment were “incomplete and unreliable.”39

Table 2: Variation among State-Reported Percentages of Training Participants Who 

Obtained Training-Related Employment in Program Year 2011

Specifically, the 

Inspector General reported that 5 of the 53 state workforce agencies it 

reviewed did not report any data and 12 state workforce agencies 

reported unreasonably high or abnormally low data on training-related 

employment. Further, DOL’s WIA data quality report for the third quarter 

of program year 2011 raised questions about training-related employment 

data for 26 states. For example, some states reported that none of their 

training participants secured training-related employment.

Percentage of training-related 

employment

Adult Programa Dislocated Worker Program

Number of states

b

c Number of states

0 to 25

c

10 13

26 to 50 9 16

51 to 75 10 23

76 to 100 24 1

Source: GAO analysis of DOL data. 

a

These figures are based on states’ reported data on 48 percent of training participants and do not 

include 52 percent of training participants, for whom data were missing.

b

These figures are based on states’ reported data on 74 percent of training participants and do not 

include 26 percent of training participants, for whom data were missing.

c

Number of states is based on the 53 state workforce agencies that reported training-related 

employment data in program year 2011.

 39 U.S. Department of Labor, Office of Inspector General, Additional Information Needed 

to Measure the Effectiveness and Return on Investment of Training Services Funded 

Under WIA Adult and Dislocated Worker Programs, Report Number 03-11-003-03-390 

(September 30, 2011). 
Page 19 GAO-14-137 WIA Training Outcomes

Workforce officials we interviewed in four of six states said that collecting 

data on credential attainment can be resource-intensive primarily 

because it requires manually tracking the information. Unlike 

employment-related outcomes—which states can generally obtain 

through the state’s UI wage record system—credentials are not generally 

recorded in a central, automated data system. As a result, case 

managers must manually collect this information from various sources 

including participants, training providers, and third-party credentialing 

organizations. DOL also requires documentation of credential attainment 

with a copy of a diploma, transcript, or other approved record in the 

participant’s case file. DOL monitors this requirement through its data 

validation process. The process of collecting and verifying a participant’s 

credential attainment generally entails one or more of the following steps:

Contacting Participants. Workforce officials in most states we 

reviewed said they generally begin their efforts to determine credential 

attainment by attempting to contact training participants, though some 

are unresponsive or inaccessible. Several local officials noted that 

they use a variety of means, including phone, mail, email, and social 

media. Some training participants readily provide evidence of their 

credentials. For example, local officials in two states estimated that for 

about 70 percent of participants, credentials are fairly easy to verify. 

Other participants may be less responsive. Workforce officials in three 

states explained that participants who have already exited the 

program have little incentive to respond to their requests. Local 

officials in two states also noted that some participants relocate 

without providing updated contact information.

Selected States Cited 

Some Obstacles to 

Collecting Data on 

Credential Attainment 

and DOL and States 

Have Taken Steps to 

Address Challenges 

and Improve These 

Data

Manual Process of 

Collecting Data on 

Credentials Can Be 

Resource-Intensive
Page 20 GAO-14-137 WIA Training Outcomes

Contacting Training Providers. Training providers are another 

potential source of credential information, though in some cases they 

may decline to share such information. If case managers cannot 

reach a participant, they generally contact the training providers to 

determine whether a credential was earned. However, workforce 

officials from three states noted that training providers often declined 

to provide this information, citing student privacy rights such as those 

established by the Family Educational Rights and Privacy Act of 1974, 

as amended (FERPA).40

Contacting Third-party Organizations. Third-party credentialing 

organizations represent an additional source of credential 

information.41

 40 20 U.S.C. § 1232g. FERPA includes requirements related to the use and disclosure of 

education records on individual students and provides that, generally, a student must 

provide written consent before an educational agency or institution may disclose 

personally identifiable information from that student’s education records. Directory 

information, including major field of study and the degrees received, may be released 

without prior consent of a student if certain conditions are met: an educational agency or 

institution must give public notice of: (1) the types of personally identifiable information the 

educational agency or institution has designated as directory information, (2) the right of 

the student to refuse to let the educational agency or institution designate those types of 

information as directory information, and (3) the period of time within which a student can 

notify the educational agency or institution of that refusal. An educational agency or 

institution may disclose directory information about former students unless, while in 

attendance, the student opted out of the disclosure of directory information and has not 

rescinded that request. 34 C.F. R. § 99.37.

For some occupations, a license or certification is 

required before a person can be employed in that capacity, such as a 

licensed practical nurse. In these cases, third-party organizations, 

such as state regulatory bodies, issue credentials. Case managers 

can sometimes search licensing databases online to confirm 

credential attainment. Local officials from two states noted that such 

data are fairly easy to obtain because the information is generally 

centrally accessible. However, an official from another state said that 

third-party organizations do not always provide information on 

credential attainment before DOL’s reporting timeframes end.

41 Officials from the Department of Education said that FERPA generally does not apply to 

third-party organizations that grant credentials to an individual who takes a test through a 

third-party organization and applies directly to that organization to receive a 

credential. However, if such a third-party organization were to provide an educational 

agency or institution covered by FERPA with a copy of the test results or credential that 

directly relate to a student attending their program and that information were maintained 

by the educational agency or institution, then that information would become subject to 

FERPA.
Page 21 GAO-14-137 WIA Training Outcomes

Because case managers may not always be able to track down the 

documentation needed to verify credential attainment, the actual number 

of participants who attain credentials may be underreported to DOL. For 

example, officials in one state we contacted said they believe their 

credential attainment rate should be about 65 percent, but the rate they 

actually report is about half of that.

Despite such obstacles to verifying data on participant credentials, 

several workforce experts and officials noted the importance of collecting 

this information. Workforce experts from one national organization noted 

that credential attainment can demonstrate the value of the funds 

invested in training and show employers the value of workforce programs

and their participants. Workforce experts from another national 

organization said that credential data could help officials determine which 

credentials are best aligned with good employment outcomes. Some 

employer groups also noted the value of credentials in some highdemand occupations, such as manufacturing and information technology. 

For example, representatives of employer groups in Illinois and Rhode 

Island said they value information technology and manufacturing 

credentials from certifying organizations because these programs prepare 

individuals to perform high-skill tasks.

In 2010, DOL provided guidance to states to increase the quantity and 

quality of credentials attained and to clarify the definition of credential for 

reporting purposes. During early WIA implementation, we reported that 

the definition of credential varied within and across states. For example, 

some states strictly defined credential as a diploma from an accredited 

institution, and other states broadly defined credential to include 

certificates of job readiness or completion of a workshop.42 DOL issued 

guidance in 2006 that provided additional clarification on which 

credentials to report, but, according to some workforce officials and 

experts, allowed for some interpretation.43

 42 See GAO, Workforce Investment Act: Improvements Needed in Performance Measures 

to Provide a More Accurate Picture of WIA’s Effectiveness, 

In 2010, DOL issued guidance 

that defined “credential” as an umbrella term that can include a range of 

GAO-02-275 (Washington, 

D.C.: February 1, 2002).

43 U.S. Department of Labor, Training and Employment Guidance Letter 17-05, Common 

Measures Policy for the Employment and Training Administration’s (ETA) Performance 

Accountability System and Related Performance Issues (February 17, 2006). 

To Address Challenges, 

DOL Has Stressed Both 

Credential Attainment 

and Reporting
Page 22 GAO-14-137 WIA Training Outcomes

postsecondary degrees, diplomas, licenses, certificates, and 

certifications.44

In addition to clarifying which credentials should be reported, the 2010 

guidance also included strategies that state and local officials can use to 

increase the quantity and quality of credentials attained. It noted that the 

first step in increasing the quantity of credentials attained is to refer more 

participants to training. DOL’s guidance also encouraged officials to take 

steps to ensure that the training programs result in an industry-recognized 

credential and that participants complete these training programs. These 

steps include shortening the duration of training and providing supportive 

services that enable participants to succeed. Further, to improve the 

quality of credentials attained, DOL suggested that state and local 

agencies build the capacity of front-line staff to identify and assess 

valuable and appropriate credentials for participants.

DOL also clarified that credentials must show attainment 

of measurable technical or occupational skills necessary to obtain 

employment or advance within an occupation. For this reason, DOL 

specified that credentials related to remedial training, such as workreadiness certificates, would not be counted for the purposes of credential 

attainment.

DOL has also stressed the importance of credential attainment by 

measuring it through an agency-wide performance goal for its workforce 

development programs including the WIA Adult and Dislocated Worker 

Programs. DOL officials noted that credential attainment rates for these 

WIA programs are higher than the rates of some other DOL programs 

included in the agency-wide performance goal. DOL first began tracking 

credential attainment data for its agency-wide performance goal in 2010 

when it set out to increase the number of training participants who attain 

credentials through any one of multiple federal workforce programs. 

Specifically, the goal was an increase of 10 percent, up to a total of 

220,000 training participants earning credentials. In fiscal year 2013, DOL 

continued to assess credential attainment through this performance goal 

and sought to increase the percentage of training participants who earn 

 44 U.S. Department of Labor, Training and Employment Guidance Letter 15-10, Increasing 

Credential, Degree, and Certificate Attainment by Participants of the Public Workforce 

System (December 15, 2010).
Page 23 GAO-14-137 WIA Training Outcomes

credentials from 57 to 62 percent.45

In addition to issuing guidance and setting credential attainment goals, 

DOL also undertook a number of other related initiatives, including some 

that are specific to credentials and others that are more broadly designed, 

such as the Workforce Data Quality Initiative. See Table 3 for a 

description of DOL’s initiatives.

DOL officials reported a credential 

attainment rate of 59.4 percent through the first two quarters of fiscal year 

2013. DOL officials also said that DOL has established a new credential 

attainment goal; specifically, that by September 30, 2015, the percentage 

of training participants who attain credentials will increase by 10 percent 

from the level reported as of the end of fiscal year 2013.

Table 3: Key DOL Initiatives Related to Credential Attainment 

Initiative Date

Guidance to workforce system 

Issued guidance that more precisely defined credentials for purposes of WIA reporting; and in 2010 

provided strategies state and local workforce agencies, partners, and grantees could take to increase the 

rate of credential attainment among workforce program participants and improve the quality of those 

credentials.

February 2006 and 

December 2010

Agency-wide initiatives

Set credential attainment as an agency-wide performance goal across various workforce programs, 

including the WIA Adult and Dislocated Worker programs.

2010-2015

Created a credentials database to disseminate information on available credentials and Certification 

Finder website to help employers identify occupational certifications that may be useful for current or 

future employees.

2005-ongoing

Held Credentials Forum for employers to identify credentials valued by employers and updated the 

credentials database.

May 2012-January 2013

Began Career Pathways Initiative (jointly with Education and Health and Human Services) to help adults 

acquire marketable skills and industry-recognized credentials through better alignment of education, 

training and employment, and human and social services among public agencies and with employers.

2010-ongoing 

Longer-term initiatives

Awarded Workforce Data Quality Initiative grants to states to help them link their statewide educational 

databases to their statewide workforce databases and improve access to data in their workforce 

databases, including data on credentials.

2011-2013

Source: GAO summary of DOL information.

 45 The fiscal year 2013 goal included a broader array of workforce programs than the prior 

goal, including National Emergency Grants, Trade Adjustment Assistance, Green Jobs 

Innovation Fund, and Job Corps, among others. 
Page 24 GAO-14-137 WIA Training Outcomes

Some states have stressed the importance of credential attainment by 

implementing broad, statewide efforts. Similar to DOL’s efforts to enhance 

credential attainment by establishing annual goals, three of the six states 

we reviewed have either implemented statewide credential attainment 

goals or are working to do so:

• Texas implemented an annual state performance measure on 

educational achievement that tracks credential attainment for multiple 

programs, including the WIA Adult and Dislocated Worker Programs. 

All WIBs in the state are held to this measure.

• Washington has made credential attainment a state performance 

measure, but defines credential more broadly than DOL. For example, 

Washington recognizes a larger range of credentials, such as 

completion of on-the-job training.46

• Illinois re-implemented a credential attainment performance measure 

during program year 2012 and, according to state officials, is in the 

process of setting credential targets for program year 2014.

According to officials from some states, their efforts to emphasize 

credential attainment and reporting may have a positive impact on 

participants’ reported rate of credential attainment.

Moreover, officials in Alabama, Illinois, Kansas, and Rhode Island told us 

they targeted their training funds more narrowly on credential-yielding 

programs by only approving training providers with programs that resulted 

in credentials that met DOL’s definition. For one Chicago WIB, this 

strategy, along with its other efforts to streamline training options from 

753 occupations to 40 in-demand occupations, reduced its number of 

training providers. This practice was one of many DOL suggested in its 

2010 guidance as a means for states and local areas to improve the 

value of credentials for participants. Officials in nearly every state we 

interviewed reported that this guidance was helpful largely because it 

more clearly defined which credentials should be reported to DOL.

In addition, selected states and local areas have taken steps to ease the 

resource-intensive process of collecting data on credentials by enhancing 

communication with participants and working to overcome privacy issues 

 46 Washington State Employment Security Department, Certificates and Credentials 

Under Common Measures, Policy Number 3690, Revision 1 (September 23, 2011). 

Selected States and Local 

Areas Have Taken Steps 

to Address Challenges by 

Enhancing Attainment and 

Reporting of Credentials
Page 25 GAO-14-137 WIA Training Outcomes

with training providers. Workforce officials in three states told us that case 

managers seek to build rapport with participants early in the process so 

they are more likely to be responsive after their training program ends. 

Regarding training providers, officials in four of six states said they have 

made efforts to address privacy concerns. Local officials in Alabama, 

Kansas, and Texas, for example, told us that they ask participants to sign 

consent forms to allow training providers to share credential information 

with officials. In Washington, state officials access some credential data 

from the National Student Clearinghouse and from their state database of 

community and technical colleges.47

While these varied efforts to mitigate challenges may help reduce the 

resources required or improve the quality of reported data, workforce 

officials from three states and three experts we interviewed raised some 

additional considerations about measuring performance on credential 

attainment (see table 4). Establishing a performance measure on 

credential attainment may affect the type of training provided and which 

participants receive training. For example, neither work readiness training 

nor on-the-job training (OJT) leads to what DOL has defined as a 

credential for reporting purposes. However, these may be the most 

appropriate types of training for participants with basic skills or for 

particular industries, according to officials from two states we 

interviewed.

They said that student privacy rights 

are generally not a barrier to accessing credential data in Washington 

because students attending their community and technical colleges are 

notified that such information can be released to other entities unless the 

student opts out of sharing it. Washington state officials noted that they 

have been refining their process for collecting data on credential 

attainment for 15 years and now have a fairly sophisticated approach.

48

 47 The National Student Clearinghouse offers access to nationwide coverage of 

enrollment and degree records — encompassing over 98 percent of all students in public 

and private U.S. institutions. None of our other selected states mentioned using this 

source for collecting data on credential attainment. 

Our December 2013 report found that participants in the 

48 WIA defines “on-the-job training” as training by an employer that is provided to a paid 

participant while engaged in productive work in a job that: a) provides knowledge or skills 

essential to the full and adequate performance of the job; b) reimburses the employer up 

to 50 percent of the participant’s wage rate for the costs associated with training and 

related supervision; and c) is limited in duration as appropriate to the particular 

occupation, taking into account the content of the training, the prior work experience of the 

participant, and the service strategy of the participant, as appropriate. 29 U.S.C. § 

2801(31).
Page 26 GAO-14-137 WIA Training Outcomes

WIA Adult and Dislocated Worker Programs often lacked the relevant 

qualifications and basic skills needed to participate in training that would 

meet the needs of employers seeking employees for in-demand 

occupations.49 In addition, work readiness certificates are generally 

valued by employers, according to several employer group 

representatives and local workforce officials we interviewed.50

Representatives from a few employer groups also noted that, in some 

cases, experience is more important than credentials. For example, local 

officials we interviewed in Illinois said that the vast majority of participants 

in OJT obtained jobs with the employers once their training was 

completed. The officials said OJT provided a good return on investment, 

despite the fact that these participants did not earn credentials.51

Currently, DOL’s credential attainment data do not include participants 

who completed these types of training programs. If credential attainment 

is established as a performance measure, it will be important to consider 

ways to address participants who are enrolled in certain types of training 

that do not lead to a credential, such as by excluding these participants 

from a credential attainment measure or considering if other measures, 

such as basic skill attainment, could capture the value of training provided 

to participants excluded from the measure.

 49 GAO, Workforce Investment Act: Local Areas Face Challenges Helping Employers Fill 

Some Types of Skilled Jobs. GAO-14-19 (Washington, D.C.: December 2, 2013).

50 Work readiness certificates generally refer to awards that demonstrate achievement of 

a certain level of workplace employability skills. One common program is ACT’s National 

Career Readiness Certificate, which can be earned through the Work Keys program. 

51 DOL grants waivers that can exempt a state from reporting credential attainment 

outcomes for participants enrolled in OJT.
Page 27 GAO-14-137 WIA Training Outcomes

Table 4: Establishing a Performance Measure on Credential Attainment: Challenges, Mitigation Strategies, and 

Considerations 

Challenges Potential mitigation strategies Related considerations

Collecting data can be 

resource-intensive

Build relationship with participant early on to 

enhance responsiveness to follow-up efforts.

Build relationship with training provider to enhance 

responsiveness to follow-up efforts.

Have participants waive student privacy rights so 

training providers can share credential information.

Seek automated or direct access to centralized 

credential databases.

Strategies do not guarantee a reduction in the 

resources required because case managers must 

still follow up with participants and training 

providers.

Because data collection can be dependent on 

multiple data sources and the cooperation of 

participants and training providers, these strategies 

do not guarantee that local areas will be able to 

document credential attainment for all participants.

This may result in underreporting. 

Some training does not 

result in a DOL defined 

credential, but may still be 

needed by participants and 

valued by employers

Exclude participants who are not in training that 

potentially leads to a DOL-defined credential from 

the calculation of the measure, by excluding those 

in OJT, for example.

If such participants are not excluded, it could create 

a disincentive to offer some types of training or to 

serve low-skilled participants.

Consider if other measures, such as basic skill 

attainment, could capture the value of training 

provided to participants excluded from the 

measure.

Source: GAO analysis of interviews with selected state and local officials and workforce experts.

Workforce officials in most states we studied identified challenges 

reporting data on training-related employment that were greater than 

those for reporting data on credential attainment, including the high 

degree of resources required and the subjective nature of determining 

whether employment is linked to training. Similar to credential attainment, 

there is no definitive source for these data, so case managers must 

generally collect participants’ employment information from various 

sources, including participants, employers, and UI wage records. Then—

States Face 

Continuing 

Challenges Reporting 

Data on TrainingRelated Employment, 

Despite Efforts by 

States and DOL to 

Improve Reporting

Reporting Data on 

Training-Related 

Employment Is ResourceIntensive and Involves 

Subjective Decisions
Page 28 GAO-14-137 WIA Training Outcomes

in a step beyond what is required for reporting on credential attainment—

they must piece this information together to determine whether 

participants’ employment is substantially related to their training. Also 

unlike reporting on credential attainment, DOL does not require that local 

WIA programs collect and retain documentation on training-related 

employment in the participants’ case files to verify the accuracy of data 

they report to states.52

Officials in four of the six states we studied, as well as at DOL, said this 

data collection process often requires considerable time and effort. 

Further, officials from DOL and four states emphasized the need to 

consider the balance between the time required to collect outcome data 

and the time case managers spend serving participants, especially in an 

environment of reduced resources. In addition, officials in all six states 

said that making training-related employment determinations can be 

subjective. According to DOL’s reporting guidance, participants’

employment is related to their training if it uses “a substantial portion of 

the skills taught in training.”53

 52 DOL does not require documentation on training-related employment because it is not a 

WIA performance measure.

However, officials in all six states, as well as 

workforce experts from one national organization, said that one case 

manager’s interpretation of what constitutes a substantial portion of the 

skills obtained in training may differ from another’s. The training-related 

employment decision can be straightforward if the training and job are 

clearly connected. For example, if a participant received training to attain 

a commercial driver’s license and was subsequently hired as a driver by a 

trucking company, the case manager can easily determine that the 

participant’s employment is substantially related to the training received. 

In other cases, however, the decision may be more subjective. For 

example, officials in one state could not agree whether a participant who 

had received aviation instruction training had secured training-related 

employment in his position as an airframe and power plant mechanic. 

Some state officials thought the skills obtained were transferable, but 

others were unsure.

53 U.S. Department of Labor, Training and Employment Guidance Letter No. 17-09: 

Quarterly Submission of Workforce Investment Act Standardized Record Data (WIASRD)

(March 10, 2010).
Page 29 GAO-14-137 WIA Training Outcomes

Collecting participants’ employment information and attempting to 

determine whether it was training-related generally entails several steps 

(see fig. 5).

Figure 5: General Process for Collecting Participants’ Employment Information and 

Determining Whether It Relates to their Training 

Contacting participants. Several state and local officials we 

interviewed said that they generally begin the process of collecting 

data on training-related employment by attempting to contact 

participants, though some can be unresponsive or inaccessible, which 

workforce experts from one national organization noted as well. If 

case managers are successful, they ask participants for information 

such as the name of their employer and their job title. Some local 

areas also ask participants directly if their new jobs are—in the 

participants’ opinion—related to the training they received. In some 

cases, case managers make their training-related employment 

determinations based solely on information the participants provide 

about their employment.

Contacting employers. Some case managers contact employers to 

obtain participants’ employment information, though employers may 

not be responsive. If case managers could not reach a participant but 

know where the person works, they may contact the employer to 

obtain their job title and description. They may also contact an 

employer to verify the information provided by a participant. Case 

managers can use the employment information obtained from an 

employer, or from both the participant and employer, to determine if a 
Page 30 GAO-14-137 WIA Training Outcomes

participant’s job is training-related. However, workforce officials in 

Illinois and Texas said employers may not be responsive because 

they are concerned about employee privacy or about the amount of 

follow-up required.

Checking UI wage records. Some local workforce officials said that if 

they are unable to gather information about a participant’s 

employment from the participant or the employer, they check the UI 

wage records, which are generally not available until several months 

after a participant exits from WIA services.54

Other steps to determine training-related employment. If 

successful in accessing a participant’s employment information, 

including job title, some workforce officials said case managers use 

DOL’s Occupational Information Network (O*NET), which provides an 

online tool to match job title occupational codes to the skills code 

associated with the participant’s training. This can help case 

managers decide whether the participant’s employment is trainingrelated.

DOL officials and 

workforce experts at two national organizations said the UI wage 

records generally provide the name of the participant’s employer and 

a code associated with the employer’s industry, but specific 

information on the participant’s occupation is rarely included. In some 

cases, the industry code has a clear connection to the training 

received, making the case manager’s training-related employment 

determination straightforward. For example, if a participant who was 

trained as a nurse was hired by a hospital, the case manager can 

reasonably assume that the employment and training are related. 

However, some officials noted that the industry code is not always a 

good predictor of a participant’s occupation. For example, if the same 

participant was hired by the health unit of a manufacturing company, 

the industry code in the UI wage records would suggest that the 

person’s job was associated with manufacturing and not related to the 

nursing training.

55

 54 UI wage data are collected quarterly, meaning that participants who find employment 

will not show up in the data until several months after they are hired. As a result, case 

managers must wait several months to use these data.

55 DOL’s O*NET database provides detailed information about many occupations. The 

database is online, available to the public at no cost, and is continually updated by 

surveying a broad range of workers from each occupation.
Page 31 GAO-14-137 WIA Training Outcomes

According to some local workforce officials, however, it can be difficult 

to find the precise occupational code that matches the participant’s 

new job. In addition, DOL officials told us that even attempting to 

match O*NET codes in this manner might not help case managers 

determine if participants’ employment is related to their training 

because the threshold of relatedness is still subjective, as mentioned 

previously. DOL officials also told us it is difficult to prescribe a 

standard definition for determining whether a job is related to training 

because it often requires some judgment on the part of local officials.

To improve reporting, some states have taken steps to increase their 

access to information about participants’ employment. Similar to their 

efforts to collect information on credential attainment, some local 

workforce officials told us that case managers seek to build rapport with 

participants. At the same time, officials we interviewed in four of six states 

have taken steps to increase access to employment-related information. 

In Illinois, Rhode Island, and Texas, local workforce officials ask 

participants to sign release forms authorizing employers to release 

employment information to officials. Local workforce staff in Kansas and 

Texas also said they obtain information on participants’ employment, 

including the names of their employers, and—unlike UI wage records—

their job titles, by subscribing to an online payroll database called The 

Work Number. This service verifies employment via a database of 

national payroll data but does not include all employers, and local WIA 

programs must pay to subscribe.

In addition, local workforce officials in four states we contacted said they 

have developed strategies to help reduce the subjectivity in determining 

whether a participant’s employment is related to their training. For 

example, local officials in Texas told us that staff may consult their 

American Job Center’s local business services office, which often has 

specific knowledge about what skills correspond with particular job titles. 

Officials in Texas and Alabama also said case managers may consult 

with their peers or supervisors to reach consensus about a trainingrelated employment decision. Further, state workforce officials in Kansas 

told us that when employers post jobs on the state’s job bank, they are 

required to enter occupational codes from DOL’s online O*NET database. 

If a training participant gets one of these jobs, case managers can 

compare the employer-provided occupational code with the training codes 

supplied by training providers to help them determine whether the job is 

States Have Taken Steps 

to Address Challenges 

by Improving Reporting 

on Training-Related 

Employment

Washington state collects additional 

information on training-related 

employment

In addition to reporting data on trainingrelated employment to DOL, Washington 

state surveys a subset of WIA participants 

three quarters after exit and asks them about:

• the extent to which their new job is 

related to their training,

• the extent to which the training they 

received helped them get their job, and

• how useful the skills they learned in 

training are in doing their job.

Officials said they conduct this survey for 

internal use to collect and share data on 

training-related employment that is not 

otherwise available to program managers and 

state officials. Another official noted that a 

significant percentage of respondents who do 

not report their job as training-related find the 

training instrumental in getting the job or that 

the skills they acquired in training are useful 

on their job.

Source: Interview with state officials in Washington and 

survey documents.
Page 32 GAO-14-137 WIA Training Outcomes

training-related. State and local officials in Washington also said 

managers routinely use O*NET when making their training-related 

employment decisions.

While DOL has recently issued guidance aimed at increasing reporting 

rates for training-related employment data, it has taken limited steps to 

address states’ ongoing reporting challenges. As previously discussed, a 

September 2011 report by DOL’s Office of Inspector General raised 

questions about the quality of these data and made recommendations to 

address this issue.56 In response, DOL issued a notice to states in 

September 2013 that reiterated the requirement for states to report these 

data and stressed the data’s importance for program analysis and 

evaluation efforts pertaining to the value of investments in WIA training.57

The DOL notice also acknowledged that reporting on training-related 

employment is challenging largely because the information must be 

collected manually. According to the DOL notice, nearly every state that 

participated in conference calls on the topic indicated that there was 

considerable cost in conducting the necessary follow-up for reporting on 

training-related employment and that this was the primary reason the data 

were not well-reported. State and local officials we interviewed also noted 

that such data may be underreported because of the difficulty of following 

up with participants and employers. DOL also concluded that states with 

larger training caseloads had less complete reporting on training-related 

employment. In the notice, DOL also described a few practices most 

common among states with higher reporting rates for training-related 

employment.58

 56 U.S. Department of Labor, Office of Inspector General, Additional Information Needed 

to Measure the Effectiveness and Return on Investment of Training Services Funded 

Under the WIA Adult and Dislocated Worker Programs, Report Number 03-11-003-03-390 

(September 30, 2011).

For example, DOL cited the practice of instituting a data 

system check to ensure that training-related employment data are 

recorded before closing a participant’s case file.

57 U.S. Department of Labor, Training and Employment Notice No. 5-13: Reporting 

Training-Related Employment under the Workforce Investment Act (WIA) Programs

(September 4, 2013).

58 DOL also established a new WIASRD response option to isolate situations in which 

case managers do not know if participants’ employment was training-related. 

DOL Has Taken Limited 

Steps to Address 

Challenges and Improve 

Data on Training-Related 

Employment
Page 33 GAO-14-137 WIA Training Outcomes

DOL has not identified and disseminated strategies for increasing access 

to employment-related data or helping to minimize the subjectivity of 

training-related determinations, but instead has focused exclusively on 

increasing data reporting rates. While DOL officials maintain that manual 

follow-up with participants is the best approach for obtaining employment 

related data, they also recognize that such data collection is resourceintensive. In addition, as officials in all six of our selected states also 

noted, DOL officials acknowledged that determining whether a 

participant’s employment is training-related can be a subjective decision. 

They noted challenges in defining training-related employment more 

precisely. For example, officials said some training is intended to develop 

broad, nonspecific skills that may help participants get jobs but are not 

associated with a specific occupation or industry. We recognize that 

utilizing professional judgment is inherent in certain tasks such as 

determining whether a participant’s employment is related to the training 

the participant received. However, minimizing the amount of subjective 

decision-making involved to the extent possible could help ensure better 

quality data on training-related employment. Reasonable approaches for 

improving the quality of performance data focus on aspects of 

completeness, accuracy, consistency, and ease of use.59

In addition to the strategies all six selected states use to mitigate 

reporting challenges, workforce officials from three states and workforce 

experts at two national organizations said some additional considerations 

should be taken into account in weighing a performance measure on 

training-related employment (see table 5). We previously noted concerns 

about the level of resources required and the subjectivity of 

determinations, both of which could affect the data’s completeness and 

consistency—key aspects of performance data quality. The state and 

local strategies we identified may help mitigate some reporting 

challenges. In addition, a participant’s successful placement in a trainingrelated job depends on both the ability and decision to pursue such a 

position. Some local workforce officials we contacted said that it may take 

By identifying 

and sharing with states practices to increase access to employmentrelated data and reduce the subjectivity of some determinations, DOL 

could help states improve their reporting of data on training-related 

employment.

 59 GAO, Performance Plans: Selected Approaches for Verification and Validation of 

Agency Performance Information, GAO/GGD-99-139 (Washington, D.C.: July 30, 1999).

Use of crosswalk not necessarily a 

solution for determining training-related 

employment

Another program DOL administers, Job 

Corps—a residential, educational, and career 

technical training program for disadvantaged 

youths—uses a crosswalk that links specific 

training codes and occupational codes to help 

staff determine training-related employment. 

However, in a September 2011 report, the 

DOL Inspector General found that this 

crosswalk included some matches that were 

either not related or poorly related. Moreover, 

DOL officials said that using a crosswalk for 

the WIA Adult and Dislocated Worker 

programs could make the trainingemployment link too restrictive and would 

require a considerable amount of resources to 

develop.

Source: Interviews with DOL officials and U.S. Department of 

Labor Office of Inspector General Audit Report No. 26-11-

004-03-370, Job Corps Needs to Improve Reliability of 

Performance Metrics and Results, September 30, 2011.
Page 34 GAO-14-137 WIA Training Outcomes

participants longer to find employment in their field of training than is 

allowed for reporting purposes.60 For example, workforce officials in 

Rhode Island said that due to the poor economy in their state, it is not 

uncommon for some participants to take 2 years or more to find a job. 

Additionally, workforce officials in three states we contacted said 

participants may decide to take a job unrelated to their training if it is the 

only job they can find or if they simply choose not to pursue a job in the 

field in which they were trained. 

 60 According to DOL officials, states must report on the number of participants who find 

training-related employment within three quarters of their exit from WIA services.
Page 35 GAO-14-137 WIA Training Outcomes

Table 5: Establishing a Performance Measure on Training-Related Employment: Challenges, Potential Mitigation Strategies, 

and Considerations

Challenges Potential mitigation strategies Related considerations

Collecting data can 

be resource-intensive

Build relationship with participant early on to enhance 

responsiveness to follow-up efforts.

Have participants sign release forms so employers 

can share credential information.

Subscribe to payroll record databases (e.g. The Work 

Number).

Require employers to report occupational codes to 

state job banks.

Strategies do not guarantee a reduction in the 

resources required to collect data because case 

managers must still follow-up with participants and 

employers.

Because data collection can depend on multiple data 

sources and the cooperation of participants and 

employers, these strategies do not guarantee that 

local areas will be able to obtain necessary 

employment information. Participants and employers 

may be unresponsive, or employers may not report 

complete data on their employees. This may result in 

underreporting.

State and local workforce agencies may not have the 

resources to subscribe to payroll record databases.

Requiring employers to report occupational codes 

could place more burdens on employers.

Determining whether 

employment is related 

to training is 

subjective

Consult the American Job Center’s local business 

services office.

Discuss internally and reach consensus with 

supervisors or peers.

Use O*NET to find occupational codes associated 

with job titles and match to training code.

Note: The strategies presented in the row above may 

also help mitigate the subjectivity of determining 

training-related employment.

Additional input may not help make a valid 

determination.

Using O*NET to find specific occupational codes and 

matching to training codes may still involve some 

subjectivity.

It is generally straightforward to make training-related 

determinations for some types of training such as OJT 

and customized training for employers.

Participants may not 

find a training-related 

job before the 

reporting deadline

Provide job search and placement assistance to help 

participants find training-related jobs.

Strategies do not guarantee a successful placement in 

a training-related job. 

Participants may 

choose to take a job 

unrelated to their 

training 

Counsel participants about related jobs before placing 

them in training to help ensure the job they are 

training for is of interest to them.

Participants may still decide to take a job unrelated to 

their training.

Source: GAO analysis of interviews with selected state and local officials and workforce experts.

To ensure that public funds invested in WIA’s Adult and Dislocated 

Worker Programs are spent wisely, program managers and policymakers 

need performance data that are accessible, complete, accurate and 

consistent. The current common performance measures—employment, 

retention, and earnings—provide a basis for assessing the overall value 

of the services the programs provide, primarily using a standardized data 

source (state UI wage records). Beyond these, data on outcomes such as 

Conclusions
Page 36 GAO-14-137 WIA Training Outcomes

credential attainment and training-related employment can potentially 

provide information more specifically on the value of training services. 

However, as we have noted, collecting data on these outcomes can be 

more resource-intensive, in part because there is no single readily 

available source of data.

DOL has taken steps to elevate the importance of credential attainment 

and improve data quality for this outcome. We found credential attainment 

data reported to DOL to be reliable. In contrast, we found the data 

reported to DOL on training-related employment to be incomplete and 

inconsistent. While DOL has acknowledged challenges in collecting data 

on and determining training-related employment, it has taken only limited 

steps to address these challenges, focusing efforts exclusively on 

improving reporting rates. This effort alone will not improve the quality of 

the data being reported. Given the nature of the challenges we identified, 

there are no easy or complete solutions. However, we also identified 

strategies some states use that may help increase access to employment 

information and reduce the subjectivity of some training-related 

determinations. Sharing such strategies with other states, as well as 

identifying and communicating other approaches, could lead to 

incremental improvement in the quality of data reported. Without such 

action, the data states are required to report on training-related 

employment are likely to remain unusable.

To provide policymakers and program managers with better quality 

information to assess the value of training provided by WIA’s Adult and 

Dislocated Worker Programs, we recommend that the Secretary of Labor 

identify and share with states strategies for collecting and reporting data 

on training-related employment that could either increase access to 

employment information or reduce the subjectivity of determining when 

training is related to employment.

We provided a draft of this report to the Secretary of Labor and selected 

draft sections to the Secretary of Education. DOL and Education provided 

technical comments, which we incorporated as appropriate, and DOL 

provided a written response (see app. II). DOL agreed with our 

recommendation and noted that having reliable data on training-related 

employment is important to effectively manage and evaluate the Adult 

and Dislocated Worker Programs. DOL also agreed that states can 

benefit from learning what other states are doing to address challenges 

regarding access to and subjectivity of these data. Toward this end, DOL 

Recommendation for 

Executive Action

Agency Comments 

and Our Evaluation
Page 37 GAO-14-137 WIA Training Outcomes

noted that it plans to conduct additional conference calls with state 

officials to reiterate the importance of identifying training-related 

employment and continue to discuss and share best practices to improve 

these data. DOL noted that this sharing of best practices would 

supplement actions it has already taken to improve data on trainingrelated employment. These actions include coding changes to the 

WIASRD to capture additional information, conference calls with state 

workforce officials to discuss reporting on training-related employment, 

and a work group considering adding more data elements to the UI wage 

records such as an occupational code. 

As agreed with your office, unless you publicly announce its contents 

earlier, we plan no further distribution of this report until 30 days from its 

issue date. At that time, we will send copies of this report to the 

appropriate congressional committees, the Secretary of Labor, the 

Secretary of Education, and other interested parties. In addition, the 

report is available at no charge on the GAO website at 

http://www.gao.gov.

If you or your staff have any questions about this report, please contact 

me at (202) 512-7215 or sherrilla@gao.gov. Contact points for our Offices 

of Congressional Relations and Public Affairs may be found on the last 

page of this report. GAO staff who make key contributions to this report 

are listed in appendix III.

Andrew Sherrill

Director

Education, Workforce,

and Income Security Issues
Appendix I: Objectives, Scope, and 

Methodology

Page 38 GAO-14-137 WIA Training Outcomes

Our objectives for this study on the Workforce Investment Act’s (WIA) 

Adult and Dislocated Worker Programs were to examine: 1) the extent to 

which training participants obtain credentials and secure training-related 

employment, 2) challenges states face reporting on credential attainment 

and what steps, if any, they and DOL are taking to address them, 3) 

challenges states face in reporting on training-related employment and 

what steps, if any, they and DOL are taking to address them.

To address these objectives, we reviewed relevant federal laws, 

regulations, and DOL’s guidance to states for reporting select data on 

participants in the WIA Adult and Dislocated Worker Programs. We 

interviewed DOL officials from the Employment and Training 

Administration and the Office of Inspector General as well as experts on 

workforce issues (see Selection of Experts, below). We also interviewed 

state and local workforce officials as well as organizations that provided 

an employer perspective from a nongeneralizeable sample of six states 

(see Selection of States and Local Areas, below). To assess the reliability 

of the data DOL collects on credential attainment and training-related 

employment, we analyzed data from the Workforce Investment Act 

Standardized Record Data (WIASRD) system for program year 2010 and 

program year 2011—the most recent years for which data were 

available—by testing the data electronically and interviewing 

knowledgeable agency officials.1 We found the data to be sufficiently 

reliable for providing estimates on: 1) participants who received training, 

2) the minimum number of training participants who earned a credential, 

and 3) the types of credentials they earned. However, we determined that 

the data on training-related employment were not reliable for the 

purposes of our report (see Analysis of DOL’s Training Outcome Data 

below).

We interviewed experts on workforce issues representing six 

organizations. We identified experts by first reviewing relevant literature 

and asking officials from DOL for recommendations. We further 

 1 Program year 2011 runs from July 1, 2011 through June 30, 2012. WIASRD is a national 

database of individual records with information on the characteristics, activities, and 

outcomes for all enrolled participants who received services or benefits under WIA. 

Participant data are typically collected by frontline staff at local American Job Centers and 

entered into a state or local data system. After the state receives data from local areas, it 

compiles and formats the information and submits it electronically to DOL.

Appendix I: Objectives, Scope, and 

Methodology

Selection of Experts
Appendix I: Objectives, Scope, and 

Methodology

Page 39 GAO-14-137 WIA Training Outcomes

developed the list by asking each expert we interviewed for additional 

names. Only experts who were mentioned more than once were selected. 

The results of these interviews are not generalizeable, but provided 

insights about the overall challenges states face in reporting on these 

outcomes and any efforts by states to overcome them.

We interviewed state and local workforce officials from a 

nongeneralizeable sample of six states. We conducted in-person 

interviews with officials in Alabama, Illinois, and Texas and telephone 

interviews with officials in Kansas, Rhode Island, and Washington (see 

table 4). These results are not generalizeable, but provided insights about 

the challenges they face in reporting data on credential attainment and 

training-related employment as well as any steps they are taking to 

address those challenges. We selected the states to reflect a mix of those 

that had leading-edge data practices (as identified by experts) and those 

with either incomplete data or relatively high rates of reporting errors (as 

identified by WIA data quality reports on credential attainment).2 We also 

factored geographic diversity into state selection. In each state, we 

interviewed state workforce officials and also local workforce officials from 

at least one workforce investment board and at least one American Job 

Center—formerly known as a one-stop center. We selected a nongeneralizeable sample of local areas based on input from state workforce 

officials and, for states we visited, proximity to the state workforce 

agency. In addition, we interviewed at least one employer organization in 

each state. In selecting these entities, we considered states’ input on 

organizations that could provide us with an employer perspective on the 

value of credentials and certain types of training for various industries. 

They included statewide business associations, regional business 

associations, individual employers, and industry-specific representatives.

 2 Error rates were not available for data on training-related employment, so we did not 

consider these data in state selection.

Selection of States and 

Local Areas
Appendix I: Objectives, Scope, and

Methodology

Page 40 GAO-14-137 WIA Training Outcomes

Table 6: Selected Site Visit States and Local Areas 

State Workforce Investment Board City of American Job Center 

Alabama Alabama Workforce Investment Area Montgomery 

Illinois Chicago Cook Workforce Partnership Chicago 

Kansas Workforce Alliance of South Central Kansas Wichita

Rhode Island Workforce Solutions of Providence/Cranston Providence

Texas Workforce Solutions: Rural Capital Area Round Rock

Workforce Solutions: Alamo San Antonio

Washington Workforce Development Council of Seattle-King County Seattle

Source: GAO state and local areas interviews

In each state, we obtained information about the state and local area’s 

process of collecting data on credentials and employment related to 

training as well as any challenges they may have encountered. We also 

asked state officials about DOL’s related guidance. We used a semistructured interview guide for the state and local interviews.

To assess the reliability of DOL’s data on training, credential attainment, 

and training-related employment in the WIASRD database for participants 

in the WIA Adult and Dislocated Worker programs, we: (1) reviewed 

documentation related to reporting these data, including DOL’s Office of 

Inspector General reports; (2) tested the data electronically to identify 

potential problems with consistency, completeness, or accuracy; and (3) 

interviewed knowledgeable DOL officials about the data. Our electronic 

testing consisted of identifying inconsistencies, outliers, and missing 

values. In addition, we analyzed the publicly-available WIASRD data file 

for program years 2010 and 2011, which was produced for DOL by its 

data contractor, Social Policy Research Associates. As part of our 

analysis, we reviewed the steps the data contractor took to address data 

errors and, to the extent possible, compared the data DOL provided for 

our analysis to the publicly-available file, and found only slight 

discrepancies. We found the data on training and credential attainment to 

be sufficiently reliable for reporting estimates of: (1) participants who 

received training, (2) the minimum number of training participants who 

earned a credential, and (3) the types of credentials they attained. We 

were not able to reliably make state-to-state comparisons because two 

states are piloting a new reporting format for DOL, and therefore would 

not have been compatible with the others. For the purposes of this report, 

we did not find the data on training-related employment reliable. We 

Analysis of DOL’s Training 

Outcome Data
Appendix I: Objectives, Scope, and 

Methodology

Page 41 GAO-14-137 WIA Training Outcomes

reached this conclusion based on our analysis of the data, an Office of 

Inspector General report, and DOL’s data quality reports. We were not 

able to determine how many training participants in the Adult or 

Dislocated Worker Programs obtained employment related to their 

training in program year 2011. For the Adult Program, we found that 

states reported data on 48 percent of training participants, but had 

missing data for the remaining 52 percent. For the Dislocated Worker 

Program, states reported data on 74 percent of training participants, but 

had missing data for the remaining 26 percent. Further, an analysis of the

reported data showed wide variation among states regarding the 

percentage of participants who obtained training-related employment 

raising questions about the data’s reliability.

We conducted this performance audit from October 2012 to January 2014 

in accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit work to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives.
Appendix II: Comments from the

Department of Labor

Page 42 GAO-14-137 WIA Training Outcomes

Appendix II: Comments from the

Department of Labor
Appendix II: Comments from the

Department of Labor

Page 43 GAO-14-137 WIA Training Outcomes
Appendix III: GAO Contact and Staff 

Acknowledgments

Page 44 GAO-14-137 WIA Training Outcomes

Andrew Sherrill, (202) 512-7215 or sherrilla@gao.gov.

In addition to the contact named above, Laura Heald, Assistant Director, 

John Lack, Jeffrey G. Miller, and Kathryn O’Dea Lamas made key 

contributions to this report. Also contributing to this report were James 

Bennett, Jessica Botsford, David Chrisinger, Elizabeth Curda, Kathy 

Leslie, Carol Patey, Rhiannon Patterson, Jerry Sandau, and Walter 

Vance.

Appendix III: GAO Contact and Staff 

Acknowledgments

GAO Contact:

Staff 

Acknowledgments
Related GAO Products

Page 45 GAO-14-137 WIA Training Outcomes

Workforce Investment Act: Local Areas Face Challenges Helping 

Employers Fill Some Types of Skilled Jobs. GAO-14-19. Washington, 

D.C.: December 2, 2013.

Workforce Investment Act: DOL Should Do More to Improve the Quality 

of Participant Data. GAO-14-4. Washington, D.C.: December 2, 2013.

Postsecondary Education: Many States Collect Graduates’ Employment 

Information, but Clearer Guidance on Student Privacy Requirements Is 

Needed. GAO-10-927. Washington, D.C.: September 27, 2010.

Workforce Investment Act: Employers Found One-Stop Centers Useful in 

Hiring Low-Skilled Workers; Performance Information Could Help Gauge 

Employer Involvement. GAO-07-167. Washington, D.C.: December 22, 

2006.

Workforce Investment Act: Labor and States Have Taken Actions to 

Improve Data Quality, but Additional Steps Are Needed. GAO-06-82. 

Washington, D.C.: November 14, 2005.

Workforce Investment Act: Substantial Funds Are Used for Training, but 

Little Is Known Nationally about Training Outcomes. GAO-05-650. 

Washington, D.C.: June 29, 2005.

Workforce Investment Act: Labor Should Consider Alternative 

Approaches to Implement New Performance and Reporting 

Requirements. GAO-05-539. Washington, D.C.: May 27, 2005.

Workforce Investment Act: States and Local Areas Have Developed 

Strategies to Assess Performance, but Labor Could Do More to Help. 

GAO-04-657. Washington, D.C.: June 1, 2004.

Workforce Investment Act: Labor Actions Can Help States Improve 

Quality of Performance Outcome Data and Delivery of Youth Services. 

GAO-04-308. Washington, D.C.: February 23, 2004.

Workforce Investment Act: Improvements Needed in Performance 

Measures to Provide a More Accurate Picture of WIA’s Effectiveness. 

GAO-02-275. Washington, D.C.: February 1, 2002.

Related GAO Products

(131213)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.

Visit GAO on the web at www.gao.gov.

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.