Office of Inspector General—Office of Audit 

EMPLOYMENT AND TRAINING 

ADMINISTRATION 

ADDITIONAL INFORMATION NEEDED TO 

MEASURE THE EFFECTIVENESS AND RETURN 

ON INVESTMENT OF TRAINING SERVICES 

FUNDED UNDER THE WIA ADULT AND 

DISLOCATED WORKER PROGRAMS 

Date Issued: September 30, 2011 

Report Number: 03-11-003-03-390 
U.S. Department of Labor 

Office of Inspector General 

Office of Audit 

BRIEFLY... 

Highlights of Report Number 03-11-003-03-390, issued 

to the Assistant Secretary for Employment and Training.

WHY READ THE REPORT

The Office of Inspector General (OIG) conducted a 

performance audit of training services provided under 

the Workforce Investment Act (WIA) Adult and 

Dislocated Worker programs. The U.S. Department of 

Labor’s (DOL) Employment and Training Administration 

(ETA) awards more than $2 billion annually in formula 

grants to State Workforce Agencies (SWA) to operate 

the WIA Adult and Dislocated Worker programs. 

Training Services is one of the three tiers, or levels, of 

services provided under WIA. However, the amount of 

funding spent on each of the service tiers is not 

available because fund allocation is left to the discretion 

of the SWAs, and ETA does not collect this information. 

WIA established a comprehensive performance 

accountability system to assess the effectiveness of 

SWAs in achieving continuous improvement of 

workforce investment activities in order to optimize the 

return on investment of federal funds. The core 

indicators of performance are entry into unsubsidized 

employment; retention in unsubsidized employment 6 

months after entry into the employment; and earnings 

received in unsubsidized employment 6 months after 

entry into the employment. 

WHY OIG CONDUCTED THE AUDIT 

The audit objectives were to answer the following 

questions: 

1. Was ETA’s comprehensive performance 

accountability system able to assess the 

effectiveness of training services provided under 

the WIA Adult and Dislocated Worker programs? 

2. What were the results of the training services 

provided under these programs, and were the 

training services linked to demand occupations? 

READ THE FULL REPORT 

To view the report, including the scope, methodology, 

and full agency response, go to: 

http://www.oig.dol.gov/public/reports/oa/2011/03-11-

003-03-390.pdf. 

 September 2011 

ADDITIONAL INFORMATION NEEDED TO MEASURE 

THE EFFECTIVENESS AND RETURN ON INVESTMENT 

OF TRAINING SERVICES FUNDED UNDER THE WIA 

ADULT AND DISLOCATED WORKER PROGRAMS

WHAT OIG FOUND 

WIA limits ETA’s ability to include the results of training 

services in its WIA performance accountability system. 

Although the system ETA developed complied with 

WIA, it did not provide sufficient information on 

performance results and the cost of training services for 

which ETA paid. For the 20 SWAs in our audit universe, 

we estimated that ETA spent about $307 million on 

training services to 103,340 exiters during the period 

April 2008 to March 2009. 

Because ETA had limited information on the outcomes 

and cost of training services, we analyzed a sample of 

362 Adult and Dislocated Worker program exiters who 

received training services. Our analysis showed that in 

almost every case the exiters received training related 

to a demand occupation and 86 percent completed their 

training. Employment outcomes for the sampled exiters 

showed 85 percent obtained employment after training. 

Of those who obtained employment, 82 percent were 

still employed in the second quarter after exiting. 

Our analysis also showed that 37 percent of the 

sampled exiters either did not obtain employment or 

their employment was unrelated to the training they 

received. Specifically, almost 13 percent did not obtain 

employment, and for those who did, almost 24 percent 

were in jobs not related to their training. Additionally, for 

another 10 percent of exiters, the SWAs could not 

provide sufficient evidence to demonstrate whether job 

placement was training related. We estimated that 

$124 million was spent on training participants who did 

not obtain training-related employment, or there was 

insufficient information to make that determination. 

WHAT OIG RECOMMENDED

The OIG recommended that the ETA Assistant 

Secretary pursue legislative authority to develop 

performance measures for training outcomes, and 

require SWAs to report training costs at the participant 

level; and to exercise oversight over SWAs to ensure 

they develop practices to increase the percentage of 

exiters who find training-related employment. 

ETA did not believe the report put the findings in the 

proper perspective and did not agree with the 

recommendations to pursue legislative authority on 

performance measures for training outcomes and to 

collect training costs and funding sources. 
 

 

U.S. Department of Labor – Office of Inspector General 

WIA Adult & Dislocated Worker Training Services 

Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

Table of Contents 

Assistant Inspector General’s Report......................................................................... 1 

Results In Brief............................................................................................................... 2 

Objective 1 — Was ETA's performance accountability system able to assess the 

effectiveness of training services provided under the WIA Adult 

and Dislocated Worker programs?..................................................... 5 

ETA did not have sufficient information to assess the performance results 

of training services for which it paid more than an estimated $307 million. .......... 5 

Finding 1 — ETA's performance accountability system was not able to 

assess the effectiveness of training services provided under 

the WIA Adult and Dislocated Worker programs............................... 5 

Objective 2 — What were the results of the training services provided under the 

WIA Adult and Dislocated Worker programs, and were the training 

services linked to demand occupations? ........................................ 10 

An estimated $124 million was spent on training Adult and Dislocated 

Worker exiters who did not obtain training-related employment or there 

was insufficient information to make the determination. ..................................... 10 

Finding 2 — More that 80 percent of the exiters who received training 

services under the Adult and Dislocated Worker program 

obtained employment but not always related to the training 

they received. ................................................................................. 10 

Recommendations ...................................................................................................... 14 

Exhibit

Exhibit Sample Results and Projections ............................................................. 17 

Appendices

Appendix A Background ..................................................................................... 21 

Appendix B Objectives, Scope, Methodology, and Criteria ................................ 25 

Appendix C Acronyms and Abbreviations .......................................................... 29 

Appendix D ETA Response to Draft Report ....................................................... 31 

Appendix E Acknowledgements ......................................................................... 37 

WIA Adult & Dislocated Worker Training Services 

Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

 PAGE INTENTIONALLY LEFT BLANK 

WIA Adult & Dislocated Worker Training Services 

Report No. 03-11-003-03-390 
 

 

U.S. Department of Labor – Office of Inspector General 

U.S. Department of Labor Office of Inspector General 

Washington, D.C. 20210 

September 30, 2011 

Assistant Inspector General’s Report 

Jane Oates 

Assistant Secretary for Employment and Training 

Employment and Training Administration 

200 Constitution Avenue, N.W 

Washington, D.C. 20210 

The U.S. Department of Labor’s (DOL) Employment and Training Administration (ETA) 

awards more than $2 billion annually in formula grants to State Workforce Agencies 

(SWA) to operate the Workforce Investment Act (WIA) Adult and Dislocated Worker 

programs. The Adult and Dislocated Worker programs provide for three tiers, or levels, 

of services — core, intensive, and training. The amount of funding spent on each of the 

service tiers is not available because allocation of the funds between the service tiers is 

left to the discretion of the SWAs and ETA does not collect the information. Within the 

SWAs, Local Workforce Agencies (LWA) are responsible for providing these services to 

eligible participants. In order to receive training services, participants must be unable to 

obtain employment through intensive services and must require assistance beyond 

what is available through other grant programs. WIA requires training services to be 

directly linked to occupations that are in demand in the local area or in another area 

where the participant is willing to relocate, although exceptions are allowed in some 

cases. WIA, Section 136, established a comprehensive performance accountability 

system to assess the effectiveness of SWAs in achieving continuous improvement of 

workforce investment activities in order to optimize the return on investment of federal 

funds. WIA specified the core indicators of performance as being entry into 

unsubsidized employment; retention in unsubsidized employment 6 months after entry 

into the employment; and earnings received in unsubsidized employment 6 months after 

entry into the employment. 

The audit objectives were to answer the following questions: 

1. Was ETA’s comprehensive performance accountability system able to assess 

the effectiveness of training services provided under the WIA Adult and 

Dislocated Worker programs? 

2. What were the results of the training services provided under these programs, 

and were the training services linked to demand occupations? 

WIA Adult & Dislocated Worker Training Services 

1 Report No. 03-11-003-03-390 
 

 

U.S. Department of Labor – Office of Inspector General 

The audit focused on training services provided and outcomes achieved under the WIA 

Adult and Dislocated Worker programs for 103,430 exiters from 20 SWAs1 between 

April 1, 2008, and March 31, 2009. This was the most current period available for review 

because outcome data is not available until 12-15 months after participants exit the 

program. The audit covered processes in place at the time of our fieldwork from January 

through July 2011. We reviewed case file documentation for a statistical sample of 362 

exiters from 6 SWAs to determine if the training provided was linked to a demand 

occupation, if the exiter completed the training, if the exiter found employment and if it 

was related to the training received, and if the exiter retained employment. The audit 

also included a determination of the extent to which SWAs and LWAs identified demand 

occupations. 

We conducted this performance audit in accordance with generally accepted 

government auditing standards. Those standards require that we plan and perform the 

audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that the evidence 

obtained provides a reasonable basis for our findings and conclusions based on our 

audit objectives. 

RESULTS IN BRIEF 

WIA limits ETA’s ability to include the results of training services in its WIA performance 

accountability system. The WIA performance accountability system that ETA developed 

complied with WIA, Section 136 in that it included performance measures for reporting 

whether exiters found and retained employment. However, it did not provide sufficient 

information on the performance results of training services for which ETA paid. WIA 

does not allow ETA to establish any new performance measures apart from the core 

employment indicators required in WIA, Section 136. Therefore, ETA could not use data 

it collected on training-related employment through its Workforce Investment Act 

Standard Record Data (WIASRD) for performance accountability purposes. Additionally, 

our review of the training service data in WIASRD found it was incomplete and 

unreliable. Moreover, ETA did not collect information on training costs and was unable 

to identify the funding source(s) of training services, which could be paid for by sources 

other than WIA. ETA claimed that WIA does not allow the agency to collect information 

on the cost of training activities, but we believe WIA does not prevent ETA from 

requesting SWAs to report these costs at the participant level. 

As a result, there is limited information available at the national level about the results 

and cost of the training services provided by the Adult and Dislocated Worker programs. 

ETA is unable to report if the training services provided by these programs are 

achieving intended results and at what cost. For the 20 SWAs in our audit universe, we 

estimated that ETA spent about $307 million on training services during the period 

April 2008 to March 2009. 

1 Exiters are participants who enrolled, received services, and exited from a WIA program. The 20 SWAs were 

responsible for 79 percent of the exiters who received training services nationwide within the audit time frame. 

WIA Adult & Dislocated Worker Training Services 

2 Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

Because ETA had limited information on the outcomes and cost of training services, we 

performed an analysis of the 362 sampled Adult and Dislocated Worker program exiters 

who received training services. Our analysis of the sampled exiters showed that in 

almost every case (98 percent), they received training related to a demand occupation. 

Although ETA did not have a measure or goal for the rate at which training was 

completed, we found that 86 percent of the sampled exiters completed their training, 

and their employment outcomes indicated that 85 percent obtained employment after 

training. Of those who obtained employment, 82 percent were still employed in the 

second quarter after exiting the programs. 

However, our analysis also showed that 37 percent of the sampled exiters either did not 

obtain employment or their employment was unrelated to the training they received. 

Specifically, almost 13 percent of the sampled exiters did not obtain employment, and 

for those who did, almost 24 percent were in jobs that were not related to the training 

they received. Additionally, for another 10 percent the SWAs could not provide sufficient 

evidence to demonstrate whether the job placement was training related. Projecting 

these results to our audit population of 103,340 exiters, we estimate that $124 million 

was spent on training participants who did not obtain training-related employment or 

there was insufficient information to make that determination. 

These results demonstrate the importance of having performance and cost information 

to assess the effectiveness of training services. ETA is not in position to report to 

stakeholders the outcomes and cost of training services and areas that could be 

improved. Also, SWAs and LWAs lacked goals for placing exiters in training-related 

employment. Information on training services is especially important because recent 

trends show the annual amount of funds allotted to SWAs for the WIA Adult and 

Dislocated Worker programs is decreasing while the number of participants is 

increasing. Usable and reliable information on training costs and training-related 

employment would assist ETA and the SWAs in determining how to best allocate 

decreasing WIA funds to those services that will achieve the desired results of enabling 

participants to pursue viable career paths leading to self-sufficiency. Such information 

would also improve accountability and transparency over WIA funds invested in training 

participants for demand occupations. 

We recommend that the ETA Assistant Secretary pursue legislative authority in the WIA 

reauthorization to develop performance measures for training outcomes; require SWAs 

to report training costs and funding sources at the participant level so stakeholders have 

adequate information to make return-on-investment decisions for WIA services; develop 

and provide guidance to SWAs and LWAs regarding the best methodology for collecting 

and reporting data for training-related employment; and exercise oversight over SWAs 

to ensure they develop and/or identify best practices to increase the percentage of 

exiters who find employment related to the training they receive. 

WIA Adult & Dislocated Worker Training Services 

3 Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

ETA RESPONSE 

In response to the draft report, ETA disagreed that “there is limited information available 

at the national level about the results and cost of the training services provided by the 

Adult and Dislocated Worker programs.” ETA said it currently collects data that allows it 

to calculate outcomes for each participant who receives training; and OIG’s suggestion 

that the data is “limited” is neither fair nor accurate. ETA also believed the report tended 

to overstate deficiencies and understate positive findings. 

ETA did not agree with the recommendation to pursue legislative authority in WIA 

reauthorization to develop performance measures for training outcomes because it may 

exert a diluting effect on accountability. ETA stated that it measures the system on its 

ability to improve outcomes for all customers regardless of the services provided. 

ETA also disagreed with the recommendation to require SWAs to report training costs 

and funding sources in WIASRD because its opinion is that WIA, Section 185(g), 

prohibits it from doing so. ETA also stated that the report does not acknowledge the 

costs necessary to modify local, state, and federal information systems, train staff, and 

use staff time to comply with the requirement. 

ETA agreed with the recommendations to provide guidance to the SWAs and LWAs 

regarding the collection and reporting of training-related employment data and exercise 

oversight to ensure SWAs develop and/or identify best practices for increasing the 

percentage of exiters who find training-related employment. 

ETA’s response is included in its entirety as Appendix D 

OIG CONCLUSION 

ETA’s response did not result in any changes to the report. While ETA does collect the 

additional information cited in its response, we believe that obtaining accurate and 

reliable data on training-related employment, the funding source of training, and the cost 

of training are key data elements needed in assessing the programs’ effectiveness and 

return on investment. The data currently collected on training-related employment is 

incomplete and unreliable and, as described by ETA, should only be used with “extreme 

care.” Additionally, this data does not differentiate training funded under the WIA Adult 

and Dislocated Worker programs from training funded by other sources. As a result of 

these data limitations, ETA cannot currently calculate outcomes for participants who 

received training paid with WIA funds; and therefore, it has no way to assess whether 

those training funds are being spent in an effective manner. As such, the data is 

“limited” in its usefulness in determining the effectiveness of training paid with WIA Adult 

and Dislocated Worker program funds. 

Elevating training-related employment results to a performance measure would increase 

accountability over the effectiveness of training services paid under the WIA Adult and 

Dislocated Worker programs and increase the transparency of ETA’s reporting to 

WIA Adult & Dislocated Worker Training Services 

4 Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

stakeholders. Although ETA’s response downplayed the importance of exiters obtaining 

training-related employment, WIA, Section 136, requires not only that this information 

be reported by the SWAs but that the outcomes be compared to those participants who 

did not receive training services. The problems we found with the completeness and 

reliability of outcome data indicated that its importance needs to be emphasized. 

WIA does not prohibit ETA from collecting cost information at the participant level and 

WIA, Section 185(g), applies to the reporting of costs on the Federal Financial Report. 

ETA’s response did not address how it is complying with WIA, Section 185(d), which 

specifically requires it to collect “the specified costs of the programs and activities” in 

which participants are enrolled. Concerning the cost necessary to require the reporting 

of training costs, we found the training costs at the participant level were readily 

available at the LWAs we visited. 

RESULTS AND FINDINGS 

Objective 1 — Was ETA's performance accountability system able to assess the 

effectiveness of training services provided under the WIA Adult 

and Dislocated Worker programs? 

ETA did not have sufficient information to assess the performance results of 

training services for which it paid more than an estimated $307 million. 

Finding 1 — ETA's performance accountability system was not able to assess the 

effectiveness of training services provided under the WIA Adult and 

Dislocated Worker programs. 

WIA limits ETA’s ability to include the results of training services in the WIA 

performance accountability system. The WIA performance accountability system that 

ETA developed complied with WIA, Section 136 in that it included performance 

measures for reporting whether exiters found and retained employment. However, it did 

not provide sufficient information on the performance results of training services for 

which ETA paid. WIA does not allow ETA to establish any new performance measures 

apart from the core employment indicators required in WIA, Section 136. Therefore, 

ETA could not use data it collected on training-related employment through its 

Workforce Investment Act Standard Record Data (WIASRD) for performance 

accountability purposes. Additionally, our review of the training service data in WIASRD 

found it was incomplete and unreliable. Moreover, ETA did not collect information on 

training costs and was unable to identify the funding source(s) of training services, 

which could be paid for by sources other than WIA. ETA claimed that WIA does not 

allow the agency to collect information on the cost of training activities. However, we 

believe WIA does not prevent ETA from requesting SWAs to report these costs at the 

participant level. 

WIA Adult & Dislocated Worker Training Services 

5 Report No. 03-11-003-03-390 
 

 

U.S. Department of Labor – Office of Inspector General 

As a result, there is limited information available at the national level about the results 

and costs of training services. Consequently, ETA is unable to report if the training 

services provided by these programs are achieving intended results or whether 

taxpayers are receiving a return on investment for the estimated $307 million 

investment spent to train participants in demand occupations during the audit period. 

ETA did not have a specific performance measure to report whether exiters got 

jobs related to the training services they received. 

While ETA did establish performance measures for employment outcomes of WIA Adult 

and Dislocated Worker exiters, these measures only reported the overall outcomes for 

WIA exiters regardless of the type of services they received. Therefore, these measures 

were not sufficient to gauge the effectiveness of the funds ETA spent on training 

services. 

WIA, Section 136 establishes a comprehensive performance accountability system to 

assess the effectiveness of SWAs in achieving continuous improvement of workforce 

investment activities in order to optimize the return on investment of federal funds. WIA, 

Section 136(b)(2)(A)(1), specifies the core indicators of performance as being entry into 

unsubsidized employment; retention in unsubsidized employment 6 months after entry 

into the employment; earnings received in unsubsidized employment 6 months after 

entry into the employment; and attainment of a recognized credential. 

After the enactment of WIA, ETA implemented the following performance measures for 

its job training and employment programs: 

• Entered Employment 

• Employment Retention 

• Average Earnings 

DOL reported the results of the above measures in its annual Performance and 

Accountability Report (PAR).2 

WIA, Section 136(d)(2), describes additional information that the SWAs are required to 

include in their annual report to ETA, including entry by participants who have 

completed training services under the WIA Adult or Dislocated Worker programs into 

unsubsidized employment related to the training received. However, only one of the six 

SWAs in our sample reported information on training-related employment in its WIA 

Annual Report. Employment-related data from the 53 SWAs’ WIA Annual Report 

appeared in a summary Excel spreadsheet on ETA’s Performance and Results website. 

This spreadsheet consisted of 17 worksheets, lacked explanatory notes or descriptions 

of the data, and was difficult to locate on ETA’s website. Using the SWAs’ WIA Annual 

Reports for reporting training-related employment was not only inconsistent but it lacked 

accountability and transparency. The lack of a performance measure relieved ETA and 

2

 Commencing in FY 2010, DOL reported performance results in its annual PAR. 

WIA Adult & Dislocated Worker Training Services 

6 Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

the SWAs of accountability over the effectiveness of training services paid for under the 

WIA Adult and Dislocated Worker programs. Additionally, performance results on a 

national level were not transparently reported in an annual performance report, which 

could be easily accessed by all interested parties but were instead buried in a confusing 

Excel spreadsheet that was not easily located. 

Since WIA does not allow ETA to implement performance measures beyond the core 

indicators specified in WIA, Section 136(b)(2)(A)(1), ETA would need to seek legislative 

authority to add performance measures that will assess the effectiveness of the SWAs’ 

use of WIA funds for training services. Establishing a performance measure for 

training-related employment would assist ETA, SWAs, and LWAs in their 

decision-making process regarding the continuous improvement of workforce 

investment activities in order to optimize the return on investment of federal funds spent 

on training. 

Data on training-related employment was incomplete and unreliable. 

Although ETA did not use training-related employment data for performance 

accountability purposes, it did collect such data in WIASRD. However, we found the 

data to be incomplete and unreliable. ETA developed WIASRD to establish a 

standardized set of data elements, definitions, and specifications that could be used to 

describe the characteristics, activities, and outcomes of individuals served by WIA 

programs. The purpose of WIASRD was to facilitate the collection and reporting of valid, 

consistent, and complete information on an individual in order to support the overall 

management, evaluation, and continuous improvement of the programs at the local, 

state, and federal levels. ETA officials stated that the program performance results from 

WIASRD can be shared with consumers, taxpayers, Congress and others with an 

interest in the WIA programs. 

ETA instructed the SWAs to report in WIASRD whether or not exiters entered 

employment in which they used a substantial portion of the skills taught in the training 

they received (data element 604, Entered Training-Related Employment). While some 

data elements in WIASRD were considered optional, ETA required that data element 

604 be completed for all training exiters in the Adult and Dislocated Worker programs. 

Our analysis of data element 604 in the WIASRD file used for the audit found 5 of the 

53 SWAs did not report anything for this element. ETA removed all the reported data 

from an additional 12 SWAs because it found the data they had reported was 

unreasonably high or abnormally low. Additionally, the record layout for the WIASRD 

public-use file stated that ETA’s data adjustments only screened out the most extreme 

cases and “extreme care should be taken when using this data.” Given that data for 25 

percent of the SWAs was removed, such a disclaimer would indicate the data ETA 

made available to the public should be considered unreliable. 

ETA officials explained that the SWAs may not be doing as much follow up as they 

should, but WIA does not require personal follow up to obtain outcomes. Instead, WIA 

WIA Adult & Dislocated Worker Training Services 

7 Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

requires the SWAs to use Unemployment Insurance (UI) wage records to verify 

employment. ETA officials stated that the problem with this method for determining if the 

employment was related to the training received was that the UI wage records only 

showed industry code, not occupational codes for the job duties being performed. 

Another issue identified by ETA officials was that it was very subjective for someone to 

decide if a participant used “a substantial portion of the skills taught in the training 

received.” 

Although WIA does not specifically require personal follow up, WIA, Section 

185(d)(1)(c), requires that outcomes of programs and activities for participants be 

reported, including the occupations of participants. 

ETA did not know what training services it paid for or how much it invested in 

those services. 

ETA did not collect the cost of training at the participant level. 

WIA, Section 185 (d)(1)(B), requires that the programs and activities in which 

participants are enrolled be reported as prescribed by the Secretary. Additionally, WIA, 

Section 136(d)(2)(C), requires cost of workforce investment activities relative to the 

effect of the activities on the performance of participants be reported by SWAs in the 

WIA Annual Report. Finally, WIA, Section 185(d)(1)(D), requires that the specified costs 

of WIA programs and activities be reported as prescribed by the Secretary. 

ETA used WIASRD to collect data elements related to participant training activities. ETA 

considered a participant to have received training under the WIA Adult and Dislocated 

Worker programs if the participant’s record had a valid date in the “Training Entered 

Date” field (data element 335) and the “Type of Training Service” field (data element 

340) indicated a valid training service was provided to the participant. Our analysis of 

these data elements in the WIASRD file used for the audit found they were unreliable 

because different funding sources were used to pay for training services. Therefore, 

ETA did not know whether the WIA Adult and Dislocated Worker programs had funded 

the training for which it was credited or if another source had funded the training. 

In selecting our sample we learned that the training data in WIASRD included training 

paid for by sources other than WIA. Our adjustments to the training data resulted in 

determining that the number of participants who were recorded as receiving training 

services under the Adult and Dislocated Worker programs was overstated by 

approximately 19 percent. 

ETA instructed the SWAs to include a discussion of “the cost of workforce investment 

activities relative to the effect of the activities on the performance of the participants” in 

the narrative section of their WIA Annual Reports. To ETA’s credit, since September 

2008, the agency has issued several Training and Employment Guidance Letters 

(TEGL) to SWAs on the costs of workforce investment activities. The TEGLs explained 

how ETA had a long-term interest in improving program efficiency so that both the 

WIA Adult & Dislocated Worker Training Services 

8 Report No. 03-11-003-03-390 
 

 

U.S. Department of Labor – Office of Inspector General 

taxpayers and customers can be better served. ETA suggested that the SWAs explain 

how the mix of core, intensive, and training services provided to participants affected 

outcomes. The TEGLs also provided examples of potential cost measures but left the 

choice of which measures to use in reporting to the discretion of the SWAs. ETA did not 

collect any cost information in WIASRD. 

For program year (PY) 2009 — the latest WIA Annual Reports available — two of the 

six SWAs in our sample did not report costs of workforce investment activities relative to 

the effect on performance in their WIA Annual Reports. The remaining four SWAs 

reported using different methodologies to determine the effect of cost on performance. 

Even if all the SWAs had reported this cost information using the same measures and 

methodologies, ETA would still lack reporting transparency on a national level because 

the information would have to be analyzed from 53 separate WIA Annual Reports. 

ETA officials stated that WIA did not allow the agency to require the reporting of specific 

cost information on training services. We believe that WIA was referring to costs 

reported on the standard government-wide Federal Financial Report (FFR).3

 Costs 

reported on the FFR are generally supported by the SWAs’ and LWAs’ accounting 

systems which may not be designed to capture and report detailed costs. Rather than 

requiring SWAs and LWAs to report training costs on the FFR, ETA could add an 

additional data element to WIASRD to capture the costs paid for the training services 

using the invoice payments to training providers (we were able to readily obtain this 

information from all the LWAs included in the audit). Adding this data element to 

WIASRD would not require revisions to the SWAs’ and LWAs’ accounting systems, and 

having such cost data would allow ETA to share it with consumers, taxpayers, Congress 

and others who have an interest in the WIA programs. Furthermore, doing so would 

also facilitate ETA’s and the SWAs’ compliance with WIA, Section 185, which requires 

that these costs be reported. 

Several SWAs and LWAs in our sample showed interest in training costs and 

outcomes. 

At the state level, three of the six SWAs in our sample were able to provide us 

information on how much they spent on training services under the WIA Adult and 

Dislocated Worker programs. Two of the three SWAs were required by their state law to 

report training expenditures. A fourth SWA planned to start collecting information on 

WIA training expenditures in July 2011 because it was constantly receiving requests for 

this information. 

The SWAs and LWAs in our sample also tracked — or showed an interest in tracking — 

whether participants obtained employment related to the training they received. One 

SWA and 10 LWAs tracked training-related employment, while an additional 2 SWAs 

and 4 LWAs indicated they were interested in tracking this information. For example, at 

1 LWA, the Workforce Investment Board actually set a training-related employment rate 

3 Formerly called the Financial Status Report 

WIA Adult & Dislocated Worker Training Services 

9 Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

as an internal performance measure to assess how well the training programs were 

doing. Also, 1 SWA that showed an interest in tracking training-related employment had 

recently conducted a longitudinal study into the outcomes of training services for 

participants. A key objective of the study was to ensure that training funds were spent 

prudently and exhibited measurable outcomes. 

We believe there is a need for a performance measure and reliable data on the costs 

and outcomes of training services so that ETA can put itself in a position to report to 

stakeholders and Congress how much WIA funding is used for training services and 

what the outcomes of those services are. 

Objective 2 — What were the results of the training services provided under the 

WIA Adult and Dislocated Worker programs, and were the training 

services linked to demand occupations? 

An estimated $124 million was spent on training Adult and Dislocated Worker 

exiters who did not obtain training-related employment or there was insufficient 

information to make the determination. 

Finding 2 — More that 80 percent of the exiters who received training services 

under the Adult and Dislocated Worker program obtained 

employment but not always related to the training they received. 

Our testing of 362 sampled Adult and Dislocated Worker program exiters who received 

training services showed that in almost every case (98 percent), they received training 

related to a demand occupation. Although ETA did not have a measure or goal for the 

rate at which training was completed, we found that 86 percent of the sampled exiters 

completed their training. Testing for employment outcomes indicated that 85 percent 

obtained employment after training, and of those who obtained employment, 82 percent 

were still employed in the second quarter after exiting the programs. 

However, our analysis also showed that 37 percent of the sampled exiters either did not 

obtain employment, or their employment was unrelated to the training received. 

Specifically, almost 13 percent of the sampled exiters did not obtain employment, and 

for those who did, almost 24 percent were in jobs that were not related to the training 

they received. Additionally, for another 10 percent we could not determine if the job was 

related to the training received because of insufficient case file evidence. We projected 

to our audit universe of 103,430 exiters that the amount of funds paid for training exiters 

who did not find employment or employment was not related to the training, and exiters 

for whom we could not determine if the employment was related to their training, totaled 

approximately $124 million. 

Because of the lack of reliable information on the costs and outcomes of training 

services provided under the WIA Adult and Dislocated Worker programs, we tested a 

random sample of 362 exiters who were reported as being served by either program 

WIA Adult & Dislocated Worker Training Services 

10 Report No. 03-11-003-03-390 
 

 

U.S. Department of Labor – Office of Inspector General 

and received training services during the 12-month period of April 1, 2008 through 

March 31, 2009 (see Appendix B for detail on the sampling methodology.) The following 

are the results of our testing of the 362 sampled exiters. The sample results and 

projections are summarized in the report exhibit. 

Cost of Training 

ETA did not track the cost of training services it provided to participants in the WIA Adult 

and Dislocated Worker programs. In order to estimate the training costs, we obtained 

the costs of the training services provided to the sampled exiters from case file 

documentation or accounting records maintained by the LWAs and projected those 

costs to the audit universe. 

Based on the cost of the training provided to the 362 sampled exiters (which totaled 

$1.1 million) we estimated that training costs for the 103,430 exiters in our universe to 

be approximately $307 million. Since the audit period covered exiters from two program 

years4

 (PYs 2007 and 2008), we averaged the annual allotments for the 20 SWAs in 

our universe to put the amount of funding paid for training services in perspective. The 

20 SWAs received 78 percent of the average annual allotment of $2 billion for bo th 

programs. ETA allotted an average of $1.6 billion annually to the 20 SWAs in our 

universe and these SWA spent 19 percent of the average allotted funds on training 

services. 

The extent to which the training was for a demand occupation 

Overall, we found SWAs and LWAs had identified the needs regarding current and 

projected employment opportunities in the respective states and local areas, and most 

of the training was related to these occupations or was exempt from the requirement 

that it be related to these occupations. 

WIA requires SWAs and LWAs to provide in their WIA plans information describing the 

needs of the state and local areas in regard to current and projected employment 

opportunities, by occupation.5

 WIA, Section 134(d)(4)(G), requires that training services 

be directly linked to occupations that are in demand in the local area, or in another area 

to which an adult or dislocated worker receiving such services is willing to relocate, 

except that a local board may approve training services for occupations determined by 

the local board to be in sectors of the economy that have a high potential for sustained 

demand or growth in the local area. Exceptions to this requirement are allowed for 

on-the-job and customized training. 

Every SWA and LWA in our sample had identified demand occupations. We reviewed 

case file documentation for the sampled exiters to determine if the training provided was 

4

 WIA provides funding through allotments to the SWAs based on program year which run from July 1 through 

June 30. 

5 WIA, Section 112(b)(4) for SWAs and Section 118(b)(1)(B) for LWAs 

WIA Adult & Dislocated Worker Training Services 

11 Report No. 03-11-003-03-390 
 

 

U.S. Department of Labor – Office of Inspector General 

related to the demand occupations identified by the SWAs and LWAS, or if the training 

was exempt from the demand occupation requirement. 

Of the 362 sampled exiters, 98 percent received training in a demand occupation when 

required. Forty-eight exiters received on-the-job or customized training; therefore the 

training did not have to be in a demand occupation and we omitted these exiters from 

our calculation. Our results showed that of the 314 remaining exiters, 309 received 

training related to a demand occupation. 

Overall, for the population of 103,430 exiters who received training, we estimated 83 

percent received training in a demand occupation, 15 percent received training that was 

exempt from the demand occupation requirement, and 2 percent did not receive training 

in a demand occupation. 

The extent to which exiters completed the training 

The rate at which exiters complete training is an indicator of how effective the LWAs 

assessed the exiters’ capability and aptitude for training. ETA does not have, and WIA 

does not require, a measure or goal for the rate at which training is completed 

Of the 357 sampled exiters who did not drop out of training for medical reasons6

, 312, 

or 87 percent, completed the training. 

Overall, for the population of 103,430 exiters who received training, we estimated 86 

percent completed training, 12 percent did not complete the training, and 1 percent 

dropped out for medical reasons. In addition, for 1 percent, we were unable to make a 

determination because of insufficient documentation. 

The extent to which exiters obtained and retained employment 

Two of the performance measures that ETA uses for the WIA Adult and Dislocated 

Worker programs are the percent of exiters who are employed in the first quarter after 

the exit quarter and the percent of exiters who are employed in both the second and 

third quarters after the exit quarter. WIA, Section 136(f)(2), requires SWAs to use UI 

wage records to document employment outcomes. 

We reviewed the sampled exiters’ case files for evidence of employment after training. 

Of the 362 sampled exiters, 305 (84 percent) obtained employment. Overall, we 

estimated that for population of 103,430 exiters who received training, 85 percent 

obtained employment. 

We also reviewed the sampled exiters’ case files for evidence of employment in the 

second quarter after exit. Of the 362 exiters sampled, 249 (69 percent) retained 

employment in the second quarter after exiting. Overall, we estimated that for the 

population of 103,430 exiters who received training, 70 percent retained employment. 

6

 We identified 4 of the 362 sampled exiters dropped out of training for medical reasons. 

WIA Adult & Dislocated Worker Training Services 

12 Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

The extent to which exiters obtained employment related to the training received 

Obtaining employment related to the training received is an important measure of the 

effectiveness and return on investment of WIA funds spent on training services. For 

each sampled exiter who obtained employment, we reviewed case file documentation to 

determine the occupation the exiter was being trained for and compared it to 

documentation that described the job the exiter obtained. 

Of the 362 sampled exiters, 305 (84 percent) found employment after completing 

training. Of these 305 exiters, 73 (24 percent) did not find employment related to the 

training they received. Additionally, LWAs did not maintain sufficient information for us 

to make a determination on whether employment was related to the training for an 

additional 29 exiters (10 percent). 

Overall, we estimated that for the population of 103,430 exiters, 51 percent obtained 

training-related employment, 20 percent did not obtain training-related employment, and 

21 percent did not obtain employment, dropped out of training for medical reasons, or 

received training while employed. For 8 percent, we were unable to determine if 

employment was training-related because the LWAs did not maintain sufficient 

documentation. 

We estimated that the cost of training provided to exiters in the population who did not 

obtain training-related employment was $56,416,119; and of those for whom we could 

not make a determination the cost was $26,123,373, for a total of $82,539,492. We also 

estimated that the cost of the training provided to the exiters in our population who did 

not obtain employment totaled $41,541,222. 

The results demonstrate the need for information on training services 

The above results demonstrate that the lack of performance and cost information 

hinders ETA’s ability to assess the effectiveness of training services. Approximately 

$124 million was spent on training exiters who were unable to find a job, did not obtain 

training-related employment, or for whom there was insufficient documentation to make 

any determination. Information on training services is especially important because 

recent trends show the annual amount of funds allotted to SWAs for the WIA Adult and 

Dislocated Worker programs are decreasing while the number of participants is 

increasing. Usable and reliable information on training costs and training-related 

employment would assist ETA and the SWAs in determining how to best allocate 

decreasing WIA funds to assist individuals in pursuing viable career paths leading to 

self-sufficiency, and would improve the accountability and transparency over WIA funds 

invested in training participants for demand occupations. 

WIA Adult & Dislocated Worker Training Services 

13 Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

RECOMMENDATIONS 

We recommend that the Assistant Secretary for Employment and Training: 

1. Pursue legislative authority in the WIA reauthorization to develop performance 

measures for training outcomes. 

2. Require SWAs to report training costs and funding sources at the participant level in 

WIASRD so stakeholders have adequate information to make return-on-investment 

decisions for WIA services. 

3. Provide guidance to SWAs and LWAs regarding the best methodology for collecting 

and reporting data for training-related employment. 

4. Exercise oversight over SWAs to ensure they develop and/or identify best practices 

to increase the percentage of exiters who find employment related to the training 

they receive. 

We appreciate the cooperation and courtesies that ETA personnel extended to the OIG 

during this audit. OIG personnel who made major contributions to this report are listed in 

Appendix E. 

Elliot P. Lewis 

Assistant Inspector General 

 for Audit 

WIA Adult & Dislocated Worker Training Services 

14 Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

Exhibit 

WIA Adult & Dislocated Worker Training Services 

15 Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

 PAGE INTENTIONALLY LEFT BLANK 

WIA Adult & Dislocated Worker Training Services 

16 Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

Exhibit 

Sample Results and Projections 

Number of 95 Percent Confidence Limit Projected to a 

Occurrences Universe of 103,430 Exiters Who Received 

from a Training Services 

Sample of Point Attribute 362 Estimate Lower Limit Upper Limit 

What was the cost of training? 

Adult $528,736 $189,957,397 $126,036,270 $253,878,524 

Dislocated Worker $583,019 $117,190,707 $51,810,072 $182,571,342 

Total $1,111,755 $307,148,197 $302,479,304 $311,817,091 

Did the exiter receive training related to a demand occupation? 

Yes 309 83.03% 78.9% 87.17% 

No 5 1.59% 0.18% 3.00% 

Exiters who received training that 

was exempt from the demand 

occupation requirement. 48 15.38% 11.38% 19.37% 

Did the exiter complete training? 

Yes 312 86.02% 82.27% 89.76% 

No 44 12.36% 8.82% 15.91% 

Cannot Determine (CND) 2 0.7% 0.0% 1.66% 

Not Applicable (1) 4 0.92% 0.0% 1.86% 

Did the exiter obtain employment? 

Yes 305 85.33% 81.65% 89.02% 

No 49 12.67% 9.2% 16.14% 

CND 4 1.08% 0.0% 2.18% 

Not Applicable (1) 4 0.92% 0.0% 1.86% 

Cost of Training for Nos $156,910 $41,541,222 $40,375,180 $42,707,263 

Did the exiter retain employment? 

Yes 249 70.21% 65.37% 75.05% 

No 40 10.96% 7.61% 14.31% 

CND 20 5.24% 2.91% 7.57% 

Exiters who did not obtain 

employment or medically unable 

to complete training. 53 13.59% 10.02% 17.15% 

Did the exiter obtain employment related to the training? 

Yes 185 51.16% 45.79% 56.53% 

No 73 19.57% 15.35% 23.78% 

CND 29 7.88% 5.00% 10.76% 

Exiters who did not obtain 

employment (49), medically 

unable to complete training (4), or 

were already employed and SWA 

received an ETA waiver to use 

funds to train them (22). 75 21.22% 16.81% 25.63% 

Cost of Training for Nos $212,434 $56,416,119 $55,103,456 $57,728,782 

Cost of Training for CND $102,977 $26,123,373 $24,932,169 $27,314,576 

(1) Exiters who did not complete training for medical reasons. 

WIA Adult & Dislocated Worker Training Services 

17 Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

 PAGE INTENTIONALLY LEFT BLANK 

WIA Adult & Dislocated Worker Training Services 

18 Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

Appendices 

WIA Adult & Dislocated Worker Training Services 

19 Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

 PAGE INTENTIONALLY LEFT BLANK 

WIA Adult & Dislocated Worker Training Services 

20 Report No. 03-11-003-03-390 
 

 

 

 

 

U.S. Department of Labor – Office of Inspector General 

Appendix A 

Background 

WIA was designed to provide employment and training services to assist eligible 

individuals in finding and qualifying for meaningful employment and to help employers 

find the skilled workers they need to compete and succeed in business. WIA goals are 

to: 

Increase employment, as measured by entry into unsubsidized employment; 

Increase retention in unsubsidized employment six months after entry into 

employment; 

Increased earnings received in unsubsidized employment for dislocated workers; 

and 

Enhance customer satisfaction for participants and employers. 

DOL’s ETA is responsible for administering WIA programs at the federal level. WIA 

services are primarily provided through state and local workforce development systems 

through grants awarded by ETA. Under WIA the Governor designates local workforce 

investment areas in which workforce activities are to be administered locally. Local 

workforce investment boards are responsible for planning and overseeing the local 

program. The local board is appointed by the local elected official and must have a 

majority of business representatives, and include representatives of education 

providers, labor organizations, community-based organizations, economic development 

agencies, and other partners. WIA requires each local area to establish a one-stop 

delivery system through which employment and training services funded under Title 1 

and other Federal programs are provided. The programs providing services through the 

one-stop system are referred to as one-stop partners. Designated one-stop partners are 

programs that must provide core services through the one-stop, and include programs 

authorized under Title I, the Wagner-Peyser Act. the Adult Education and Literacy title 

of WIA, the Vocational Rehabilitation Act, Title V of the Older Americans Act, 

postsecondary vocational education under the Perkins Act, Trade Adjustment 

Assistance, veterans employment services, unemployment compensation laws, 

Community Service Block Grants, and employment and training activities carried out by 

the Department of Housing and Urban Development. 

The Adult and Dislocated Worker programs are authorized under Title IB of WIA. WIA 

provides formula-based funding to SWAs to design and operate these training 

programs. The Adult program provides employment and training activities for adults, 

including low-income individuals and public-assistance recipients. The Dislocated 

Worker program provides reemployment services and retraining assistance to 

individuals permanently dislocated from their employment. 

WIA provides the following three tiers, or levels of service: 

WIA Adult & Dislocated Worker Training Services 

21 Report No. 03-11-003-03-390 
 

 

 

 

 

 

 

U.S. Department of Labor – Office of Inspector General 

Core services – include career counseling, job search and placement assistance, 

labor market information identifying job vacancies, skills necessary for 

occupations in demand, and relevant employment trends. 

Intensive services - include comprehensive assessments, development of 

individual employment plans, counseling, case management, and short-term 

prevocational services. 

Training Services – include occupational skills training, on-the-job training, 

entrepreneurial training, skill upgrading, job readiness training, and adult 

education and literacy activities in conjunction with other training. 

Core services funded by the Adult program stream are available on a universal basis 

with no eligibility requirement. Funds for the Dislocated Workers program are to be used 

exclusively for services to such workers. However, the amount of funding spent on each 

of the service tiers is not available because fund allocation is left to the discretion of the 

SWAs, and ETA does not collect this information. Intensive services can be provided to 

unemployed workers who are unable to obtain employment through the core services 

and to employed workers who are determined to need additional assistance to obtain or 

retain employment. 

Training services may be provided to individuals who met the eligibility requirements for 

intensive services and: 

have been unable to obtain or retain employment through such services; 

have the skills and qualifications to successfully participate in a selected 

program; 

select programs that are directly linked to employment opportunities in the local 

area; and 

are unable to obtain other grant assistance, including Pell grants, or need 

assistance above the levels provided by such other grants. 

If Adult program funds are limited in a local area, priority for intensive and training 

services must be given to recipients of public assistance and other low-income 

individuals. WIA requires (with three exceptions) that training be provided through the 

use of Individual Training Accounts (ITA), through which a participant chooses among 

eligible providers. Specifically, the one-stop system is to provide participants with the list 

of eligible providers and related performance information. The participant then is to 

choose the program after consultation with a case manager, with payment arranged 

through the ITAs. 

Training may be provided through a contract for services in lieu of an ITA for: on-the-job 

training and customized training; where there are an insufficient number of providers to 

WIA Adult & Dislocated Worker Training Services 

22 Report No. 03-11-003-03-390 
 

 

U.S. Department of Labor – Office of Inspector General 

meet the competitive purposes of ITAs; and for programs offered by community-based 

organizations or other private agencies that serve special participant populations that 

face multiple barriers to employment. 

Funds allocated to local areas under the adult and dislocated worker funding streams 

are to be used at the local level to provide employment and training services through 

the one-stop system. The following table provides appropriated funding for the Adult 

and Dislocated Worker programs for FYs 2008 through 2010 and requested for 2011. 

Table: Funding for Adult and Dislocated Worker Programs for FYs 2008-2011 

Program FY 2008 FY 2009 FY 2010 Recovery Act FY 2011 

Adult $ 849,101,000 $ 861,540,000 $ 861,540,000 $ 500,000,000 $ 770,992,000 

Dislocated $ 1,446,189,000 $ 1,341,891,000 $ 1,413,000,000 $ 1,450,000,000 $ 1,287,544,000 

Worker 

Total $ 2,295,290,000 $ 2,203,431,000 $ 2,274,540,000 $ 1,950,000,000 $ 2,058,466,000 

Congress intended that Recovery Act funds for employment and training be spent 

concurrently with regular formula funds to increase the number of workers who could be 

served and not be not be used to replace state or local funding currently dedicated to 

workforce development and summer jobs. ETA expected states and local areas to fully 

utilize Recovery Act funds to substantially increase the number of customers served, 

provide more in-depth services, as needed, and to substantially increase the number 

and proportion of those customers who receive training, especially training in emerging 

and high demand occupations and industries. 

ETA adopted the common measures methodology for calculating the entered 

employment, employment retention, and earnings measurement in the WIA 

performance accountability system for WIA Adult and Dislocated Worker programs. The 

following are the Common Measure for the WIA Adult and Dislocated Worker programs. 

•	 Entered Employment – percentage of exiters who are employed in the first 

quarter after the exit quarter. 

• Employment Retention – percentage of exiters who obtained employment and 

are employed in both the second and third quarters after the exit quarter. 

•	 Average Earning – for the exiters employed in the first, second, and third 

quarters, their total earnings for the second and third quarters divided by the 

number of participant who exit during the quarter. Number of adult participants 

who exit during the quarter. 

SWAs are required to submit performance data no later than October 1 following the 

end of each program year (July-June) in their Annual Report (9091). The performance 

data reported in the WIA Annual Report comprises information provided by each SWA 

from their individual WIASRD files. ETA developed WIASRD to establish a standardized 

WIA Adult & Dislocated Worker Training Services 

23 Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

set of data elements, definitions, and specifications that can be used to describe the 

characteristics, activities, and outcomes of individuals served by WIA Title IB programs, 

including services financially assisted by National Emergency Grants. The purpose of 

WIASRD is to facilitate the collection and reporting of valid, consistent, and complete 

information on an individual in order to support the overall management, evaluation, and 

continuous improvement of the programs at the local, state, and federal levels. The 

program performance results from WIASRD can be shared with consumers, taxpayers, 

Congress and others with an interest in the WIA programs. The program outcomes are 

reported in WIASRD about 18 to 24 months after the person exits the programs, 

depending on the timeframes of the exit as well as the reporting timelines. 

WIA Adult & Dislocated Worker Training Services 

24 Report No. 03-11-003-03-390 
 

 

U.S. Department of Labor – Office of Inspector General 

Appendix B 

Objectives, Scope, Methodology, and Criteria 

Objectives 

The audit objectives were to answer the following questions: 

1. Was ETA’s performance accountability system able to assess the 

effectiveness of training services provided under the WIA Adult and 

Dislocated Worker programs? 

2. What were the results of the training services provided under these 

programs, and were the training services linked to demand occupations? 

Scope 

The audit covered ETA policies and the SWAs’ in our sample practices and procedures 

over identifying demand occupations and training services provided under the Adult and 

Dislocated Worker programs. The audit covered processes in place at the time of our 

fieldwork from January through July 2011. We used a statistical random sample of 

exiters7

 as reported in WIASRD as of March 31, 2010, the most recent data available at 

the time of our audit. We randomly selected the sample from a universe of 103,430 

Adult and Dislocated exiters who received training services and exited WIA between 

April 1, 2008 and March 31, 2009. This was the latest period on hand in order to obtain 

employment outcomes which are not available until 12-15 months after the exit. Since 

this was aged data, we selected a more recent sample covering those who exited 

between October 1, 2009 and March 31, 2010, to determine if there were changes in 

the extent to which LWAs provided training related to demand occupations and the 

extent to which exiters completed training. The results between the two time periods 

were comparable. Therefore, for clarity, we decided to only report the results of the 

participants who exited WIA between April 1, 2008 and March 31, 2009. 

The universe of SWAs consisted of 20 SWAs that together were responsible for 

providing training services to 79 percent of the exiters within the audit time frames. We 

reduced the SWA universe from 53 to 20 in order to make a more manageable selection 

in the terms of the number of SWAs and LWAs we had to visit. We randomly select the 

following six SWAs — California, Florida, Louisiana, New Jersey, North Carolina, and 

Utah. From these SWAs we randomly selected 23 LWAs and then exiters. We selected 

a sample of 362 exiters. 

We performed audit work at ETA’s National Office in Washington, DC and at the 

following statistically selected SWAs and LWAs: 

7

 Exiters are participants who enrolled and received services and the exited from WIA programs. 

WIA Adult & Dislocated Worker Training Services 

25 Report No. 03-11-003-03-390 
 

 

U.S. Department of Labor – Office of Inspector General 

List of SWAs and LWAs Selected for Audit 

California Employment Development 

Department, Sacramento 

New Jersey Department of Labor and 

Workforce Development; Trenton 

County of Los Angeles, Community and 

Senior Services; Los Angeles 

Bergen County One Stop Career Center; 

Hackensack 

City of Los Angeles Community 

Development Department; Los Angeles 

Passaic County Workforce Investment 

Board; Clifton 

Foothill; Pasadena Camden County Workforce Investment 

Board; Cherry Hill 

San Bernardino Department of Workforce 

Development, San Bernardino 

Cumberland-Salem Workforce 

Investment Board; Bridgeton 

Santa Anna Community Development 

Agency, Santa Anna 

Florida Agency for Workforce 

Innovation, Tallahassee 

North Carolina Division of Workforce 

Development, Raleigh 

Chipola Regional Workforce Development 

Board, Incorporated, Marianna 

High Country Workforce Area; Boone 

Workforce Central Florida; Orlando Regional Partnership Workforce Area; 

Asheboro 

Worknet Pinellas Incorporated; St. 

Petersburg 

Region C Workforce Area; Rutherfordton 

First Coast Workforce Development 

Incorporated; Jacksonville 

Mountain Area Workforce Area; 

Asheville 

Workforce One; Fort Lauderdale 

Louisiana Workforce Commission, 

Baton Rouge 

Utah Department of Workforce 

Services, Salt Lake City 

First Planning District; Covington Utah does not have LWAs 

Second Planning District; Hammond 

Fifth Planning District; DeRidder 

Jefferson Parish; Gretna 

We conducted this performance audit in accordance with generally accepted 

government auditing standards. Those standards require that we plan and perform the 

audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that the evidence 

obtained provides a reasonable basis for our findings and conclusions based on our 

audit objectives. 

Methodology 

To determine the extent state and local workforce investment boards identified demand 

occupations we gained an understanding of the applicable WIA and ETA requirements 

and guidance. We also reviewed SWA and LWA WIA planning documents and 

interviewed appropriate officials at the state and local workforce investment boards 

WIA Adult & Dislocated Worker Training Services 

26 Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

about the processes they used and reviewed documentation they provided. 

To determine what extent did WIA Adult and Dislocated Workers program exiters 

received training in demand occupations, completed their training, found and retained 

employment, and found employment related to the training received, we reviewed case 

file documentation for a statistical random sample of 362 exiters from 6 SWAs who 

received training services. 

Reliability Assessment 

We assessed the reliability of ETA’s WIASRD data by (1) performing tests for 

completeness, accuracy, and consistency of the data elements used in the audit, and 

(2) reviewing existing information about the data. We did find some problems with 

several of the data elements and they are described in the report findings. Otherwise, 

we determined the data was sufficiently reliable for the purposes of this audit. 

Sampling 

We determined a universe of 103,430 exiters who received training services under the 

Adult and Dislocated Worker programs between April 1, 2008, and March 31, 2009. The 

audit used a three-stage stratified cluster statistical sampling design to randomly select 

the SWAs, LWAs, and exiters. We stratified the universe of SWAs into the following 

three strata based on the total number of Adult and Dislocated Worker program exiters 

for each of the time periods within the audit scope. 

Stratification of SWAs Based on the Universe of Exiters 

Stratum 

Percent of the Total Population of Adult and 

Dislocated Worker Participants Who Exited WIA 

Either April 2008 through March 2009 or October 

2009 through March 2010 

Number 

of SWAs 

Percent of 

Population 

for Stratum 

1 More than three percent 10 56 

2 Two to three percent 10 23 

3 Less than 2 percent 33 21 

We narrowed the universe to the 20 SWAs from stratums 1 and 2 because they were 

responsible for providing training services to 79 percent of the participants in who 

received training services under the Adult and Dislocated Worker programs and exited 

WIA within the audit scope time frames. We randomly selected 6 SWAs, three from 

each of the two stratums. The number of LWAs for the 20 SWAs in our population 

totaled 397. Using a 90 percent confidence level and a sampling precision of plus or 

minus 10 percent, we determined the sample size of the LWAs to be 23. The 23 LWAs 

were randomly selected from the 6 SWAs sampled. At the LWAs we randomly selected 

a total of 362 exiters. In determining the sample size for the exiters, we used a 95 

percent confidence level and a sampling precision of plus or minus 7 percent. 

WIA Adult & Dislocated Worker Training Services 

27 Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

In planning and performing our audit, we considered whether internal controls significant 

to the audit were properly designed and placed in operation. This included reviewing 

ETA’s policies and guidance related to identifying demand occupations and providing 

and reporting training services under the Adult and Dislocated Worker programs. We 

confirmed our understanding of these controls and procedures through interviews and 

case file review and analysis. Our consideration of these internal controls would not 

necessarily disclose all matters that might be significant deficiencies. Because of 

inherent limitations in internal controls, misstatements or noncompliance may 

nevertheless occur and not be detected. 

Criteria 

•	 WIA of 1998 

•	 Code of Federal Regulations Title 20 Part 652 et. al, August 11, 2000 

•	 TEGL 17-05 – Common Measures Policy of ETA’s Performance Accountability 

System and Related Performance Issues 

•	 TEGL 14-00 – WIA Performance Reporting System 

•	 TEGL 14-00, Change 1 – Guidance on WIA Management Information and 

Reporting System 

•	 TEGL 14-00, Changes 2 and 3 – WIA Annual Report Narrative 

•	 WIA Annual Report: General Reporting Instructions and ETA Form 9091 

WIA Adult & Dislocated Worker Training Services 

28 Report No. 03-11-003-03-390 
 

 

U.S. Department of Labor – Office of Inspector General 

Appendix C 

Acronyms and Abbreviations 

CND Cannot Determine 

DOL U.S. Department of Labor 

ETA Employment and Training Administration 

FFR Federal Financial Report 

FY Fiscal Year 

ITA Individual Training Account 

OIG Office of Inspector General 

PAR Performance and Accountability Report 

PY Program Year 

SWA State Workforce Agency 

TEGL Training and Employment Guidance Letter 

UI Unemployment Insurance 

WIA Workforce Investment Act 

WIASRD Workforce Investment Act Standard Record Data 

WIA Adult & Dislocated Worker Training Services 

29 Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

 PAGE INTENTIONALLY LEFT BLANK 

WIA Adult & Dislocated Worker Training Services 

30 Report No. 03-11-003-03-390 
 

U.S. Department of Labor 

SEP 30 2011 

MEMORANDUM FOR: 

FROM: 

SUBJECT: 

Assistant Secretary for 

Employment and Training 

Washington, D.C. 20210 

ELLIOTP. LEWIS 

Assistant Inspector General 

Office of Audit 

JANE 

Assistant 

OATES 

Secretary {J 

I~ ~ 

Response to the Office of the Inspector General's Audit, 

Additional Information Needed to Measure the 

Effectiveness and Return on Investment of Training 

Services Funded Under the Adult and Dislocated Worker 

Programs, Audit Report 03-11-003-390 

The Employment and Training Administration (ETA) appreciates the opportunity to 

provide a response to the Office of the Inspector General's (OIG) audit report on 

Workforce Investment Act's performance accountability of job training services. We 

acknowledge the time and effort that the OIG spent examining data ofWIA participants 

who received training services and for helping to confirm that the majority of participants 

are receiving placement in demand occupations. 

As we begin our response, we would like to correct for the record some of the context for 

the report before addressing the specific OIG findings and recommendations. 

The premise of the overarching finding in this report is based on the OIG'sjudgment that 

ETA should collect additional data and establish additional measurements around training 

outcomes. To see this conclusion in its proper context, it is important to understand that 

state WIA programs already collect and report hundreds of data elements to ETA, 

including data on training services. We respectfully disagree with the assessment made 

in the report that "there is limited information available at the national level about the 

results and cost of the training services provided by the Adult and Dislocated Worker 

programs." Current data collections already allow us to calculate outcomes for each 

participant who receives training. 

ETA can answer any number of more detailed programmatic and management questions 

for these participants, such as: (a) What are the characteristics of individuals receiving 

training (race/ethnicity, age, employment status, income status, etc.)? (b) How many 

attained a credential upon completion of a training service? (c) What was the success of 

training participants in finding a job, retaining a job, and what were their earnings? (d) 

What type of training was provided (e.g., On-the-Job Training, occupational skills 

training, customized training, basic skills training, etc.)? To suggest that this is a 

U.S. Department of Labor – Office of Inspector General 

Appendix D 

ETA Response to Draft Report 

WIA Adult & Dislocated Worker Training Services 

31 Report No. 03-11-003-03-390 
 

2 

"limited" dataset is neither a fair nor accurate assessment. The audit report appears to 

define "limited" through a narrow lens based on two facts: 1) state workforce agencies 

(SWAs) uneven reporting of one data element (training-related employment); and, 2) the 

lack of cost data. 

In addition, in our opinion, the overall tone of the report is very misleading. The report 

tends to overstate deficiencies and understate positive findings. For example, the report 

cites that "almost 13 percent of the sampled exiters did not obtain employment .... " 

Although not stated as such, the conclusion is that most (87 percent) of the individuals 

who received training services actually became employed. This is significant --

especially considering the high unemployment and slow job growth during the period of 

observation. Another significant finding is that 98 percent of the training received was in 

demand occupations. One more example is where the report states that "ETA is not in a 

position to report to stakeholders the outcomes and cost of training services ... " As stated 

above, the current reporting system allows us to report the outcomes of training services 

and outcomes by type of training, occupation of training, demographics, and a variety of 

other useful cross-tabulations. This information is available for both the national level as 

well as by state and/or local area. As a final note, while ETA believes training is 

certainly a valuable service, training in and of itself is not an outcome; rather training is 

an output. WIA's stated purpose is to provide activities that increase employment, 

retention, and earnings. In accordance with that purpose, and with the provisions of 

section 136 of the law, ETA implemented the performance accountability system. In 

other words, the outcome measures for which states are held accountable are the required 

outcomes in WIA' s purpose. 

Finding I-ETA' s performance accountability system was not able to assess the 

effectiveness of training services provided under the WIA Adult and Dislocated Worker 

programs. 

ETA Response: States report annually on the outcomes for individuals who receive core 

and intensive services, as well as on outcomes for those who receive training services, 

and ETA makes such information publicly available. This information is reported in the 

states' annual performance progress report, required by WIA section 136(d). 

Finding 2-More than eighty percent of the exiters who received training services under 

the Adult and Dislocated Worker programs obtained employment but not always related 

to the training they received. 

ETA Response: As stated above, that 87 percent of those who received training services 

became employed is a noteworthy accomplishment, and ETA believes this performance 

metric should be highlighted. 

Further in the finding, the report states that "Approximately $124 million was spent on 

training exiters who were unable to find a job, did not obtain training-related 

employment, or for whom there was insufficient documentation to make a determination 

on if employment was related to training." In your letter you state more than $2 billion is 

appropriated for WIA adult and dislocated workers. While our goal is to assist all 

U.S. Department of Labor – Office of Inspector General 

WIA Adult & Dislocated Worker Training Services 

32 Report No. 03-11-003-03-390 
 

3 

jobseekers with finding employment, the fact that only six percent of program funds was 

used to train exiters who were unable to find ajob speaks to the efficacy of the programs, 

and should be highlighted. 

Recommendation 1: Pursue legislative authority in the WIA reauthorization to develop 

performance measures for training outcomes. 

ETA Response: Although we understand that it is nice to have more data, especially 

data to answer a specific question in mind, we respectfully disagree with this 

recommendation for the following reasons. While we believe it is valuable to collect 

outcome data on training participants (as explained in the opening to the memo), 

continually adding more units of "measurement" to the existing "common performance 

measures" (Le., entered employment rate; retention rate; and earnings) may exert a 

diluting effect on accountability. Regardless of service or intervention (e.g., guided job 

search, resume writing, skills assessment, training), ETA measures the system on its 

ability to improve outcomes for all customers, not just the smaller sample of those who 

have the opportunity to receive training services. Over the past two years, ETA, in 

concert with the Office of Management and Budget (OMB) and the Domestic Policy 

Council (DPC), has provided in-depth technical assistance to a Senate subcommittee 

working on WIA Reauthorization. One of the guiding principles of these reauthorization 

efforts has been to streamline performance measures and allow better synchronization 

across the WIA Titles which builds on the work of the "common measures initiative." As 

Congress moves forward to reauthorize WIA, it and the Administration may consider 

additional policy positions such as that contained in this recommendation; however, ETA 

can not predict whether that is likely, nor can it commit to pursue a law change within 

the context of this response. 

For some time, ETA has been at the forefront in the implementation of the "common 

performance measures" among Federal government programs that help individuals 

receive education, job training, and employment services. The use of common measures 

enables ETA to describe in a similar manner the core purposes and results of the public 

workforce investment system - how many individuals got ajob, how many stayed 

employed, and what were their earnings. By 2006, ETA's employment and training 

programs had fully implemented common measures. 

Multiple sets of performance measures have burdened state and local grantees, as they 

have been required to report performance outcomes based on varying definitions and 

methodologies. By minimizing the different reporting and performance requirements, the 

use of common performance measures can facilitate service delivery integration, reduce 

barriers to cooperation across programs, and enhance our ability to assess the 

effectiveness and impact of the workforce investment system. 

ETA recognizes that the information obtained through these common measures provides 

a starting point for understanding the performance of numerous workforce programs 

within a single context. ETA's policy on common measures acknowledges that good 

management and oversight of programs will necessitate the capture and analysis of 

additional information relevant to each program, such as outcomes of training services. 

U.S. Department of Labor – Office of Inspector General 

WIA Adult & Dislocated Worker Training Services 

33 Report No. 03-11-003-03-390 
 

4 

This is why additional information about participant characteristics, services, and 

outcomes are collected by states and reported to ETA to supplement the "story" told by 

the common measures. 

Recommendation 2: Require SWAs to report training costs and funding sources at the 

participant level in WIASRD so stakeholders have adequate information to make returnon-investment decisions for WIA services. 

ETA Response: We respectfully disagree with this recommendation for the following 

reason. Our understanding of WIA Section 185 prohibits ETA from collecting the level 

of detailed cost data that has been suggested in the report. The law states that "the 

Secretary shall require only that the costs be categorized as administrative or 

programmatic costs." It is the agency's opinion that requiring recipients to record 

additional cost categories, such as training, is prohibited by WIA sec. 185(g). 

Aside from legal considerations there are practical considerations. The report asserts that 

the ETA investments necessary to collect cost data would be minimal. However, the 

report does not acknowledge the costs necessary to modify management information 

systems at the local, state and federal levels, the costs for training staff, and the 

"opportunity costs" of spending more staff time on administrative requirements in place 

of serving the historically high customer volume now seen in One-Stop Career Centers 

throughout the country. 

Recommendation 3: Provide guidance to SW As and L WAs regarding the best 

methodology for collecting and reporting data for training-related employment. 

ETA Response: ETA agrees with the OIG report that the agency should continue to 

pursue methods for collecting and reporting data for training-related employment. The 

one caveat is that almost any "best practice" to improve reporting of where participants 

find employment will require One-Stop staff to engage in time-consuming follow-up on 

participants who have exited services rather than on the high numbers of individuals 

waiting for training and training- and employment-related services. In recent years, ETA 

has worked to reduce the time that One-Stop staff needs to conduct time-consuming 

follow-up of exiters by requiring the use of administrative records (Le., unemployment 

insurance wage records) to verify participant outcomes after they leave our programs. 

This "gold standard" data source allows states to verify employment status of exiters and 

has been instrumental in ensuring consistency and accuracy of the data across the states. 

While the administrative records can provide details on whether or not a person is placed 

in a job, these records do not provide information on whether a participant found 

employment related to their WIA training. Interestingly, in ETA's analysis of the audit 

report data, we found that a very high number (76 percent) of the participants sampled 

did find work in employment related to their WIA training. 

As part of our continuing guidance to the system, ETA plans to publish a Training and 

Employment Guidance Letter (TEGL) which provides information on improving data 

collection for the "training-related employment" data element. ETA staff also will 

U.S. Department of Labor – Office of Inspector General 

WIA Adult & Dislocated Worker Training Services 

34 Report No. 03-11-003-03-390 
 

5 

continually examine data trends to detennine how quickly improvements are made within 

the system. 

Recommendation 4: Exercise oversight over SW As to ensure they develop and/or 

identify best practices to increase the percentage of exiters who find employment related 

to the training they receive. 

ETA Response: ETA agrees with the OIG report recommendation that SW As develop 

and/or identify best practices to increase the percentage of exiters who find trainingrelated employment. ETA will begin by canvassing the best perfonning states for 

insights into the most practical ways of improving reporting on this data element. This 

infonnation will be encompassed in the TEGL noted in response to Recommendation 3. 

ETA will use its Regional Offices to compile infonnation on how states deemed as 

"under reporting" on this element are tightening their data collection protocols in this 

area. 

Thank you again for the opportunity to comment. If you have any questions, please 

contact Christine Ollis at (202) 693-3937. 

U.S. Department of Labor – Office of Inspector General 

WIA Adult & Dislocated Worker Training Services 

35 Report No. 03-11-003-03-390 
 

U.S. Department of Labor – Office of Inspector General 

 PAGE INTENTIONALLY LEFT BLANK 

WIA Adult & Dislocated Worker Training Services 

36 Report No. 03-11-003-03-390 
 

 

U.S. Department of Labor – Office of Inspector General 

Appendix E 

Acknowledgements 

Key contributors to this report were Michael Hill, Audit Director, Anthony Grice, Audit 

Manager, Jen Varvel, Ted Lawson, Steve Sovich, Gloria Collazo-Scotland, Susan 

Rosenblum, Dave Halstead, Goleda Sutton-Watson, Janet Cucunato, and Ajit Buttar. 

WIA Adult & Dislocated Worker Training Services 

37 Report No. 03-11-003-03-390 
 

 

TO REPORT FRAUD, WASTE OR ABUSE, PLEASE CONTACT: 

Online: http://www.oig.dol.gov/hotlineform.htm 

Email: hotline@oig.dol.gov 

Telephone: 1-800-347-3756 

202-693-6999 

Fax: 202-693-7020 

Address: Office of Inspector General 

U.S. Department of Labor 

200 Constitution Avenue, N.W. 

 Room S-5506 

Washington, D.C. 20210