Report to the Ranking Minority 

Member, Committee on Health, 

Education, Labor, and Pensions, 

U.S. Senate 

United States General Accounting Office 

GAO 

February 2004 WORKFORCE 

INVESTMENT ACT 

Labor Actions Can 

Help States Improve 

Quality of 

Performance Outcome 

Data and Delivery of 

Youth Services 

GAO-04-308 
www.gao.gov/cgi-bin/getrpt?GAO-04-308.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact David Bellis 

(415) 904-2272 or bellisd@gao.gov. 

Highlights of GAO-04-308, a report to the 

Ranking Minority Member, Committee on 

Health, Education, Labor, and Pensions, 

U.S. Senate 

February 2004

WORKFORCE INVESTMENT ACT

Labor Actions Can Help States Improve 

Quality of Performance Outcome Data 

and Delivery of Youth Services 

Local areas primarily used the WIA program for dropout prevention and 

other efforts to improve academic achievement for in-school youth. 

Nationally, about 70 percent of youth served were in school, but percentages 

ranged from 38 to 86 percent by state. Officials in the five states GAO visited 

said that they focused on in-school youth because serving out-of-school 

youth was much more difficult and expensive, and less effective. Local areas 

emphasized learning-related summer employment for in-school youth and 

occupational skills training and supportive services for out-of-school youth. 

Over half of local boards nationwide used providers that had subcontracting 

arrangements with others to deliver youth services. The majority of youth 

were served primarily from educational institutions and community 

organizations. 

Despite Labor’s guidance, local areas continue to face implementation 

challenges in identifying and retaining out-of-school youth, providing youth 

with mentoring and follow-up services, and using interim measures for 

ongoing program assessment. While Labor supports information exchange 

forums, a promising practices Web site, and technical assistance, some local 

areas may have difficulties gaining access to and using these resources. 

Little is known about the effectiveness of the WIA youth program because 

Labor has not yet conducted an impact evaluation. In addition, while the 

youth program exceeded most of its performance goals, these data were 

questionable because of problems with state information systems and 

inadequate oversight of data quality. While states will be required to verify 

data, concerns remain about their ability to fully implement the requirement 

and Labor’s ability to monitor implementation consistently. 

Source: GAO.

WIA youth gain summer work experience by landscaping a local high school’s grounds. 

The Workforce Investment Act 

(WIA) has been in effect for several 

years and is currently undergoing 

reauthorization. In order to provide 

the Congress with information on 

the implementation and 

effectiveness of the WIA youth 

program, GAO was asked to 

explore how services have been 

delivered at the local level, whether 

the Department of Labor’s 

guidance has addressed challenges 

faced by local areas, and how 

effective the program has been. 

GAO is recommending that the 

Departments of Labor and 

Education coordinate efforts to 

clarify how schools can work with 

workforce officials to help connect 

school dropouts to local WIA youth 

programs. GAO is also 

recommending that the 

Department of Labor provide states 

and local areas with technical 

assistance necessary to address 

ongoing implementation challenges 

and establish standard monitoring 

procedures to improve the quality 

of data reported by states. 

In formal comments on a draft of 

this report, Education concurred 

with our recommendation to work 

with Labor to connect out-ofschool youth to local WIA youth 

programs. In its informal 

comments, Labor said that many of 

the findings corroborated its own 

observations and that the 

recommendations are consistent 

with Labor’s current program 

direction. 
Page i GAO-04-308 WIA Youth Service Delivery 

Letter 1

Results in Brief 3

Background 4

WIA Program Used Primarily for Dropout Prevention and 

Emphasized Summer Services and Skills Training Coordinated 

among Multiple Service Providers 9

Labor’s Guidance Has Not Fully Addressed Implementation 

Challenges Faced by Local Areas 21

Lack of Program Evaluations and Questionable Performance Data 

Prevent Assessments of WIA Youth Program 26

Conclusions 30

Recommendations for Executive Action 31

Agency Comments and Our Evaluation 31

Appendix I Comments from the Department of Education 33 

Appendix II GAO Contacts and Staff Acknowledgments 35 

GAO Contacts 35

Staff Acknowledgments 35

Related GAO Products 36 

Tables 

Table 1: WIA Youth Performance Measures 8

Table 2: Percentage of Local Boards Using Different Types of 

Service Delivery Arrangements, by Number of Youth 

Served 17

Figures 

Figure 1: Percentage of All WIA Youth Served Who Were in School, 

by State (including the District of Columbia) 10

Figure 2: WIA Services Provided to In-School and Out-of-School 

Youth 13

Figure 3: Number of WIA Youth Service Providers Used by Local 

Boards 16

Figure 4: Primary Providers of Services to WIA Youth 18

Contents 
Page ii GAO-04-308 WIA Youth Service Delivery 

Abbreviations 

OIG Office of Inspector General 

JTPA Job Training Partnership Act 

WIA Workforce Investment Act 

WIASRD Workforce Investment Act Standardized Record Data 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 
Page 1 GAO-04-308 WIA Youth Service Delivery 

February 23, 2004 

The Honorable Edward M. Kennedy 

Ranking Minority Member 

Committee on Health, Education, 

 Labor, and Pensions 

United States Senate 

More than 5 million young people—15 percent of the nation’s youth 

population between ages 16 and 24—are out of school and out of work, 

according to a recent study. Further, many teenagers and young adults 

who are in school are at risk of dropping out. According to some experts, 

this indicates that a considerable portion of the country’s emerging 

workforce may face significant difficulty making the transition to 

productive adulthood.1 At the same time, the Department of Labor projects 

that some labor demands will go unmet because there will be too few 

workers in the labor market with the necessary skills. Enacted in 1998, the 

Workforce Investment Act (WIA) aims to address these issues by assisting 

the nation’s emerging workforce in realizing its full potential. As the 

administering agency, Labor has budgeted about $1 billion annually on 

WIA Title I-B youth employment and training programs to serve an 

estimated 721,000 of the nation’s most at-risk young people. 

WIA services are based on promising practices in the fields of youth 

development and employment. Research suggests, for example, that youth 

can achieve positive outcomes when they interact with caring adults, 

engage in hands-on education and training activities, and receive support 

for personal growth. The WIA program has sought to make these sorts of 

experiences available to both in-school and out-of-school youth 

participants. Under WIA, local areas can tailor their approach to the types 

of youth served, the services provided, and how they are delivered. To 

ensure that youth programs are tailored to local areas, WIA requires the 

participation of a wide variety of people—youth policy experts, 

representatives from youth-serving agencies, parents, and others with a 

vested interest in the local youth programs. These participants serve on 

local workforce investment boards created by WIA to establish workforce 

 1

Center for Law and Social Policy, Leave No Youth Behind: Opportunities for Congress to 

Reach Disconnected Youth, (Washington, D.C.: July 2003). 

United States General Accounting Office

Washington, DC 20548 
Page 2 GAO-04-308 WIA Youth Service Delivery 

development policies and oversee implementation at the local level, or on 

youth councils, subgroups of the local board that plan and coordinate the 

local youth program. 

We previously reported on the implementation challenges local areas 

faced during the first few years of implementation.2

 Now that the program 

has been ongoing for several years and WIA is undergoing reauthorization, 

you asked us to review (1) what approaches local areas have taken to 

serve at-risk youth, (2) whether Labor’s youth program guidance has 

addressed ongoing implementation challenges, and (3) what is known 

about the effectiveness of the WIA youth program. 

To obtain information on what approaches local areas have taken to 

deliver youth services, we administered a survey to the directors of all 

604 local workforce investment boards across the nation, including those 

in the 50 states, the District of Columbia, Puerto Rico, and U.S. territories. 

We received responses from 496 local workforce investment boards 

(82 percent) and relied on self-reported data. To further understand local 

area approaches to service delivery, we visited nine local workforce 

investment areas in five states: California, Louisiana, New Hampshire, 

Ohio, and Virginia. We selected these states to obtain a mix based on their 

differences in geographic location, number of workforce investment areas, 

amount of youth funding, and presence of a state youth council. In 

addition, our selection was informed by recommendations from Labor, 

youth policy experts, and state workforce officials. We interviewed 

officials representing state and local boards, youth councils, one-stop 

centers, youth-service providers, business representatives, and state and 

local education agencies. We reviewed Labor’s program evaluation agenda 

and published guidance letters from program year 2000 to the present. We 

also assessed the reliability of the performance data submitted by states in 

their annual reports and compiled in Labor’s WIA database known as the 

Workforce Investment Act Standardized Record Data (WIASRD) by 

performing checks for internal consistency, reviewing Office of Inspector 

General reports and other relevant documents, and speaking with the 

contractor for WIASRD. We determined that the data were not sufficiently 

reliable to use for the purposes of this report. We also interviewed officials 

from Labor, including each of its six regional offices, as well as from the 

 2

U.S. General Accounting Office, Workforce Investment Act: Youth Provisions Promote 

New Service Strategies, but Additional Guidance Would Enhance Program Development, 

GAO-02-413 (Washington, D.C.: April 5, 2002). 
Page 3 GAO-04-308 WIA Youth Service Delivery 

Department of Education and national associations. Our work was 

conducted between January and December 2003 in accordance with 

generally accepted government auditing standards. 

Local WIA youth programs primarily focused on preventive strategies to 

help in-school youth avoid academic failure and dropping out of school, 

emphasizing summer enrichment services that were coordinated through 

individual case managers and multiple service providers. Our survey of 

local boards showed that about 70 percent of WIA youth were in school, 

although this percentage varied widely, from 38 percent in South Dakota 

to 86 percent in Nebraska. State and local education officials said that the 

WIA program allowed communities to provide complementary services 

needed to support at-risk in-school youth. For example, local officials 

from one urban area in Virginia said that while a local school provided 

1 counselor for as many as 300 students, the WIA program funded 1 

counselor for every 50 WIA participants. In addition, local officials in four 

states we visited said they focused on serving in-school youth because it 

was easier and less costly than recruiting and retaining youth once they 

had dropped out of school. Local boards reported that more than half of 

in-school youth received summer employment services that were linked 

to classroom learning. For example, a service provider in a rural area of 

California enrolled in-school youth in a 6-week summer enrichment 

program where students worked part-time while learning reading skills. In 

contrast, WIA youth who were out of school were more likely to receive 

occupational skills training and supportive services, such as assistance 

with child care, transportation, and housing. While in-school and out-ofschool youth usually participated in programs separately, both groups 

worked with case managers who helped develop individual service 

strategies and coordinate delivery of services. The majority of WIA youth 

were served primarily by community organizations or by educational 

institutions such as high schools, colleges, and universities. 

Despite guidance issued by Labor, local areas continued to face challenges 

in serving out-of-school youth, providing mentoring and follow-up 

services, and establishing and using optional interim performance 

measures. One reason serving out-of-school youth continued to be 

challenging was that such youth can be difficult to locate within the 

community. Some local workforce and regional Labor officials said that 

identifying youth was problematic, in part because schools did not always 

ensure that dropouts were linked with the WIA program. Another reason 

cited was the difficulty in retaining youth who were primarily interested in 

immediate employment rather than in participating in WIA’s long-term 

Results in Brief 
Page 4 GAO-04-308 WIA Youth Service Delivery 

youth development activities. Local officials added that mentoring and 

follow-up services were challenging to provide, in part because of the 

difficulty in finding enough qualified mentors to work with at-risk youth 

and sustaining a connection with youth once they had exited the program. 

Some state workforce officials said that to more effectively serve out-ofschool youth, they needed better-targeted guidance from Labor that 

addressed their local areas’ particular service delivery issues. Local areas 

also faced challenges establishing interim measures, such as enrollment 

and service participation, to assess program performance and improve 

service delivery in a timely manner. Despite Labor’s encouragement, some 

regional Labor officials said that interim performance measures were not 

widely used by local areas. Labor has established WIA youth learning 

exchanges, a Web site, and a technical assistance program to help address 

WIA youth program implementation challenges, but some states and local 

areas may not have access to or be aware of these resources. 

Little is known about the effectiveness of the WIA youth program because 

program impact evaluations have not been performed and performance 

outcome data that have been collected may not be reliable. Impact 

evaluations provide information on program effectiveness by 

differentiating between outcomes that result from the program itself and 

those that result from other factors. Because of possible legislative 

changes to the WIA youth program, Labor does not plan to undertake an 

impact evaluation until 2006, with initial results expected by 2009. While 

states annually report performance data to Labor, these data are 

questionable because of insufficient state monitoring of data quality and 

the inadequacies of some state management information systems. Labor 

officials said that they will require states to verify local area data beginning 

with program year 2002 data, but concerns remain about states’ ability to 

fully implement validation requirements and Labor’s ability to monitor 

implementation consistently. 

We are recommending that Labor and Education coordinate efforts to 

clarify how schools can work with workforce officials to connect dropouts 

to WIA youth services. We are also recommending that Labor provide 

guidance to address specific concerns identified by local implementers 

and establish standards to monitor data quality. 

Enacted in 1998, WIA replaced the fragmented and overlapping programs 

under the Job Training Partnership Act (JTPA) with a system that sought 

to connect employment, education, and training services to better match 

job seekers to labor market needs. The youth program falls under Title I 

Background 
Page 5 GAO-04-308 WIA Youth Service Delivery 

of WIA along with programs for adults and dislocated workers; Title II 

deals with adult and family literacy; Title III pertains to employment 

services under the Wagner-Peyser Act; and Title IV addresses vocational 

rehabilitation. To better prepare low-income youth who face barriers to 

employment and education, WIA requires youth programs to focus on 

long-term, comprehensive services delivered year-round through a 

coordinated network of service providers. In addition to meeting the lowincome requirement, to be eligible for WIA services youth must be 

between the ages of 14 and 21 and faced with one or more of six barriers 

to employment.3

 WIA serves both in-school and out-of-school youth. A 

significant portion of out-of-school youth are high school dropouts and 

unemployed youth who are struggling to succeed in the public education 

system and lack financial, family, or social support. Thus, at least 

30 percent of local WIA youth funds must be spent on out-of-school youth.4

Since WIA’s full implementation in 2000, funding for the youth program 

has ranged between approximately $1 billion and $1.4 billion a year. Labor 

follows a formula to allocate WIA funds to states, which in turn distribute 

money to their local workforce investment areas.5

 At the state level, the 

governor can reserve up to 15 percent of the annual WIA allotment for 

such statewide activities as developing the WIA strategic plan for the state. 

States allocate the remaining 85 percent of funds to local boards that use 

the funds to develop the local strategic plan, establish a youth council, and 

award competitive contracts to youth service providers that have been 

recommended by the youth council. In addition to recommending eligible 

providers, the youth council coordinates youth activities in the local area, 

oversees providers, and carries out other duties authorized by the local 

board such as forging linkages with educational agencies. Youth councils 

 3

A youth is considered to face employment barriers if he or she is (1) deficient in basic 

literacy skills; (2) a school dropout; (3) homeless, a runaway, or a foster child; (4) pregnant 

or a parent; (5) an offender; or (6) an individual who requires additional assistance to 

complete an educational program or to secure and hold employment. Up to 5 percent of 

WIA youth participants are not required to meet the income eligibility requirements, but all 

youth served by WIA must meet barrier requirements. 

4

Out-of-school youth do not include youth enrolled in alternative schools at the time of WIA 

registration. 

5

For a discussion of WIA formula funding, see U.S. General Accounting Office, Workforce 

Investment Act: Issues Related to Allocation Formulas for Youth, Adults, and Dislocated 

Workers, GAO-03-636 (Washington, D.C.: April 25, 2003). 

WIA Youth Funding 
Page 6 GAO-04-308 WIA Youth Service Delivery 

may also leverage additional public and private funds to supplement their 

WIA funding in order to provide comprehensive youth services. 

Once they are determined to be WIA eligible, youth receive an objective 

assessment of their academic level, skills, and service needs. Local youth 

programs then use the assessment to create each participant’s individual 

service strategy, which lays out employment goals, educational objectives, 

and necessary services. Every local youth program must offer the 

following 10 services, known as program elements, to eligible youth, 

though participants may receive different combinations of these elements 

depending on their service strategy. Labor groups the 10 required program 

elements around four major themes: 

Improving educational achievement

1. tutoring, study skills training, and instruction leading to completion 

of secondary school, including dropout prevention, 

2. alternative school services, 

Preparing for and succeeding in employment

3. summer employment linked to academic and occupational learning, 

4. paid and unpaid work experiences, including internships and job 

shadowing, 

5. occupational skills training, 

Developing the potential of young people as citizens and leaders

6. leadership development, which may include community service and 

peer-centered activities encouraging responsibility, 

Supporting youth

7. supportive services (such as child care and housing assistance), 

8. adult mentoring for at least 12 months that may occur both during 

and after program participation, 

9. follow-up upon program completion for at least 12 months, and 

Youth Services 
Page 7 GAO-04-308 WIA Youth Service Delivery 

10. comprehensive guidance and counseling. 

Eligible youth may obtain these services directly from approved youth 

service providers or through WIA’s one-stop system, which serves as a 

gateway to a variety of employment and training services.6

 In addition to 

helping WIA youth gain access to the 10 program elements, the one-stop 

system also provides all youth with basic services, whether or not they are 

eligible for WIA. Any young person may walk into a one-stop center and 

make use of the center’s career resources and obtain information on and 

referrals to other providers. While one-stop centers are designed primarily 

to serve those 18 and older, some states have established one-stops that 

serve only youth, as we reported previously.7

Labor provides guidance to help states and local areas implement WIA. 

Labor’s guidance includes issuing annual Training and Employment 

Guidance Letters on the youth program in general as well as on specific 

topics, sponsoring WIA Learning Exchanges8

 in every region, and 

maintaining a Web site for promising and effective practices. Labor 

responds to requests for clarification or additional information by phone 

or e-mail or at conferences. Labor’s guidance often includes promising and 

 6

To create a more comprehensive workforce investment system, WIA requires states and 

localities to bring together 17 federally funded employment and training services into a 

single system, called the one-stop center system. Locally, the one-stop system must include 

at least one physical site offering a comprehensive array of WIA services as well as those of 

the other partners, and may be supplemented by satellite sites that provide one or more 

WIA services. For more information on the one-stop system, see U.S. General Accounting 

Office, Workforce Investment Act: One-Stop Centers Implemented Strategies to Strengthen 

Services and Partnerships, but More Research and Information Sharing Is Needed, 

GAO-03-725 (Washington, D.C.: June 18, 2003). 

7

Local boards in Alaska, Arizona, Arkansas, California, Colorado, the Federated States of 

Micronesia, Florida, Idaho, Louisiana, Missouri, New Jersey, New York, Puerto Rico, 

Tennessee, Texas, and Wisconsin reported having one or more youth-only one-stop 

centers. See GAO-02-413. 

8

WIA Learning Exchanges for Youth Systems are two-day regional meetings designed to 

highlight promising practices at exemplary WIA sites, foster peer-to-peer information 

sharing, and develop team-driven action plans in three areas: (1) recruiting and retaining 

out-of-school youth; (2) building and sustaining partnerships, especially with the education 

community; and (3) defining and aligning assessments, skill achievement, and credentials. 

Along with the National Youth Employment Coalition and the American Youth Policy 

Forum, Labor conducted seven learning exchanges around the country, one of which was 

specifically for rural local areas. 

Youth Program Guidance 
Page 8 GAO-04-308 WIA Youth Service Delivery 

effective practices to reinforce state and local flexibility under WIA to 

tailor programs to best meet youths’ needs. 

The law requires that states and local areas collect performance 

information on seven youth measures, which are separated for younger 

and older youth (see table 1). All seven youth measures apply to both 

statewide and local performance. 

Table 1: WIA Youth Performance Measures 

Younger youth measures (ages 14-18) Older youth measures (ages 19-21) 

1. Skill attainment rate 

2. Diploma or equivalent attainment rate 

3. Placement and retention in 

 postsecondary education, advanced 

 training, or employment 

1. Entered into employment rate 

2. The employment retention rate at six 

 months 

3. Increase in average earnings 

4. The credential rate 

Source: GAO analysis of the Workforce Investment Act. 

WIA performance information is collected from service providers and 

flows upward to the local boards, then to state boards or agencies, and 

finally to Labor. Local performance data are aggregated and entered into a 

state’s own automated data system that tracks the activities of individual 

WIA participants. States use two mechanisms to report performance to 

Labor: WIASRD, a database of individual records containing activity and 

outcome information for each registered participant that has exited WIA, 

and state annual reports. Labor uses the state annual reports to track 

states’ progress in meeting negotiated performance goals. Labor then 

awards monetary incentives to states that meet or exceed their 

performance goals and sanctions states that fail to meet at least 80 percent 

of each goal in two consecutive years. 

Performance Measures 
Page 9 GAO-04-308 WIA Youth Service Delivery 

Most local areas used WIA youth program services for a range of 

approaches to prevent academic failure and school dropouts, emphasizing 

learning-related summer services that were coordinated through case 

managers and multiple service providers. Nationally, about 70 percent of 

youth served were in school, according to local boards responding to our 

survey. For in-school youth, local boards provided summer employment 

services linked to classroom learning more often than for out-of-school 

youth, while services for this latter population more often included 

occupational skills training and supportive services. Local boards reported 

that most youth received services primarily from community organizations 

and educational institutions. These providers were most likely to 

subcontract or make informal arrangements with other organizations to 

deliver the full range of WIA services to youth participants. 

Overall, local areas’ approach to serving youth was to supplement schools’ 

dropout prevention efforts to keep youth connected to an educational 

system, according to state and local workforce and education officials. 

WIA allows states and local areas to determine the proportion of in-school 

and out-of school youth to be served, requiring only that they spend at 

least 30 percent of funds for out-of-school youth. Labor reported that with 

some exceptions, states chose to focus the majority of resources on inschool youth,9

 and our survey showed that about 81 percent of local areas 

served more in-school youth than out-of-school youth. Nationally, about 

70 percent of youth served were in school, according to local boards we 

surveyed. However, this percentage varied across the nation, as shown in 

figure 1. For example, 38 percent of youth served in South Dakota were in 

school, compared with 86 percent in Nebraska. 

 9

Excluding the territories, Labor’s data show that only eight states had spent 50 percent or 

more of their program year 2001 WIA youth allotment on out-of-school youth as of 

September 30, 2003: Iowa, Idaho, Illinois, Maryland, Maine, New Hampshire, North 

Carolina, and South Dakota. 

WIA Program Used 

Primarily for Dropout 

Prevention and 

Emphasized Summer 

Services and Skills 

Training Coordinated 

among Multiple 

Service Providers 

Local Areas Focused on 

Dropout Prevention for InSchool Youth 
Page 10 GAO-04-308 WIA Youth Service Delivery 

Figure 1: Percentage of All WIA Youth Served Who Were in School, by State 

(including the District of Columbia) 

a

Includes states where less than 65 percent of boards responded. 

According to officials in four states that we visited, WIA youth programs 

primarily targeted in-school youth because recruiting and retaining out-ofschool youth for the WIA program was much more difficult and expensive. 

For example, officials from a rural area in Ohio had difficulty identifying 

and retaining out-of-school youth whose contact information changes 

frequently, and Labor reports that the average cost of serving an out-ofschool youth under the WIA program is about $4,000 a year, twice as much 

as for an in-school youth. Officials from other local areas we visited 

considered other factors. A local official in Louisiana said that in-school 

youth are interested in the WIA program, unlike out-of-school youth, who 

Sources: GAO survey of local board directors (data); copyright © Corel Corp. All rights reserved (map).

Data unavailablea (4)

Less than 50% (3)

50 to 70% (30)

Greater than 70% (14)
Page 11 GAO-04-308 WIA Youth Service Delivery 

are difficult to engage. In contrast, a rural area in Virginia chose to focus 

primarily on out-of-school youth because so few services were available 

for this population. 

The Administration has proposed amending the WIA youth program to 

focus more resources on out-of-school youth. Department of Education 

officials said that WIA’s services to in-school youth were not unique, since 

schools already offer various services to their students to deter them from 

dropping out. While Education officials said that the department’s grants 

geared exclusively toward dropout prevention would be insufficient to 

address the national dropout problem, they also stated that funding is 

available under many Education programs that could be used for dropout 

prevention activities.10 However, local workforce and education officials 

we spoke with in three states we visited said that they were either 

unaware of or unable to gain access to other available federal resources 

that could be used to provide intensive services to at-risk youth. In New 

Hampshire, for example, officials told us that WIA provides the only 

dropout prevention program, and that they were unaware of other 

available education dropout prevention resources. 

State and local workforce and education officials we spoke with in the 

five states we visited said that WIA funding complemented rather than 

duplicated education services and was critical in preventing in-school 

youth from dropping out of school. For example, in a rural area in Ohio, 

workforce officials stated that the WIA program was the only dropout 

prevention program and that WIA provided students with their only 

chance at academic and career success. In addition, they said that WIA’s 

services were more intensive and comprehensive, and were delivered in a 

one-on-one setting where each student received individualized attention. 

In one urban school in Virginia, there was 1 WIA counselor for 50 eligible 

in-school youth, compared to 1 school counselor for as many as 

300 students. 

 10For a discussion of Education and other federal programs to assist youth at risk of 

dropping out of school, see U.S. General Accounting Office, School Dropouts: Education 

Could Play a Stronger Role in Identifying and Disseminating Promising Prevention 

Strategies, GAO-02-240 (Washington, D.C.: February 1, 2002). 
Page 12 GAO-04-308 WIA Youth Service Delivery 

Local areas tailored services provided to youth based on their needs, 

emphasizing different services, depending on whether youth were in or 

out of school. To meet youth’s many developmental needs, the WIA youth 

program requires that local areas offer the same menu of 10 academic, 

employment, and support services to all eligible youth, which WIA 

providers choose from when tailoring services to an individual’s service 

strategy based on an assessment of needs.11 As shown in figure 2, services 

such as work experience and leadership development were provided fairly 

equally to both youth populations, but there were differences in other 

areas. For example, in-school youth were more likely to receive tutoring 

services and summer employment linked to classroom learning, while outof-school youth were more likely to receive occupational skills training 

and supportive services to help prepare them for employment. 

 11One of the 10 required services, follow-up, is required for all youth for at least 12 months 

after they exit the WIA program. 

In-School Youth Received 

More Summer 

Employment; Out-ofSchool Youth Received 

More Occupational Skills 

Training 
Page 13 GAO-04-308 WIA Youth Service Delivery 

Figure 2: WIA Services Provided to In-School and Out-of-School Youth 

Note: Follow-up services are not depicted in the figure because they are mandatory for all participants 

when they exit the WIA youth program. 

WIA requires that summer employment programs be linked to academic 

learning, and Labor guidance promotes meaningful summer enrichment 

0 10 20 30 40 50 60

Tutoring

Guidance and

counseling

Mentoring

Supportive services

Leadership

development

Paid or unpaid work

experience

Alternative secondary

school

Occupational skills

training

Summer employment

linked to learning

In-school youth

Out-of-school youth

Source: GAO survey of local board directors.

Percentage of youth

Services received
Page 14 GAO-04-308 WIA Youth Service Delivery 

experiences to help youth attain skills throughout the school year.12 Local 

youth programs in the five states we visited were providing youth with 

various types of summer experiences. For example, a service provider in a 

rural area of California enrolled in-school youth in a 6-week summer 

enrichment program where students worked part-time while learning 

reading skills. Another service provider in a rural area in Louisiana offered 

in-school youth summer services that included academic enrichment, 

community service, and exposure to different career options. 

Local youth programs provided out-of-school youth training in 

occupational skills and job readiness, as well as offered them supportive 

services. Labor’s guidance states that enrolling youth in occupational skills 

training and retaining them until program completion leads to better 

outcomes.13 Local areas provided several different types of occupational 

skills training, depending on local labor market needs. For example, one 

urban local area in Ohio trained out-of-school youth in nursing, welding, 

and computer repair, while another urban local area in Virginia offered 

technology certification training. Local areas also provided job readiness 

skills cited by employers as lacking in many youth seeking employment. 

For example, a program for out-of-school youth in an urban area of Ohio 

taught participants such job readiness skills as customer service, conflict 

management, and other interpersonal skills. Supportive services were also 

important for out-of-school youth who needed additional assistance to 

help them overcome their multiple barriers to employment. A service 

provider in New Hampshire provided out-of-school participants with 

support services such as child care, transportation, and housing 

assistance. 

 12Research has shown that engaging youth during the summer provides them with an 

opportunity to learn new skills that they can apply in school and also learn the value of 

work. See Westat, The 1993 Summer Youth Employment and Training Program, 

(Rockville, MD, April 1994). In addition, Labor’s guidance highlights work-based learning 

during summer employment as a strategy to help out-of-school youth acquire work 

experience while making the connection between staying in school and pursuing a career. 

See U.S. Department of Labor, Training and Employment Guidance Letter No. 3-99, 

(Washington, D.C., 2000). 

13See U.S. Department of Labor, Training and Employment Guidance Letter No. 18-00, 

(Washington, D.C., 2001). 
Page 15 GAO-04-308 WIA Youth Service Delivery 

Almost all local boards we surveyed reported using case managers to 

coordinate services for youth in and out of school. Because at-risk youth 

often require services from a host of departments, including Education, 

Health and Human Services, Housing and Urban Development, Justice, 

and others, Labor’s guidance promotes the use of a case manager to 

coordinate services among all these youth-serving partners to provide 

seamless access to and delivery of services.14 Nationwide, all but 3 percent 

of local boards responding to our survey reported that their youth program 

uses case managers. Local boards reported that, in addition to performing 

other duties, case managers assessed youth upon enrollment in the WIA 

program. These assessments typically included a review of educational 

attainment, work readiness, work experience, and career interests. The 

assessments are then used to develop an individual service strategy for 

each youth participant. Labor’s guidance requires that each strategy 

identify employment goals and educational objectives, and prescribe 

appropriate services for each participant.15 About three-fourths of local 

boards reported that youth were greatly or very greatly involved in the 

development of their individual service strategies. 

Most local areas used multiple service providers to deliver the youth 

services spelled out in each participant’s individual service strategy. WIA 

does not specify how services must be provided to youth, allowing local 

areas to determine how many providers they will fund and hold 

responsible for delivering the services outlined in each youth’s individual 

service strategy. Over half of local boards responding to our survey 

reported using 4 or fewer service providers, but 15 percent used more than 

10 providers, as shown in figure 3. 

 14See Training and Employment Guidance Letter No. 18-00. 

15See U.S. Department of Labor, Training and Employment Guidance Letter No. 9-00

(Washington, D.C., 2001). 

Case Managers Developed 

Service Strategies and 

Providers Coordinated to 

Deliver Services 
Page 16 GAO-04-308 WIA Youth Service Delivery 

Figure 3: Number of WIA Youth Service Providers Used by Local Boards 

Note: Some local board directors reported using zero providers because the board itself delivered 

youth services. WIA permits local boards to provide training services if there is an insufficient number 

of eligible providers to meet local demand and if the board has demonstrated that it meets the 

requirements for an eligible service provider. 

Over half of local boards nationwide used providers that did not deliver all 

services themselves, using formal or informal subcontracting 

arrangements to provide the range of services needed. The extent that 

providers coordinated with others to deliver services was related to the 

number of youth served in the local area. In local areas with 250 or fewer 

youth, providers delivered all services themselves more often than in areas 

with more than 250 youth, as indicated in table 2. 

0

10

20

30

40

0 to 1 2 to 4 5 to 10 >10 

Percentage of local boards

Number of service providers

Source: GAO survey of local board directors.
Page 17 GAO-04-308 WIA Youth Service Delivery 

Table 2: Percentage of Local Boards Using Different Types of Service Delivery 

Arrangements, by Number of Youth Served 

0–250 

youth 

251–750 

youth 

Over 750 

youth 

Providers deliver all services without using 

subcontractors 44 34 30 

Providers are responsible for delivering all 

services but use subcontractors 23 31 39 

Providers do not deliver all services themselves, 

but make noncontractual arrangements with 

other organizations to make sure all services are 

delivered 22 25 17 

Other arrangement 10 10 14 

Source: GAO survey of local board directors. 

Note: Columns may not add to 100 because of rounding. 

Regardless of the level of coordination used to provide youth services, 

workforce officials told us that providers often informally collaborated 

with one another through regular meetings to discuss problems, train one 

another, share best practices, or share other resources. 

Local boards responding to our survey reported using a range of public 

and private entities to deliver youth services. Over half of all youth 

received services primarily from educational institutions or community 

organizations, while less than 1 percent of youth received services 

primarily from faith-based organizations, as shown in figure 4. 
Page 18 GAO-04-308 WIA Youth Service Delivery 

Figure 4: Primary Providers of Services to WIA Youth 

a Youth opportunity centers were established under the Youth Opportunity Grant program to enable 

youth living in high-poverty areas to gain access to a wide range of services. 

Local boards reported that about one-fourth of youth received services 

primarily from WIA one-stops—whether the one-stops focused on serving 

adults or youth. However, among the one-stops we visited, these entities 

Types of providers

Percentage of youth

In-school

Out-of-school

Source: GAO survey of local board directors.

0 5 10 15 20 25 30 35

Other

Local or state government

Private employer

Programs sponsored by federal

agencies besides Labor

Faith-based organizations

Community organizations

Colleges or universities

Secondary schools

Youth opportunity centersa

Youth one-stop

Adult one-stop
Page 19 GAO-04-308 WIA Youth Service Delivery 

were frequently used for supplemental services such as information and 

referrals. Officials we spoke with in New Hampshire and Ohio said that 

service providers or schools usually took youth on a field trip to the local 

one-stop for basic orientation to expose them to employment services they 

could use in the future. Although most youth were served by the WIA 

program through other providers, one-stop usage varied considerably by 

local area. For example, the majority of WIA youth living in an urban area 

of California were served by youth-exclusive one-stops, while youth in a 

rural area of the state rarely interacted with the local adult one-stop. 

In-school and out-of-school youth usually did not participate in WIA 

programs together. Nationwide, 43 percent of local boards reported that 

these two types of youth received services from different providers. In 

another 27 percent of cases, the two groups received services from the 

same providers but participated separately. Another 24 percent of local 

boards reported that all youth were served by the same service providers 

and participated together.16

Local areas developed partnerships with the local business and education 

communities to identify employer needs and provide comprehensive youth 

services related to academic and employment preparation. In many cases, 

youth councils helped facilitate these partnerships. 

Local areas partnered with businesses to identify and provide employment 

and training services for WIA youth needed to fill high-demand, highgrowth occupations.17 Over 85 percent of local boards reported using each 

of the following methods to develop and maintain relationships with local 

businesses: consulting with businesses about their job needs, training 

participants in skills needed by local businesses, training program 

participants to understand the values and attitudes local businesses look 

for in employees, and providing employment experience to participants 

that suits the jobs available in local businesses. Officials we spoke with at 

the local youth programs in all five states we visited provided such work 

readiness training as punctuality, teamwork, respect for others, and 

 16The remaining 6 percent of local boards reported their arrangement for service delivery to 

in-school and out-of-school youth as “other.” 

17Over 70 percent of local boards reported that their programs encouraged youth to seek 

employment in each of the following sectors: retail, hospitality and food service, health 

care, and information technology. 

Local Areas Developed 

Partnerships with Business 

and Education to Deliver 

Youth Services 
Page 20 GAO-04-308 WIA Youth Service Delivery 

appropriate dress. In addition to identifying needed skills, businesses also 

provided employment opportunities. Nationally, 34 percent of local boards 

reported that businesses subsidized work experience for WIA youth. In 

New Hampshire, for example, a financial services firm employed out-ofschool youth and taught them about personal financial management. In 

addition, an appliance-store owner, once an at-risk youth himself, hired 

WIA youth and hoped to groom one of them to take over the business. 

Finally, officials in three states we visited noted that businesses donated 

in-kind assistance such as building materials, work clothes, work 

readiness training workshops, and a financial management curriculum. 

For example, in an urban area in Virginia, businesses donated and 

remodeled the space for the one-stop center. 

Local areas also partnered with schools to provide academic preparation 

services to WIA youth. In a local area of Louisiana, for example, a service 

provider helped eighth graders explore classes leading to high school 

industry-based certifications. In all five states we visited, representatives 

from local postsecondary institutions made presentations to WIA youth, 

informing them about higher education opportunities. In some instances, 

WIA youth were allowed to take advanced level courses at their 

institutions and even earn college credit. In addition, a community college 

in California provided a pathway to facilitate the transition from high 

school to higher education by hosting youth for a one-day college 

experience of classes and conducting workshops on financial aid. Officials 

in four states we visited also said that schools provided in-kind assistance 

such as office space and tutoring. 

Local boards reported that youth councils performed a number of 

important functions that facilitated partnerships between local boards and 

the community. Over 70 percent of local boards reported that the youth 

council served as a forum to bring together key community partners who 

may never have collaborated with each other. Almost two-thirds of local 

boards reported that they would keep their youth council even if it became 

optional.18 In addition, over half of local boards reported that their youth 

councils elevated the importance and visibility of local youth issues and 

programs and added value to the youth program. 

 18Of the local boards that reported that they would not keep their youth council, 73 percent 

reported that they would use a youth committee of the local board to perform the functions 

of a youth council. 
Page 21 GAO-04-308 WIA Youth Service Delivery 

Local areas continue to face challenges in implementing aspects of the 

WIA youth program despite guidance issued by Labor. Some of these 

ongoing challenges include recruiting and retaining out-of-school youth, 

providing mentoring and follow-up services, and designing interim 

performance measures that can be used to continually improve aspects of 

program performance before youth exit the program. Labor supplements 

guidance on these issues through information forums, a Web site, and a 

technical assistance program. However, regional Labor officials said that 

state and local areas’ access to these resources had been limited. 

Labor has issued guidance on recruiting and retaining out-of-school youth 

for the WIA program during the last two years, but local areas continue to 

face challenges in serving this population. Labor issued a guidance letter 

to states in April 2001 outlining strategies for recruiting youth to the 

program, suggesting such methods as engaging youth to recruit their 

peers, collaborating with community organizations that already work with 

disadvantaged youth, and offering incentives for recruiting new 

participants.19 In subsequent guidance issued in May 2002, Labor described 

ways to keep out-of-school youth engaged in the program, such as by 

helping youth make the link between career potential and education and 

skills training as well as by providing support services like child care.20

Despite this guidance, local officials from most of the states we visited 

said that recruiting out-of-school youth continues to be a problem. 

According to data from Labor, the District of Columbia, Delaware, and 

New Mexico had not met WIA’s requirement to spend at least 30 percent of 

WIA funds on out-of-school youth for program year 2001 as of September 

30, 2003.21

Some local workforce officials indicated that closer coordination with 

schools to immediately connect dropouts to the WIA program would help 

them identify and recruit more out-of-school youth. Some local workforce 

and regional Labor officials said that one reason schools may not share 

information on dropouts directly with the WIA program was because of 

concerns about student privacy restrictions. However, schools we visited 

 19See Training and Employment Guidance Letter No. 18-00. 

20See U.S. Department of Labor, Training and Employment Guidance Letter No. 28-01, 

(Washington, D.C., 2002). 

21Under WIA, states have three years to spend their annual allotment, therefore program 

year 2001 funds must be expended by June 30, 2004. 

Labor’s Guidance Has 

Not Fully Addressed 

Implementation 

Challenges Faced by 

Local Areas 

Out-of-School Youth Were 

Hard to Recruit and Retain 
Page 22 GAO-04-308 WIA Youth Service Delivery 

in four states shared dropout information by referring students who had 

been expelled or had dropped out directly to the WIA service provider, 

notified or provided contact information to the WIA provider when a 

student had dropped out, or worked with the WIA program to develop a 

list of dropouts. None of the schools, however, had procedures in place to 

routinely connect all dropouts with local WIA programs. Labor and 

Education officials agreed that schools could do more to work closely 

with local workforce officials to help connect dropouts to local WIA youth 

programs. 

Local officials from most of the states we visited said that retaining out-ofschool youth in the WIA program was also difficult. According to some 

local officials, one reason was that out-of-school youth tend to prefer 

immediate employment instead of training and academic learning. Other 

officials said that both in-school and out-of-school youth face difficulties 

with transportation—particularly in rural areas—-that limit their 

involvement in WIA programs. For example, local officials in a rural area 

in Ohio said that transportation was their biggest issue. To mitigate this 

challenge, the local area offered services through a mobile one-stop unit 

that traveled to WIA clients throughout the county. Some state workforce 

officials said that to be more effective in serving this population, they 

needed guidance and technical assistance that was more focused on the 

specific service delivery issues within their local areas. 

Nearly all local areas we visited indicated that providing mentoring 

services continued to be a problem for the WIA youth program, but Labor 

has not addressed mentoring in its annual youth program guidance or 

shared best practices on the provision of high-quality mentoring services. 

Local areas identified several reasons why mentoring has been difficult. In 

one instance, a Virginia official in a rural area said that it was difficult to 

identify sufficient numbers of qualified mentors to work with eligible 

youth. In another instance, a local area official in California noted that 

geographical distances within the county resulted in long commutes and 

discouraged adults from mentoring youth. A service provider in New 

Hampshire said that finding mentors was especially difficult for out-ofschool youth, as some adults are uncomfortable working with school 

dropouts. Labor youth program officials said that mentoring is an 

important way of connecting youth to caring adults and said they would 

take action to provide guidance on this topic. 

Another service element that remains challenging for local areas is 

providing complete and thorough follow-up services to help youth succeed 

Mentoring and Follow-up 

Were among the Most 

Difficult Services to 

Provide 
Page 23 GAO-04-308 WIA Youth Service Delivery 

after they have exited the program. WIA regulations require that follow-up 

services last for at least 12 months, and Labor’s guidance states that 

follow-up may include regular contact with a youth’s employer to track 

progress made, assistance in addressing work-related problems, and help 

in securing better-paying jobs and further education. Labor’s policy 

guidance for program year 2001 provided some principles from best 

practices in the field of youth development to help local areas develop 

strategies for follow-up. The principles included developing a systematic 

approach for maintaining contact and interaction with the young person; 

evening and weekend social activities for informal support; helping youth 

access services to fulfill physical, emotional, and vocational needs; and 

visiting the job site soon after the youth has started employment. 

Nevertheless, several local officials cited continued difficulties in 

sustaining a connection with youth and identifying outcomes for them 

once they exited the WIA program. 

Labor’s guidance underscored the importance of establishing and using 

optional interim performance measures to monitor the success of 

delivering WIA youth services, but some regional Labor officials said that 

such measures were not widely used. Six of the seven required WIA youth 

measures are collected only after youth exit the program. However, 

because youth may remain in the program for many years, local areas with 

long-term youth retention strategies may have limited means of gauging 

progress without interim measures. Labor issued guidance in May 2002, 

encouraging the use of interim measures, which may be tracked and 

reported separately from the required annual performance measures, to 

provide ongoing feedback on which aspects of the youth program were 

working well or needed modification.22 The guidance identified the 

following interim measures that local boards could use to monitor 

progress as youth move through the WIA program: 

• intake, including development of a comprehensive individual service 

strategy with short-term and long-term goals; 

• participation in program elements to help ensure youth meet short-term 

skill attainment goals; 

 22See Training and Employment Guidance Letter No. 28-01. 

Few Local Areas Used 

Interim Measures to Gauge 

Program Success 
Page 24 GAO-04-308 WIA Youth Service Delivery 

• skill attainment to help ensure successful completion of short- and longterm goals; 

• exit data, including number of participants exiting the program; and 

• follow-up services received to help measure youth performance outcomes. 

Some local areas we visited were using interim measures to hold providers 

accountable for delivering services. For example, New Hampshire 

workforce officials said that they tracked enrollment and expenditure 

levels to monitor activity levels across their contracted service providers. 

In a local area in California, workforce officials monitored the progress of 

youth by tracking enrollment, participation, work readiness skills, and the 

rate at which youth made a successful transition to other activities once 

they exited the WIA program. 

Despite the potential usefulness of these data, interim measures were not 

widely used, according to Labor officials in two regions. These officials 

said that states lacked the resources to properly track them. Additionally, 

the guidance did not explain how states and local areas could collect, 

analyze, and use the data to assess progress and make needed 

adjustments. For example, officials in one state we visited said that they 

were unclear about how to apply the interim measures. 

Labor has disseminated information related to these and other issues 

through forums that allow local areas to exchange information with one 

another, but access to these forums has been limited. From December 

2002 to April 2003, for example, Labor sponsored peer-to-peer WIA 

Learning Exchanges with two national youth organizations that provided 

venues around the country for local areas to share information and 

observe an exemplary program firsthand. However, state budgetary cuts 

prevented many local areas from attending some of these forums in 

person, according to an official from the sponsoring organization. 

Although information and ideas shared at the Learning Exchanges were 

later summarized and made available to all local areas, those that were 

unable to attend missed the opportunity to network with their colleagues 

and develop an action plan to take back and apply to their program.23

 23Labor officials told us that their regional offices have also sponsored youth conferences 

and other forums where guidance and technical assistance have been provided. 

Access to and Awareness 

of Labor’s Assistance Has 

Been Limited 
Page 25 GAO-04-308 WIA Youth Service Delivery 

Labor has also supported the development of an online resource to 

facilitate information sharing among local areas. The agency’s Promising 

Practices Web site was intended to provide a mechanism for local areas to 

post and share promising workforce development practices, including 

those pertaining to WIA youth programs. However, some state workforce 

and regional Labor officials said that states and local areas may not even 

have been aware of the site, that it has been difficult to enter practices for 

inclusion, and that some users found it difficult to access the documents 

described in the narrative.24 We also had difficulty accessing and using the 

Web site to find information. For example, when we conducted a search 

on the key word “mentoring,” the five results did not satisfactorily address 

the topic. 

Labor guidance encourages local areas to tap into other resources that can 

help them develop quality youth programs.25 The Promising and Effective 

Practices Network, for example, provides a useful listing of promising 

practices categorized by the specific strategy and the 10 required WIA 

youth services that local areas can use for improving their WIA youth 

programs.26

Labor has provided technical assistance to states through its Performance 

Enhancement Project, initiated in program year 2002. Under this initiative, 

Labor grouped into three tiers states that failed or were at risk of failing to 

meet their performance goals, based on their reported outcomes, 

according to Labor officials.27 These officials also said that states in the 

first tier received priority for targeted technical assistance to improve their 

youth programs, with the goal of improving performance outcomes. Labor 

officials said that states could use this technical assistance to address any 

of the challenges they faced in implementing their WIA youth program. 

However, unless the state falls into one of these three tiers, most local 

areas may not receive such assistance to help them increase the 

 24We previously reported on problems with the Promising Practices Web site. See 

GAO-03-725. This Web site can be found at http://www.promising-practices.org. 

25See Training and Employment Guidance Letter No. 18-00. 

26The National Youth Employment Coalition’s Promising and Effective Practices Network 

Web site can be found at http://www.nyec.org/pepnet. 

27According to Labor, the first tier includes those states that were financially sanctioned. 

The second tier includes those states that failed one or more performance measures but 

were not sanctioned. The third tier includes those states that did not fail any measures but 

were deemed at risk of failing them. 
Page 26 GAO-04-308 WIA Youth Service Delivery 

proportion of out-of-school youth served, improve mentoring and followup, and use interim measures. 

Little is known about the effectiveness of the WIA youth program because 

Labor has not yet initiated an impact evaluation, and results from a 

planned evaluation will likely not be available until 2009, according to 

Labor officials. While Labor’s performance data for program year 

2001 indicate that five of seven youth measures were exceeded, these 

results cannot be used to infer program outcomes because of insufficient 

monitoring of state data quality and inadequacies of some state 

management information systems. Labor officials said that they will 

require states to validate local area data beginning with program year 

2002 data, but some implementation concerns remain. 

According to department officials, Labor intends to initiate an impact 

evaluation28 of the WIA youth program in 2006, and preliminary results will 

not be available until about 2009. While Labor officials said that the youth 

program’s 3 years of full implementation was sufficient time to initiate a 

comprehensive evaluation, they were anticipating significant changes to 

the WIA youth program as a result of reauthorization. They did not plan to 

begin an impact evaluation of the program until after these changes had 

been implemented. 

If reauthorization is completed by spring 2004, Labor officials said they 

anticipate that the process of awarding the contract for the study will be 

completed by 2006 when the impact evaluation is scheduled to begin. The 

evaluation will proceed with 5 or 6 years of data collection with an 

additional 3 or 4 years of follow-up activity. Labor officials said they 

expect to issue a series of interim reports before the final product. If the 

project begins in 2006, Labor expects to issue the first report on short-term 

impacts in 2009, with a final report on long-term impacts available in 

2011. According to officials, this evaluation is part of Labor’s proposed 

 28By isolating a program’s effects, impact evaluations provide policymakers with key 

information for determining program effectiveness. To isolate a program’s effect, impact 

evaluations divide participants into two groups: one that receives program services and a 

similar group that does not. Some impact evaluations assign participants randomly to one 

or the other group; the group that does not receive services is called the control group. The 

use of random assignment allows researchers to compare outcomes for the two groups and 

attribute any differences to the program services rather than other factors. 

Lack of Program 

Evaluations and 

Questionable 

Performance Data 

Prevent Assessments 

of WIA Youth 

Program 

Labor Plans to Initiate 

First Impact Evaluation in 

2006, and Results Are At 

Least Five Years Away 
Page 27 GAO-04-308 WIA Youth Service Delivery 

research plan to study all of its major employment and training programs, 

including the WIA youth program, on a regular cycle.29

Performance data submitted by states to Labor in quarterly and annual 

reports were not sufficiently reliable to determine outcomes for the WIA 

youth program. Labor’s national aggregation of WIA performance data for 

program year 2001 indicated that the program exceeded its goals for five 

of the seven youth measures.30 According to Labor’s Office of Inspector 

General (OIG), however, there is little assurance that the states’ 

performance data for all WIA programs, including the youth program, are 

either accurate or complete because of inadequate oversight of data 

collection and management at the local, state, and federal levels. The OIG 

also found that just 2 of 12 local areas and none of four states it reviewed 

had formal policies on documentation requirements for participant 

activities and outcomes.31 At the local level, for example, another OIG 

report found that WIA youth program outcomes were adequately 

documented only 37 percent of the time for a sample of 420 participants 

across 14 local areas.32 An official in one of Labor’s regional offices added 

that documentation requirements are inconsistent among states and local 

areas. 

At the state level, the OIG reported that two of four states it reviewed had 

not monitored local areas’ performance data at the case file level, and that 

none of the four states had adequate procedures in place to ensure the 

accuracy of their performance data.33 According to regional Labor officials, 

some states had insufficient procedures in place for verifying the accuracy 

of the data collected by their local areas. For example, officials in one 

region said that state monitoring rarely addressed data accuracy or 

 29Labor anticipates that the WIA youth program will be evaluated on a 10-year cycle. 

30States did not meet goals for the earnings change measure and credential rate for older 

youth. States reported actual earnings change in 6 months as $3,109, compared with their 

negotiated goal of $3,396. For the credential rate, about 40 percent of older youth earned 

credentials, compared with the goal of 44 percent. 

31U.S. Department of Labor, Office of Inspector General, Workforce Investment Act 

Performance Outcomes Reporting Oversight, 06-02-006-03-390 (Washington, D.C., 

Sept. 30, 2002). 

32U.S. Department of Labor, Office of Inspector General, Workforce Investment Act: 

Evaluation of Youth Program Enrollments, Services, and Recorded Outcomes, 

06-03-006-03-390 (Washington, D.C., September 30, 2003). 

33U.S. Department of Labor, Office of Inspector General, 06-02-006-03-390. 

State Performance 

Outcome Data Were 

Questionable, Partly 

because of Inconsistent 

Data Monitoring 
Page 28 GAO-04-308 WIA Youth Service Delivery 

included the verification of a sample of data items against original records. 

In addition, some state information systems had significant flaws that 

caused them to produce incorrect information, casting further doubt on 

the quality of WIA performance data. For example, Ohio state officials said 

they were in the process of replacing their old information system that had 

caused such errors as data changing or disappearing entirely after it had 

been entered in the system. 

At the federal level, Labor did not have a standard data-monitoring guide 

in place, and officials in some regional offices—who, according to agency 

officials, are responsible for overseeing the quality of states’ reported 

data—-said they followed their own oversight procedures. These 

procedures did not usually include verifying the accuracy of a sample of 

the data submitted by states. For example, an OIG report stated that while 

regional offices conducted some data accuracy reviews, such as computer 

edit checks, they did not verify the data’s accuracy with such tests as 

comparing the data with participant case files. Consistent with the OIG’s 

findings, officials in all six of Labor’s regional offices said that they 

examined state data submissions through desk reviews, which included 

checking for errors such as incorrect calculation of performance 

measures, extreme outliers, and miscoding of data. However, only the 

Atlanta regional office checked a sample of data records against source 

documentation. In its review of data records from six of the states it 

oversees, the Atlanta office examined a sample of participant records 

across all WIA programs from each state and found errors that affected the 

calculation of one or more performance measures. In one state, for 

example, 17 percent of participant records had at least one error, 

compared with 83 percent of participant records in another state. The 

regional office also found that two of six states it reviewed computed the 

younger youth skill attainment measure incorrectly. 

Labor recognizes these problems with data reliability, but in accordance 

with WIA regulations, uses states’ annual performance reports—-the only 

reports that depict states’ progress in achieving or exceeding negotiated 

performance levels—-to make incentive grant and sanction decisions. 

States that exceed the WIA performance goals negotiated between Labor 

and the states are eligible to receive incentive grants that generally range 

from $750,000 to $3 million.34 States that fail to meet 80 percent of their 

 34To be eligible for incentive grants, states must also meet performance goals for the Adult 

Education and Literacy programs under Title II of WIA and programs authorized under the 

Carl D. Perkins Vocational and Applied Technology Education Act. 
Page 29 GAO-04-308 WIA Youth Service Delivery 

WIA performance goals for 2 years in a row are subject to sanction of up 

to a 5 percent reduction in their annual WIA formula grant. However, 

Labor’s use of questionable performance data reported by the states may 

hinder its ability to negotiate realistic performance goals and make 

appropriate incentive grant and sanction decisions. Ohio state officials 

questioned the appropriateness of being sanctioned 2 percent of its 

program year 2002 WIA youth allotment. They believed that poor 

performance levels were due to problems with its information system, 

not its workforce development system. 

To address the data issues described above, Labor is implementing a new 

data validation policy requiring states to ensure the accuracy of their 

annual reports and verify a sample of the data they submit. According to 

Labor’s policy, these requirements will be phased in over a 3-year period. 

In the first year, states will be required to validate their annual reports and 

data submissions for the program year 2002 period.35 According to agency 

officials, Labor does not plan to publicly release these initial reports but 

will use their findings to work with the states to correct their data 

accuracy problems. In the second year, Labor will use validation reports 

covering program year 2003 data to establish acceptable error rates. In the 

third year, Labor will require states to meet the acceptable error rates for 

their program year 2004 data submissions. Labor will consider failure to 

meet the standard a violation equivalent to failing to submit a report, for 

which states may be subject to corrective action or financial sanction, as 

appropriate. In addition, Labor’s data validation guidance indicates that 

states that do not meet data accuracy standards will receive technical 

assistance. 

While the data validation initiative may improve the reliability of WIA 

performance data, several implementation concerns remain. First, officials 

in some of Labor’s regional offices said that the states they oversee will 

have difficulty implementing the data element verification requirement 

because of limited staff resources. Further, these officials said that in 

states where local providers keep the original documentation on-site, 

retrieving the documentation to check it against records in their 

information system will be time-consuming and expensive. Second, a 

Labor official said that the agency does not plan to issue a program-

 35Program year 2002 started on July 1, 2002, and ended on June 30, 2003. 

Data Validation Initiative 

May Improve Reliability 

but Concerns Remain 
Page 30 GAO-04-308 WIA Youth Service Delivery 

monitoring guide to standardize procedures across regional offices for 

several more years. 

Many youth are struggling to be successful in the public education system 

and often face substantial obstacles to obtaining a high school diploma 

and going on to college or getting jobs with career advancement 

possibilities. Many of these youth may lack meaningful social and family 

supports and may require comprehensive, intensive services to remain 

engaged in society and avoid risky behaviors that can lead to chronic 

unemployment, criminal activity, and other adverse outcomes. WIA 

currently provides both the education and workforce systems with 

strategies and resources to engage youth in academic and job training. 

The opinion held by some federal education officials that WIA in-school 

services overlap with existing education programs is not necessarily 

seconded by officials in local areas. State and local workforce and 

education officials believe WIA’s educational, occupational, and support 

services provide critical support and services to at-risk youth and that 

without WIA’s comprehensive services, schools may face an increasing 

burden to keep these youth in school and ensure their academic success. 

The connection between WIA youth services and schools could be made 

more effective if Labor worked with Education to find ways to connect 

school dropouts with local WIA youth programs. 

Since WIA’s passage, Labor has provided general guidance and promising 

practices in addressing implementation issues, but increased availability 

of technical assistance may be necessary to overcome some of the more 

difficult challenges some states and local areas face in providing youth 

services. This will be especially critical for states and local areas that are 

reportedly meeting performance goals but still need assistance in 

improving delivery of youth services and shifting program resources to 

target different populations such as out-of-school youth. In addition, local 

areas will need guidance, including specific strategies, to help them 

provide effective mentoring and follow-up services and use interim 

measures to track program performance. 

Robust research and reliable performance data are needed to obtain a 

complete picture of the WIA youth program’s effectiveness and outcomes 

and to make quality decisions about managing the program. However, 

none currently exist. Labor has not yet initiated an impact evaluation and 

does not expect to report on program impacts from a planned evaluation 

until 2009. In addition, states and local areas continue to struggle to collect 

and document accurate and complete participant and performance data 

and maintain data systems that can yield reasonably reliable outcome 

Conclusions 
Page 31 GAO-04-308 WIA Youth Service Delivery 

information. For its part, Labor is taking action to improve data integrity 

by requiring states to validate WIA performance data, beginning with data 

from program year 2002. However, Labor’s inconsistent monitoring 

processes across regions will continue to challenge Labor’s capability to 

ensure that all states validate and report data consistently and effectively. 

In the short term, the lack of accurate outcome data will continue to 

hinder Labor’s ability to negotiate realistic state performance goals and 

use data to make sound decisions about financial incentives or sanctions. 

In the long term, lack of accurate and complete information will keep 

Labor from obtaining a true picture of how effectively the youth program 

is working. 

To promote information sharing that improves local WIA youth programs’ 

ability to identify and serve out-of-school youth, we recommend that the 

Secretaries of Labor and Education coordinate efforts to clarify how 

schools can work with workforce officials to connect school dropouts 

with local WIA youth programs. 

To assist state and local WIA youth programs address ongoing 

implementation challenges, we recommend that the Secretary of Labor 

• increase availability of guidance and technical assistance to local areas 

that continue to face challenges in serving out-of-school youth; 

• disseminate guidance, including specific strategies, to help local areas 

provide effective mentoring services; and 

• develop additional guidance on providing follow-up services and using 

interim measures to track program performance. 

To obtain an accurate and complete gauge of WIA outcomes and 

determine whether local programs are operating successfully, we 

recommend that the Secretary of Labor establish standard monitoring 

procedures that Labor’s regional offices could use to oversee state data 

validation efforts. 

We provided a draft of this report to Labor and Education officials for 

their review and comment. Education’s comments are reprinted in 

appendix I. In its formal comments, Education concurred with our 

recommendation to work with Labor to connect out-of-school youth to 

local WIA youth programs. Labor responded informally, and said that 

Recommendations for 

Executive Action 

Agency Comments 

and Our Evaluation 
Page 32 GAO-04-308 WIA Youth Service Delivery 

many of the findings corroborated its own observations and that the 

recommendations are consistent with Labor’s current program direction. 

Both agencies also provided technical comments, which we incorporated 

as appropriate. 

We will send copies of this report to the Secretaries of Labor and 

Education, relevant congressional committees, and other interested 

parties. Copies will be made available to others upon request. In addition, 

the report will be available at no charge on GAO’s Web site at 

http://www.gao.gov. 

Please contact me at (415) 904-2272 if you or your staff have any questions 

about this report. Other major contributors to this report are listed in 

appendix II. 

Sincerely yours, 

David D. Bellis 

Director, Education, Workforce, 

 and Income Security Issues 
Appendix I: Comments from the Department 

of Education 

Page 33 GAO-04-308 WIA Youth Service Delivery 

Appendix I: Comments from the Department 

of Education 
Appendix I: Comments from the Department 

of Education 

Page 34 GAO-04-308 WIA Youth Service Delivery 
Appendix II: GAO Contacts and Staff 

Acknowledgments 

Page 35 GAO-04-308 WIA Youth Service Delivery 

Lacinda Ayers, Assistant Director (206) 654-5591 

Meeta Sharma, Analyst-in-Charge (206) 287-4806 

In addition to the individuals mentioned above, the following staff made 

key contributions to this report: Karyn Angulo, Susan Baker, Andrew 

Bauck, Paula Bonin, Maya Chakko, Keira Dembowski, Joel Grossman, 

Corinna Nicolaou, Rebecca Woiwode, and Monica Wolford. 

Appendix II: GAO Contacts and Staff 

Acknowledgments 

GAO Contacts 

Staff 

Acknowledgments 
Related GAO Products 

Page 36 GAO-04-308 WIA Youth Service Delivery 

Transportation-Disadvantaged Populations: Some Coordination Efforts 

Among Programs Providing Transportation Services, but Obstacles 

Persist. GAO-03-697. Washington, D.C.: June 30, 2003. 

Workforce Investment Act: One-Stop Centers Implemented Strategies to 

Strengthen Services and Partnerships, but More Research and 

Information Sharing is Needed. GAO-03-725. Washington D.C.: June 18, 

2003. 

Workforce Investment Act: Issues Related to Allocation Formulas for 

Youth, Adults, and Dislocated Workers. GAO-03-636. Washington, D.C.: 

April 25, 2003. 

Workforce Investment Act: States’ Spending Is on Track, but Better 

Guidance Would Improve Financial Reporting. GAO-03-239. Washington, 

D.C.: November 22, 2002. 

Workforce Investment Act: Youth Provisions Promote New Service 

Strategies, but Additional Guidance Would Enhance Program 

Development. GAO-02-413. Washington, D.C.: April 5, 2002. 

Workforce Investment Act: Improvements Needed in Performance 

Measures to Provide a More Accurate Picture of WIA’s Effectiveness. 

GAO-02-275. Washington, D.C.: February 1, 2002. 

School Dropouts: Education Could Play a Stronger Role in Identifying 

and Disseminating Promising Prevention Strategies. GAO-02-240. 

Washington, D.C.: February 1, 2002. 

Related GAO Products 

(130214) 
The General Accounting Office, the audit, evaluation and investigative arm of 

Congress, exists to support Congress in meeting its constitutional responsibilities 

and to help improve the performance and accountability of the federal 

government for the American people. GAO examines the use of public funds; 

evaluates federal programs and policies; and provides analyses, 

recommendations, and other assistance to help Congress make informed 

oversight, policy, and funding decisions. GAO’s commitment to good government 

is reflected in its core values of accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and fulltext files of current reports and testimony and an expanding archive of older 

products. The Web site features a search engine to help you locate documents 

using key words and phrases. You can print these documents in their entirety, 

including charts and other graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site 

daily. The list contains links to the full-text document files. To have GAO e-mail 

this list to you every afternoon, go to www.gao.gov and select “Subscribe to e-mail 

alerts” under the “Order GAO Products” heading. 

The first copy of each printed report is free. Additional copies are $2 each. A 

check or money order should be made out to the Superintendent of Documents. 

GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a 

single address are discounted 25 percent. Orders should be sent to: 

U.S. General Accounting Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800 

U.S. General Accounting Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

GAO’s Mission 

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Mail or Phone 

To Report Fraud, 

Waste, and Abuse in 

Federal Programs 

Public Affairs