Employment and Training Administration Occasional Paper 2003-10 

Review of Recent Pilot, Demonstration, 

Research, and Evaluation Initiatives to 

Assist in the Implementation of Programs 

under the Workforce Investment Act 

Burt S. Barnow 

Daniel B. Gubits 

Johns Hopkins University

This report has been funded, either wholly or in part, with Federal funds from the U.S. 

Department of Labor, Employment and Training Administration under Contract Number 

K-6826-8-00-80-30. The contents of this publication do not necessarily reflect the views 

or policies of the Department of Labor, nor does mention of trade names, commercial 

products, or organizations imply endorsement of same by the U.S. Government. 
1 

his report provides a review of recent research and evaluation studies in the 

employment and training area. The report has several objectives. First, we provide 

policy makers, government officials, researchers, and others with an interest in 

employment and training a summary of the major findings from recent studies. Second, 

when warranted, we suggest the implications of the studies for program and policy 

development. Finally, we suggest areas where additional research and evaluation would 

be helpful to help formulate and improve policies and programs. 

Because the literature on employment and training is so extensive we made several 

decisions on how to limit the scope of the creport and which areas to emphasize. First, 

we have generally included only studies published after 1995. Second, we have focused 

almost exclusively on studies funded by the Employment and Training Administration 

(ETA). Our rationale is that the ETA-funded studies reflect the areas of greatest interest 

to ETA officials. Third, we have focused more on evaluations of current employment 

and training programs, and, consequently, paid little attention to general labor market 

research and programs administered by other agencies. It should be stressed that this 

decision was based on time and budget limitations and not on the merits or importance of 

the literature in these areas. Specific topics that are given scant attention in this report 

include wage subsidy programs such as the earned income tax credit, minimum wage and 

living wage laws, the public education K-12 system, vocational education, vocational 

rehabilitation, and employment and training programs funded by the U.S. Department of 

Housing and Urban Development. 

The report is organized as follows. We first discuss studies dealing with the Workforce 

Investment Act (WIA) and its predecessor, the Job Training Partnership Act (JTPA). 

Within this section, we first cover impact studies and then process and implementation 

studies. The following sections deal with the employment service and unemployment 

insurance. Next we cover employment and training programs dealing with target groups 

of particular interest: dislocated workers, welfare recipients, youth, and special target 

groups. The final section provides caveats and our recommendations for additional 

studies, including topics and programs not covered by recent research. 

A. THE JOB TRAINING PARTNERSHIP ACT AND THE WORKFORCE 

INVESTMENT ACT 

he Job Training Partnership Act (JTPA) served as the nation’s major employment 

and training program between 1983 and 2000. The Comprehensive Employment and 

Training Act (CETA), which was in operation between 1973 and 1982, preceded it and 

the Workforce Investment Act (WIA), which was passed in 1998 and became effective in 

July 2000, succeeded it. Training for economically disadvantaged adults was provided 

under Title II-A of JTPA, and other components of JTPA covered special target groups 

such as disadvantaged youth, Indians and Native Americans, migrant and seasonal farm 

workers, and dislocated workers. This section deals solely with the program for 

disadvantaged adults, and other programs are discussed in later sections. 

T 

T 
2 

Evaluations of CETA and earlier employment and training programs had been carried out 

using nonexperimental methods, and the impact evaluations provided a wide range of 

estimates.1

 Random assignment of individuals to treatment status can avoid many of the 

problems that arise in impact evaluations, so an advisory panel recommended that an 

experimental design be used to evaluate the JTPA Title II-A program. The National 

JTPA Study was carried out as a classical experiment in 16 sites across the country 

between November 1987 and September 1989. Abt Associates and the Manpower 

Demonstration Research Corporation conducted the evaluation, and the findings are 

available in several publications.2

 

The National JTPA Study included disadvantaged adults and out-of-school youth. It was 

considered impractical to include in-school youth because it would be difficult to use 

random assignment for this group, and we believe dislocated workers were excluded 

because of budget constraints. Although the design called for random selection of sites, 

the 16 sites in the study consisted of all local service delivery areas (SDAs) that were 

willing to participate in the project. Data for the evaluation were obtained from a brief 

baseline survey, two waves of telephone follow-up surveys, and administrative data from 

state agencies. Because not all those assigned to receive training actually participated, 

true experimental findings can only be obtained comparing those assigned to the training 

group with those assigned to the control group. Under the plausible assumption that 

treatment assignment had no effect on outcomes, the researchers were also able to 

estimate the impact on those who enrolled in the program.3

The National JTPA Study found JTPA to have a statistically significant but modest 

impact on the earnings of adult men and women for the 30-month follow-up period and 

no impact for out-of-school youth. The total impact over the 30 months following 

random assignment, which includes the period when the treatment group was in training, 

was $1,837 per assignee for women and $1,599 per assignee for men, both statistically 

significant. The impacts for months 19 through 30, which correspond roughly to the first 

year following training for the treatment group, were $847 for women and $856 for men, 

with both figures statistically significant. The estimated impacts for youth were negative 

and insignificant. 

The study also reported the impact by “assigned service strategy.” For adult women, 

those who were assigned to receive on-the-job training (OJT) or job search assistance 

(JSA) had a statistically significant impact of $2,292 over the 30 months following 

random assignment, but for women selected to receive classroom training, the estimated 

impact was an insignificant $630 over the period. For adult women assigned to receive 

“other services,” the impact was a statistically significant $3,949. For adult men, the 

estimates for all three service strategies were positive but not statistically significant--

$1,287, $2,109, and $941 for classroom training, OJT/JSA, and other services, 

respectively. 

 

1

 See Barnow (1987) for a review of this literature. 

2

 See Bloom et al. (1994), Bloom et al. (1997), and Orr et al. (1996). 3

 Heckman, Smith, and Taber (1998) verified that the assumptions appeared to be met for the JTPA 

evaluation. 
3 

It is important to keep in mind that although the control group was barred from 

participating in JTPA, they frequently received other services. Nearly one-quarter of the 

adult men and one-third of the adult women in the control group received some 

employment and training services during the 30 months covered by the study. Thus, the 

impacts reported do not provide estimates of the impact compared to no services, but 

rather, compared to the services received by the control group. 

Finally, the National JTPA Study estimated the net present value of the program. For 

adult men and women, the program produced net social benefits of about $500, but for 

youth, the program had a net social cost of $1,180 for men and $2,923 for women. These 

costs and benefits do not extrapolate the earnings gains beyond the 30-month observation 

period, so the actual net benefits for adults are likely to be larger. 

The General Accounting Office (GAO) (1996) extended the National JTPA Study by 

merging social security earnings data with study data to develop estimates of the program 

impact on earnings five years after random assignment. In their report, GAO stated: “we 

found no significant effect of JTPA on earnings or employment rates after 5 years.” 

Although technically correct, what GAO fails to point out is that the point estimate of the 

earnings impact remained fairly consistent over the five-year follow-up period. What the 

data in an appendix show is that the standard errors of the estimates increased steadily 

over time as the earnings in both the treatment and control groups tended to disperse over 

time. Contrary to the negative conclusions of GAO, the evidence in their study is 

supportive (but not conclusive) of the hypothesis that the earnings gains for adult men 

and women persist for at least five years. 

Although the National JTPA Study showed that the mix of services offered under JTPA 

had positive impacts for adults and were not useful for out-of-school youth, there were 

some important questions that the study did not answer. First, because the sites were not 

selected randomly, there has been some controversy about how representative the 

findings are.4

 This problem might be reduced or avoided in future evaluations by using 

the methodology employed in the recent Job Corps evaluation where more sites were 

used and only a small number of individuals were assigned to the control group in each 

site. Second, the National JTPA Study did little to shed light on which service strategies 

are most effective. Because random assignment occurred after the service strategy was 

selected, the individuals in various activity streams are likely to have quite different 

characteristics, and the impact identified for a particular service strategy might not be the 

same for individuals referred to other service strategies. It is unlikely that this selection 

problem can be resolved for a large national evaluation, so research on alternative service 

strategies should be undertaken in smaller-scale efforts or using nonexperimental 

methods. 

 

4

 There are several reasons why local programs may have chosen not to participate in the study: (1) they 

may have believed that the use of random assignment is unethical; (2) they may have been ineffective and 

were reluctant to have the evaluation bring this to light; and (3) they may have had problems recruiting 

sufficient participants without the experiment, and the experiment would have required them to admit 50% 

more participants. 
4 

One modification that would be useful is to better distinguish the service strategies 

studied. Specifically, lumping job search assistance, which is the cheapest strategy, with 

on-the-job training (OJT), which is one of the most expensive strategies, should be 

avoided in future studies. In effect, the National JTPA Study grouped participants by 

how job ready they were rather than by service stream. In addition, it would be useful to 

separate basic skills classroom training from occupational classroom training. Finally, 

the National JTPA Study made use of the service strategy assigned rather than the service 

strategy received; a more policy-relevant question is the impact of service strategies 

received, and this can only be investigated using nonexperimental methods. 

Eberwein, Ham, and LaLonde (1997) used the National JTPA Study data to explore 

whether the earnings gains found in that study for adult women enrolled in classroom 

training resulted from a reduction in the time unemployed or an increase in the length of 

employment spells. Using hazard models, the researchers estimate the probabilities that 

individuals will exit a spell of employment or unemployment in a particular month as a 

function of program status (measured as assigned to treatment or control status, or 

alternatively based on receipt of training) and personal characteristics (e.g., education, 

age, race and ethnicity, prior work experience, marital status, and welfare status). They 

find that receipt of training significantly reduces the length of unemployment spells but 

does not affect the length of employment spells. Their findings reinforce the common 

concern that traditional employment and training programs have not done enough to 

foster employment retention among participants. 

Welfare Recipients in JTPA 

Several researchers have examined how particular groups, such as welfare recipients, 

have fared in JTPA. Heinrich (1998a) analyzed data on welfare recipients from the Cook 

County, Illinois service delivery area from 1984 through 1994. This area serves much of 

the suburban area surrounding the City of Chicago. Heinrich first used multinomial logit 

models to determine if, other things equal, welfare recipients were likely to receive the 

same services as other JTPA recipients. She found that welfare recipients were more 

likely to receive classroom training and job search assistance than other participants, and 

they were less likely to be placed in OJT positions. She noted the absence of sequences 

of training whereby those with minimal skills at entry would receive OJT after 

completing remedial education classes. She then looked at the labor market outcomes for 

welfare recipients compared to other JTPA participants, and she found that their postprogram labor market experiences were considerably worse than the experiences of other 

participants. This is consistent with the JTPA performance standards regression models, 

which were based on data for the universe of participants. These models have 

consistently showed worse labor market outcomes for welfare recipients in terms of 

placement rates and wages at placement. Heinrich then examined the relative 

effectiveness of OJT, vocational training, and basic skills training. She consistently 

found that OJT had large, statistically significant impacts on earnings and that basic skills 

training had a negative, statistically significant impact. She found mixed results for 

vocational training. She then looked further at the occupations trained for, and she found 
5 

that although training for high-skill occupations led to slightly lower placement rates, the 

impact on earnings was large and statistically significant. Heinrich also looked at how 

the impact of training varied for “repeaters.” 

Although one might hypothesize that those who required a second dose of training simply 

did not absorb the treatment, she found that those who enrolled a second time had higher 

placement and retention rates than first-time enrollments. Heinrich concludes by noting 

that it is not wise to invest in job search assistance and low-skill training for welfare 

recipients with serious labor market barriers; instead she suggests that more emphasis on 

OJT and high-skill vocational training are well worth the investment. 

Several other recent studies of welfare recipients’ experiences in JTPA and other training 

programs support Heinrich’s conclusions. In another paper, Heinrich (1998b) analyzes 

the effectiveness of a Chicago area program that was intended to serve people in the most 

disadvantaged part of the SDA with intensive services. The pilot project did not use 

random assignment, so Heinrich developed two comparison groups: adult Title II-A 

participants who were in regular JTPA activities, and individuals who applied to 

participate in the pilot but did not become participants. She found significant earnings 

gains during the post-program year for pilot participants relative to those in regular 

programs and those who applied but did not participate. She also found evidence that 

OJT and other more intensive services were more effective. Although this study in 

interesting, it should not in itself be given a great deal of weight in determining policy 

because of the small sample sizes, the lack of random assignment to treatment status, and 

the fact that neither of the comparison groups used were ideal.5

Plimpton and Nightingale (2000) review the recent literature and synthesize the findings 

on recent programs intended to assist welfare recipients in obtaining employment. Based 

on their review of 14 programs, Plimpton and Nightingale conclude: 

• Most welfare programs that offer low-cost, low-intensity services (like job search 

assistance and short-term unpaid work experience) have positive impacts on 

employment and earnings, and in some cases reduce welfare costs; 

• More comprehensive training programs offering services like supported, paid 

work experience and occupational training generally have larger and longerlasting impacts; 

• Even those interventions with the greatest impacts have been unable to move 

individuals and families out of poverty or permanently off the welfare rolls, nor 

have they produced economic self-sufficiency. 

Nudelman (2000) used the National JTPA Study data to analyze the experiences of 

welfare recipients in depth. Regression analysis was used to control for differences 

between the treatment and control groups and to improve precision of the estimates. 

Overall, Nudelman found that earnings for the treatment group exceeded earnings for the 

 

5

 There were 42 participants in the demonstration, and one of the comparison groups had only 55 members. 
6 

control group by $1,200 and $988 in the first and second post-program years, 

respectively. She conducted subgroup analyses to determine how the impact on earnings 

varied by several characteristics. Among the ethnic groups examined, only the white and 

other groups had a statistically significant impact on earnings over the 30-month followup period ($3,493 per participant). The point estimate of the impact was higher for those 

without a high school diploma than for participants with a diploma, $3,127 compared to 

$1,583, but only the estimated impact for participants with a diploma was statistically 

significant. When she examined how the impact varied by time on welfare, Nudelman 

found that those on AFDC for less than two years had a small estimated impact that was 

not significant, but those on welfare two to five years had an impact of $4,577 per 

participant over the 30-month follow-up period. The impact dropped to $2,887 for those 

on welfare more than five years, and the finding was significant only at the .10 level. 

Nudelman found that those assigned to the OJT/job search assistance service strategy and 

those assigned to receive other services had statistically significant earnings impacts in 

both the first and second post-program years, but the estimated impacts for those in 

classroom training (vocational and academic) were generally small and never statistically 

significant. 

The studies reviewed here on welfare recipients in JTPA provide a fairly consistent 

message: longer-term, more intensive training strategies appear to be considerably more 

effective. Although this suggests some broad policy strategies, there is considerable 

room for refining the strategies to make sure they are effective. Specifically, the 

Department of Labor should consider testing, using a classical experiment, strategies that 

foster intensive investments in participants rather than the short-term training typically 

offered. 

Early Implementation of the Workforce Investment Act 

As was noted above, the Workforce Investment Act (WIA) was enacted in 1998 and 

became operational in July 2000. Thus, there are only a few studies currently available 

that deal exclusively with WIA. Unlike its predecessors, JTPA and CETA, WIA was 

enacted with a relatively short authorization period, and decisions on whether to 

maintain, modify, or replace WIA must be made in 2003. At this time, three process 

studies analyze the early implementation of WIA--one written by ETA staff (2001), one 

by Buck (2002), and the third by D’Amico et al. (2001a). We also include one study, 

Grubb et al. (1999), that covers workforce system reform that was carried out after WIA 

was enacted, but prior to its implementation; a study on workforce development program 

potential overlap by the U.S. General Accounting Office (2000); and a study by 

Balducchi and Pasternak (2001) that was prepared for an international conference. 

Because these studies are process rather than outcome oriented, what is of most use for 

reauthorization purposes, and for further research as well, are some of the areas where 

implementation has not gone as planned and where there is substantial variation in 

implementation across states or local areas. Thus, although the reports are very positive 

about the early implementation of the program, we focus here more on the findings that 

are most important for formulating the research agenda. 
7 

Grubb et al. (1999) looks at workforce system reform efforts in 10 states during the 

1990s.6

 Within each state, two local areas were studied. The states were selected 

purposively, with an effort to pick states that had made a conscious effort to reform their 

workforce development system through a variety of means. The key to all the reforms 

was an attempt by states to reduce the duplication and fragmentation that is fostered by 

the presence of the plethora of employment and training programs that currently exist. 

Grubb et al. note that states use a variety of means to get programs to interact. For 

example, Florida uses a “zipper” approach where agencies have a stake in the 

performance of other agencies. Michigan has a “no wrong door” policy with three tiers 

of coordination: 

• Programs under control of the local workforce boards; 

• Programs where the local board has planning authority; and 

• Programs where the boards are supposed to use their authority to coordinate. 

Grubb et al. note that the reform efforts use two broad approaches to implementing their 

policies—institutional and market-based. States that use an institutional approach rely on 

agencies and boards to perform functions such as serving as an advisory board, providing 

technical assistance, and promoting technical change. Market-based strategies rely on 

market incentives to encourage efficiency. Market-based strategies include: 

• Performance standards; 

• Performance-based contracting; 

• Competition among local providers; 

• Competition through subcontracting; 

• Vouchers for customers; and 

• Provision of vendor information to consumers. 

Notably, WIA requires performance standards, vouchers, and provision of vendor 

information to customers. 

The Grubb et al. study identifies a number of barriers to implementing workforce system 

reform. Key barriers include an unstable environment, with changes in program focus or 

formation of new agencies; resistance from educational institutions that prefer to keep the 

rules of the game constant; inconsistencies in state policies; competing priorities for 

 

6

 The 10 states are Florida, Iowa, Maryland, Massachusetts, Michigan, North Carolina, Oklahoma, Oregon, 

Texas, and Wisconsin. 
8 

individual programs; multiple advisory committees and substate entities; and changes in 

the roles for state agencies. 

Employer involvement is a key to workforce system reform, and the Grubb et al. study 

identifies a number of roles that employers can fill in workforce development systems: 

• Policy making and governance; 

• Identifying training occupations and industries; 

• Identifying non-vocational skills (“soft skills”) that participants will need; 

• Curriculum development; 

• Setting skill standards for occupations; 

• Serving as contractors/vendors for workforce development; and 

• Hiring workers who complete the programs successfully. 

Finally, Grubb et al. make a useful contribution by developing a classification hierarchy 

for workforce coordination. The lowest level of coordination is information sharing, and 

they describe three levels of sharing information: having sign-off provisions across 

programs; having cross membership on local advisory boards, and sharing information on 

clients. The second level of coordination involves referrals among agencies. The three 

levels in this category are general referrals, subcontracts, and developing feeder systems 

to facilitate the flow of clients among programs. The third level of coordination is joint 

service delivery. The categories here are division of labor (where agencies specialize in 

particular facets of workforce development), co-delivery of services, and consolidated 

service delivery. 

The General Accounting Office (2000) does not analyze a particular program, but instead 

looks at the entire legislative infrastructure of employment and training programs to 

explore the extent to which the programs overlap in targeting and services. The report 

provides a detailed listing of all programs whose focus is on employment and training. 

Omitting financial grant and loan programs, the GAO report finds that there are 40 

employment and training programs administered by seven federal agencies and 

departments. They note that six of the 35 programs for which they had data account for 

about two-thirds of the funding and that 33 of the programs are focused on a single target 

group, such as Native Americans, youth, or veterans. The report correctly notes that 

overlap and duplication have the potential to lead to waste, but this is not necessarily the 

result: 

Programs overlap when multiple agencies administer programs that are 

designed to achieve similar outcomes. Such overlap creates the potential 

for duplication of, or gaps in, service delivery, as well as administrative 

inefficiencies. While program overlap is sometimes necessary to meet 
9 

federal priorities, it can create an environment in which programs do not 

serve participants as efficiently and effectively as possible. To determine 

whether duplication and inefficiencies result from overlap among 

employment and training programs, one would have to obtain and analyze 

a considerable amount of additional information beyond that in our 

review, such as participant eligibility requirements, whether programs are 

providing similar services to similar groups of individuals, information on 

program effectiveness, and the relative costs and benefits of different 

service delivery mechanisms. In addition, because effective coordination 

among programs can reduce the likelihood of duplication and inefficiency, 

information on the degree of coordination among programs with similar 

objectives is also needed. (p. 5) 

This GAO report is precisely on target in suggesting that a major research study is needed 

to determine if the large number of programs with overlapping target groups and services 

are wasteful or beneficial. Simply consolidating programs is no guarantee of increased 

efficiency, and services to some groups may be hindered, but the current system may 

indeed be wasteful in some ways. 

The internal ETA evaluation of WIA implementation is based on reports from regional 

office staff on all states and 126 local programs. Additional information was obtained 

from a series of “road shows,” where ETA staff traveled around the country and met with 

state and local program officials to discuss and provide technical assistance on various 

aspects of WIA. The study found that the program was being implemented fairly well 

across the country, albeit at a slower pace than the legislation anticipated. 

One area of note in the internal report is that many local boards had problems recruiting 

private sector representatives to participate. ETA has long recognized the importance of 

private sector involvement in its training programs, beginning with the Private Sector 

Initiative Program during the CETA program, and ETA has modified the requirements 

for involving the private sector in training programs and funded several studies over the 

years to address this issue. Because this issue remains important and unresolved, 

additional demonstrations, research, and evaluations are likely to be necessary. 

The internal ETA study also notes that in the adult programs some programs do not 

register some potential customers for fear of failing to meet the performance standards 

(particularly the earnings gain measure) and that some local boards appear to be engaged 

in “creaming,” i.e., selecting participants on the basis of their likely outcomes on 

performance measures rather than on need or value added by the program. (We deal with 

performance management issues in a separate section.) 

WIA introduced the concept of sequencing core, intensive, and training services, and it is 

not surprising that some local boards have had problems adapting. In part, this may stem 

from confusion about whether WIA, like Temporary Assistance for Needy Families 

(TANF), is a “work first” program or whether the intent is to channel customers to the 

most appropriate services as quickly as possible. Technical assistance on the program’s 

mission should help clear up any confusion. It is likely, however, that some problems 
10 

may stem from a lack of knowledge about how to vary the mix of intensive services 

among customers and when it is appropriate to move someone to the next level of 

intervention. Demonstrations to test promising practices may be useful here, as well as 

further research and evaluation on sequencing strategies. It would be useful to know, for 

example, if customers who do move on to the next level of service do so because they are 

successful or if they leave the system because they are discouraged. 

WIA makes substantial changes in services to youth, so it is not surprising that local 

boards have been somewhat slow in implementing youth services. The slowness 

experienced may result from startup problems due to changing eligibility requirements 

and required services rather than any inherent lack of knowledge on the part of local 

WIBs. 

D’Amico et al. (2001a) also analyzed the early implementation of WIA, roughly speaking 

the first six months of the program. Their study was based on a 90-item tracking 

instrument that was completed by ETA regional office staff every six weeks and case 

studies of state and local WIB staff in six states that were early WIA implementers. This 

study is ongoing, with additional state and local areas added for the case studies, and 

additional reports are forthcoming. 

The D’Amico et al. study found that much of the JTPA infrastructure was retained. For 

example, one-half of the states initially retained their state planning council, and slightly 

more than one-half set up optional youth councils. Slightly less than one-half the states 

reconfigured their local areas, and there was a modest reduction in local program areas 

from about 620 under JTPA to 598 under WIA. 

WIA discourages local WIBs from operating One-Stop centers and directly providing 

services, and only a minority of local boards have sought waivers to engage in such 

activities: only 3% received a waiver to provide training, 14% were approved to provide 

core and intensive services, and 18% were approved to serve as One-Stop operators. An 

important research question that will need to be addressed is whether the separation of 

control and service delivery affects performance. 

The WIA requirements for developing an approved provider list and consumer report 

card on providers has had mixed results. Almost all states have developed an approved 

provider list, but only 33 had electronic consumer reporting systems at the time of the 

study. Many states expressed concerns about their ability to obtain relevant data from the 

providers. 

The case studies provided findings similar to the internal ETA study. For example, some 

local programs required fixed sequences of services to move from core to intensive to 

training services, and the researchers questioned the wisdom of constraining the flow to 

be the same for all customers. The study found significant variation in reactions to 

ETA’s new performance standards system, with some sites indicating that the process 

worked well, but others were concerned. 
11 

The D’Amico et al. (2001a) study identified several challenges to WIA implementation 

that should be monitored: 

• There is a need to expand and enrich the core and intensive services provided by 

local boards; 

• Local boards have been too cautious about introducing group workshops and 

individual counseling for customers receiving core services; 

• The mix of intensive services has been limited, and a broader array of services is 

required; 

• Employers need to be more involved in the programs; and 

• Relationships among One-Stop partners need to be clarified, particularly around 

issues such as how customers, data, and resources should be shared. 

The study by Buck (2002) is the most limited in scope of the early implementation 

studies discussed here. It is based on the experiences of five cities in states that were 

early implementers: Houston, TX; Boston, MA; Orlando, FL; Charlotte, NC; and 

Philadelphia, PA. Massachusetts was technically not an early implementer, but state 

legislation made the environment similar to that in early implementing states. 

Buck found that although the types of service providers varied significantly across the 

cities studied, the philosophy on including providers was fairly similar: make it easy to 

get on the list, but hard to stay there. For example, Philadelphia approved all 64 

applicants, and Charlotte approved 15 of 16 applicants. Several concerns were raised 

about the provider process. First, several of the cities indicated that it might be difficult 

to obtain reliable data on vendor performance for all enrollees, particularly if the vendor 

provided the data. Second, some of the cities were not certain whether short-term 

educational training was correctly classified as an intensive service or training. At least 

two of the cities considered short-term training to be an intensive service because the 

providers were solely interested in literacy rather than job placement. In one state, there 

was a reluctance of colleges to apply to be on the approved vendor list, as they 

considered it inappropriate to grade their performance based on placement data. 

Universal access raised concerns at the sites studied. Some of the local boards were 

worried about how they could afford to assess the large number of people expected to 

receive core services so that they could determine who would get intensive services and 

training. In addition, some of the sites indicated that the universal access would make it 

difficult for them to train as many disadvantaged customers as they would like. One site, 

for example, expected to reduce its training by about 75%, and the only reason the site 

could maintain its training level was a large contract from the TANF agency. 

There was a great deal of variety in how the One-Stops were managed, so no 

generalizations could be made in the study. In some sites there was concern that the 

employment service staff would ignore the One-Stop system and continue operating 
12 

independently. A management problem common to all the One-Stops was the issue of 

managing staff from different levels of government with very different cultures, pay 

scales, and even dress codes. 

The Balducchi and Pasternak (2001) paper was prepared to familiarize representatives at 

a conference sponsored by the Organization for Economic Cooperation and Development 

(OECD) with the operation of the One-Stop system authorized by WIA. They begin by 

tracing the history of federal-state-local relationships in U.S. employment and training 

programs. They point out that employment security programs (the unemployment 

insurance and employment service) are state programs, while the WIA program is largely 

run through local boards with some state and federal oversight. In the 1990s, an effort 

was made to consolidate service delivery through One-Stops, and over $826 million was 

appropriated to establish One-Stop centers throughout the country. 

In addition to consolidating service delivery through One-Stops, Balducchi and Pasternak 

note that several other important trends were taking place: 

• The programs began following current management philosophy and began 

implementing customer-driven practices dealing with quality, reliability, and 

service; 

• Rapid changes in technology led to a complete overhaul of labor exchange 

functions and the distribution of labor market information, exemplified by the 

development of four key Internet sites for job seekers and employers (America’s 

Job Bank, America’s Talent Bank, America’s Career Information Network, and 

America’s Learning Exchange); 

• Resource rooms were developed where job seekers could make use of the new 

technology and other information to seek employment on their own; 

• In addition to expanding self-service resources, the employment service began 

offering workshops where groups of people could learn job search techniques and 

methods of acquiring labor market information; 

• WIA requires services to be provided through One-Stops and makes several other 

changes relative to JTPA, particularly universal access; the sequential core, 

intensive, and training services; and the use of individual training accounts. 

Balducchi and Pasternak also suggest some areas for further research: 

• It will be important to monitor the performance of the One-Stops during a 

recession to see if they are effective in such an environment; 

• The resource rooms and job search workshops are recent innovations, and 

evaluations should be undertaken to see if they are effective; 
13 

• Given the growth of private employment agencies in recent years and the 

technological advances that enable people to search for work on their own, 

research should be conducted to determine if there is still a rationale for a public 

labor exchange program. 

• For the One-Stops to function efficiently, the partners must be willing to share 

power; research should be undertaken to see if the agencies are cooperating and 

sharing power; and 

• There is interest by some governors to have more authority vested in states rather 

than local areas; research may shed light on what structure is most efficient. 

Self-Directed Services at One-Stop Centers 

The introduction of self-service core services at One-Stop centers under WIA is a major 

departure from the way JTPA programs operated. A major function of the One-Stop 

centers is to provide resources for individuals from all backgrounds to obtain labor 

market information, explore career choices, and seek employment, all with little 

intervention by One-Stop staff unless the job seekers need assistance. D’Amico et al. 

(2001) conducted a process study of exemplary practices in One-Stop centers that were 

early implementers. The sample of eight One-Stops was selected purposively and 

included the following sites: Brevard County, FL; Bloomington, IN; Boston, MA; Anoka 

County, MN; Austin, TX; Morrisville, VT; Renton, WA; and Racine, WI. The study 

analyzed four features of the One-Stop centers: facilities and design, access to selfservices, staffing and staff roles, information and resources, and employer services. 

Exemplary facilities were what common sense would suggest. The centers should be 

located so that potential users can easily get to them by public transportation and by auto. 

Once inside, One-Stop centers should be designed so that customers can easily find their 

way around and locate the services they wish to access. A key aspect of self-services is 

obtaining data from the Internet and making use of labor market information (LMI) and 

diagnostic software, so the researchers concluded that having adequate Internet and other 

computer facilities was important. All the exemplary sites took care to assure that they 

were in compliance with the Americans with Disability Act (ADA) and that customers 

with vision, hearing, mobility, and other disabilities are able to make use of the center. 

Finally, the exemplary centers all made an effort to maintain a friendly appearance and 

avoid the drab, dismal look of many government offices. 

The exemplary sites made an effort to market all One-Stop services, not just the selfservices. Marketing approaches included advertisements on television and radio, Internet 

web pages, brochures, videos, and job fairs. Some sites used unattended kiosks, but they 

were generally not considered successful. The exemplary sites also performed targeted 

outreach, soliciting referrals from their partners and linking with other systems, such as 

school-to-work. Other approaches used to attract customers included presentations at 

special events, targeted mailings, tours, and allowing community groups to use the OneStop space for other purposes. Efforts were also made to attract employers, including 

offering to let them recruit at the One-Stop center, holding “meet the employers” 
14 

sessions, holding workshops for employers, and providing an employer resource room for 

the exclusive use of employers. 

The One-Stop centers in the study used a variety of ways to staff the self-service 

component, and the report does not recommend a single strategy. Some of the centers 

had full-time staff assigned to the self-service center; others rotated staff through the selfservice center, and still others had staff come by on an as-needed basis. Some sites found 

it useful to have staff offices near the self-service center, so that the staff stationed there 

would serve as resources to customers using the self-service center. 

In addition to having staff available when customers encounter problems with the selfservice center, the other key to success is having the right resources available. 

Information and resources that should be available include: 

• Assessment and career planning tools; 

• Listings of education and training opportunities, including the names of vendors 

and relevant information about the training provided and its results; 

• Labor market information (LMI) on the prospects for employment and earnings 

for occupations available in the local area; 

• Job search tools and job listings; and 

• Tools for preparing resumes and cover letters. 

Although the report gives examples of the types of products used, it does not provide 

comparative ratings on alternative products. The report notes that customers are most 

interested in job search, so it is wise to meet the job search needs adequately. 

This report provides a useful starting point for One-Stop centers that wish to assess their 

self-service components. Several extensions of this project would be useful. First, as the 

report notes, there are a variety of commercial tools available for assessment, career 

choice, job search assistance, resume writing, and all the other activities that take place at 

these centers. Just as report cards are now prepared on the performance of training 

vendors, it is worth considering developing report cards for these products. The report 

notes that many of the users tend to be highly educated and employed. Thus, the 

D’Amico et al. study does not do much to inform us of how effective self-service 

programs are for the less educated customers.7

 There is a danger that the most 

disadvantaged customers might become frustrated and abandon the One-Stop if they 

cannot make use of computer-based tools and resources, even though they are the best 

candidates for training. Thus, a useful follow-on study would be to assess how well the 

disadvantaged are served in self-service centers and to identify promising materials for 

less educated customers and approaches to assisting such customers. 

 

7

 Additional information will be gathered for an ongoing study of self-services being conducted by Social 

Policy Research Associates. 
15 

Software Models to Assist Staff and Customers in One-Stops 

Eberts, O’Leary, and DeRango (2001) have developed computer models that can help 

both staff and customers at One-Stop centers make better decisions on employment 

prospects, future earnings, and services to receive. The authors note that One-Stop 

centers are likely to have a large number of customers who will be undertaking job search 

plus customers who will need the right combination of services if they cannot find a job 

on their own. The Frontline Decision Support System (FDSS) illustrates how statistical 

models can be developed and used to improve decisions made by customers and OneStop staff. The models presented in the study reviewed here illustrate applications of the 

models for dislocated workers in Georgia and Washington. Eberts et al. indicate that 

their models are intended to supplement the decisions made by customers and staff, not 

replace them. By using models that are custom developed for a particular labor market, 

decision makers will have a good starting place, and they can add additional information 

on preferences, individual characteristics, and the environment to reach a better decision. 

The models are based on recent information from JTPA/WIA and other programs to 

provide predictions, based on prior experience, on what current customers will 

experience. 

The search module, which is intended to help the customer make decisions related to job 

search, has three components: an industry transition matrix, an earnings algorithm, and a 

related occupations component. Workers generally like to remain in the same occupation 

and industry as they were employed previously, so the industry transition matrix can be 

of assistance by informing workers of the probability that workers in their industry who 

have been laid off have been able to obtain new jobs in the same industry. The earnings 

algorithm estimates earnings functions for workers with various characteristics to predict 

what their earnings will be upon reemployment. Helping dislocated workers make 

realistic assessments of their earnings prospects is important because studies have shown 

that dislocated workers may lose 25% of their old wage rate when they find a new job.8

 

Awareness of wage prospects can help prevent dislocated workers from searching for 

jobs that simply are not available. 

The related occupations component helps customers who recognize that they should seek 

new occupations identify which occupations are potentially the best match. Two 

approaches are used to identify new occupations. First, O*NET, the occupational 

information system developed by ETA, provides information on what occupations are 

similar to a particular occupation based on skills and aptitudes required. The problem 

with this approach is that what experts believe are similar occupations may not translate 

into actual mobility, so the authors also provide a matrix based on occupational transition 

experience. As the authors point out, this approach has limitations as well because the 

occupations people go into may reflect the only occupations that are growing in an area 

rather than any particular closeness in the occupations. 

The second component of the FDSS is the service referral module. It is intended to assist 

One-Stop staff in determining the appropriate mix and sequence of services for customers 

 

8

 See Jacobson, LaLonde, and Sullivan (1993). 
16 

who require more than core services. This system includes equations to estimate the 

customer’s employability and the barriers that limit employability and models to estimate 

outcomes conditional on alternative services and sequences and customer characteristics. 

Because assignment to service strategies is not random, the researchers plan to add a 

correction for potential selection bias. FDSS is currently being piloted in the Athens and 

Cobb-Cherokee Georgia One-Stops, and it is expected to be implemented on a statewide 

basis in 2003. 

The FDSS illustrates some of the uses that statistical modeling can play in providing 

information to customers and staff as well as in making service strategy decisions. 

Because the models make use of past experience, they can only be useful if relationships 

remain the same in the future. Research could address how stable such models are over 

time. The ability of the FDSS to determine optimal service strategy depends on how well 

the models can deal with potential selection bias. As the report points out, these models 

rely on differential impact estimates. It would be very useful for the Employment and 

Training Administration to take the lead in conducting evaluations, especially ones 

involving random assignment, to determine the effectiveness of alternative service 

strategies. 

Vouchers and Individual Training Accounts 

Another key principle underlying WIA is customer choice. In JTPA and other prior 

employment and training programs, participants often had little say about what field they 

were trained in or which vendor they used. By requiring that most training for adults be 

done through the use of individual training accounts (ITAs), there is considerable 

emphasis on customer choice. In this section we review several recent studies that 

analyze the actual or potential use of vouchers and ITAs in WIA programs. 

Barnow (2000) reviews the theory and empirical evidence on vouchers in employment 

and training programs prior to experience with WIA. He notes that there are several 

important arguments favoring the use of vouchers. The most basic rationale for vouchers 

is that by permitting consumer choice, the well being of the customers should be 

increased. In addition, vouchers foster competition among training providers as they 

compete for customers, and this should lead to improved performance. Finally, vouchers 

fit well with the movement to “reinvent” government popularized by Osborne and 

Gaebler (1992). Barnow notes that there are about as many theoretical arguments against 

vouchers as there are in support of the concept. First, if choice is such a good idea, critics 

argue that rather than provide vouchers that are restricted to purchasing a particular type 

of good or service, providing cash should enhance well being even more. Also, the 

voucher holders may not have the same objectives as the government and may, therefore, 

use the vouchers inappropriately, from a social perspective. Finally, imperfect 

information about the labor market, vendor quality, or the individual’s capabilities may 

lead the voucher holder to make an inappropriate choice on how to use the voucher. 

Because the theories underlying vouchers is inconclusive, one must rely on the empirical 

literature. Barnow reviews evaluations of programs using vouchers for the poor and 
17 

dislocated workers, and he finds the evidence to be mixed, with some studies showing 

modest benefits from the presence of vouchers and others showing no impact or even 

economic harm to participants. He concludes by suggesting that ITAs be restricted so 

that individuals can only use ITAs for programs where the individual’s qualifications and 

aptitudes are strong. In addition, he notes that all the interventions that had positive 

estimated impacts made extensive use of assessment and counseling. 

Trutko and Barnow (1999) review the experience of JTPA training programs that used 

vouchers or voucher-like instruments. They conducted site visits and telephone surveys 

with eight service delivery areas that made use of vouchers to see what lessons could be 

learned for the ITAs that would soon be required under WIA. One site used a pure 

voucher, where participants received a voucher with a dollar limit ($840-$4,500) based 

on which programs the person was eligible for. Although this local program believed 

that its voucher program worked very well, the other eight sites believed that the 

“constrained choice” individual referral approach was preferable. Under the constrained 

choice approach, the vouchers were subject to a number of conditions: 

• Assessment and counseling were used to determine what fields were appropriate, 

given the participants’ qualifications and aptitudes; 

• Training was generally limited to high-demand occupations; 

• Vendors were screened for quality, placement rates, and cost;9

• The vouchers were generally limited to one year or less, but they could often be 

extended; and 

• The decision on what program to attend was generally made jointly between the 

counselor and the participant. 

The WIA regulations permit the constrained choice arrangement for ITAs, and the 

legislation either requires or is consistent with the other recommendations of most of the 

sites using vouchers under JTPA, e.g., screening vendors for quality, restricting training 

to high-demand occupations, and requiring assessment prior to training. 

Although not an impact study, Trutko and Barnow (1999) inquired about how the use of 

vouchers affected outcomes. Perhaps surprisingly, most of the sites interviewed believed 

that vouchers had little or no impact on outcomes or costs, although they had no data to 

back up their beliefs. Nonetheless, the sites were enthusiastic about the use of vouchers 

for several reasons: 

• Vouchers expanded the range of training programs and vendors from which 

participants could select; 

 

9

 Screening for quality sometimes involved having experts review curricula and facilities. 
18 

• Vouchers increased participant involvement in making career choice decisions 

and gave participants a sense of empowerment; and 

• Because of these changes, vouchers resulted in increased customer satisfaction. 

Several studies provide preliminary information about the implementation of ITAs under 

WIA, including a demonstration project involving early implementation of ITAs in 13 

sites and an ongoing experiment to test alternative voucher strategies.10 Patel and Savner 

(2001) describe the early policies and experiences of a sample of 76 local workforce 

boards. Their sample includes the 50 largest urban areas plus all boards in New Jersey 

and Michigan, so their results may not be representative of the nation. The authors found 

that, consistent with the statute, training in the areas studied will only be available after 

caseworkers have assessed potential participants. They found that many of the local 

boards planned to limit training to low-income individuals, as suggested by the statute, 

but at least one board included a “most likely to benefit criterion.” About one-third of the 

boards had established limits on ITAs in terms of duration or cost. 

Because there is a range of options for implementing vouchers and we have little 

evidence on how the structure of the vouchers affects their impacts, ETA has sponsored 

an experiment to test the efficacy of various approaches, using random assignment to the 

three basic strategies identified. The experiment is to take place in six local workforce 

investment areas. Perez-Johnson and Decker (2001) describe the three voucher strategies 

that are being tested: 

• “Structured customer choice” is the most directive of the three approaches. 

Customers will be guided through a series of cost-benefit calculations on 

alternative strategies, and counselors can reject ITA uses that they believe are 

inappropriate; 

• “Guided customer choice” involves substantial counseling and the ITAs may have 

monetary limits, but ultimately the customer decides what training to pursue and 

does not require approval from the counselor; and 

• “Maximum customer choice” where the customer is eligible but not required to 

participate in any counseling activities, and the customer may choose any training 

from vendors on the state’s approved provider list. 

This experiment, which will include a process study and impact analysis, should provide 

evidence on the trade-offs between customer choice versus front-line worker guidance for 

training under the Workforce Investment Act. 

Sectoral Approach to Workforce Development 

In recent years, some workforce development specialists have advocated using a 

“sectoral” approach to workforce development. Unfortunately, the concept is not always 

 

10 See D’Amico et al. (2001b) for a discussion of the ITA demonstration. 
19 

easy to comprehend, and its meaning is likely to vary depending upon the user. In this 

section we review two research papers on the use of a sectoral approach to workforce 

development. 

Dresser and Rogers (1998) argue that recent changes in the economy make labor markets 

function less efficiently than they had in the past. In particular, firms have reduced the 

number of job descriptions, and jobs now have overlapping sets of skills. Left to its own 

devices, they say that the labor market increasingly leads to dead-end jobs, few 

opportunities to increase skills, and access to jobs and promotions based on informal 

networks. They then argue that a sectoral approach, using intermediaries that focus on 

one or a few industrial sectors in the economy, will “provide solutions to industry 

problems and use those solutions to improve training for incumbent workers and increase 

access to the industry of disadvantaged workers.” The authors claim that a sectoral 

approach leads to a “win-win” situation for workers and firms by generating efficiencies 

in three areas: economies of scale (because of the large number of firms involved), 

economies of scope (because the consortium of firms will span various segments of the 

economy), and network externalities (because firms will band together to solve common 

problems). 

To support their case, Dresser and Rogers describe three recent sectoral efforts in 

Wisconsin. Two of the efforts were mostly at the planning stage when the paper was 

written, so they will not be discussed here.11 The operational project that is described is 

the Wisconsin Regional Training Partnership (WRTP). The WRTP consists of over 40 

firms employing about 60,000 workers in the metalworking industry. The organization 

was founded in 1992, and during its first five years of operation, WRTP focused on three 

areas: incumbent worker training, modernization, and future workforce development. 

The paper does not provide specific results, but it concludes: 

The apparent results of the WRTP have included significant improvement 

in the skill levels of the workforce, stabilization of employment in this 

hard-hit and highly competitive industry, wage improvements for 

incumbent workers, clear markers for entry level and incumbent workers 

regarding job expectations and career advancement, and considerable 

improvement in the general quality of labor-management relations in 

affected firms. 

Elliott and King (1999) summarize what they learned from 13 case studies of sectoral 

projects across the nation.12 They define successful sectoral programs as “programs that 

understand and intervene in local labor markets in order to benefit low-income workers.” 

Although the approaches used vary, Elliott and King (1999) find that successful programs 

share three characteristics. 

 

11 The Milwaukee Jobs Initiative is a seven-year project that is intended to connect at least 240 central city 

residents to jobs in manufacturing and other sectors; and the Community Career Ladder Project is intended 

to work with the manufacturing, health, and insurance and finance sectors in Dane County. 

12 The 13 programs studied are Project QUEST in San Antonio, TX; Good Faith Fund in Pine Bluff AR; 

Garment Industry Development Corporation in New York, NY; Cooperative Home Care Associates in 

South Bronx, NY; Primavera Services, Inc. in Tucson, AZ; New Hampshire Community Loan Fund in 
20 

• They target an occupation or cluster of occupations within an industry or sector of 

the economy; 

• They seek to become an influential actor in that sector; and 

• They intervene to benefit low-income workers by connecting individual 

participants to better jobs and by achieving systemic changes in the labor market 

that benefit low-income workers more broadly. (p. 4) 

Elliott and King find that sectoral programs use two broad strategies to accomplish their 

objectives: they either improve access of their target group to the jobs, or they restructure 

the jobs so that the jobs pay better, have improved fringe benefits, and include a job 

ladder to better jobs. Although they are enthusiastic about the sectoral approach, Elliott 

and King identify several risks to the strategy. If the program does not perform well, it 

will quickly get a bad reputation and will have great difficulty obtaining cooperation 

from employers. In addition, if the sector declines in importance, the work of the sectoral 

program may become irrelevant to the community. 

The sectoral programs must alter employer behavior if they are to succeed. Elliott and 

King identify five broad types of behavior and interactions with employers: 

• Restructuring jobs so that they offer higher pay for more skilled work; 

• Making participating employers into “yardstick” companies that are recognized 

for their leadership; 

• Rewarding exemplary employers through recognition; 

• Removing artificial barriers to employment, such as irrelevant qualifications; and 

• Training the next generation of business owners by improving curricula in 

community colleges and other locations where future owners gain their skills. 

Elliott and King identify four specific types of intervention strategies: 

• Vocational training; 

• Business development activities; 

 

Concord, NH; Working Partnerships USA in San Jose, CA; ARCH Training Center, Inc. in Washington, 

DC; Focus: HOPE in Detroit, MI; Westside Industrial Retention and Expansion Network in Cleveland, 

OH; Training, Inc—Essex County College in Newark, NJ; Direct Action for Rights and Equality in 

Providence, RI; and Philadelphia Area Accelerated Manufacturing Education, Inc. in Philadelphia, PA. 
21 

• Organizing workers and/or community residents; and 

• Research and policy analysis. 

In addition, Elliott and King point out that it is important to use several of the strategies 

and to integrate the strategies into a coherent package. 

Both of the papers reviewed here are enthusiastic about the sectoral approach, but both 

are stronger in providing theoretical reasons on why the approach should work and 

anecdotes on its success than in providing rigorous (or even non-rigorous) impact 

evaluations. Dresser and Rogers go even farther, and they are highly critical of programs 

that focus on individual employers. Both papers point to a number of appealing features 

of sectoral programs, so it would be useful for the Department of Labor to sponsor 

process and impact studies to better understand how the programs work and how 

effective they are. It would also be useful to consider evaluations contrasting a sectoral 

approach with the individualized approach criticized by Dresser and Rogers but found 

useful by several other authors and discussed immediately below. 

Customized Training 

Sectoral programs are only one approach to involving employers in training programs. In 

this section we review studies on other approaches to involving employers, particularly 

customized training. 

One group of researchers prepared three reports on employer-based training for the 

Employment and Training Administration in 1996. Pindus and Isbell (1996) prepared a 

literature review on the topic, and Isbell et al. prepared a report on 17 case studies 

(1996a) and a report on best practices (1996b). We focus here primarily on the best 

practices report; the literature review is now somewhat dated, and the lessons from the 

case studies are provided in the best practices report. 

Some of the promising practices suggested in the literature remain promising but 

potentially underused today. For example, the study cited uniform occupational skills 

standards as a promising approach for standardizing training content, but the President’s 

proposed FY 2003 budget calls for abolishing the National Skills Standards Board. Even 

if the current approach to developing skills standards is not working well, that does not 

mean that the concept lacks merit. Depending upon the fate of the Board, research on 

how to foster the development of skills standards in the absence of a national board may 

be very useful. 

The literature review also suggests improving the formal linkages between employers and 

schools through formal employment and training mechanisms referred to as “youth 

apprenticeship programs.” A retrospective analysis on how the youth apprenticeship 

programs have fared would be useful to assess whether this approach was simply a fad or 

a useful addition to the menu of approaches available. The review notes that employer 

associations are natural partners for training programs, but their use has been limited. 
22 

Research on how to increase their involvement and possibly pilot projects to test out 

promising ideas along these lines would be useful. Finally, the literature review suggests 

that joint labor-management activities should be used more. Although the proportion of 

the work force that is unionized continues to drop, registered apprenticeship programs 

represent a way to create skilled workers at virtually no cost to taxpayers. Thus, research 

and pilot projects that can promote and expand apprenticeship should be useful. 

Finally, the literature review indicated that employer involvement in government training 

programs remains uncommon. This makes the case studies and best practices discussed 

below all the more important. 

The best practices were drawn from nine training programs affiliated with JTPA 

programs and eight privately funded programs; the focus here is on the JTPA funded 

programs. All nine JTPA affiliated programs used the training program to recruit, screen, 

assess, and train new workers. All the participating firms indicated that they were having 

trouble finding a sufficient number of skilled entry-level workers, and they hoped that the 

program would help them solve this problem. 

Although increased profits was one goal for employers, the participating companies’ 

immediate objectives also included: 

• Increased productivity; 

• Increased quality of output; 

• Improved timeliness of product delivery; 

• Improved customer service; 

• Facilitating hiring and training workers in the community; and 

• Increased worker morale and reduced turnover. 

The programs all involved customized training, but there were major differences across 

the nine case studies. One program required 26 weeks to train low-skill workers for 

highly technical electronic manufacturing positions, while another trained welfare 

recipients in five weeks to be customer service representatives in an auto parts and 

service store. The programs shared a number of common features: 

• The companies were heavily involved in curriculum development to assure that 

the training would allow successful participants to be immediately productive; 

• Recruiting, screening, and assessing potential trainees was a shared responsibility 

between the firm and the vendor, where at a minimum the firm set the standards 

and in other cases the firm was actively involved in selecting each participant; 
23 

• The training was tailored to the jobs to be filled and customized for the 

employer;13

• The programs used a combination of classroom training and laboratory, hands-on 

training; 

• Participants were assessed for adequate reading and math skills; 

• When needed, basic skills training was provided for participants who were 

slightly below the required reading or math level for the job; 

• Most of the programs included preemployment and workplace skills; and 

• Most of the companies provided a written or oral commitment to hire successful 

program completers. 

The case studies did not include an impact evaluation, but the outcomes exceeded those 

of most JTPA programs at the time. In all the case studies, over 80% of the participants 

completed the training, and in most cases over 90% finished. The participating 

employers hired almost all those who completed the training, and the wages were above 

the average for JTPA training at that time. The employers believed that retention was 

higher than usual, although no records were available to confirm this. Finally, the 

programs were not particularly costly—they ranged from $900 to $2,700 with an average 

cost of $2,000. The programs were characterized as “win-win-win” situations, with 

participants gaining employment and earnings, firms obtaining much needed skilled 

labor, and the JTPA programs looking good on their performance standards. 

With all these benefits, one might wonder why customized training is not used more 

often. According to the report, the major problem with this approach is that there are 

high fixed costs for both the local training program and employers, so the approach may 

not be feasible for small and mid-size employers. In addition, firms are sometimes wary 

of government programs, and they are hesitant make commitments to hire program 

graduates. 

Duscha and Graves (1999) analyze state-financed customized training programs. These 

programs started in the late 1950s, and initially they were intended to help states cope 

with occupational labor shortages by training new workers for the jobs. As of 1999, 

Duscha and Graves find that 45 states operated such programs, and the majority of the 

programs target incumbent workers—they estimate that 60% of the funding is now for 

incumbent workers. In 33 states, the firms are permitted to select any vendor they choose 

or to do the training themselves, and in the other 12 states the firms are restricted to 

community colleges and similar institutions. 

 

13 For example, participants training to be customer service workers would use the same cash registers and 

software as they would use on the job. 
24 

The state programs differ from federal programs such as JTPA and WIA because they are 

focused exclusively on the employer; they rarely have targeting requirements for 

workers. The programs are intended to help firms deal with technical change, to attract 

new businesses to the state or prevent current businesses from leaving, and to foster 

economic development. Duscha and Graves have identified only one impact evaluation 

of these programs, and that evaluation was very positive. 

Although the programs are generally popular with the states, the authors identified 

several problems. The foremost problem is that the programs are sometimes 

characterized as “corporate welfare,” where employers are paid for activities that they 

should be paying for. This criticism is particularly relevant in programs aimed at 

attracting and retaining businesses, and similar criticisms are made of tax exemptions and 

zoning variances. Duscha and Graves note that none of the state programs are of a large 

scale, and they suggest that states might have problems expanding the programs, 

particularly to small and medium size firms. As a remedy to this problem, they suggest 

that states encourage firms to form consortia to improve the economies of scale. 

Customized training is permissible under WIA, and given its potential, the Employment 

and Training Administration should consider additional research, demonstration, and 

evaluation projects. First, an outcome or, if possible, an impact evaluation should be 

conducted on customized training programs to determine if they are as good as they 

appear to some observers or if the criticisms of Dresser and Rogers are valid. It would 

also be useful to mount some demonstration programs to encourage the use of such 

programs. Finally, research should be conducted to learn about ways of removing the 

barriers to implementing customized training and to avoid corporate welfare. 

The Role of Faith-Based Providers in Providing Services 

In recent years there has been increased interest in the role of faith-based organizations 

(FBOs) in providing social services as vendors to government programs. To learn more 

about the role that FBOs are currently playing in the delivery of employment and training 

services, ETA funded Kramer et al. (2002) to conduct an exploratory study. Kramer et 

al. selected a convenience sample of five major cities for their analysis: Baltimore, MD; 

Ft. Worth, TX; Milwaukee, WI; Pittsburgh, PA; and San Diego, CA. In the study, a 

faith-based organization is defined as one that “holds religious or worship services or is 

affiliated with a religious denomination or house of worship.” 

In each city, the WIA administrator or a designated official was interviewed about their 

contracts with FBOs. The nine largest WIA contractors were interviewed in each city to 

learn about subcontracts with FBOs. Interviews were conducted with the nine largest 

congregations in each city and nine smaller congregations. Finally, nine faith-based 

nonprofit organizations were identified and interviewed in each city to learn about their 

employment and training activities. Although the approach used in the study provided 

information from a variety of FBOs, it was not done in a manner so that reliable estimates 

for the cities studied or the nation could be generated. 
25 

Major findings from the study include the following: 

• All five of the local boards contract with FBOs, but the range of activity is wide—

from $36,000 in Milwaukee to $3.6 million in San Diego; 

• The proportion of funds going to FBOs was small in all the cities, ranging from 

about 1% in three cities to 6% in Pittsburgh; 

• Most of the congregations do not provide formal employment and training 

services, although they may help individual members on an informal basis; 

• In each city, one to three congregations do provide employment and training 

services, but funding is minimal and public funds are not used; 

• A few of the congregations do provide substantial employment and training 

services and, in that respect, resemble the nonprofit FBOs; 

• Nonprofit FBOs offer a wide range of employment and training services, and 

nearly half have federal funding, with the Department of Housing and Urban 

Development (HUD) being the most common provider of funds; and 

• All the local workforce boards were satisfied working with the FBOs and might 

expand their contracting with them. 

Performance Management 

The Employment and Training Administration has been one of the pioneers in applying 

performance management to its programs, rating local JTPA programs on their 

performance well before the Government Performance and Results Act (GPRA) of 1993 

required all federal agencies to develop performance management systems. The JTPA 

statute required ETA to establish performance measures and standards for acceptable 

performance, and WIA includes performance management requirements as well. In this 

section we review four recent papers on the use of performance management systems for 

workforce development programs. The papers by Barnow (2000) and Heckman, 

Heinrich, and Smith (2002) focus on how the JTPA performance management system 

performed in terms of ranking areas by their impact. The paper by Barnow and Smith 

(2002) provides a review of the literature on incentive effects of the JTPA performance 

standards, and King et al. (2002) review the efforts by states to develop systemic 

performance measures. 

The primary goal of the performance management system under JTPA was to encourage 

local programs to maximize the program impact on earnings. If each local program had 

its own control group of non-participants, then impact could be directly measured at each 

site. In the absence of such control groups, ETA had to rely on a system where postprogram levels of employment and wages were used for the performance measures, and 

satisfactory performance for each local program was determined by using a regression 

model that adjusted expected levels of employment and wages for characteristics of the 
26 

population served and local economic conditions. Barnow (2000) points out that some 

strong assumptions are required for such a system to rank local programs the same as 

they would be ranked based on the actual program impacts. The National JTPA Study, 

carried out between 1987 and 1989 included a randomly assigned control group in each 

of the 16 local programs included in the experiment, and this provided an opportunity to 

determine how well measured performance corresponded to actual impacts in the 16 sites 

in the study. Barnow found a weak relationship between program impacts and measured 

performance. Based on these findings, he recommended the following: 

• ETA should consider its performance goals carefully, and particularly consider 

the tradeoff between program impact and equity of services as performance 

criteria; 

• ETA should acknowledge the weaknesses of the current performance 

management system and make the rewards and sanctions relatively weak; 

• Periodic efforts should be made to anchor the performance management system 

by conducting experimental or (if necessary) quasi-experimental evaluations to 

calibrate measured performance with impacts; and 

• An effort should be made to develop better measures of local economic 

conditions, as the models used by ETA appeared to be weak in this area. 

Using the same database, Heckman, Heinrich, and Smith (2002) look at the relationship 

between measured performance and long-term impacts (30 months after random 

assignment), and they find an even weaker relationship than Barnow found for short-term 

impacts (18 months after random assignment). Heckman, Heinrich, and Smith also 

consider the impacts of “cream skimming” or “creaming” on program efficiency. 

Creaming is the term used to describe the behavior whereby local programs seek to enroll 

individuals who will lead to high measured performance. Because the JTPA measures 

were based on post-program levels of employment and earnings, local programs might 

select individuals likely to do well in the labor market even without the training program. 

The authors find no reliable evidence that impacts vary by the characteristics of 

participants; so cream skimming is not likely to have a significant effect (positive or 

negative) on the overall impact of the program. 

Barnow and Smith (2002) review the literature on performance management under JTPA 

to see what is known about the overall incentives of the program. Their conclusions 

include the following: 

• The literature is clear that local program managers respond to the incentives they 

face. In the employment and training context, this takes the form of changes in 

who gets served, what services they receive, the types of contracts entered into 

with vendors, and strategic manipulation of performance measurement. 

• It appears that all types of local agencies—governmental, private nonprofit and 

private for-profit, respond to the performance incentives presented to them. They 
27 

may do different things with the rewards they reap from doing so, but they all 

appear to act on the incentives. 

• Agencies appear to be able to identify characteristics associated with positive 

labor market outcomes and to select persons to serve based on those 

characteristics if they choose to do so. 

• The literature clearly shows that in the employment and training context, outcome 

levels have at best a very weak relationship with program impacts. This implies 

that performance incentives based on outcome levels do not encourage agencies 

to serve those who benefit most from the program (nor do they discourage such 

behavior). This also implies that cream skimming has little or no efficiency 

benefits. 

• Performance incentives in employment and training programs often get passed on 

to providers in the form of performance-based contracts. 

• Performance-based contracts can provide effective incentives to induce vendors to 

focus on the desires of the funding organization. This is not to say that all 

performance-based contracting arrangements will align the interests of funding 

providers and vendors, but they can be useful. 

King et al. (2002) critically examined the performance management system in Texas and 

several other states with the goal of recommending changes to the system, with emphasis 

on the development of “system” measures that gauge the performance of the entire 

workforce development system, not just the programs funded under WIA. Currently, 

local Texas boards are responsible for 31 performance measures, and this will rise to 35 

when the new employment service standards become effective.14 In addition to concern 

about the number of standards, the authors raise the following issues: 

• Some participants served under WIA, those who only make use of self-service 

activities, are not counted in the performance standards system; 

• The lack of adjustments under WIA for participant characteristics and local 

economics conditions, as was done under JTPA, may lead local programs to 

cream in order to improve their measured performance; 

• Termination status is “somewhat arbitrarily and artificially defined” under WIA 

while the choice among activities is constrained under the “work first” philosophy 

that some see in the statute;15

 

14 Texas workforce boards face more performance standards than is typically the case because the boards 

are responsible for many programs that are run by other agencies in most states, e.g., food stamps 

employment and training (FSE&T). 

15 The Employment and Training Administration has indicated that the sequencing of services does not 

imply a work first philosophy for the program. 
28 

• The performance measures cover different time periods, with some on an October 

1 begin date and others on a July 1 cycle; 

• The time lag for obtaining unemployment insurance wage records combined with 

the six-month follow-up period required in WIA mean that the performance 

measurement is sometimes long delayed from the period being rated; 

• The short-term measures being used may not reflect long-term impacts (as was 

discussed above); 

• There are too few measures dealing with employers; 

• The current mix of measures may not be well matched to Texas’s system needs; 

and 

• There is not enough emphasis on developing system goals for the local boards. 

Although King et al. believe that Texas is a leading edge state in performance 

management; they review the approaches used by Florida, Pennsylvania, Maryland, 

Oregon, and Washington for guidance, as well as an approach developed by the National 

Governors’ Association. The specific approaches vary greatly across the states reviewed, 

but the common thread is a limited number of system-wide goals and crosscutting 

measures that are not attached to a single program. 

King et al. propose four specific system goals (or “ends,” as they call them) plus an 

overarching goal of maximizing the return on investment for the programs. The four 

ends they recommend the state strive to achieve are: 

• A better educated and skilled workforce; 

• More competitive employers; 

• More and better jobs; and 

• Higher per capita earnings. 

The authors then attempt to map the existing performance measures to the proposed ends. 

They suggest that performance measures that do not map to any of the desired ends are 

candidates for dropping, and they suggest that measures be developed for any ends 

without good performance measures. 

The performance management literature suggests many research topics that can be 

profitably explored. The literature makes it clear that incentives matter, so having the 

“wrong” performance measures can be as bad or worse than not having the “right” 

measures. The recent WIA experience dropping adjustments for participant 

characteristics and local economic conditions has led to great concern among state and 

local officials, and we do not need formal research studies to show that the system under 
29 

WIA is a big step backward.16 Some of the interesting research topics that arise from the 

literature include: 

• How can we reasonably measure return on investment when the payoff may 

extend for many years for some interventions but decay quickly for others? 

• What is the optimal follow-up period for performance management purposes, 

taking account of the tradeoff between longer periods capturing more of the postprogram experience, but shorter periods are better for giving quick reinforcement 

of performance? 

• How can we develop reliable outcome measures for low-cost and self-service 

activities? 

• When system measures are developed, who should be held responsible for them? 

B. WAGNER-PEYSER ACT LABOR EXCHANGE SERVICES

ince 1933, the employment service (ES), or job service as it is sometimes called, has 

provided labor exchange services for employers and job seekers. Although the ES 

also provides labor market information and performs some assessments of aptitudes and 

capabilities, its major function is as a labor exchange—taking job listings from employers 

and referring qualified applicants to the employers. Because the ES is a low-cost 

intervention and because the Department of Labor’s attorneys have determined that the 

authorizing legislation, the Wagner-Peyser Act, prohibits denial of services, evaluating 

the ES has proven very difficult. Jacobson and Petta (2000) conducted the only recent 

evaluation of the ES for the states of Washington and Oregon.17

Through the use of a mail survey in Washington and administrative data in Washington 

and Oregon, Jacobson and Petta make some assumptions that permit them to estimate the 

impact of the ES on weeks of unemployment. Specifically, in analyzing the mail survey 

data, they assume that the primary reason that individuals were not interviewed is that the 

referral was made after the job was filled. For the administrative data analyses, the 

authors compare workers who were placed with those who were referred but not placed. 

Jacobson and Petta stress the methodological implications of their work as well as the 

empirical findings. Their major impact finding, based on Washington survey data, is that 

 

16 One of the authors of this report chairs a state workforce investment board performance committee, and 

he has met with officials from about 20 states. The greatest dissatisfaction with WIA in every instance has 

been with the way that the performance management system has been implemented. 

17 Currently, USDOL has contracted with Westat, Inc., to study the impact of the Wagner-Peyser Act labor 

exchange services in state one-stop delivery systems. The study, Evaluation of Labor Exchange Services in 

a One-Stop Delivery System Environment, will examine the delivery of public labor exchange services in 6 

states (WA, OR, NC, MA, MI and CO) and provide findings and estimates on the value of labor exchange 

services under the Wagner-Peyser Act. Evaluation results should be available in 2004. 

S 
30 

for individuals with a “strong” work record (three or four quarters of employment the 

previous calendar year), receiving a placement reduced unemployment by 7.2 weeks, and 

for those with a “spotty” work record, a placement reduced unemployment by 3.4 weeks. 

Jacobson and Petta performed a cost-benefit analysis, and they estimated the cost per 

placement to be $542 and the benefit for a placement to be $978, leading to a benefit/cost 

ratio of 1.8. In addition, they included some simulations that indicated that most (80%) 

of the impact was due to vacancy time reduction rather than displacement of other 

workers. They also concluded that the impact of the ES is lower in Oregon, possibly 

because Oregon has a more stringent unemployment insurance work test than 

Washington. 

The authors are very enthusiastic about the approach used in the study, and they believe 

that surveys of ES users can be used to obtain highly accurate estimates of the impact of 

the ES. Furthermore, they believe that the surveys can be used to calibrate evaluations 

based on administrative data. 

An unusual (and admirable) feature of the Jacobson and Petta evaluation is that they 

included detailed comments by four reviewers18 in the report. The reviewers found the 

approach used in the study to be interesting and innovative, but several of the reviewers 

were concerned about the low response to the mail survey (about 25%) and the strong 

assumptions required for unbiased estimates (e.g., that the only reason individuals 

referred were not interviewed is that they received the referral after the job had been 

filled). Thus, several of the reviewers suggested that the estimates not be given too much 

credence but that the general approach could be refined in future studies. 

With core services being a major component of WIA and self-administered services 

becoming more common for both WIA and the ES, it is important to continue efforts to 

evaluate labor exchange services. In addition to dealing with the difficulties of 

evaluating a low-cost treatment, where even a fairly small impact could yield benefits 

that exceed the costs, it is important to capture all the benefits of the program. Thus, 

future evaluations should try to estimate benefits to employers as well as workers and to 

estimate the displacement that occurs. The ES and other labor exchange services may 

provide benefits even if vacancies are not shortened if the program results in a better 

match of workers to jobs. 

C. UNEMPLOYMENT INSURANCE 

he Unemployment Insurance (UI) program has long been an institutional fixture in 

the American labor market. Authorized by the Social Security Act of 1935, UI 

provides for the involuntarily unemployed to receive cash benefits to alleviate hardship 

during unemployment. Federal and state governments jointly administer the UI program. 

One long-time UI trend is that many eligible unemployed workers do not receive 

benefits. Another is that many recipients exhaust their benefits before finding work. The 

question of whether increased UI benefits provide a disincentive to finding work and so 

 

18 Including one of the authors of this report. 

T 
31 

prolong insured unemployment has been much researched over the past 30 years. The 

evidence shows that higher wage replacement rates do extend insured unemployment 

spans to a certain degree.19

Bassi and McMurrer (1997) note that the percentage of workers covered by UI has 

steadily increased at the same time that the percentage of covered unemployed workers 

who receive benefits has steadily decreased. The decline in recipiency means that the 

system has not become more responsive to the needs of workers despite the increase in 

coverage. An individual’s receipt of UI is a function of coverage provisions, the 

individual’s decision to apply for benefits, and state eligibility standards. Employers pay 

UI taxes for all their employees or for none of them. Since the vast majority of 

employers pay UI taxes, the vast majority of employees in the labor force (over 90%) are 

covered under UI. An individual must meet both monetary and non-monetary 

requirements of the state to be deemed eligible for UI. The monetary requirements (total 

wages within a certain time frame) are in place to make sure that the individual has had 

sufficient attachment to the labor force. The non-monetary standards require that an 

individual is involuntarily unemployed (or left work for a good reason), is able and 

available to work, and is actively looking for work. The monetary requirement to check 

attachment to the labor force means that low-wage workers have to work more hours than 

high-wage workers to demonstrate attachment and be UI eligible. Bassi and McMurrer 

suggest that either this monetary requirement be reduced in amount, or that the 

requirement be changed to total hours of work, rather than total earnings. If the 

requirement were changed to total hours, as many as 15% of the total unemployed might 

move from UI ineligible to UI eligible.20

Vroman (2001) considers the wide state-to-state variation in recipiency rates and 

especially focuses on states with low recipiency rate. He examined state-level data since 

1967 and also conducted site visits to gather information about administrative features of 

states. States have wide variation in UI application rates, rate of first payments to 

applicants, and duration in UI benefits status. Vroman finds that varying rates of inflow 

into benefit status generally have larger effects on overall benefit recipiency than 

variation in duration of benefit receipt. The states that have low recipiency rates can be 

sorted into three groups based on the dynamics underlying their recipiency. One group of 

five states has both low inflow rates and short benefit duration. A second group of nine 

states (eight of which are contiguous in the south-southwest) has inflow rates low enough 

to overcome higher than average benefit duration. A third group of five states has 

average inflow rates but short benefit duration. Vroman also finds that even taking into 

account obvious factors in the macro-labor markets of the states, UI program variables 

have important effects on all aspects of UI recipiency. The site visits generated several 

interesting observations: 

• High recipiency states have made much more accommodation to non-English 

speakers in filing for UI benefits; 

 

19 Blaustein, O’Leary, Wandner (1997), 26-27. 

20 Bassi and McMurrer (1997), 81. 
32 

• Requirements for monetary eligibility are generally easier to satisfy in high 

recipiency states; 

• Rates of adjudication on separation issues, both quits and misconduct, are 

generally lower in states with higher recipiency; 

• Quits are more likely to be compensated in high recipiency states; 

• Disqualifying and deductible income denials are less frequent in high recipiency 

states; 

• Eligibility reviews generally occur less frequently in high recipiency states while 

penalties for failure to meet reporting requirements are more likely to be enforced 

in low recipiency states; and 

• Rates of employer appeals of nonmonetary determinations are much lower in high 

recipiency states, less than half the rate of appeals in low recipiency states. 

Vroman also suggests a number of topics for future research. These include:21

• What are the reasons for increased unemployment duration in the 1980s and 

1990s as compared to earlier decades? 

• How many people combine work with the receipt of UI benefits in the same 

week? 

• What share of weeks compensated is paid to persons who quit their last jobs? All 

that is known at present is the number of allowances of claims where there was a 

voluntary quit separation issue. Subsequent first payments and weeks 

compensated are not tracked. 

• What is the net effect of interstate commuter claimants on recipiency by state 

when all commuters are classified according to their state of residence? 

• Why do voluntary quit and misconduct determination rates vary so widely across 

states? 

• How much is recipiency increased by having language accommodation to nonEnglish speakers? 

• What role do third party representatives play in influencing UI benefit recipiency? 

• Do high rates of employer appeals affect UI benefit recipiency? 

 

21 Vroman (2001), 160-1. 
33 

Another feature of UI receipt is that a sizable number of recipients exhaust their benefits 

without finding employment. In order to better assist these recipients and to possibly 

save UI trust fund monies, Congress amended the Social Security Act in 1993 to require 

states to establish the Worker Profiling and Reemployment Services (WPRS) system. 

Congress mandated that states identify UI claimants who are likely to exhaust their 

benefits, refer them to reemployment services in a timely manner, and collect information 

about the services they receive and their subsequent outcomes. Claimants who are 

referred to reemployment services must participate in WPRS to be eligible for UI.22 

 

Dickinson et al. (1997) examined the initial implementation of WPRS and found that all 

states had implemented a two-step screening process to identify workers likely to exhaust 

their benefits. The first step screened out claimants on recall status, claimants attached to 

union hiring halls, and claimants working in seasonal industries. The second step usually 

entailed a statistical method that assigned a likelihood that a claimant would exhaust his 

or her benefits based upon his or her characteristics. About 80% of the states used a 

statistical model while the remaining 20% used a less sophisticated characteristic screen 

that did not assign likelihood. Individuals who are not screened out are referred to 

reemployment services depending upon available funds. The percentage of claimants not 

screened out who were referred to services ranged from a low of 1% to a high of 100% 

across states. Across the nation, only a third of those in the WPRS selection pool get 

referred to services.23 In addition to orientation and assessment, reemployment services 

include counseling, job search assistance such as job search workshops, referrals to jobs 

and job placement, and other similar services. The Employment Service (ES) was the 

major provider of WPRS services. 

 

Delaware, Kentucky, and New Jersey, were early implementers of WPRS and were 

closely examined by the evaluators. Claimants who were referred to services were 

compared with a group of claimants who passed the WPRS screen but were not referred 

for services. In Kentucky and New Jersey, new claimants with the highest probabilities 

of exhausting their benefits were referred to services each week. The predicted 

probabilities for treatment group members and comparison group members overlap 

considerably, however, because of week-to-week and office-to-office variation in the 

capacity constraints (which determined the weekly cutoff probability for service referral). 

Delaware used a characteristic screen and randomly chose claimants from the screened in 

group for referral. In Kentucky and New Jersey, WPRS reduced benefit receipt by 

slightly more than half a week per claimant, translating into a UI savings of about $100 

per claimant. In Delaware, estimated impacts on UI receipt were not significant 

primarily because the relatively small sample size prevented generating precise estimates. 

Increased earnings due to WPRS in New Jersey were equal to $190 in the first quarter 

after the initial claim and $225 in the following quarter. Neither Delaware nor Kentucky 

showed significant effects in employment or earnings from WPRS. 

 

 

22 Referred claimants must participate in services unless they already previously completed them or they 

have a “justifiable cause” for not participating. 

23 Wandner and Messenger, eds. WPRS Policy Workgroup (1999), 3-4. 
34 

Administrators from around the country were surveyed about their perspectives on 

WPRS: 

 

About two-thirds of all administrators felt that WPRS met its goal of 

reducing the length of UI receipt among profiled and referred claimants. 

Most felt that the mandatory nature of services was justified; however, 

about half of administrators expressed some concern that many profiled 

and referred claimants may not need services. Administrators indicated 

that WPRS had other benefits, including improving coordination among 

their agencies overall.24

In 1999, a workgroup on WPRS policy issued some recommendations for improving the 

system. It recommended that the statistical models used by states be regularly revised to 

remain as accurate as possible and that the Department of Labor provide technical 

assistance and disseminate best practices to the states. The workgroup also 

recommended that states profile all who apply for UI and that states be given additional 

resources for reemployment services so that some people who do not receive UI can still 

receive services. Following the spirit of the Workforce Investment Act, the workgroup 

recommended that linkages between ES, workforce training, and UI be strengthened. 

Finally, the workgroup suggested that UI data and WSRP administrative data be linked 

electronically. 

 

The Job Search Assistance Demonstration was implemented during 1995-96 to determine 

the most effective services for claimants who are targeted as likely to exhaust benefits 

based on profiling. This DOL-sponsored demonstration was conducted in the District of 

Columbia (D.C.) and Florida, and these sites had about 8,000 and 12,000 claimants, 

respectively. In each site, claimants were randomly assigned to one of three treatment 

groups or the control group. The three treatments were: 1) Structured Job Search 

Assistance (SJSA), 2) Individualized Job Search Assistance (IJSA), and 3) Individualized 

Job Search Assistance With Training (IJSA+). Claimants assigned to SJSA were 

required to participate in an orientation, testing, a job search workshop, a one-on-one 

assessment interview, and two additional contacts with staff after completion of services. 

Claimants assigned to IJSA were required to participate in an orientation and a one-onone assessment interview. During the interview, the claimant and staff member 

developed a service plan for the claimant and any additional demonstration-specific 

services in this plan became mandatory. IJSA+ was identical to IJSA except for the 

addition of a coordinated effort with local Economic Dislocation and Worker Adjustment 

Act (EDWAA) staff to enroll interested claimants in training. 

 

All assigned claimants were first identified by profiling as being likely to exhaust 

benefits. This fact allowed evaluators to test if the profiling was accurate by comparing 

control group members with non-eligible claimants (those not identified by profiling). 

Decker et al. (2000) find that differences in UI receipt between these two groups were 

about 2 weeks in Florida and 1.5 weeks in D.C. with the profiling regression models 

correctly identifying those likely to remain on UI longer. Very few claimants in either 

 

24 Dickinson et al. (1997), Executive Summary. 
35 

site who were assigned to IJSA or IJSA+ participated in any services other than the 

orientation and assessment. Claimants were reluctant to volunteer for additional services. 

In D.C., the SJSA treatment reduced claimants’ average weeks of UI benefits by 1.13 

weeks, which is statistically significant at the .01 level using a one-tailed test. IJSA and 

IJSA+ treatments reduced weeks of UI benefits by about half this amount (significant at 

the .05 level using a one-tailed test). In Florida, all three treatments reduced weeks of UI 

benefits by about one-half of a week (significant at the .05 level using a one-tailed test). 

In a benefit-cost analysis, the treatment was cost effective in D.C. but not in Florida. The 

difference in results was driven by the significant earnings impact of the services in D.C. 

The earnings impact in Florida was actually negative, though this result was not 

statistically significant. Decker et al. recommend that if states want to expand services 

received by claimants through WPRS, then states should make particular services 

mandatory for all claimants referred to WPRS. This study does not find strong evidence 

that SJSA has a greater impact than just an orientation and an assessment (which is what 

most received under IJSA and IJSA+ treatments). SJSA had a greater impact than IJSA 

and IJSA+ in D.C., but not in Florida. Therefore, it is not clear from this study that 

expanding mandatory services under WPRS would have much effect. WPRS already 

mandates more reemployment services than control group members received in this 

demonstration. 

An earlier attempt to reduce UI receipt by claimants was made in the 1980s. DOL 

funded experiments that made use of reemployment bonuses for those claimants who 

found employment within a certain time span.25 The first test of reemployment bonuses 

took place in Illinois in 1984-85. Before that test was evaluated, DOL funded another 

demonstration in New Jersey in 1986-87 that included reemployment bonuses. In 

Illinois, the use of bonuses was very cost effective, generating a benefit/cost ratio of 

2.32:1. The New Jersey Demonstration was not cost effective, however. These divergent 

results led DOL to sponsor two additional demonstrations modeled after the Illinois 

program in the states of Pennsylvania and Washington in 1988-89. The programs in 

Pennsylvania and Washington failed to replicate the Illinois results. These 

demonstrations did not use profiling to target those claimants most likely to exhaust their 

benefits. O’Leary et al. (1998) use statistical analysis to examine simulated results of 

profiling if it had been used in targeting the reemployment bonus offers. They find that 

the targeting of offers would increase cost effectiveness. They also find that the best 

reemployment bonus program would be structured with a low bonus amount, a long 

qualification period (such as the first 12 weeks of receiving UI benefits), and with 

targeting to the half of profiled claimants most likely to exhaust benefits. A 

reemployment bonus would be a “carrot” to add to the current “stick” of the WPRS 

program. Corson and Haimson (1996), in their six-year follow-up to the New Jersey 

Demonstration, find that adding the carrot of a reemployment bonus would be no more 

effective than mandatory job search assistance alone. 

The Maryland Unemployment Insurance Work Search Demonstration was designed to 

test policies for all UI recipients rather than just for those claimants likely to exhaust 

benefits. At the time of the demonstration, UI recipients had to report two employer 

 

25 A brief summary of these experiments is found in O’Leary et al. (1998). 
36 

contacts per week to sustain their UI eligibility. UI staff did not verify these contacts. 

This basic procedure was similar to the policies of many other states. The demonstration 

tested four alternative policies: 1) increasing the work search requirement to four contacts 

per week, 2) supplementing the normal search requirement with a mandatory four-day 

job search workshop early in one’s unemployment spell, 3) leaving the work search 

requirement at two contacts but informing claimants that contacts would be verified, and 

4) leaving the work search requirement at two contacts but no longer requiring the 

claimant to report the contacts. In 1994, over 27,000 new claimants were randomly 

assigned to one of the four treatment groups or two control groups.26

 

Klepinger et al. (1997) find that Treatment 1 reduced UI payments per claimant on 

average by 0.7 weeks and $116. Both of these amounts are statistically significant. 

Treatment 2 reduced UI payments per claimant by 0.6 weeks and $75. It appears as if 

this impact is due to raising the costs of remaining on UI, so some claimants left UI early 

without finding employment. Treatment 3, making contacts subject to verification, 

reduced UI payments by 0.9 weeks and $113. About 10% of contacts were actually 

verified by UI staff. Treatment 4 increased payments per claimant by 0.4 weeks and $34. 

Only the increase in weeks was statistically significant, and these increases were much 

smaller than found in another demonstration in Washington State. The results of 

Treatments 1 and 3 show that making work search requirements more stringent can 

reduce unemployment spells. These briefer job searches were not found to have come at 

the cost of lower earnings. 

 

Finally, three UI studies are currently being conducted by ETA that may provide valuable 

insights into counter-cyclical approaches to assist jobless workers. The first study will 

examine the effectiveness of the Temporary Extended Unemployment Compensation Act 

(TEUC) of 2002. The TEUC program provides temporary benefits to unemployed 

workers during the current period of recession and slow growth. The TEUC Act also 

provided for distribution of $8 billion of special Reed Act funds to state accounts in the 

unemployment trust fund. ETA will study how states have utilized these Reed Act funds. 

The third study will investigate the effect of economic downturns since World War II on 

the performance of UI and ES services to unemployed workers. 

D. SERVICES FOR DISLOCATED WORKERS 

lthough many of ETA’s programs have focused on poor youth and adults, there has 

also been an interest in serving dislocated workers who have lost their job due to 

plant closings or major layoffs. Currently, major ETA programs serving dislocated 

workers include WIA (which has a separate funding stream for dislocated workers), trade 

adjustment assistance (TAA), and North American Free Trade Agreement Trade 

Adjustment Assistance (NAFTA TAA). In this section we discuss several studies on 

dislocated workers. 

 

26 One control group was informed they were part of a demonstration and one was not. This was to test for 

the presence of the “Hawthorne effect,” where individuals behave differently when they know their 

behavior is under scrutiny. The data showed that no Hawthorne effect was present. 

A 
37 

Fallick (1996) reviews the literature on dislocated workers except for the literature on 

training programs. His major conclusions are: 

• Job displacement is widely spread across regions, occupations, industries, and 

time, but it is concentrated in occupations requiring little education and industries 

and states that are doing worse than average; 

• Displaced workers experience more time without employment than other workers, 

but the differences fade after about four years; 

• Earnings losses, by contrast, appear to persist at a level of 14% or more even four 

years after displacement; and 

• Legal requirements to provide advanced warning of layoffs through the Worker 

Adjustment and Retraining Notification (WARN) Act do not appear to have 

significantly increased advanced warning of layoffs, and the literature on the 

effectiveness of advanced notice is ambiguous. 

Leigh (2000) complements Fallick by reviewing the literature on the impacts of 

employment and training interventions for dislocated workers. Leigh distinguishes 

between interventions intended to speed up reemployment (job search assistance, 

reemployment bonuses, and relocation assistance) and interventions intended to 

compensate for earnings loss (classroom training, on-the-job training, earnings subsidies, 

and self-employment subsidies). 

He finds that for many of the early demonstrations (the Downriver demonstration in 

Michigan, the six demonstrations in Buffalo, and the Workforce Adjustment 

Demonstration in Texas), job search assistance had a modest impact that faded fairly 

quickly (often after six months). Leigh found the New Jersey demonstration results to be 

the most credible because the sample size was larger and the follow-up period was 

longer. The impact for classroom training steadily increased over the follow-up period, 

and it was statistically significant (ranging between $400 and $600 per quarter) in 

quarters 8, 9, and 10 following random assignment. When annual data were used, 

however, the impacts were never statistically significant. The findings were stronger for 

OJT—annual impacts of over $11,000 and statistically significant over all follow-up 

periods up to six years after training. Leigh notes, however, that the large impacts for 

OJT may be due in part to self-selection into the program. The only evaluation of the 

trade adjustment assistance program (TAA) was inconclusive; the study used 

nonexperimental methods, and the findings were sensitive to how the analysis was 

conducted. 

Leigh concludes that job search assistance, classroom training, and OJT all have a place 

for dislocated workers. Job search is inexpensive and is all that is needed for many 

workers. Leigh notes that in some cases the assistance was not provided in a timely 

manner or was not properly tailored to the participant’s needs. He is “cautiously 

optimistic” about classroom training. Although some evaluations did not find positive 
38 

impacts, the largest and most rigorous evaluation did find significant impacts more than 

two years after the intervention. Leigh finds the results for OJT in New Jersey to be 

impressive, but he believes they may be somewhat inflated due to self-selection into the 

program. 

Jacobson, LaLonde, and Sullivan (2001) evaluate the impact of community college 

attendance for dislocated workers in the State of Washington and Allegheny County, 

Pennsylvania. The Washington sample consists of approximately 21,000 workers with 

significant job attachment who lost their jobs between 1990 and 1994. The Pennsylvania 

sample includes approximately 3,200 workers who lost their jobs between 1978 and 

1985. For both samples, the authors have five or more years of earnings data following 

the job loss. This study does not make use of random assignment nor does it focus on a 

particular dislocated worker program (e.g., WIA or TAA). The authors use regression 

analysis to control on observable worker characteristics, and they make use of first 

differences to avoid bias resulting from any time-invariant characteristics. The study 

focuses on the larger Washington sample, but some results are provided for the 

Pennsylvania sample as well. 

In their simplest models, the authors find that the long-term impact of attending 

community college is $261 per quarter or 4.7% of post-displacement earnings for men, 

and $147, also about 4.7%, for females. The authors find that each credit taken leads to a 

$6 increase in quarterly earnings for males and $5 increase for females. For a person 

who takes the equivalent of one full year of schooling, their findings imply a 5.0% 

increase in male earnings and a 6.5% increase in female earnings. The authors also 

conduct subgroup analyses, and they find that minority men gain less from community 

college courses than white men, and that the impacts of community college courses are 

much higher for workers with greater job tenure. The authors find that when they 

analyze technical courses, including science, health, and math courses, the technical 

courses yield high returns, but no return is detected for non-technical courses. Finally, 

Jacobson, LaLonde, and Sullivan find that more than half the gain in earnings is due to an 

increase in hours worked rather than an increase in the wage rate. 

The authors devote little attention to their findings for dislocated workers in 

Pennsylvania, perhaps because the findings are puzzling. In their simplest model, they 

find quarterly earnings gains of $1,047 for males and $812 for females for attending 

community college, but the estimated impact per credit is negative, though not 

statistically significant. 

As Jacobson, LaLonde, and Sullivan point out, there is surprisingly little evidence on the 

effectiveness of programs for dislocated workers. Moreover, most of the evidence is 

based on nonexperimental methods for a limited sample. Thus, ETA should consider 

mounting impact evaluations for WIA dislocated worker training and TAA training. If 

possible, it would be very useful to use random assignment to remove potential bias due 

to nonrandom selection into programs. In addition, it would be informative to see if 

some of the interesting findings of Jacobson, LaLonde, and Sullivan can be replicated in 

other sites and using stronger methods. 
39 

H-1B Training Projects 

Employers use H-1B visas to bring high-skilled workers to the United States for a period 

of up to three years. Part of the fee collected, currently $1,000 in total, is used for U.S. 

Department of Labor (DOL) demonstration programs and projects that provide technical 

skills training for U.S. workers in an attempt to increase the pool of workers in the United 

States with the skills necessary to fill high-tech jobs. Demonstration projects have been 

funded through a series of grant competitions. Barnow, Kaiser, and Trutko (2002) 

provide an assessment of six of the 43 projects funded in the first three rounds of 

competition awarded in 2000: Pima County, Arizona; Hampden County, Massachusetts; 

Anchorage, Alaska; New York City, New York; the State of Vermont; and Clarksville, 

Tennessee. 

Sites were identified based on whether they were close to meeting their enrollment 

targets, occupations targeted for training, employer involvement, training methods, and 

location. This study provided an early look at the H-1B projects, so outcome data were 

not available. Instead, the study focused on identifying interesting practices that might be 

of use to other local areas implementing similar projects. The report documents notable 

practices in the categories of applications, screening, and paperwork requirements; 

recruitment; employer involvement; job commitment by employers and participants; 

matching contributions for training; utilization of workforce investment boards, advisory 

boards, and industry associations; training technology, training management; institutional 

development, data systems, and service area covered. Although the H-1B training 

projects are not authorized under the Workforce Investment Act, the projects reviewed 

here were all operated by local workforce investment boards, and the legislation calls for 

three quarters of the projects to be operated by local WIBs. 

The projects shared similar goals, including upgrading incumbent and 

unemployed/underemployed workers’ skills to enable these workers to fill jobs in highskill occupations for which local employers face shortages; facilitating worker job 

retention, career advancement, and wage growth; offsetting training costs for local 

employers to encourage investment in training; and increasing worker productivity. Most 

of the training was for information technology (IT) and health occupations and included a 

mix of incumbent workers and unemployed workers. 

The study reported several factors that facilitated the design and implementation of the 

projects: 

• Established relationships with employers. All grantees mentioned that they had 

very little time to get projects started, and if they had not had existing, positive 

relationships with employers and employer associations, it would have been very 

difficult to meet the H-1B grant requirements. 

• Prior experience in operating a government-financed training activity. Certain of 

the requirements under the grant, such as providing matching funds, could have 
40 

presented a problem for those not familiar with government grants. Each of these 

grantees had operated programs before or was currently involved with WIA or 

other workforce development activities. One site (Massachusetts) had piloted its 

training program using a prior government grant. 

• Shortage of trained individuals in the target occupation coupled with a pool of 

candidates meeting minimum requirements. Clearly, there needed to be a demand 

for workers in the occupations for which training was planned. In addition, the 

training programs would have failed in H-1B training sites if there were not an 

adequate supply of suitable candidates for training. Sites would also have had 

difficulty reaching full scale and achieving participation goals if they had 

attempted to mount their programs for only unemployed individuals because of 

strong economic conditions and low unemployment rates at the time the 

initiatives started. 

• Available curricula to use as base for establishing training program. All grantees 

used standardized, pre-existing curricula for their training programs. This enabled 

them to avoid spending significant time developing curricula. Several sites also 

worked with post-secondary institutions to develop a degree program (AA or 

BA), but even those programs relied to a large extent on existing courses that 

were modified. 

• Established working relationships with training providers, both public and private. 

Where there were existing relationships with training providers, it was easier for 

the grantees to develop tailored training programs for employers or to develop 

more non-traditional approaches to the delivery of credit courses. 

Although the site selection process was designed to yield sites that were relatively 

successful in enrolling participants, all sites encountered some problems that hindered 

implementation. These problems included: 

• Availability of funds. Some grantees indicated that they did not have access to 

grant funds when the program was officially started. For some grantees, there 

were local grant approval processes that further slowed down start–up efforts. 

• Grant period. A two-year grant period hindered implementation of some degree 

programs, even AA degree programs, because there needed to be time devoted to 

recruitment and assessment prior to enrolling people into training. In addition, 

there was little time at the end of longer training to provide placement assistance 

for those needing it. Some site administrators indicated that it would be more 

useful if the programs were for 3 years to 5 years. 

• Employer fear of government paperwork and audits. Some employers feared that 

if they became involved with government sponsored training programs, that they 

would be engulfed by a mountain of paperwork and/or would open the door to 

government audits. 
41 

• Inflexible training providers. Incumbent worker training requires maximum 

flexibility on the part of training providers, and some just could not adapt. Small 

institutions and community colleges seemed to be more attuned to dealing with 

the working student. 

• Deteriorating labor market. A number of programs started with a list of 

employers willing to partner in the program, but some did not follow through on 

their commitments to the program because of deterioration in the business climate 

and/or cutbacks in staffing. In response, programs sought to replace these 

employers with others not as affected by the economic downturn. 

E. PROGRAMS FOR WELFARE RECIPIENTS 

elfare-to-Work is a term that has come to have several meanings. At its most 

sweeping, it sums up the spirit of the welfare reform law, the Personal 

Responsibility and Work Opportunity Reconciliation Act (PRWORA) of 1996. The term 

also applies to funds that are used to help welfare recipients move into the labor market 

as part of the Temporary Assistance for Needy Families (TANF) block grants channeled 

to states. Finally, the term refers to the categorical Welfare-to-Work Grants Program that 

was administered by the Department of Labor and that began in 1997 and is currently 

being phased out. 

PRWORA changed welfare in the United States in several important ways. It altered the 

funding mechanism so that states had much greater latitude in deciding how to assist their 

citizens in need of income support. Along with this greater freedom for states, PRWORA 

imposed goals on what proportion of welfare recipients had to be engaged in work 

activities and tied yearly funding to the achievement of these goals. Also, PRWORA 

instituted a five-year lifetime time limit on an individual’s receipt of welfare. Virtually 

all states moved to a “work first” approach that encouraged current welfare recipients to 

find work immediately rather than enroll in education or training classes to improve their 

skills. The results of these changes in the welfare system have been remarkable—

dramatically reduced caseloads and surprising success of former recipients in finding 

employment. It is likely that the strong national economy of the 1990s also played an 

important role in the reform success. 

While welfare reform has had very positive results in caseload reduction and 

employment, the advancement and retention of current and former recipients are still 

areas that need attention. Holzer and Wissoker (2001) present evidence from a new 

survey of about 3,000 employers in Chicago, Cleveland, Los Angeles, and Milwaukee. 

They find that women on welfare can be divided into three groups. The first two groups 

have had success in being hired initially. The first group, which is the majority of hired

recipients, performs at least adequately in their jobs and is able to find and keep jobs. 

Nevertheless, they suffer from low wages (about $7 per hour) and limited prospects for 

advancement either in their current jobs or in movement to better positions. The second 

group has problems with performance and retention and/or has problems on the job such 

W 
42 

as absenteeism (up to one-half of hired recipients). This group’s difficulties have more to 

do with “soft skills” such as attitude and relations to coworkers than their ability to 

perform the work. The third group includes the “hardest-to-serve” recipients who have 

difficulty obtaining employment, let alone retaining it. This third group appears little in 

the survey data. 

Holzer and Wissoker find that low wages and advancement are more pressing issues than 

retention—most hired recipients have fairly high retention rates. Their ability to achieve 

self-sufficiency, however, is tied to their wage rate and their ability to advance. The 

authors suggest different services for the first two groups described above. The group 

with few problems at work needs education and training to enhance their human capital 

and so to eventually advance into higher paying jobs. The group that has more problems 

at work needs help with soft skills, which can be learned either through preemployment 

training, community service placements, or post-employment support services. 

 

Patel (2001) also finds that retention and advancement are the major issues now that 

many welfare leavers have found employment. She suggests that the low wages and 

limited prospects for advancement can be overcome through better program design. In 

addition to a primary focus on rapid entry into the workforce, programs should include 

improved job matching, better use of labor market information, closer links to employers, 

and increased access to skill-building activities. Patel points to Portland, Oregon’s Steps 

to Success program as one that offers a mix of job search, education, job training, and 

work activities. This mix contrasts with most “work-first” approaches, which attempt to 

increase employment and earnings solely through building a work history, rather than 

increasing skill levels through classes or training. However, this mix also contrasts with 

education-focused approaches that de-emphasize job search and look first to place clients 

in skill-building activities. In the DHHS-sponsored National Evaluation of Welfare-toWork Strategies, which incorporated an experimental design, the Portland program stood 

out in substantially increasing employment and earnings, lowering welfare receipt, and 

being effective across a cross-section of sample members.27

The Freedman et al. (2000) results on the Portland program which find such a strong 

earnings impact need to be interpreted with caution, however. First, the study mentions 

that the Portland program worked with a less disadvantaged caseload because staff 

excluded those with serious barriers to participation. Thus, the hardest-to-serve 

participants were not included in either the program or control groups in Portland. 

Second, Portland was the only program looked at by Freedman et al. that was 

employment-focused yet offered a varied first activity (not solely job search). Therefore, 

it is difficult to attribute the strong results to that type of approach as opposed to other 

idiosyncratic details of the Portland site. If another program besides Portland had been of 

this type and had also produced sterling results, it would be more convincing evidence 

that these results were due to program design alone. 

 

27 The National Evaluation of Welfare-to-Work Strategies studied programs funded with TANF dollars. 

The findings from this evaluation are summarized in Freedman et al. (2000). 
43 

Patel makes a few suggestions for federal policy. She proposes that the TANF goal of 

promoting work be expanded to include employment retention and workforce 

advancement and that states use TANF funds and other resources to further retention and 

advancement. She also suggests that states be allowed to count some education and 

training activities in meeting federally-mandated work participation targets. 

More evidence that a work-first approach might not be optimal is found in Hotz et al. 

(2001). This study takes a long-term look at the participants of the Greater Avenues to 

Independence (GAIN) study that originally showed the benefits of work-first in the 

Riverside County, California program. The data used in Hotz et al. contains nine years of 

employment and earnings outcomes following the initial random assignment in four 

California counties: Alameda, Los Angeles, Riverside, and San Diego. The welfare-towork programs in Alameda, Los Angeles, and (to a lesser extent) San Diego contrasted 

from the one in Riverside by putting much more emphasis on human capital development 

and were much more likely to have participants engaging in basic skills activities. In the 

early years, the Riverside program had much greater impacts on earnings and 

employment outcomes. Over time, however, the relative advantage of Riverside 

diminished, and in later years Alameda’s and Los Angeles’ programs show the same or 

greater level of benefits as Riverside. This evidence could be read in support of the 

mixed strategy approach discussed in Patel (2001). 

If more effective welfare-to-work programs were implemented that enabled recipients to 

raise their human capital in a meaningful way, it is possible that they might attract more 

people to welfare rolls if they felt these programs would really help them. Moffitt (1996) 

discusses this possibility, namely that attractive employment and training programs might 

increase the entry rate to welfare. Presumably, a high-quality program would increase 

the exit rate as well, making the overall effect on caseloads unclear. The mixed strategy 

approach, if implemented on a large scale, might induce recipients to stay in the program 

somewhat longer while improving their human capital. The additional time on the rolls 

would lead to greater expenditures in addition to the funds necessary to expand services 

beyond work-first. 

Helping the hardest-to-serve welfare recipients has been an area of concern during 

welfare reform. Congress created the Welfare-to-Work (WtW) Grants program in 1997 

to assist in this effort. These funds are administered by the Department of Labor and 

have primarily been granted to local Workforce Investment Boards (WIBs).28 

Nightingale (2001) looks at the implementation of 11 of these grants over the first two 

years of the program. She finds that local WtW programs are expanding service options 

for hard-to-serve groups, which include people with substance abuse problems, homeless 

individuals, non-custodial parents (mainly fathers), and people with disabilities. The 

programs are highly decentralized at the local level with the WIBs relying heavily on 

contracted service providers, usually community-based organizations. Services include 

job search assistance and job placement, internships with partnering employers, paid 

work experience, wage supplements, and post-employment skills development. The first 

 

28 WIBs are the local entities that replaced Private Industry Councils when the Workforce Investment Act 

of 1998 replaced JTPA as the primary workforce development legislation. 
44 

two years at the WtW sites were dominated by recruitment and enrollment problems. 

The reasons for enrollment problems include the strict eligibility criteria in the 

authorizing statute, a lack of familiarity with new programs on the part of TANF workers 

resulting in fewer referrals, the fact that people with relatively serious personal and 

employment problems are difficult to recruit and retain in programs, and simply a 

shrinking pool of welfare recipients, in part because of a strong economy providing 

employment opportunities.29 All the sites had stepped up their recruiting efforts and were 

having better results by the end of the two-year period. The new recruiting methods 

included direct and proactive outreach approaches, scheduling activities and components 

more closely together, and incorporating special financial provisions into service 

providers’ contracts. The effectiveness of the WtW programs on participants’ 

employment outcomes will be examined in a future study. 

Many have commented on the wide range of job-readiness found within welfare 

recipients as a group. Some recipients need much more help in finding employment than 

others. Realizing this fact, and that resources are probably better used on those who need 

more help, Eberts (1997, 2002) develops a model to target those less likely to find jobs on 

their own. His profiling approach follows the lines established by the Worker Profiling 

and Reemployment Services program used by the unemployment insurance system. 

Eberts estimates the parameters of a logit model to profile recipients entering WtW 

programs. This work was subsequently used in a random assignment experiment within 

the “Work First” program of Kalamazoo, Michigan. In the experiment, single parents 

entering the Work First program were given an employability score based on Eberts’ 

estimated parameters which determined which of three (high, middle, or low) 

employability groups they were classified into. The earlier model estimation and 

qualitative observations indicated that the high group was best served by the Behavioral 

Foundation, the middle group by Youth Opportunities Unlimited (YOU), and the low 

group by Goodwill Industries. Each employability group was split evenly into randomly 

assigned treatment and control groups. The treatment group members were referred to 

the specific service provider thought to best serve that employability group, while control 

group members were randomly assigned to one of the three service providers. 

Eberts (2002) describes how 90-day employment retention rates among the control group 

can be used to examine different allocation schemes of the three employability groups 

among the three service providers.30 He demonstrates that the matching of employability 

groups to service providers that was believed to be optimal before the experiment was in 

fact the optimal match, thus validating the profiling approach. Eberts overstates the 

strength of the evidence, however. The retention rate of middle employability group 

members in the control group assigned to the YOU service provider was 0.370 (10 out of 

27) compared to 0.170 (33 out of 194) for middle employability group members in the 

treatment group assigned to YOU. The discrepancy is puzzling as these two rates should 

 

29 In 1999, the legislation was amended to loosen the eligibility requirements and broaden the mix of 

services that could be provided. 

30 The Work First program was contractually obligated to use all three service providers, so after the 

experiment each service provider would serve exactly one employability group. This yielded six possible 

allocation schemes. 
45 

have been identical or very close to one another. A conservative approach would have 

been to take an agnostic point of view and simply pool the results to arrive at a retention 

rate of 0.195 (43 out of 221) for middle employability group members assigned to YOU. 

Instead, Eberts uses the 0.370 figure to demonstrate the optimality of the allocation 

scheme used with the treatment group. He also uses the 0.370 figure in the benefit-cost 

analysis, which more than doubles the benefits produced when using the conservative 

approach described here. Nevertheless, Eberts’ profiling approach is sensible and holds 

promise for more efficient use of WtW funds and other funding streams targeted on 

diverse groups of participants. 

Many welfare reform observers have felt that it is important to continue to extend 

services to current and former recipients even after they have found employment. To test 

this idea, the Department of Health and Human Services sponsored the Post Employment 

Services Demonstration (PESD), which was located in Chicago, Illinois; Portland, 

Oregon; San Antonio, Texas; and Riverside, California. Rangarajan and Novak (1999) 

evaluated the demonstration. All sites provided counseling and support, job search 

assistance (in case of job loss or to find a better job), help with benefits, service referrals, 

and support service payments for work-related expenses. The evaluation used a random 

assignment design, and between 60 and 80 percent of PESD clients (the program group) 

received some support services during the six months after enrollment. Unfortunately, 

the programs had little effect on earnings or on reducing welfare receipt. This might 

partly have been due to the fact that control group members in two of the sites had access 

to services similar to those that the PESD was providing. There is some evidence in the 

study that more disadvantaged recipients might benefit more from post-employment 

services than recipients who are better off. 

ETA is currently supporting three notable studies on WtW strategies: two long-term 

evaluation projects in conjunction with the Department of Health and Human Services 

(HHS) and a shorter term project with the Oregon Employment Department. The first of 

these is the Employment Retention and Advancement (ERA) study. The main goal of 

this experimental project is to determine how to best serve former welfare recipients in 

retaining jobs and advancing their careers. Phase I of the study developed innovative 

retention and advancement methods in selected states and has already been completed. 

Phase II is an experiment to evaluate the relative effectiveness of alternative employment 

retention and advancement strategies in ten states around the country. The study is 

scheduled to be completed in 2007. The second jointly sponsored study with HHS is the 

Enhanced Services for the Hard-to-Employ (HtE) Demonstration and Evaluation Project. 

This will be a nine-year study at six sites that evaluates promising programs designed to 

enhance employment outcomes for current and former TANF recipients and other lowincome parents who face serious obstacles to steady work. An important component of 

this demonstration is the participation of One-Stop Career Centers and the evaluation of 

the centers in serving this group effectively. Finally, ETA is co-sponsoring the Oregon 

Low-Wage Worker Retention and Advancement Demonstration Project to determine 

effective methods to assist employers in retaining and advancing low-wage workers. 

Services will be delivered by One-Stop Career Centers in Beaverton and Corvallis; initial 

study results are expected in 2003. 
46 

F. PROGRAMS FOR YOUTH 

Job Corps, established in 1964, is the largest youth employment and training program run 

by the federal government. It serves disadvantaged youth aged 16-24 in 119 centers 

across the country. Job Corps enrolls 60,000 new participants each year and has an 

annual budget of about $1 billion. It is an intensive and comprehensive program, 

combining academic and vocational activities with a wide range of support services in a 

residential environment. Job Corps was shown to have a strong positive impact on 

participants’ earnings in an evaluation that studied the program’s operations in the late 

1970s. 

During the 1990’s, the U.S. Department of Labor sponsored the National Job Corps 

Study. This large-scale study used a randomized experimental design to measure the 

impacts of the program. Individuals were randomly assigned to program and control 

groups in 1994 and 1995, and they were followed for four years after assignment. The 

study measured the impacts of Job Corps on educational attainment, employment and 

earnings, receipt of public assistance, and criminal behavior. Complete responses were 

collected from over 11,000 individuals, almost 7,000 of whom were in the program 

group. Seventy-three percent of the program group (those individuals randomly allowed 

to participate in Job Corps once they were determined eligible) actually participated in 

Job Corps. 

Burghardt et al. (2001) summarize the main results of the National Job Corps Study. The 

study found that the program group received substantially more education and training 

than the control group, even though many control group members received a sizable 

amount of education and training. Ninety-three percent of the program group engaged in 

some education or training as compared to 72% of the control group. Jobs Corps 

participants received an average of 1,000 more hours of education and training than nonparticipants over the 48-month follow-up period. This additional education and training 

is roughly the equivalent of one high school year. About 42% of the program group 

attained a GED during the study period as compared to 27% of the control group. 

However, only 5% of the program group earned a high school diploma as compared to 

8% of the control group. This is not entirely surprising, though, as participation in Job 

Corps precludes high school attendance. Additionally, 38% of the program group 

received a vocational certificate as compared to only 15% of the control group. Job 

Corps did not appear to have an effect on college attendance or college completion. Over 

2,000 members of the sample were given skill tests at the time of the 30-month followup, and there was some evidence that Job Corps participants had higher functional 

literacy in the quantitative and prose areas than non-participants. 

In addition to effects on education and training, Job Corps was found to have positive 

impacts on employment and earnings. Early in the follow-up period, control group 

members had higher average weekly earnings than program group members. After two 

years, this difference disappeared, however. In the third year, program group members 

had higher average weekly earnings, and in the fourth year the differential increased. In 

the fourth year, program group members had average weekly earnings of $211, compared 
47 

to $195 for control group members. This $16 differential can be decomposed into $11 

due to program group members working more hours and $5 due to program group 

members earning higher hourly wages. If the assumption is made that all positive effects 

of Job Corps only accrue to actual participants, then adjusting for the fact that only 73% 

of the program group actually enrolled gives the result that Job Corps participation 

increases average weekly earnings by $22. Employment rates and hours of work showed 

the same pattern of the program group at first catching up to, and later surpassing, the 

control group over the four years. Employed program group members earned an average 

of $0.22 more per hour in their most recent job than employed control group members. 

Also, program group members earned a higher wage across occupational categories. 

Study results showed that Job Corps reduced the receipt of public assistance over the four 

years. Program group members received an average amount of $3,696 in public 

assistance (cash welfare plus food stamps) over four years, compared to $4,156 for 

control group members. If the adjustment for actual Job Corps participation is made, the 

impact per Job Corps participant is $639. Differences in public assistance receipt were 

greater at the beginning of the follow-up period but were still present in the fourth year. 

The study also concluded that Job Corps reduced criminal behavior. About 29% of 

program group members were arrested during the four-year follow-up period, compared 

to 33% of control group members. This difference is statistically significant at the .05 

level. Arrest rate reductions were largest in the early part of the follow-up, when most 

program group members were enrolled in the program. About 22% of program group 

members were convicted of a crime, while 25% of control group members were 

convicted. Jobs Corps also reduced the number of times participants were victims of 

crimes by about 20%. Reductions in victimization were found for almost all types of 

crimes. 

These positive impacts in the areas of educational attainment, employment and earnings, 

receipt of public assistance, and criminal behavior are encouraging. Job Corps is an 

expensive program, however, costing the Federal government over $1 billion per year. 

As part of the national study, evaluators performed a benefit-cost analysis to determine if 

Job Corps delivered benefits of greater value than its high cost. The study finds that Job 

Corps produces benefits worth $30,957 per participant versus costs of $14,128, so that 

benefits exceed costs by $16,829 per participant. A key assumption in this analysis is 

that the positive impact on earnings that Job Corps provides in the fourth year of the 

follow-up period will continue over the full working lifetime. 

Dollar figures were attached to several different types of benefits. These included the 

benefits of increased output from the improved productivity of participants, the resources 

saved from the reduced use of programs and services by participants, and the benefits 

from reduced crime committed by and against participants. Costs included explicit 

program costs as well as the economic costs stemming from the usage of land, buildings, 

and capital by the program. The largest benefit by far of the Job Corps is that of 

increased output due to the additional skills of participants. The study valued this 

increased output, as reflected in increased earnings, at about $27,500. Reduced usage of 
48 

other services produced a benefit valued at about $2,200 and the benefit from reduced 

crime is valued at about $1,200. 

McConnell and Glazerman (2001) provide further details of the benefit-cost analysis. 

Benefits exceed costs for most subgroups of participants with the puzzling exceptions of 

two subgroups. Benefits do not exceed costs for Hispanics or for those who were ages 18 

and 19 at time of random assignment. In fact, Schochet et al. (2001) report that both of 

these subgroups showed no earnings impact over the follow-up period. Age groups 16-17 

and 20-24 showed much larger earnings gain over the follow-up period. 

It is reasonable to question the assumption that the positive Job Corps impact on earnings 

in the fourth year will continue unabated over the working lifetime, because this is the 

assumption that generates almost $27,000 of hypothesized benefits per participant. The 

evaluators offer a few reasons why this assumption is justified. First, the impact on 

earnings never declined during the four-year study. Second, the additional education and 

training received by Job Corps participants was about the equivalent of an extra school 

year. Several studies have shown that the benefits of an additional year of schooling 

persist over the lifetime. Finally, the evaluators point to the types of skills that Job Corps 

teaches, such as literacy, numeracy, workplace, and social skills and argue that these 

types of skills are unlikely to become obsolete. These reasons appear sensible. The 

study also states that benefits of Job Corps would exceed costs if the earnings impact 

continued for just nine years without change or if the impacts declined at no more than 

8% per year over the course of the working lifetime. 

For measures such as earnings impact and receipt of public assistance, the study adjusts 

the results for actual program participation because only 73% of the program group 

actually participated in Job Corps. The study bases this adjustment on the assumption 

that no benefits accrued to program group members that did not enroll in the program. 

This assumption would be on a more solid footing if testing supported it. The nonenrollers were part of the four-year follow-up, so it should be possible to use a statistical 

methodology to test whether any benefits did accrue to this subgroup.31 

The bulk of the evidence from the National Job Corps Study leans toward the conclusion 

that even given its large expense, Job Corps is a worthwhile program. Are there ways to 

make it even more cost-effective, however? One reason why Job Corps is so expensive is 

because it is a residential program.32 Thus, it is of great interest whether Job Corps could 

be effective without its residential aspect. Some clues about this question can be gleaned 

from the study. Females with children in Job Corps who lived at home instead of 

dormitories had earnings 24% higher than the comparable group in the fourth year of the 

study. Male nonresidents had earnings 26% higher than their counterparts in the fourth 

 

31 See Heckman, Smith, and Taber (1998). 

32 It has been argued that the primary reason for Job Corps’ high cost is its intensity of services rather than 

its residential aspect. This may well be the case. Table VI.6 in McConnell and Glazerman (2001) reports 

that residential slots in Job Corps cost $23,468 while non-residential slots cost $21,514. The figures in this 

table were prepared by the National Office of Job Corps rather than by the evaluators. Because $11,024 in 

non-residential costs were lumped into the category “Other Center Costs,” it is not possible to delve deeply 

into the accounting decisions made by the national office. 
49 

year. However, female nonresidents without children had no earnings impact. Arrest 

rates were reduced for female nonresidents, but not for male nonresidents. The generally 

positive impacts for nonresidents need to be interpreted with care as nonresidents differed 

from residents in several ways. Nonresident males and females tended to be older than 

residents, were more likely to have a child, more likely to have a high school credential, 

and less likely to have been previously arrested. So, in general, nonresidents tended to be 

more mature than residents, and the additional responsibility that comes from parenthood 

might have made nonresidents more motivated participants than residents. Though 

nonresident participants had access to the full range of support services that were 

available to residents, they were less likely to utilize center services outside of the 

academic and vocational areas. 

One component of the National Job Corps Study evaluated the program’s effect on 

participants’ literacy skills. Glazerman et al. (2000) report the findings in this area. 

Literacy testing was administered to 1,117 program group members and 1,156 control 

group members at the time of the 30-month follow-up. The test given was the 

Educational Testing Service instrument that is used for the National Adult Literacy 

Survey. This particular test was chosen because it measures functional rather than 

academic skills. The test measures prose literacy, numeric literacy, and document 

literacy. 

Some positive impacts were found when comparing program group average scores with 

control group average scores, although these impacts were small. Program group scores 

were higher than control group scores for all three types of literacy, and the differences 

for prose and numeric literacy were statistically significant at the 0.10 level. One might 

have expected a greater difference given that program group members had on average 

received the equivalent of an extra year of schooling through participation in Job Corps. 

However, the small gap can be explained by the fact that the greater work experience of 

control group members in part makes up for their smaller amount of schooling. 

The Workforce Investment Act of 1998 highlights Job Corps’ mission of assisting 

students in either attaining a GED or completing a vocational training program. One 

component of the National Job Corps Study, described in Gritz and Johnson (2001), 

examines what effect achieving these milestones has on students by comparing them to 

similar students in the control group who did not achieve these milestones. The study 

looks at GED attainment and vocational training separately. Matching methods were 

used to make the comparisons between key milestone achievers and similar nonachievers. The strongest finding from this component of the study is that almost all 

positive earnings impacts from Job Corps accrued to those participants who either 

attained a GED or completed vocational training. Participants who did not achieve either 

key milestone derive no earnings benefit from the program. The study cautions that there 

is no way to disentangle the effects of the longer time spent in the program necessary to 

achieve a milestone with the actual graduation milestone. Still, this study confirms that 

the WIA directive to focus on these two types of graduation is appropriate. 
50 

In another report on the Jobs Corps, unconnected with the National Job Corps Study, 

Ginsburg et al. (2000) examine retention in the program and what can be done to reduce 

attrition. Of the 343,000 youth who enrolled in Job Corps nationally between July 1993 

and the end of 1998, 86% remained in the program after 30 days, but only 64% remained 

after 90 days. The study finds that there are no easily identified characteristics of 

enrollees that would allow administrators to predict who is most at risk of dropping out of 

the program. Instead, the decision to stay in the program is linked to characteristics such 

as attitude and motivation that are more difficult to measure. The study also finds that 

the performance of the staff and the relationships between the staff and the youth are very 

influential in determining the rates of attrition. Hence, the main recommendation of the 

study is that a national effort should be undertaken to further staff development and 

insure that staff have the necessary skills to best assist the participants. These skills 

include ability to teach positive coping methods, ability to communicate effectively with 

youth from different backgrounds and at different stages of development, and ability to 

build participants’ confidence and feelings of connectedness to the program. A 

secondary recommendation is that centers should try to achieve balance in the gender of 

participants. Both male and female students are more likely to drop out when there are 

less than 40% female students at a center. 

Other Programs for Youth 

In the past decade, the ETA has sponsored several youth demonstrations in order to test 

new approaches to preparing youth for the labor market. These demonstrations include 

the School-to-Work Out-of-School Youth demonstration, the Youth Offender 

Demonstration Project, JOBSTART, Youth Fair Chance, the Youth Opportunity 

program, the Quantum Opportunities Program (QOP) replication, and the Center for 

Employment Training (CET) replication.33 Some of these demonstrations have finished 

while others (including the Youth Offender Demonstration Project, the Youth 

Opportunity program, QOP replication, and CET replication) are still ongoing. Most of 

these demonstrations have already been the subjects of completed evaluations with the 

exceptions of the QOP and CET replications. Studies on these latter two are 

forthcoming. 

In 1997, the Department of Labor funded two demonstration projects to examine how 

School-to-Work (STW) principles could be incorporated into programs already serving 

out-of-school youth. One demonstration involved 11 diverse non-residential out-ofschool youth programs, while the other was located at 30 Job Corps centers. School-towork has three basic components: 1) school-based learning (SBL), which incorporates 

work ideas into academic classroom setting, 2) work-based learning (WBL), which 

teaches skills at well-monitored worksites, and 3) connecting activities, which include 

creating relationships to employers and post-secondary education and training institutions 

and professional development for staff. 

 

33 The QOP and CET were not originally ETA programs. Their success in past evaluations has led the ETA 

to replicate these program models in additional locations. 
51 

While process evaluations were completed by D’Amico et al. (2000a, 2000b), no youth 

outcomes were evaluated. The process evaluations find that, as one might expect, 

organizational change is difficult and they offer recommendations to facilitate the change 

process. The 11 non-residential programs were granted $100,000-$140,000 each over a 

period of about 21 months. In general, the programs were more successful in 

incorporating school-to-work principles into vocational classroom settings than into 

academic classroom settings. In the academic courses, there was a tension between an 

integrated instructional curriculum and preparing for the GED test or a high school 

diploma. About half of the grantees in this demonstration were judged to have undergone 

substantial systemic change.34

The Job Corps sites showed similar mixed results. Thirty sites were named “Model 

Centers” and granted $60,000 per year over two years. Although Job Corps already 

shares many components with school-to-work, the evaluators believe that Job Corps 

could improve its classroom and worksite experiences and create a more cohesive and 

integrated system for learning. D’Amico et al. (2000a) found that about one-third of the 

sites understood the STW vision, about half viewed STW primarily in terms of workbased learning alone, and about 15% did not understand STW beyond getting more 

employers to participate. As with any organizational change, strong leadership is critical. 

In some cases, center directors were skeptical about STW, making it very difficult for 

change to take root. In other cases, change efforts were derailed by high staff turnover; 

one-half of the centers experienced turnover of the STW coordinator. Also, some centers 

had turf issues between academic instructors and vocational instructors, making 

collaboration difficult. 

Incorporating school-to-work principles into programs that serve disenfranchised out-ofschool youth is a promising idea. Perhaps it would be sensible for DOL to identify the 

organizations that have made the most strides in adopting the STW philosophy and then 

to conduct an outcome evaluation for the participants. This would provide evidence on 

whether this promising idea actually results in better service and outcomes for youth than 

current programs offer. 

Of course, STW principles were originally developed for in-school youth. The Schoolto-Work Opportunities Act (STWOA) of 1994 provided five years of funding to create 

school-to-work systems through state initiatives and local partnerships of schools, 

employers, and others. The final report of the National Evaluation of School-to-Work 

Implementation has not yet been released. In an earlier report of the National Evaluation, 

Hershey et al. (1999) write, “Although progress has been made, the practices that the 

STWOA promotes may be difficult to sustain. STW implementation is rarely at the core 

of states’ high priority education reforms to increase school accountability and academic 

standards.” The authors find that the expansion of career development activities is the 

 

34 D’Amico et al. (2000b) define “substantial systemic change” as having some key element of the 

grantee’s service strategy be noticeably changed in a way that aligned its project design in closer 

conformance with DOL’s threshold criteria. These changes showed every indication of being sustained and 

built upon after the demonstration grant funding period. Examples of such changes were adding an 

additional career pathway for students to choose or enhancing classroom curricula to further integrate the 

teaching of an array of skills in context. 
52 

most visible change engendered by STW, while academic classes with a career focus and 

intensive work-based learning activities (such as internships and apprenticeships) are less 

common. They conclude that without continued federal interest in STW, “the overall 

vision of a STW system may slip into the shadows of the many other competing demands 

on schools and teachers.” 

An important subgroup of the youth population that has attracted much attention is 

composed of youth who are already involved with the criminal justice system. In 1997, 

100,000 youth were in custody in residential correctional facilities.35 These youth will 

return or have returned to their communities, and they are in great need of intervention in 

order to redirect their lives. In order to address the needs of those in the juvenile justice 

system, the Departments of Labor and Justice sponsored the $13.1 million Youth 

Offender Demonstration Project (YODP) that ran from fall 1999 to fall 2001. In 2001, a 

second round of funding was awarded to 10 of the original sites and 9 new sites. A third 

round of funding may take place in 2002. Round 1 of the demonstration included 14 

projects around the nation of three distinct types: Category I-Model Community Projects 

(comprehensive community initiatives), Category II- Education and Training for Youth 

Offenders Initiative (school-to-work education and training within juvenile facilities and 

follow-up services and job placement when they return to communities), and Category 

III- Community Wide Coordination Projects (to strengthen coordination of after-care and 

prevention services for youth in small- and medium-sized cities). The first six months of 

the YODP were to be used for planning, and the remaining 18 months were for 

implementation. Research and Evaluation Associates (REA) contracted to provide 

technical assistance to the sites as well as to produce a process evaluation of 12 of the 14 

sites in Round 1. Services provided included intake and assessment, case management, 

diploma or GED help, soft and life skills training, barrier removal, vocational education, 

job search support, job placement support, post-placement followup, and substance abuse 

and personal counseling. Youth at risk of involvement with the juvenile justice system 

and youth already involved with the system are eligible for services. 

In general, the sites were only partially successful at putting in place a system of services 

for youth offenders. Two years may have been too brief a time for the goals of this 

demonstration. Securing funding after the grant period was a big issue at the sites, and 

the short-term nature of the grant contributed to staff turnover, because staff did not view 

their own jobs as long-term. Also, the wide age range of participants (ages 14-24) meant 

that there were different kinds of services needs. Labor market outcomes for the 

youngest juvenile participants will not be apparent for several years. It is clear that youth 

offenders have deep needs and for many, preparation for the labor market will not happen 

quickly. The participants did understand the necessity of preparing for the world of 

work, however, and it was the lure of decent-paying jobs that attracted them to and kept 

them engaged with the projects. 

REA reported that their double role of technical assistance provider and evaluator 

produced some tension in their relationships with the sites. In addition, some sites 

viewed requesting technical assistance as a sign of weakness rather than as part of a 

 

35 Research and Evaluation Associates (2002), Preface. 
53 

process of continuous improvement. Later in the two-year period, it became more 

difficult to get sites to share information and practices because they were competing for 

follow-on grants. Complete and accurate data collection was a problem at many sites. 

Perhaps DOL should provide an MIS in future multi-site demonstration projects to insure 

proper data collection. This MIS should also be consistent with WIA data collection. 

The Office of the Inspector General at DOL (2002) also performed an evaluation of the 

Category I and III sites. Their results show a correlation between length of time spent in 

services and length of job retention. This correlation could mean that more participants 

should be receiving services over a longer period of time. However, it is not clear 

whether the length of service is determined by the service provider, the participant, or 

both. If the length of service receipt is a choice made by the participant, the correlation 

could be a simple case of selection bias, with more motivated participants both staying 

longer in the program and performing better in their employers’ workplaces. If the 

correlation is largely due to selection bias, then simply increasing the length of services 

for participants might not result in improved employment duration outcomes. The Office 

of the Inspector General also found that one of the Category I sites had significantly less 

recidivism among participants than the other sites. This site offered more counseling and 

mentoring than the other sites, which might explain its greater success in preventing 

repeat criminal involvement. 

Round 1 of the YODP seemed to suffer from its short length (and perhaps its low 

budget). Unfortunately, the evaluations provide little evidence as to the effectiveness of 

this type of program. Which of the many services included in the YODP are the most 

crucial for youth offenders, and what should expectations be as to the length of time these 

juveniles need to prepare for the labor market? Are residential programs more effective 

for youth offenders than non-residential programs? Perhaps a future study of the sites 

that have managed to secure funding and to continue with their initiatives would shed 

additional light on these questions. 

While experimental evidence is scarce on the best methods to deal with youth offenders, 

there nevertheless seems to be a consensus that programs should be structured along 

holistic, comprehensive youth development principles. This consensus is reflected in the 

report issued by the Task Force on Employment and Training for Court-Involved Youth, 

which was jointly funded by the Departments of Labor and Justice (USDOJ 2000). The 

seven principles to improve youth programs noted in this report are: academic and work 

related skills, age appropriate programs to match stage of development, long-term followup, effective implementation, trust and effective adult support, small family-like setting 

and positive peer relationships, and work-based learning. Walker James (1997) and 

Brown et al. (2001) emphasize similar principles. While economic self-sufficiency is 

crucial to growing out of delinquency, not just any employment program will be 

successful. Support services that attend to juvenile offenders’ many needs are essential. 

Brown et al. (2001) note that the tug between rehabilitation and punishment in juvenile 

justice has moved far in the direction of punishment. The tilt toward punishment has 

consequences, as increased use of incarceration is correlated with reduced career 

prospects, which in turn is correlated with increased criminality. 
54 

Walker James (1997, 1999), in two compendiums produced for the American Youth 

Policy Forum, describes many promising programs that have extremely positive 

outcomes in reducing recidivism. All the outcome statistics have to be viewed with some 

caution, due to the possibility of selection bias. Brown et al. summarize the results of a 

project using random assignment that has been performed on a component of 

comprehensive youth development approach, i.e., mentoring. Public/Private Ventures 

evaluated Big Brother/Big Sister programs in eight cities over an 18-month period. 

Control group members were placed on a waiting list, while program group members 

were assigned mentors. The study concluded: 

Youth who worked with mentors were 46% less likely than those on the 

waiting list to initiate drug use and 27% less likely to initiate alcohol use 

during the study period. Mentored youth were one-third less likely to hit 

someone, skipped half as many days of school and performed better at 

school, and reported better relationships with their parents and peers than 

youth in the control group.36

When interpreting this evidence, it is important to keep in mind that the participant 

population in the mentoring study consisted of all youth who apply to Big Brother/Big 

Sister programs, rather than solely court-involved youth. Other experimental evidence 

supporting a comprehensive approach to disadvantaged youth is the National Job Corps 

Study, which was discussed above. Again, a caveat is that Job Corps has had a tendency 

to enroll court-involved youth in fewer numbers than disadvantaged youth with less 

juvenile justice system involvement.37 Brown et al. make the sensible recommendation 

that juvenile offenders have bonds secured for them before they are released so they can 

more easily find employment. The DOL sponsors the Federal Bonding Program (FBP) 

that can be used for this purpose.38 Although the FBP has operated for many years, it has 

never been evaluated and both a process study and impact evaluation should be 

considered. 

Lerman (2000) reviews different approaches to assisting at-risk youth and also seeks to 

draw lessons for the United States from other O.E.C.D. countries. In many of the 

O.E.C.D. nations there is a growing emphasis on apprenticeship and employer-based 

training, as well as vocational education based in the schools. In the United States, the 

schools standards movement has meant that schools are moving further away from 

vocational education. Between 1982 and 1992, the proportion of high school seniors in 

vocational education dropped from 27% to 11%. Most out-of-school training programs 

have not had proven success. The JTPA evaluation of the early 1990s showed that outof-school youth served by JTPA received on average about 180 extra hours of training 

and related services as compared to a control group, but that their earnings were no 

higher than the control group. 

 

36 Brown et al. (2001), 28. 

37 Brown et al. (2001), 17. 

38 The Federal Bonding Program provides bonds that insure employers against loss of money or property 

due to any type of employee stealing (such as theft, forgery, larceny, or embezzlement). 
55 

Another demonstration program, JOBSTART, was a test of whether intensive services in 

a non-residential setting will work; the motivation was to see if a program similar to Job 

Corps except for the residential component would be as effective as the Job Corps. Three 

of the 13 JOBSTART sites were Job Corps centers. Services included basic skills, 

vocational training, training related support services (such as transportation, child care, 

and life skills training), and job placement assistance. The experimental results showed 

that participants received more training than control group members, but that the program 

had no overall effect on earnings, employment, child-bearing, or criminal activity. The 

subgroup of males with prior arrests did exhibit a positive earnings impact. Only one of 

the sites, Center for Employment and Training (CET) of San Jose, CA, had significant 

impacts: earnings during the period 24 to 48 months after assignment were 42% higher 

than control group. DOL is currently replicating CET around the country. 

Lerman suggests that training programs linked to specific industry sectors have delivered 

promising results. Examples of these programs are CET, Focus: Hope in Detroit, and 

PROJECT CRAFT sponsored by the Homebuilders Institute. Lerman also emphasizes 

the findings of Paul Ryan: that if the United States wants to move toward an 

apprenticeship system, it might be better served by developing an institutional 

infrastructure to support the system, rather than sponsoring a continuing series of 

demonstration projects. 

Another comprehensive approach to youth development even more ambitious than the 

YODP was the Youth Fair Chance (YFC) demonstration program. This initiative was 

located in 16 sites, and was designed to saturate small high-poverty areas with a wide 

range of services available to all youths within the areas. Each site received about $4 

million from the Department of Labor over the first two years. The programs provided 

education (literacy and GED classes), employment preparation and training, counseling, 

and support services to young people of ages 14-30. A centerpiece of the program was 

the establishment of a “learning center” at each site that served as One-Stop centers for 

referral and provision of services. YFC was organized around case managers who were 

stationed at the learning centers. YFC was also to establish School-To-Work (STW) 

programs (along the lines of the School-To-Work Opportunities Act of 1994) in one or 

two local schools. 

Originally, YFC funding was to last five years, after which sites were expected to secure 

their own funding to continue the initiative. After only two years, however, 

congressional priorities shifted, and the program’s funds for the final three years were 

eliminated. The interim two-year evaluation by Mathematica Policy Research became 

the only evaluation. Corson et al. (1998) chose comparison areas for each site as a way 

of examining the impacts of the program. After two years, all the sites had learning 

centers open, school-to-work set up (to some extent), and community advisory boards in 

place. About one-third of the sites had extensive employer involvement with school-towork. About 10% of eligible youth were enrolled over the two-year period, and the 

average site enrolled 270 participants over the time period April-October 1996. 
56 

Corson et al. report that there were no outcome differences between eligible young 

people in YFC areas and comparison areas. Neither was use of services higher in YFC 

sites. The authors point out that this was not a full test of the underlying concepts of the 

program, especially of the comprehensive nature of the approach. The evaluators had not 

expected to see measurable results at the two-year point. Therefore, YFC provides little 

to no evidence as to what works best for disadvantaged youth. It does serve as an 

example of what not to do when launching a comprehensive community initiative, 

however. Most sites did not have enough time to properly plan their programs nor 

coalition-build before they opened up their learning centers. Eliminating funding after 

two years meant that YFC never had itself a fair chance of succeeding. Local word-ofmouth about the programs did not have much chance to develop. Programs intended to 

deliver long-term results need time to improve and fine-tune their operations. Programs 

that involve the community as much as YFC did need time to allow relationships to form 

and trust to build. Another problem in urban areas was that the YFC boundaries 

corresponded with census tracts, rather than neighborhoods, resulting in youth from just 

outside the boundaries being turned down for services—from their viewpoint, arbitrarily 

so. 

G. SPECIAL TARGET GROUPS 

he Employment and Training Administration has traditionally taken special interest 

in serving groups with particular barriers to labor market participation. In addition to 

the groups covered above in sections of their own, these groups include the homeless, 

people with disabilities, Indians and Native Americans, and older workers. Some of 

these groups have specific categorical programs (e.g., Indians and Native Americans) as 

well as the ability to participate in more general programs, while others do not have 

programs of their own and are served through the general ETA programs (e.g., the 

homeless). 

Homeless Persons 

The Job Training for the Homeless Demonstration Program (JTHDP) was a pilot program 

authorized under the McKinney Act to provide employment and training services to 

homeless individuals. The pilot program operated between September 1988 and 

November 1995, involving over 60 local programs. Reports from the project include a 

final report by Trutko et al. (1998) documenting the implementation of the JTHDP and a 

best practices guide by Beck et al. (1997). Major findings from the demonstration 

include the following: 

• Employment and training programs can successfully serve a wide spectrum of 

homeless individuals. Over the course of the demonstration, over 45,000 

individuals were served, almost 35,000 received employment and training 

services, and about 16,500 obtained employment; about half of those placed were 

still employed 13 weeks after placement. 

T 
57 

• A wide variety of public and private organization can successfully establish and 

operate training programs for the homeless. Grantees included mental health 

organizations, homeless shelters, JTPA service delivery areas (now workforce 

investment areas under WIA), and city government agencies. 

• Homeless participants in employment and training programs require 

comprehensive assessment and ongoing case management. Homeless participants 

generally have greater barriers to employment than most training program 

participants, particularly mental health and substance abuse problems. 

Consequently, programs serving this population must conduct comprehensive 

assessments and monitor the participants’ progress throughout their participation. 

• Employment and training programs serving the homeless must offer or arrange 

for a wider array of services than they may offer to other participants. Housing is 

the primary example of a service that must be made available, but programs also 

must be prepared to deal with the conditions that led to homelessness, including 

mental health and substance abuse. 

• Work readiness and job search assistance are important components for successful 

job placement and retention. 

• Careful screening is required to identify homeless individuals who can benefit 

from classroom training. Specifically, the candidate should have stable housing, 

be motivated enough to undertake training, be free of drugs and alcohol, and have 

sufficient financial support to get by during the training period. 

• Housing support and long-term follow-up are needed to assist homeless 

individuals retain their jobs. 

• Data from the JTPA Title II-A program shows that homeless participants can be 

served in mainstream programs for the disadvantaged. The entered employment 

rate for homeless individuals enrolled in JTPA in program year 1994 was 54%, 

compared to 62% for non-homeless terminees, and the average wage at placement 

was $7.13 per hour compared to $7.00 for non-homeless terminees. 

Since the time of the JTHDP, major responsibility providing employment and training 

services to the homeless has shifted from the Department of Labor to the Department of 

Housing and Urban Development. Given the major changes in program structure and the 

shift from JTPA to WIA with its One-Stop delivery system and universal access, if there 

is a desire to increase services to the homeless, it is likely that pilot projects for the 

homeless would be useful to stimulate interest by local workforce areas. 

Indians and Native Americans 

Many Indians and Native Americans face a number of barriers to labor market success 

including cultural differences, low educational attainment levels, and geographic 

isolation from jobs and supportive services. WIA, as did its predecessor JTPA, includes 
58 

a special program to provide employment and training services to this population. 

D’Amico et al. (1999) conducted a three-year process evaluation of the Indian and Native 

American program. The evaluation notes that in 1999 the program had annual funding of 

about $50 million and served approximately 19,000 participants through 175 grantees. 

Information for the evaluation was gathered by two methods: a mail survey of all 

grantees and field visits to 23 randomly selected grantees. The mail survey generated 

113 respondents, for a response rate of about 70%. Grantees were selected randomly for 

the site visits. Each site visit lasted two to three days and included interviews with 

program staff, participants, recent terminees, and staff from community organizations. 

Overall, the evaluation found the program to be fairly successful, and that relations 

between the grantees and ETA had improved markedly in recent years. The site visits led 

to a number of conclusions: 

• Grantees had a strong service orientation, but they may have been unduly focused 

on meeting participants’ short-term needs at the expense of dealing with longterm problems; 

• Service designs were often limited by lack of adequate funding and a lack of jobs 

in the area; 

• Grantees were very successful at linking efforts with other tribal programs and 

obtaining tribal funds to help leverage their employment and training funding; 

• Grantees tended not to target on specific subgroups because they believed that all 

tribal members were in need; 

• Participants were more often recruited through indirect means, such as word of 

mouth, rather than directly; 

• Participants often focused on immediate problems and did not want to participate 

in long-term programs that may have been more appropriate for solving long-term 

problems; 

• Grantees generally offered both basic skills and occupational classroom training; 

• Basic skills were generally not taught in context, and active learning methods 

were not often used; 

• Occupational classroom training was much more prevalent, and a variety of 

occupational fields were available, with training lasting from a few weeks to two 

years; 

• The evaluators found the training to be of uniformly high quality; 

• On-the-job training (OJT) was used by 14 of the 23 grantees visited, but it was not 
59 

a common treatment; 

• Work experience and community service employment were commonly used 

services, and they were used to introduce participants to the world of work and as 

a short-term measure between major activities; 

• The work-oriented services served as important sources of labor for the tribe and 

community organizations, and in many instances, good performance on a 

community service job or work experience position was the best way to obtain a 

permanent position with the government; 

• Supportive services providing life skills, preemployment skills, and workplace 

skills were commonly provided. 

Because the process study indicated that the Indian and Native American Program is 

functioning fairly well, it would be appropriate to conduct an impact evaluation of the 

program. Such an evaluation would not be easy because identifying an appropriate 

control group would likely prove difficult. 

H. CAVEATS AND CONCLUSIONS 

his report has provided a review of the recent literature on employment and training 

programs and policies. Because of the great interest in employment and training, the 

literature is too vast for our review to be fully inclusive. We have given priority to 

studies dealing with ETA programs, demonstrations, and pilots over programs funded by 

other government agencies at the federal or state levels, by foundations, and by other 

parties. Thus, this review does not cover studies of programs operated by the Department 

of Health and Human Services, the Department of Housing and Urban Development, the 

Equal Employment Opportunity Commission, the Office of Federal Contract Compliance 

Programs, and the Veterans Administration. We have also largely skipped literature on 

employment and training programs in other nations. In addition, we have given priority 

to studies dealing with programs serving individuals and employers over research on the 

labor market or the economic situation of particular groups. 

Primarily because of time and space limitations, we have also omitted studies on 

programs providing financial incentives to workers and employers. These programs 

include the minimum wage, “living wage” programs, targeted tax credits for employers 

such as the work opportunity tax credit, wage subsidies for workers such as the earned 

income tax credit, and programs to encourage investment in education and training, such 

as Pell grants, student loan programs, and education tax credits. A substantial literature 

exists on these programs and some of them appear to be quite effective. 

Finally, we have not reviewed research on certain topics because there is no recent 

literature that we could identify. For example, we could find no overall impact 

evaluations of the JTPA dislocated worker program, we could find no evaluations of 

T 
60 

recent pilots or demonstrations dealing with adult ex-offenders, and we could not find 

any recent evaluations of the migrant and seasonal farmworker program. 

I. RECOMMENDATIONS FOR FUTURE RESEARCH 

n this section we summarize our suggestions for future research to be sponsored by the 

Employment and Training Administration. We have restricted our recommendations 

to the general topics covered by this report. 

The Workforce Investment Act 

A number of studies have already been conducted or are underway to analyze the 

implementation of WIA. In June 2002, the Department of Labor funded the Rockefeller 

Institute of Government to study issues of importance for reauthorization. What has not 

been conducted is an impact evaluation of the WIA program. Given the time required to 

design and mount such as effort, it is not feasible to conduct an impact evaluation for 

WIA (before reauthorization), but the lesson is that once DOL understands what the 

program will look like after reauthorization, it is important to start planning for an impact 

evaluation quickly. 

Important management and implementation issues that should be studied if they are not 

adequately covered by current studies include an examination of the overlap between 

WIA and the employment service for labor exchange/core services; the best ways to 

manage and allocate costs for the One-Stop centers; an assessment of self-directed job 

search and other non-assisted core services to determine if customers most in need are 

adequately served; an evaluation of alternative assessment tools, preferably using random 

assignment; an evaluation of how well diagnostic tools correspond to labor market 

information (including O*NET39) produced by the federal and state governments; and an 

assessment of exemplary job retention and advancement strategies used by local 

workforce investment boards. 

In recent years statistical models have been developed to help identify customers most in 

need of services and the services most likely to be of benefit to customers. The most 

widely known results of this approach are the profiling studies developed to predict 

which unemployment insurance recipients were most likely to exhaust benefits, but the 

FDSS extended the approach to identifying promising service strategies and assisting 

customers make better choices. Important research in this area includes pilots to extend 

the approach to more local areas and evaluations to assess if and how such “expert 

systems” should be mixed with staff judgment. 

Several studies touted the use of a “sectoral strategy” for focusing training programs on a 

limited number of industries. Although the concept has some appeal, the evidence to date 

is more anecdotal than rigorous. Although there is some evidence on the value of 

customized training, these evaluations also lack rigor. The Department of Labor could 

 

39 Occupational Information Network 

I 
61 

improve our knowledge of these approaches by mounting pilot projects with strong 

evaluations attached to them. 

Performance Management 

Recent literature on performance management has surfaced some important issues that 

require further research. First, DOL needs to carefully assess what the objectives are that 

it is trying to promote—is net impact the only goal, or is equity of service also important? 

Second, there is increasing interest in including one or more measures of the return on 

investment (ROI) as a performance measure for ETA programs. Research is needed to 

clarify what costs and benefits should ideally be included in a ROI measure and what 

compromises must be made to develop a feasible measure. Third, the follow-up period 

has been extended significantly since the inception of performance measures for JTPA. 

Research should be conducted to determine what the optimal follow-up period is for 

performance management purposes where one must balance the need for timeliness 

against a desire for long-term measures. Fourth, performance management is important 

for low-cost services such as core services under WIA, but research is needed to measure 

the impacts of these services with tolerable error levels. Fifth, the abandonment of 

adjustments for participant characteristics and local economic conditions and setting 

standards through “negotiations” rather than use of objective measures has been one of 

the most unpopular, if not the most unpopular, aspects of WIA. ETA should begin the 

research needed to reintroduce regression models for performance standards adjustments 

immediately. Finally, several studies have suggested the importance of thinking of 

workforce development as a system; this calls for the development of system measures 

that go beyond individual programs. Research should be conducted to assess the system 

measures already in place in some states and suggest how these measures could be 

improved. 

The Employment Service and WIA Core Services 

Although recent evaluations of the employment service have used creative approaches to 

attempt to measure the impact of labor exchange services, the assumptions underlying 

such approaches need to be tested if possible. Although DOL attorneys once interpreted 

random assignment to be illegal under the Wagner-Peyser Act, this issue should be 

revisited, as random assignment is the best approach for avoiding untested strong 

assumptions. 

Past evaluations have focused on the impact of the employment service on those who 

receive services. The most recent evaluation of the employment service included 

simulations to better understand if those placed by the employment service were simply 

displacing other workers, and actual empirical studies would be useful to avoid the need 

for relying on simulations. Finally, by focusing only on workers, past evaluations have 

missed the effects of the employment service on employers. Future evaluations of the 

employment service should attempt to deal with this admittedly difficult issue. 

Unemployment Insurance 
62 

Much past research on unemployment insurance has focused on getting the benefit level 

“right”—high enough to adequately alleviate hardship but low enough to control work 

disincentives. A similar tension currently exists in recent operational UI initiatives. Call 

centers have been established to streamline the application for and receipt of UI benefits, 

but they reduce the ease of obtaining services from the employment service. At the same 

time, the Worker Profiling and Reemployment Services has made it more difficult for 

some unemployed workers to receive benefits while providing strong incentives to 

receive services. A general question for future research is whether the attempt to increase 

operational simplicity and efficiency for UI recipients has made recipients less likely to 

receive services that might help them find employment more quickly and reduce benefit 

duration. The complementary question is whether WPRS has discouraged some 

unemployed workers from taking advantage of UI, subverting the system’s goal of 

providing assistance to the unemployed. A benefit-cost analysis on the use of call centers 

may be appropriate at this point. Other high priority research questions on UI include: 1) 

Why has unemployment duration increased in the 1990s? 2) What are the costs and 

benefits of permitting work and UI to be combined? How many people do both at once? 

3) How do particular aspects of UI rules (non-monetary rules, separation, etc.) affect 

recipiency? 4) Are the current monetary eligibility requirements too stringent and result 

in the denial of benefits to many former welfare recipients and other low-wage workers? 

If so, what is an appropriate way to restructure the eligibility requirements and what 

would the cost be for the increased eligibility? 

Dislocated Workers 

Perhaps the most important research issue on dislocated workers is determining the 

impact of employment and training programs for this target group. An interesting 

nonexperimental evaluation of the TAA program was conducted a number of years ago 

and several single-site evaluations have been conducted, but a major impact evaluation of 

dislocated worker programs under JTPA or WIA has never been conducted. If possible, 

such an evaluation should use random assignment and seek to determine which types of 

workers benefit from training rather than just job search assistance. In addition, research 

would be useful to identify the advantages and disadvantages of having a variety of 

dislocated worker programs; the study could seek to develop ways to reduce the number 

of programs but retain any desired distinctions in services or cash benefits due to the 

circumstances of the layoff. 

Programs for Welfare Recipients 

Welfare-to-work programs have been funded through TANF and through the Welfare-toWork (WtW) Grants Program administered by the Department of Labor (which is 

phasing out). Research should be conducted to ascertain whether the phase out of the 

WtW Grants Program is creating a gap in services to current and former welfare 

recipients. A point of much contention has been whether labor force attachment (work 

first) models are more effective at increasing the employment and earnings of former 

recipients than models focused on human capital development. In spite of a large number 
63 

of research studies, this is not yet a settled question. One of the criticisms leveled at the 

labor attachment approach in the late 1990s was that a subsequent economic downturn 

would leave former recipients out of work and without increased skills. The current 

unfortunate state of the nation’s economy allows researchers to compare the fortunes of 

former recipients who went through the two types of programs in order to further clarify 

the strengths and weaknesses of the two approaches. The state of the economy also 

underscores the importance of job retention and advancement for former recipients. A 

demonstration to replicate Portland, Oregon’s Steps to Success program might be in 

order, as this program has performed well in terms of both retention and earnings gains. 

The Department of Health and Human Services will continue to produce more research 

on these questions. 

Programs for Youth 

The National Job Corps Study has shown that Job Corps continues to be an effective 

program. One of the crucial assumptions involved in its benefit-cost analysis is that the 

Year 4 earnings impact would continue unabated over the working lifetimes of treatment 

and control group members. DOL will extend the impact evaluation by examining the 

Social Security records of study participants subsequent to Year 4. The General 

Accounting Office successfully used this methodology to extend the National JTPA 

Study. While the Job Corps remains a relatively expensive program, its residential 

component may be the key to its success, confounding efforts to create effective but less 

expensive youth programs. Attempts to develop a profiling methodology to forecast who 

is most at risk of dropping out of Job Corps should be continued, given the increase in 

resource use efficiency that could result from successful profiling and a reduction in 

attrition. 

Apart from Job Corps, little solid evidence of effective youth programs exists. The 

National JTPA Study found that JTPA had zero impact on out-of-school youth, and 

neither Youth Fair Chance nor the Youth Offenders Demonstration Project have had 

promising results. Rather than attempting another large-scale demonstration of a new 

concept, it might be more fruitful to conduct rigorous research (including randomized 

experiments) on some of the approaches profiled in Donna Walker James’ two 

compendiums of youth programs. Many of these programs have had a chance to get up 

to speed and move past initial implementation difficulties, making them ready for welldesigned impact evaluations. 

School-to-Work was one of the major education reform waves of the 1990s. Funding 

under the School-to-Work Opportunities Act has now ended and it is time now to assess 

the results of this reform effort. Specifically, research should address what lasting 

institutional changes have occurred because of School-to-Work and what are the impacts 

on youth that are due to the implementation of School-to-Work principles. 

Other Target Groups 
64 

Research and pilot projects will be useful to help refine and develop policies and 

programs for specific target groups. Adult ex-offenders were once the subject of a 

number of innovative interventions in the 1970s, involving treatments such as training 

opportunities upon release, cash payments similar to unemployment insurance, and 

subsidized jobs for those who could not find positions. Some of the programs were 

promising, but all work along these lines ceased in the early 1980s, so we still know little 

about what to do for ex-offenders about to re-enter society. 

Although ETA operated some interesting employment and training programs for the 

homeless in the 1990s, a decision was made to mainstream the homeless into JTPA 

programs rather than maintain separate programs for them. Research is needed to see if 

the strategy has worked and if homeless individuals are receiving appropriate 

employment and training services from WIA or HUD-sponsored efforts. 

Finally, we note that a systematic effort has never been made to see if services to 

individuals with disabilities have been adequate in employment and training programs. 

Research is needed to determine if individuals with various types of disabilities are 

enrolled in proportion to their need and/or prevalence in the population, whether the 

programs accommodate their disabilities adequately, and what the impacts of these 

programs have been for individuals with disabilities.