1330 Broadway, Suite 1426

Oakland, CA 94612

Tel: (510) 763-1499

Fax: (510) 763-1599

www.spra.com

• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •

S O CIA L P O L ICY R ESEA R C H

A S S O CIATE S

Prepared for:

The U.S. Department of Labor

Employment and Training 

Administration

200 Constitution Ave, NW

Room N-5637

Washington, DC 20210

Under a subcontract from 

Mathematica Policy Research, 

Inc.

N-7731-9-00-87-30

Project No. 1132

An Evaluation of the 

Individual Training Account/Eligible 

Training Provider Demonstration

Final Report

December 2004

Prepared by:

Social Policy Research Associates

Ron D’Amico

Jeffrey Salzman

Under the Direction of:

Mathematica Policy Research, Inc.

Paul Decker, Project Director
Prepared for:

The U.S. Department of Labor

Employment and Training 

Administration

200 Constitution Ave, NW

Room N-5637

Washington, DC 20210

Under a subcontract from 

Mathematica Policy Research, Inc.

N-7731-9-00-87-30

Project No. 1132

An Evaluation of the 

Individual Training Account/Eligible 

Training Provider Demonstration

Final Report

December 2004

Prepared by:

Social Policy Research Associates

Ron D’Amico

Jeffrey Salzman

Under the Direction of:

Mathematica Policy Research, Inc.

Paul Decker, Project Director
ABSTRACT

The Workforce Investment Act of 1998 requires that, with certain exceptions, 

training services be delivered through the use of Individual Training Accounts (ITAs), 

which participants can use to procure the training of their choice, so long as the training 

program is on a state’s eligible training provider (ETP) list. In March 2000, the U.S. 

Department of Labor made grant awards to thirteen states and local areas as part of the 

ITA/ETP Demonstration, to provide support for ITA and ETP system development and 

encourage innovative approaches and practices. This report presents findings from the 

evaluation of these grantees’ efforts. As such, it describes the grantees’ 

accomplishments with their grant funds, the ITA policies and practices they formulated, 

how ETP lists were assembled, and what information was available about eligible 

programs that customers could use to make their training choices.
This page intentionally left blank.

Insert blank page here when making double-sided copies
i

CONTENTS

EXECUTIVE SUMMARY ........................................................................ ES-1

I. INTRODUCTION .............................................................................. I-1

INDIVIDUAL TRAINING ACCOUNTS IN THE CONTEXT OF WIA ....................................... I-1

Services Available to Adults and Dislocated Workers.............................................. I-2

Role of Individual Training Accounts and Training Providers.................................... I-4

CONCEPTUAL MODEL FOR THE EVALUATION ........................................................... I-6

The Prelude to Training.................................................................................. I-7

Training Content, Duration, and Quality ............................................................. I-8

GRANTEES SELECTED FOR THE DEMONSTRATION ...................................................... I-9

RESEARCH DESIGN AND DATA COLLECTION ...........................................................I-11

II. GRANT PLANS AND IMPLEMENTATION PROGRESS .............................. II-1

THE GRANTEES’ PRIOR EXPERIENCE AND READINESS ................................................ II-1

Prior Experience with Using Vouchers .............................................................. II-1

Overall WIA Readiness ................................................................................. II-5

GRANT PLANS AND GRANT IMPLEMENTATION......................................................... II-7

Grant Plans and Implementation Progress ........................................................... II-7

Sustainable Elements and Replicable Innovations .................................................II-15

SUMMARY ....................................................................................................II-22

III. CUSTOMERS’ USE OF THE ITA........................................................III-1

EMPHASIS ON TRAINING AND ITAS...................................................................... III-2

Emphasis on Training in General .................................................................... III-2

Use of Pre-Vocational Services....................................................................... III-7

ITA versus Non-ITA Training ........................................................................ III-8

ESTABLISHING CUSTOMER ACCESS TO ITAS........................................................... III-9

Content of Core and Intensive Services ............................................................III-10

Customer Flow through Pre-Training Services...................................................III-13

Establishing Eligibility for Training ................................................................III-15

MAKING ITA DECISIONS..................................................................................III-18

Types of Customer Choice ...........................................................................III-18

Processes Used in Making Choices .................................................................III-22
ii

STAFF TRAINING AND ORGANIZATION TO SUPPORT DECISION-MAKING ........................III-25

LIMITS ON CUSTOMER CHOICE THROUGH LOCAL POLICIES .......................................III-27

Dollar and Time Limits ...............................................................................III-27

Permitted Uses for the ITA...........................................................................III-29

Demand Occupations ..................................................................................III-30

Residency Preferences.................................................................................III-30

SERVICES PROVIDED TO TRAINEES DURING AND AFTER TRAINING ..............................III-31

Services during Training ..............................................................................III-31

Placement Services.....................................................................................III-33

COORDINATING THE ITA WITH OTHER SERVICES....................................................III-34

SUMMARY ...................................................................................................III-37

ATTACHMENT TO CHAPTER III: CUSTOMER PROFILE ...............................................III-39

IV. DEVELOPING THE ELIGIBLE TRAINING PROVIDER LIST ................... IV-1

WHAT COUNTS AS A “PROGRAM”?......................................................................IV-2

GENERAL APPLICATION AND INITIAL ELIGIBILITY PROCEDURES ..................................IV-5

Marketing Efforts by States and Local Areas ......................................................IV-5

Application Procedures.................................................................................IV-7

Information Required for Application ............................................................. IV-10

Performance and Other Requirements for Initial Eligibility................................... IV-13

State, Local, and Regional Approval Systems................................................... IV-16

SUBSEQUENT ELIGIBILITY PROCEDURES AND REQUIREMENTS................................... IV-20

When Subsequent Eligibility Begins ............................................................... IV-21

Performance Requirements as Part of Subsequent Eligibility ................................. IV-23

Local Area Exceptions and Options................................................................ IV-30

Additional Problems and Concerns ................................................................ IV-31

DATA CAPTURE AND DEFINITIONAL ISSUES ......................................................... IV-33

Systems for Data Capture ............................................................................ IV-33

Defining Key Terms .................................................................................. IV-38

SUMMARY .................................................................................................. IV-43

V. NATURE AND QUALITY OF THE CONSUMER REPORT 

SYSTEM........................................................................................ V-1

DESIGN OF INFORMATION SYSTEMS ......................................................................V-1

Vision for the Consumer Report System.............................................................V-1
iii

Formats, Searches, and Information Displays ......................................................V-4

VENDOR CHOICES AND THE QUALITY AND COMPLETENESS OF 

INFORMATION.................................................................................................V-7

Composition of ETP Lists ..............................................................................V-8

Quality and Completeness of Performance Data ................................................. V-10

SUMMARY ................................................................................................... V-14

ATTACHMENT TO CHAPTER V: CRS PROFILES....................................................... V-15

VI. VENDORS: INVOLVEMENT AND PERSPECTIVES ............................... VI-1

VENDOR NETWORKS........................................................................................VI-1

Types of Vendors........................................................................................VI-1

Number of Vendors.....................................................................................VI-6

VENDORS’ RESPONSES TO CUSTOMERS .................................................................VI-8

Recruiting.................................................................................................VI-8

Structuring Programs ................................................................................. VI-11

VENDORS’ RESPONSES TO THE ITA SYSTEM ......................................................... VI-12

Application Process ................................................................................... VI-12

Subsequent Eligibility ................................................................................ VI-13

Administrative Relationships with Local Areas.................................................. VI-16

RESHAPING THE TRAINING MARKETPLACE FROM JTPA........................................... VI-17

Market Structure and Competition ................................................................. VI-17

Tuition Costs ........................................................................................... VI-18

SUMMARY .................................................................................................. VI-19

VII. SUMMARY OBSERVATIONS AND CONCLUSIONS .............................. VII-1
This page intentionally left blank.

Insert blank page here when making double-sided copies
LIST OF TABLES AND EXHIBITS

EXHIBIT I-1: List of Grantees and Local Sites Visited .................................I-12

EXHIBIT II-1: Grant Objectives and Progress to Date .................................. II-9

EXHIBIT II-2: Sustainable Elements from Grant Activity .............................II-19

EXHIBIT III-1: Emphasis on ITA and Other Traning................................... III-4

EXHIBIT III-2: Definitions of Self-Sufficiency for Selected Local Areas............III-17

EXHIBIT III-3: Information and Tools to Make Training Decisions .................III-19

EXHIBIT III-4: Local Limits on Amount or Duration of ITA.........................III-28

TABLE IV-1: Target Start Dates for Subsequent Eligibility ..........................IV-21

TABLE IV-2: Minimum Performance Thresholds for Subsequent 

 Eligibility...................................................................IV-25

TABLE IV-3: Planned or Actual Procedures for Data Capture .......................IV-34

TABLE IV-4: Definitions of Key Terms ................................................IV-40

EXHIBIT V-1: Characteristics of Consumer Report Systems ........................... V-6

EXHIBIT V-2: Composition of Eligible Training Provider Lists ....................... V-9

EXHIBIT V-3: States’ ETP Listings for Five Selected Occupations...................V-12
This page intentionally left blank.

Insert blank page here when making double-sided copies
ES-1

EXECUTIVE SUMMARY

This report has been prepared as part of a contract awarded by the U.S. 

Department of Labor (DOL) to conduct an Evaluation of the Individual Training 

Account/Eligible Training Provider (ITA/ETP) Demonstration. This summary reflects 

the findings reported in the Final Report for the evaluation. As such, it describes what 

grantees accomplished with their grant funds, the ITA policies and practices they 

formulated, how ETP lists were assembled, what information was available in 

Consumer Report Systems, and how training providers responded to the ETP system.

BACKGROUND

The Workforce Investment Act (WIA) of 1998 brought about substantial changes 

in services provided to persons seeking employment and training assistance. An 

important element of WIA is the requirement that training services be provided, with 

certain limited exceptions, through individual training accounts (ITAs), which can be 

thought of as a voucher that customers can use to pay for training of their choice, so 

long as the training program is on an approved list (the eligible training provider list). 

To help customers make prudent training choices, eligible programs must meet 

minimum levels of performance established by the states, and information about them 

(e.g., costs of the training, its duration, and the employment and other outcomes 

achieved by prior cohorts of trainees, among other things) is to be assembled in a 

Consumer Report System (CRS) maintained by the state and distributed throughout the 

state’s One-Stop system. The establishment of ITAs and eligible training provider 

(ETP) lists is intended to empower customers, while promoting accountability among 

states, local areas, and service providers in meeting customers’ needs.

In the summer of 1999, DOL issued a Solicitation for Grant Applications for the 

Individual Training Account/Eligible Training Provider (ITA/ETP) Demonstration. 

This announcement emphasized that DOL was interested in identifying “a national 

group of vanguard sites” who were committed to implementing ITAs and establishing 

ETP systems “informed by best practice and insight from the field.” Chief goals of the 

demonstration as outlined by DOL include support for system-building at the state and 

local levels, rigorous testing of several key models or approaches to the establishment 

of an eligible training provider process and ITA payment system, identification of key 

components of effective ITA implementation, and the development of a learning 
ES-2

network for information sharing, both across demonstration sites and to the larger 

employment and training system. Each grantee was to receive an amount of up to 

$500,000 for a grant period that was to last 18 months. 

In March of 2000, DOL announced that it had selected thirteen grantees to 

participate in the demonstration project, located in as many different states. Six of 

these grantees are local workforce investment areas that applied individually or on 

behalf of a group of local areas. The other seven grantees are states. Of these seven 

states, four are collaborating with some subset of the state’s local areas, while the other 

three states are developing statewide systems and strategies.

The evaluation of the ITA/ETP Demonstration, being undertaken by Mathematica 

Policy Research (MPR) and Social Policy Research Associates (SPR), consisted of a 

process study that entailed two rounds of multi-day site visits to each of the thirteen 

grantees and their local-area partners, including visits to 28 separate local workforce 

investment areas over both rounds. The first round occurred in the summer and fall of 

2000, and the second round occurred in the fall of 2001 through early 2002. Each site 

visit involved interviews at both the state and local levels, regardless of whether the 

grantee was itself a state or local area. The Final Report for the evaluation, on which 

this summary is based, draws on both rounds of data collection and thus represents a 

comprehensive accounting of the evaluation’s findings.

Also to gain an understanding of ITA issues, DOL has funded a separate

experimental evaluation in which the efficacy of three different training regimens is 

being compared. These three vary according to (a) the maximum amount of the ITA 

that can be awarded a customer for training and (b) how much guidance and direction 

customers must receive from their case managers before their training choice will be 

approved. Interim results from the ITA experiment, to be available in late 2004, will 

provide important additional information to complement the findings reported here, 

with final results due in late 2005.

GRANT PLANS AND IMPLEMENTATION OF THE ITA/ETP DEMONSTRATION

With respect to the ITA/ETP Demonstration, most grantees were not starting 

their ITA system development from scratch when their demonstration grants were 

awarded. In fact, almost all had moved sharply away from the exclusive use of 

contract training in the waning years of JTPA and towards individual referral methods, 

and most claimed previous experience with using vouchers for training, either as a 
ES-3

grantee under the former Career Management Account demonstration or as part of 

similar state-funded pilot programs. A few had also developed lists of local training 

providers that were viewed as preferred, and they had assembled some basic 

performance information about them. One-Stop implementation grants that states had 

received from DOL during the mid- to late-1990s also helped establish the 

infrastructure that grantees needed to serve adult and dislocated worker customers in a 

WIA framework. In a sense, then, the ITA and ETP requirements mandated by WIA 

were not entirely new.

This prior progress notwithstanding, the transition to the use of ITAs and the 

development of ETP lists entailed substantial additional work that funds from the 

ITA/ETP Demonstration grant helped support. When we examined their grant plans, 

grantees identified a wide variety of ways in which they would be using their grant 

funds to help facilitate these changes. Their grant objectives ranged from the very 

broad activities associated with WIA implementation that all states and local areas 

needed to carry out (e.g., develop ITA policies, build a Consumer Report System), to 

quite specific features of these systems (e.g., develop a code of ethics for vendors). 

Grant objectives can also be categorized with respect to their major area of focus. 

The most common cluster of objectives related to efforts to build state Consumer 

Report Systems (CRS), including assembling performance and other data about vendors 

and developing the computer systems needed to make this information readily 

accessible to customers. A few grantees additionally were interested in building a local 

CRS, either as a supplement to the state’s emerging system or as a temporary substitute 

to make up for the fact that a functioning state system was not expected imminently. 

Capacity building was another key objective specified by many grantees, but the 

specifics varied. One grantee wanted to hire experts in assessment and counseling to 

work with customers and coach case managers; another wanted to develop a curriculum 

for a peer-managed workshop; others wanted to develop and deliver training workshops 

for staff or develop computer modules that staff could access as a resource. The third 

largest category of grant objectives related to developing or testing ITA policies; for 

example, one grantee was trying to facilitate coordination and joint policy development 

with adjacent local areas. Finally, some grantees were using their grant funds for a 

variety of other purposes, such as to develop fiscal or tracking software, automate the 

training provider application process, or develop alternative ITA payment mechanisms, 

among other things.
ES-4

As the above description suggests, grantees typically used their funds as part of 

the broader system-wide transformation of their workforce systems associated with the 

enactment of the WIA legislation, including at the level of both state-wide system 

design and local-level implementation. In this context, grantees found their grant funds 

very helpful in supporting this change; concretely, for many grantees the grants meant 

the difference in being able to hire an additional staff member or two specifically 

devoted to some facet of ITA or ETP issues (e.g., reviewing vendors’ applications for 

the ETP list, developing software, etc.), and thereby enabled grantees to devote much 

more attention to particular implementation challenges than might have been possible 

otherwise. Moreover, as another tangible benefit of the demonstration, grantees greatly 

benefited from the exchange of ideas and information among each other, and within 

their states and regions. 

In some cases, we can also point to specific innovations that grant funds 

supported that might not have been attempted otherwise. Although not many of them 

were fully implemented at the time the grants ended, these local innovations include:

• Strengthening support for customers in making training choices by 

hiring experts in the assessment of occupational skills and interests, 

developing local Consumer Report Systems, managing peer-to-peer 

counseling workshops, and developing online customer messaging 

boards. 

• Increasing training choices, such as by incubating Individual Learning 

Accounts (a broader variant of ITAs in which employers contribute 

funds for their employees’ use) and encouraging employers to add inhouse training to the provider lists.

• Refining software, such as for systems to automate the vendor 

application process, track ITA payments and obligations, and allow 

vendors to submit comments to case managers regarding students’ 

attendance and academic progress.

• Experimenting with alternative ITA payment mechanisms, such as 

“smart-cards” and checkbooks.

Many of these innovations will merit further scrutiny when they are fully functioning.

CUSTOMERS’ USE OF ITAS

All of the grantees had embraced the ITA model for providing training services 

and generally seemed enthusiastic about its possibilities for empowering customers. At 

the same time, they varied in the emphasis they placed on training services, as opposed 
ES-5

to core and intensive services, in the WIA context. Thus, some states and local areas 

sought to continue the high levels of funding for training that they had experienced 

under the Job Training Partnership Act (JTPA), while others adopted a strategy of 

emphasizing core and intensive services, and as a consequence cut back on investments 

in training considerably. 

The number of local areas that fell into these two categories changed from the 

first-round to second-round site visits, reflecting an evolving policy context. In the first 

round of site visits, we found a greater number of local areas that were limiting funding 

for training because they perceived WIA to embody a strong “work-first” philosophy. 

However, by the second round of site visits, the interest in work first and a 

corresponding de-emphasis on training had clearly waned. One reason for the shift was 

that states and local areas had come to a greater sense that a focus on meeting 

customers’ needs with whatever services were deemed appropriate and necessary was 

wholly consistent with WIA. As other reasons for the change, some sites were 

generally more reticent to use funds for training early in WIA implementation, because 

they had fears of funding shortfalls, encountered a strong economy that made job 

placements relatively easy to obtain, and were to some degree unsure of when and for 

whom training could be authorized. By the second round of site visits, these concerns 

had lessened considerably.

Given the overall emphasis they placed on training, sites also needed to decide the 

extent to which they would use ITAs as opposed to non-ITA training alternatives, such 

as contract training and customized training. We found that nearly all of the 28 local 

areas we visited were planning on using ITAs predominantly—and in many cases 

exclusively—for their training for adults and dislocated workers. However, three local 

areas were expecting to make heavy use of non-ITA alternatives. Of these, some cited 

the advantages of contract training under some circumstances, such as in meeting the 

training needs of customers that were hard to serve (e.g., those with limited Englishlanguage skills). Others made considerable use of customized training with employers, 

noting that it was virtually assured of leading to job placements for training customers, 

typically provided them with an income stream while they underwent training, and 

advanced the areas’ economic development objectives very directly.

As part of WIA implementation, all sites also needed to develop policies to guide 

the way that customers move through core and intensive services before reaching 

training. No local area established fixed durations during which customers needed to 
ES-6

stay at one service level before advancing to the next one. Nonetheless, on average it 

took customers a few weeks, up to several months, to complete at least one core and 

one intensive service, make a decision to train, and select a training program. The 

average duration varied across sites depending on the rhythm of case management 

appointments and the extensiveness of the assessment and counseling that sites normally 

provided to their customers. Within-site variation in durations was even more 

pronounced and was due to a number of factors. Among the most important of these 

was the customer’s own diligence in conducting the necessary research on alternative 

training programs and scheduling and keeping appointments with the case manager. 

Additionally, case managers adopted a flexible approach to dealing with their 

customers, so that durations varied depending on customers’ unique needs. Thus, 

customers who were evidently in need of training and who had clear expectations for 

training that were recognized by One-Stop staff as being reasonable tended to navigate 

the early service levels quickly. Overall, then, a central tenet of WIA that services 

should be customer driven and based on the individual’s own needs appears to have 

been realized.

The customer focus is evident as well in the process that sites used to help 

customers make training choices. We characterize the approach that nearly all sites 

used as corresponding to an “informed choice” model. According to this model, OneStop centers ensure that those authorized for training receive ample information, 

guidance, and assistance, so that they can make prudent choices for themselves. 

Operationally, it meant that customers would be required to undertake an assessment of 

their skills and abilities, and engage in labor market and other research, before an ITA 

would be issued. Similarly, most sites required customers to conduct field research, 

such as by visiting several vendors and interviewing former trainees and employers who 

hire in the career area in which the customer wants to undertake training. Other sites 

required that customers attend workshops that were either given by case managers or 

were peer-managed. Sometimes also customers needed to submit a formal application, 

in which they identify the training field and vendor they have chosen and justify their 

decision on the basis of assessment results and the research they have conducted. 

Because customers followed these steps, case managers felt that customers largely came 

to identify appropriate training choices on their own. Meanwhile, front-line staff 

played a key role in serving as “guides” or “facilitators,” striking what seemed to be an 

appropriate balance between lending the benefit of their expertise while not being 

overly directive. Given the predominance of the informed choice model, our 
ES-7

interviews and observations lead us to the conclusion that customers are effectively the 

decision-makers almost always.

Their choice, however, is subject to certain limitations established by state and 

local policy. For example, in keeping with the WIA legislation, training can only be 

funded if it is for an occupation in demand. Some local areas meet this requirement by 

drawing on lists developed by the state’s labor market information research unit, other 

sites used locally-developed lists, and many others relied on the judgement of the case 

managers, who were recognized as being well informed about the local labor market. 

Other restrictions related to dollar or time limits. Nearly all of the sites set a 

dollar cap on the amount of the ITA that would be funded, but these varied widely 

across sites, from a low of $1,700 to a high of $10,000, with an average of about 

$5,000. Nearly all sites also had time limits on the duration of training that they would 

support, which they usually set at two years. Tuition and fees, as well as books, 

uniforms, and equipment, would normally be funded by the ITA, and supportive 

services would be provided from a separate pot of money. Overwhelmingly, these 

policies were imposed by local areas; although states were allowed to impose limits of 

their own, they generally felt that these decisions should be left as a local prerogative.

Even with these limits, sites could be investing a substantial amount on each 

trainee. For this reason, and in keeping with WIA’s requirements, trainees were 

always expected to seek out other sources of funding, such as by applying for a Pell 

grant. Typically, the amounts they received from these sources were applied to the cost 

of the training, with the ITA paying any balance due. Some sites, however, used the 

ITA to pay for the tuition and allowed customers to keep Pell funds for their living 

expenses. All sites required customers to apply for a Pell grant, but coordination with 

other funding sources, such as the trade programs and vocational rehabilitation, was 

more limited.

As a way of ensuring that their training dollars represented worthwhile 

investments, and because performance accountability is so central to WIA, sites have an 

interest in doing what they can to ensure that their ITA holders complete the training 

and obtain a well-paying job afterwards. Thus, all sites made provisions for keeping 

abreast of the trainee’s progress and attempted to address problems as they arose. 

Some sites were more proactive than others were, but virtually all maintained at least 

monthly contact with WIA customers in training. Sites also varied in whether they 
ES-8

primarily relied on the vendors to assist customers with post-training job placements, or 

assumed that burden themselves.

Given that their performance is publicly displayed as part of the Consumer Report 

System, vendors also have a clear stake in the trainee’s success, and thus they too 

played a part in monitoring the customer’s progress. Along these lines, proprietary 

schools—at least those that we visited as part of this study—seemed very attentive to 

students’ needs for extra assistance, and were aggressive in helping their students find 

jobs once the training was completed. By contrast, although community colleges 

offered counseling and placement services, they were typically less proactive in their 

approach. 

DEVELOPING THE ETP LIST

Consistent with the WIA legislation, ITAs can be redeemed only by vendors 

whose programs are “eligible”—certified by states and local workforce areas as 

meeting acceptable levels of performance.

Although called an eligible training provider list, the ETP list should more 

properly be thought of as a list of eligible training programs, since it is individual 

programs and not providers that need to be approved. In keeping with this, vendors 

need to apply for eligibility for each of the training programs for which they are 

seeking ITA eligibility. This requirement poses the question of what should count as a 

training program for eligibility purposes. In one difference across states, some require 

the same vendor to submit separate applications for programs that were offered at 

separate locations (e.g., branch campuses), even when the curriculum was identical. 

Beyond that, most states we studied allow vendors to self-define programs, so long as 

they do so within the confines of guidance issued by DOL that defines programs as 

courses that, upon successful completion, lead to skills or competencies needed for a 

job. A few states adopt the narrower restriction that the program must also lead to a 

degree or certificate, on the grounds that doing so promotes the state’s ability to meet 

the credentialing rate, one of the core indicators of performance on which states are 

judged. To adopt this approach without unduly limiting customer choice, some states 

encourage vendors to develop a certificate specifically for course offerings that would 

otherwise not result in one. 

Regardless of the definitions they used, sites were generally eager to widely 

publicize the ETP application process and have as many vendors apply for initial 
ES-9

eligibility as possible. Thus, although their relative roles varied across the states we 

studied, both states and local areas took active roles in marketing to vendors, such as by 

sending them application packets, communicating with former JTPA providers, or 

holding informational sessions in the community. Almost all states automated the 

application process, both to make it easier for vendors to apply and also to expedite the 

state’s and local areas’ roles in processing applications and entering the data into an 

electronic ETP listing. Those states without an electronic application found the process 

substantially more burdensome. 

All of the states require the same basic vendor and program information on the 

application form, including the institution’s name and address, the name of the program 

for which eligibility is being sought, whether certificates or degrees are awarded, hours 

of instruction, credits to be earned, and costs. Some application forms are very simple 

and streamlined, asking for just this basic information. Other states have somewhat 

more elaborate information requests that include other attributes of the institution (e.g., 

whether it provides job search assistance, on-site child care, and counseling services) 

and program (e.g., program schedules, criteria for admission, qualifications of 

instructors, average class sizes, curriculum, and accessibility to public transportation). 

Several states also asked for information on program performance, but only two states 

of the 13 we visited established performance requirements for initial eligibility. The 

others did not do so, because they felt ill equipped to make decisions regarding 

performance benchmarks at such an early stage and because they wanted to minimize 

the burden on vendors at the outset. In general, vendors found the initial application 

process to be easy and straightforward, but some found it to be very burdensome. The 

latter response was more likely in those states with more complex application forms and 

processes.

In keeping with the legislation, the approval process for initial eligibility basically 

worked the same way in all the sites we visited—local areas would first review the 

applications, make a judgement of whether the application should be approved, and 

then pass the application on to the state, along with the local area’s recommendation, 

for the state’s final disposition. One difference was that some states had vendors 

submit their applications directly to the local areas, while other states had vendors 

submit their applications to a central state clearinghouse, which then forwarded the 

applications to the local areas for their review. The latter approach was viewed as 

easier for vendors, in that it standardized the application process.
ES-10

Because states were eager to develop an extensive ETP list and performance 

requirements were not usually imposed at the outset, nearly all applications for initial 

eligibility were approved virtually everywhere. Moreover, in their efforts to help 

vendors through the application process, local areas often made every effort to figure 

out how to fix an application (e.g., by phoning vendors to retrieve missing information 

from application forms), rather than summarily dismiss it. Perhaps because of the extra 

effort that locals put forth, some local-area respondents reported that the review process 

was extremely tedious and time-consuming.

Apart from the amount of effort involved, some local areas were also concerned 

about what might happen if different areas set different local standards for approval. 

Several of the local areas we studied were attempting to develop a regional approach to 

ETP review to eliminate this possibility. To accomplish the same objective, one state 

issued a policy that explicitly prohibited a vendor whose application was denied 

eligibility by one area from resubmitting elsewhere, and another stipulates that a vendor 

can apply only to the local area in which its main administrative offices are located. 

Still another requires vendors to submit to the multiple local areas in which they want 

to provide training services and gives each local area the chance to approve or reject the 

application and display its decision as part of the ETP list. More informally, case 

managers in a number of states admitted that they could not envision authorizing an 

ITA if the customer had selected a vendor that the local area did not deem to be 

acceptable.

Contentions around vendor approval were expected to be much more acute when 

states began subsequent eligibility. In only one state of the thirteen we visited had 

subsequent eligibility begun at the time our second-round site visits were conducted. 

Most other states were planning on starting subsequent eligibility on January 2002, or 

later in that year. Two states, however, asked for a waiver from DOL to delay 

subsequent eligibility for several years more. 

Seven of the eleven states that were intending to start subsequent eligibility by 

2002 had reached decisions about what their performance requirements for vendors’ 

programs would be. Two states set standards on just a few of the seven performance 

measures mentioned in the legislation but not the others. Three states set standards on 

all or most of the seven measures, but expected vendors to meet the standards on just 

some of them (e.g., four of the seven), while two states established standards on all 

seven and expected vendors to meet them all.
ES-11

Not only did the general approach vary across states, but the absolute levels of 

performance that vendors were expected to meet for their programs varied widely as 

well. For example, standards established on the all-student completion rate ranged 

from a low of 25% in one state to a high of 70% in another, and standards on the allstudent wage rate at placement ranged from the federal minimum wage to $7.50 per 

hour or more. In keeping with the legislation, local areas are able to establish 

performance requirements beyond those imposed by the states. For example, some 

states allow local areas to establish higher (but not lower) performance standards, or 

add additional measures. In fact, however, almost none did so. A few states also 

granted locals the discretion to grant waivers to vendors who miss the state’s standards, 

so long as they can justify their decision (e.g., because the vendor serves an especially 

hard-to-serve population). 

Rationales for the decisions that states made with respect to their approaches 

reflected some similar themes, even if they did often lead to very different decisions. 

Among the most common considerations was the states’ effort to strike a balance by 

establishing performance criteria that are rigorous enough to ensure high quality, while 

not establishing them so high that so many vendors will be excluded from the ETP list 

that customer choice will be unduly restricted. Some states also mentioned that they 

were endeavoring to base standards on existing performance requirements for vendors 

(e.g., from other licensing bodies). At the same time, many states voiced the fear that,

whatever choices they made, the eligibility list would contract substantially once 

subsequent eligibility actually began, because of vendors’ reluctance to undertake the 

effort involved in assembling their performance information. 

Partly to address vendors’ concerns in this regard, ten of the thirteen states we 

studied were planning on bearing most of the burden for measuring performance 

outcomes on the vendors’ behalf, through matching with Unemployment Insurance 

wage systems. Thus, vendors submit “seed records” containing students’ Social 

Security Numbers and identifying information about the training program to a state 

agency, who will in turn conduct the UI matching to measure the five of the seven 

performance measures that relate to employment outcomes. Two states are also 

attempting to measure the vendors’ program completion rates as well, at least for public 

training institutions, by conducting matches with state postsecondary enrollment 

databases maintained at the state level. In contrast to the above strategies for data 

capture, three states rely primarily on having vendors self-report data for all or most 
ES-12

measures, which they typically are expected to gather through trainee or employer 

surveys.

Regardless of what approach they are taking, most states gave some attention to 

establishing common definitions of key terms to ensure comparability of the 

performance data that result, including defining who counts as a participant or a 

completer. States greatly vary in the decisions they have made and the extent to which 

they have provided clear and consistent guidance to the vendors in their states. Thus, 

the comparability of performance data across states, and to some degree even within 

states, could be compromised.

COMPLETENESS OF CONSUMER REPORT SYSTEMS

A key element of the training system envisioned by WIA is for there to be strong 

information systems to support customer choice through the Consumer Report System 

(CRS). During the several years of our study, the states’ Consumer Report Systems 

have been evolving dramatically. From nascent or non-existent systems at first, all of 

the thirteen demonstration states now have databases with at least basic information 

about training programs. Of these, all but one are searchable, in that a customer can 

identify search criteria to generate a list of matching programs. Key search criteria 

available in most or all of the states include program or provider name, location (e.g., 

city or zip code), and training occupations. 

Information about the programs available on the CRS typically includes program 

costs and program duration, as well as basic information about the vendor. Some 

systems additionally provide information on the course schedule (e.g., such as whether 

day, evening, or weekend schedules are available), occupational training areas, course 

content, and performance information (for some, but rarely all, of the seven required 

measures). Less common are systems that provide information on entrance 

requirements, additional services offered (e.g., on-site child care, counseling), and 

accessibility (e.g., whether parking is available, access from public transportation), 

among other things. 

With respect to the array of choices available to customers, our Internet searches 

revealed that the absolute size of the ETP lists varies dramatically across the thirteen 

demonstration states, from a low of about 100 vendors and fewer than a thousand 

programs in smaller states, to a high of hundreds of vendors and nearly 10,000 

programs in the larger states. The composition of these lists also varies dramatically. 
ES-13

For example, public educational institutions make up just 10% of all institutions on the 

list in one state, but 45% of them in another. However, public institutions have a much 

greater number of program offerings on average than do vendors of other types. Thus, 

in twelve of the thirteen states we studied, the public educational institutions account for 

at least half of all program offerings, and in one case almost 90%. Put differently, 

private for-profit vendors make up the majority of all vendors on most states’ lists, but 

public educational institutions offer the majority of all eligible programs. Finally, nonprofit organizations and other public agencies constitute a small but still appreciable 

percentage of both vendor and program lists in most states. Overall, these findings 

suggest that states’ eligible training provider lists offer a substantial number of training 

choices and a rich array of offerings by institutions of a variety of different types. In 

other words, customer choice seems well supported by these figures, at least when one 

looks within each state as a whole.

Internet searches were also used to gauge the adequacy of information available in 

Consumer Report Systems to support customers’ choice, this time through searches of 

all 50 states’ Consumer Report Systems (not just the thirteen states represented by 

demonstration grantees). We identified five training fields that, based on the site visits, 

appeared to represent frequently made training choices. We then used the Internet 

searches to identify in each state how many different training programs were available 

in each field and what sorts of performance and other data were available about the 

programs.

Results affirm the notion that ETP lists in most states offer customers ample 

choices. Thus, for most of the training fields we researched, the majority of states 

offer a dozen or more separately listed programs, and some states list more than 50 or 

even 100 training programs in each field. Thus, at least for these common training 

occupations, customers have an array of training programs from which to pick, at least 

within the state as a whole.

The array of information available about these programs on the states’ Consumer 

Report Systems is generally spotty. Encouragingly, most states’ systems provide 

information on the costs and program duration for almost all of the programs that we 

researched. Data are much sparser on performance measures, however, especially the 

WIA-student (as opposed to the all-student) measures. In short, some basic program 

information appears to be available in most states’ systems relating to the cost and 
ES-14

duration of training, but information on performance-related measures is much less 

frequently available.

VENDORS’ REACTION

As the above review suggests, nearly all training vendors certified as eligible for 

ETP purposes are either public institutions (predominantly community colleges) or 

proprietary schools. These two major types of institutions have very different missions 

and define themselves very differently. The public institutions have traditionally filled 

an important role in providing training under JTPA, because of the breadth of their 

offerings and generally low tuition, and it is expected that they will be similarly 

important under WIA. However, many are balking at the eligibility requirements that 

WIA imposes, especially the need to submit performance information about their 

programs. In their view, the low volume of ITA-funded trainees that they can 

anticipate does not warrant the time and expense that such a requirement would entail. 

They also fear that their performance would be unfairly characterized, given the mix of 

customers that many of them serve and their open-enrollment policies. In the face of 

these complaints, some states fear the prospect of losing a substantial number of 

vendors and programs once full subsequent eligibility goes into effect, but are working 

aggressively to ward off this possibility by addressing the public institutions’ concerns 

as best they can. 

By contrast, representatives of the proprietary schools we met characterized their 

institutions as being active in the marketplace and highly performance driven. For their 

own purposes, or to meet other state or federal certification requirements, these schools 

had been accustomed to collecting and reporting performance data and saw no difficulty 

with doing so for ETP purposes. 

Community-based institutions constitute the third major group of potential 

vendors. Many of these organizations depended heavily on providing training to 

economically disadvantaged adults and dislocated workers under JTPA, but they are 

less likely to be successful under an ITA system because of their traditionally narrow 

customer base. Further, they are usually thinly capitalized and are thus likely to have 

difficulty coping with an irregular flow of ITA students. 

Consistent with the principle of customer empowerment, all vendors will need to 

be responsive to the training customer in order to be successful. In recognition of this, 

many vendors try to actively market their services to potential trainees. As a 
ES-15

consequence, reverse referral are fairly common, in that customers come into the OneStop center seeking an ITA because they were referred there directly by a training 

vendor. Quite prevalent under JTPA, reverse referrals seem to be common under WIA 

as well, despite the uncertainty vendors face that a customer referred to a One-Stop 

center will select another vendor from the eligible provider list.

Also as a way of appealing to customers in a competitive marketplace, vendors 

can structure their programs to make sure that they are responsive to customers’ needs, 

such as by improving accessibility, adjusting the timing, location, or duration of 

programs, or adding auxiliary services. Proprietary schools are widely considered to 

be quite responsive in this regard. Although it is less common, community colleges 

also restructured their programs to make them more flexible to ITA holders in some 

cases.

CONCLUSIONS

The evaluation’s two rounds of data collection span a period of tremendous 

growth and maturation of states’ and areas’ ITA and ETP systems. During the first 

round of site visits, in the summer and fall of 2000, ITA policies were just being 

formulated, as sites were still feeling their way and not quite sure to whom they could 

issue an ITA and when. Similarly, although ETP lists had been developed, states had 

barely begun to think about their subsequent eligibility policies, and fully functioning 

Consumer Report Systems were virtually nonexistent. A year later, by contrast, during 

our second round of site visits, these systems had evolved considerably. Thus, sites 

were much more comfortable in authorizing training for their customers, the 

groundwork for subsequent eligibility had been laid (at least in most states), and 

virtually every state had a searchable, web-based Consumer Report System with at least 

basic information about vendors’ programs.

Based on our review of this remarkable trajectory, we can offer some final 

thoughts about remaining questions and potential next steps. We have divided these 

into issues relating to general system development and grant funding, ITA issues, and 

ETP and CRS issues.

General System Development and Grant Funding Issues

1. The development of ITA and ETP systems in the demonstration states 

represented a tremendous mobilization of effort and resources—for example, 

in reviewing and processing vendors’ applications for eligibility, establishing 

procedures to measure vendor performance and track ITA expenditures, and 
ES-16

developing software to display Consumer Report Systems. For this reason, 

the extra resources represented by the demonstration grant funds were, if not 

indispensable, certainly very appreciated and were put to good use 

everywhere. 

2. The grantees endeavored to design their systems such that they would be 

sustainable once grant funds ran out, with minimal support from WIA 

formula allocations. The extent to which they were successful in doing so, 

however, remains unclear. Thus, some investigation of the ongoing costs 

associated with maintaining these systems would be helpful, so that an 

assessment of how costs compare to expected benefits can begin.

3. For the very reason that grantees used much of their funding for critical 

system development, the timing of the awards made this less like a 

demonstration project—in the sense of testing departures from standard 

practice—than it otherwise would have been. Nonetheless, some grantees 

were developing some innovative features of their ITA/ETP systems, such as 

experimenting with alternative ITA payment mechanisms, online customer 

messaging systems, and the like. Few of these were fully functioning at the 

time the demonstration ended, and thus none could be properly assessed. 

Nonetheless, some of them seem to hold substantial promise, and their 

efficacy will be worth exploring further.

4. State and local-area flexibility is a key tenet of WIA, and this principle 

seemed to be clearly on display in the systems that we studied. Thus, 

although there are obvious broad similarities in the systems that are 

developing, states and local areas are making unique decisions regarding key 

features of ITA policies and ETP systems.

5. That flexibility, although clearly embraced by states and local areas as a good 

thing, caused some confusion and uncertainty at early stages of system 

development. Thus, some local areas were unsure exactly what their policies 

and systems should look like, and were eager to compare their experiences 

with those of other states and local areas. Although much of this uncertainty 

has since been resolved, states and local areas are likely to highly value 

forums for the continued exchange of information across the workforce 

development system.

ITA Issues

6. The demonstration grantees, just as the workforce system as a whole, 

realized a substantial drop-off in the number of persons undertaking training 

in PY 2000, WIA’s first year of implementation. This occurred for a variety 

of reasons, including a strong economy that made job opportunities plentiful, 

competing priorities for using scarce WIA funds, and case managers’ 

uncertainty regarding for whom training could and should be authorized. 

However, training levels rebounded sharply upward subsequently, reflective 

of the changing economic climate, one-time expenses in establishing a One-
ES-17

Stop infrastructure that were behind them, and local areas’ greater degree of 

comfort in authorizing training in response to customers’ needs.

7. Most grantees have strongly embraced ITAs as the preferred method of 

delivering training services to adults and dislocated workers under WIA. 

Some, however, point to the strong role that customized training can still 

play in meeting the workforce system’s diverse needs, such as in directly 

meeting an area’s needs for economic development targeted to employers, 

and providing trainees with an income to support themselves through training 

and a nearly guaranteed job afterwards. Similarly, other grantees noted the 

important role that contract training plays in meeting the needs of their hardto-serve customers, for whom an ITA might be less effective. Yet neither of 

these alternatives to the ITA was used very widely in the local sites we 

visited. It is perhaps worth considering in a broad way the proper role for 

customized and contract training in the WIA context, and how these 

alternatives can be developed while still promoting customer choice.

8. As is their prerogative, local areas have established caps on the amount of 

the ITAs they would issue. These vary widely from a low of about $1,700 

per customer in some local areas to as much as $10,000 in others. In setting 

these caps, local areas struggle with the tradeoff between ensuring that a 

diverse array of high quality training choices are available to customers, 

while spreading their available training dollars over as many customers as 

possible. A quantitative analysis of how these different decisions impact the 

choices that customers make, and how the workforce system’s return on 

investment is impacted, would be very informative.

9. In keeping with one of WIA’s major principles, sites maintained a strong 

customer focus in the way they approach training and pre-training services. 

Thus, although sites have guidelines for how customers should move through 

the service levels, it was apparent that those guidelines were not meant to be 

followed rigidly and that customers’ obvious needs were taking precedence.

10. Similarly, customer choice is clearly apparent in the way that sites are 

working with customers to help them select training programs and vendors. 

This choice is structured within a framework that requires that customers 

undertake a careful assessment of their skills and abilities and conduct labor 

market and other research. To this degree, customers are making choices 

only after being exposed to a range of good information. Notwithstanding 

this fact, although all local areas make some provision for structured decision 

making, in actuality the degree of assessment and counseling that customers 

are likely to undergo varies greatly from area to area. Further, there are 

wide disparities in the way in which the local areas respond to the very large 

percentage of customers who enroll in WIA with already established training 

choices. Thus, additional information on optimal approaches for dealing 

with customers of different types would be helpful.
ES-18

11. Front-line staff are generally playing roles that support informed customer 

choice. In most local areas we visited, case managers were playing the role 

of “facilitators,” and were lending the benefit of their expertise without being 

overly directive. To this degree, customer empowerment was being 

promoted. At the same time, the exacting role that case managers are 

expected to play by providing assessment and counseling to a diverse mix of 

customers, while still respecting customer choice, places great demands on 

their expertise. For this reason, opportunities for capacity building among 

front-line staff must remain a high system priority.

12. The quality of the choices that customers are ultimately making is unknown. 

Nor is it known whether customers would benefit from more or less 

structured progressions through pre-training services and greater guidance 

and direction from their case managers. The ITA Experiment, being 

undertaken by Mathematica Policy Research, should help resolve these 

important questions. This experiment is using experimental methods to test

three alternative procedures for advising customers about their training 

choices:

• Structured Customer Choice: The counseling provided under this 

approach is intensive and mandatory. Case managers are expected to 

steer customers to training programs with the highest net benefit.

• Guided Customer Choice: This approach is similar to the one most in 

evidence in the sites we visited as part of the ITA/ETP Demonstration, 

in that case managers guide customers without being overly directive.

• Maximum Customer Choice: Under this regimen, customers are not 

required to undertake any research or counseling as aids in making 

training choices (though counseling is made available to them if they 

request it), and they are free to select any program on the state’s ETP 

list.

ETP and CRS Issues

13. Training providers play an obvious and critical role in the workforce 

development system. The demonstration states clearly recognized this, and 

thus most made every effort to recruit them to the eligible training provider 

list and ease the burden they bear in applying for eligibility. Their strategies 

for doing so included automating the vendor application process and 

conducting data capture on their behalf for performance information 

associated with the Consumer Report System. Such efforts remain critical if 

a rich array of training choices is to remain available to ITA holders. These 

efforts are especially important to ensure the continued engagement of 

community colleges, which are at once important providers of low-cost highquality training for WIA customers, yet who often view the ETP 

requirements as very burdensome. 
ES-19

14. Also as a way of easing the burden for training vendors, efforts should be 

explored to align as much as possible the ETP eligibility requirements to 

those of other licensing bodies to which training vendors are subject. These 

efforts might include adopting similar definitions of common terms and 

building on existing reporting and performance requirements where possible. 

15. At this point, little is known about how states should best establish 

performance levels for vendors in the eligible provider system that promote 

high quality without undermining customer choice unduly by curtailing the 

size of the ETP list. The demonstration states have largely been making 

their decisions in an information vacuum, but they clearly display a broad 

range of approaches to this issue. Understanding how very different 

performance thresholds affect the size and composition of the eligible 

provider lists and how vendors react to the different requirements constitute 

high priorities for research once subsequent eligibility takes hold.

16. The underdeveloped state of most Consumer Report Systems has meant that 

concrete and comparable information on vendor performance has not been 

one of the sources that most customers have been using in making their 

training decisions. When these systems are mature, additional information 

on how customers use them and how they can be configured to best meet 

customers’ needs would be desirable. 

17. Along these same lines, to ensure that performance data for different vendors 

can be interpreted meaningfully, ways must be explored for systematically 

considering the characteristics of customers that vendors serve and of their 

local labor markets, both at the stage of reviewing vendors for eligibility to 

the ETP list and helping customers interpret performance data in making 

their training choices. Although WIA clearly allows for this, we found little 

evidence that such efforts are thus far occurring in a systematic way.

18. Similarly, states vary in the way they define key terms related to 

performance measurement (e.g., participant, completer) and in the clarity 

and completeness of the guidance they give their vendors about them. This 

variability can strongly impair the comparability of performance data once it 

becomes available in Consumer Report Systems, even within individual 

states. DOL and the states must confront the dilemma of needing to impose 

greater uniformity to promote comparability, without being unduly 

prescriptive. Without such comparability, it remains to be seen how useful 

the performance data in the Consumer Report Systems will be or how they 

can best be used to help guide customers’ choices.

19. The ITA/ETP process is presenting substantial challenges to training vendors 

that relied heavily on workforce development funding under JTPA. Reliant 

for so long on contract training for serving special populations, these 

organizations are finding that their customer flow has been gravely 

interrupted since the enactment of WIA. This challenge has fallen especially 
ES-20

heavily on community-based organizations. Without an alternative customer 

base, many of these institutions are facing insolvency. To the extent they 

provide an array of valued social services, their absence can represent a 

substantial loss to the high-poverty communities they serve and could reduce 

the number of appropriate training options for customers with special needs. 

Notwithstanding these remaining research issues and concerns, the ITA/ETP 

Demonstration project has been very valuable in shedding light on key issues related to 

ITA and ETP implementation. Moreover, whatever problems they have encountered or 

challenges that remain, the demonstration grantees have all made important 

contributions to our knowledge of these systems. They have demonstrated as well the 

extraordinarily mobilization of effort that has been entailed in developing their systems. 

Indeed, their progress in building their systems in the period encompassed by our site 

visits has been truly remarkable.
I-1

I. INTRODUCTION

The Workforce Investment Act (WIA) of 1998 brought about substantial changes 

in services provided to persons seeking employment and training assistance. An 

important element of WIA is the requirement that training services be provided, with 

certain limited exceptions, through individual training accounts (ITAs), which provide 

vouchers or related funding vehicles that customers can use to pay for training. The 

establishment of ITAs is intended to empower customers to make their own choices, 

while promoting accountability among states, local areas, and service providers in 

meeting customers’ needs.

To examine how ITAs are currently functioning and how they might be structured 

for maximum benefit, DOL has funded Mathematica Policy Research (MPR) and its 

subcontractor, Social Policy Research Associates (SPR), to conduct an evaluation. The 

evaluation consists of two distinct components. The first is a process study of thirteen 

federally-funded demonstration projects whose progress in establishing ITA and eligible 

training provider (ETP) systems and structures was to serve as a learning laboratory as 

states and local areas began implementing ITA and ETP systems. The second 

component consists of an experimental evaluation of three distinct models of ITAs that 

vary in the amount of the voucher and the degree to which customer choice is guided 

by the case manager. This report describes the findings from the first of these two 

evaluation components; interim results from the ITA experiment will be detailed in a 

separate report due in late 2004, with final results due one year later.

The remainder of this chapter first describes the policy background related to 

ITAs and presents a conceptual framework for the evaluation. Then we summarize the 

process by which DOL selected the demonstration projects to be studied, and we briefly 

identify their salient commonalties and differences. The chapter concludes with a 

description of the research design and methods of data collection associated with our 

study.

INDIVIDUAL TRAINING ACCOUNTS IN THE CONTEXT OF WIA

The Workforce Investment Act substantially reshaped the nation’s employment 

and training system. One of the primary goals of the legislation is to empower 

customers to take control of their own career and training choices while providing them 

with the information and other supports that they need to choose wisely. One way that 
I-2

local job training agencies promote choice is by issuing individual training accounts to 

adults and dislocated workers who are undertaking training.

Services Available to Adults and Dislocated Workers

The WIA legislation promotes the use of ITAs for the adult and dislocated worker 

programs funded under Title I. These programs are separately funded, but both were 

authorized to increase the employment, retention, earnings, and occupational skills of 

customers and, as a result, “improve the quality of the workforce, reduce welfare 

dependency, and enhance the productivity and competitiveness of the nation” (WIA 

Section 106).1 To achieve these goals, adult and dislocated worker programs establish 

a hierarchy of three service levels that consists of core services, intensive services, and 

training services.

Core services consist of providing basic information primarily intended to assist 

individuals in conducting job search or accessing training or other services on their 

own. In keeping with this, activities authorized by the legislation [WIA Section 134(d)] 

as core services include:

• Providing an orientation to services offered under WIA, 

• Providing an initial assessment of skill levels, aptitudes, abilities, and 

supportive services needs, 

• Providing job search and placement assistance, and, where appropriate, 

career counseling,

• Providing information on jobs in demand in the local economy and on 

eligible providers of training services,

• Describing procedures for how one can access unemployment 

compensation, and 

• Providing follow-up assistance to individuals who were served by the 

program and were placed in unsubsidized employment. 

All adults and dislocated workers are eligible to receive core services. Customers can 

access these services on their own (e.g., through electronic listings of information) or 

with minimal staff assistance.

 

1

 References to either the WIA legislation or regulations, as specified in the Code of Federal 

Regulations (CFR), are provided throughout this report.
I-3

Intensive services consist of activities involving somewhat greater staff 

involvement than is typical for core services, but the focus is still on providing 

customers with guidance or information that they need to succeed in the labor market. 

For example, activities authorized as intensive services can include:

• Providing comprehensive and specialized assessments of customers’ 

skills or service needs, 

• Developing an individual employment plan for them,

• Providing group or individual counseling and (for customers undergoing 

training) case management,

• Delivering short-term prevocational services (e.g., assisting customers 

in developing interviewing skills, communication skills, personal 

maintenance skills, and the like). 

Persons can receive intensive services if they have received at least one core service 

and (a) if unemployed, were unable to obtain employment through core services and 

need more intensive services to obtain employment, or (b) if employed, need intensive 

services to obtain or retain employment that leads to self-sufficiency (20 CFR 663.220). 

Moreover, when funds for adult services in the local area are deemed to be limited, 

priority for intensive services should be given to recipients of public assistance and 

other low-income individuals [WIA Section 134(d)(4)(E)].

Training services include skill-building activities that are designed to make 

customers more employable. Training activities that are explicitly authorized include:

• Occupational skills training,

• On-the-job training,

• Training programs operated by the private sector,

• Skill upgrading and retraining,

• Entrepreneurial training,

• Job readiness training, 

• Adult education and literacy activities (but only if provided in 

conjunction with another training service),

• Customized training for individuals to be hired by a specific employer.

In keeping with the notion that there is a hierarchy of service levels, training services are 

available to those who have met the eligibility requirements for intensive services and 

have undertaken at least one intensive service without being able to “obtain or retain 
I-4

employment.” Additionally, persons funded for training should be deemed to have the 

skills necessary to succeed in training, select a program of training for jobs that are 

available, and require assistance beyond what is available from other sources (e.g., Pell 

grants). As with intensive services, when a local area’s funds are deemed to be limited, 

public assistance recipients or other low-income individuals should receive priority for 

training services in the adult program.

With respect to service delivery, core services are to be provided through a OneStop delivery system, which includes at least one comprehensive One-Stop Center in 

each local workforce investment area and potentially a network of affiliated sites. 

Intensive services are also to be made available through the One-Stop delivery system, 

either directly or through contracts with service providers. In contrast, training 

services will generally be delivered by establishing an individual training account that 

the customer can use to procure training services from eligible training providers. 

Local workforce investment areas are required to use ITAs to provide training, except 

in some specified circumstances. As specified by WIA [Section 134(d)(4)(G)], these 

exceptions include contracts that the local area can write to fund:

• On-the-job training (OJT) or customized training provided by an 

employer,

• Training programs by community-based organizations or other private 

organizations that are of demonstrated effectiveness in serving “special 

participant populations that face multiple barriers to employment,” or

• Other providers, if it is deemed that there are too few providers in the 

area to fulfill the intent of ITAs. 

However, even providers of contract training must be deemed to be “eligible 

providers,” by virtue of having met performance benchmarks established by the state or 

local area (20 CFR 663.430 and 663.595).

Role of Individual Training Accounts and Training Providers

ITAs are intended to transform the delivery of training services to adults and 

dislocated workers by ensuring customer choice. In this regard, once issued an ITA a 

customer will generally be free to use it to procure the training services of his or her 

choice, subject to some restrictions. First, as noted above, training must be directly 

linked to employment opportunities in the local area (or in another area in which the 

customer is willing to reside). Second, states and local areas may place restrictions on 

the duration of training that can be undertaken and on its costs (20 CFR 663.420). 
I-5

Third, customers may use their ITA only for approved training programs. To attain 

this classification for its training programs, the training provider must: 1) submit an 

application for each program for which the provider is seeking eligibility, 2) agree to 

provide information on program costs and on the performance of recent trainees for 

each program, and 3) meet certain performance benchmarks for each program that are 

established by the state and local area. 

With respect to provider eligibility, the state is responsible for developing the 

overall procedures. As outlined in the Act (WIA Section 122), these responsibilities 

include establishing procedures for initial and subsequent eligibility. With respect to 

initial eligibility, postsecondary educational institutions that award a degree or certificate 

and are eligible to receive funds under the Higher Education Act, as well as programs 

operating under the National Apprenticeship Act, are to be considered automatically 

eligible, as long as they submit an application to the local area containing the information 

required by that area. However, other providers, in their application for initial 

eligibility, may be asked to submit program-specific information about the costs of their 

training services and the outcomes achieved by former trainees, and must meet 

performance levels if required to do so by the state.2 The state is to establish these 

requirements after a consultation process involving Local Boards, business, and labor.

Initial eligibility can last from 12 to 18 months. Thereafter, all providers who seek 

certification, including postsecondary degree-granting institutions, will be expected to 

submit performance and cost information and meet performance criteria annually, in 

order to maintain subsequent eligibility [WIA Section 122(c) and (d)]. This performance 

information should cover:

• Outcomes achieved by all individuals who participated in the programs 

for which the provider is seeking certification, including:

− The percentage of those participating who completed the training.

− The percentage of those participating who obtained unsubsidized 

employment.

− The wages at placement of all individuals who participated.

 

2

 However, new training programs (i.e., those that have not previously provided training services) 

can be exempted from meeting performance criteria or providing performance data, at least initially 

[WIA Section122(b)(2)(D)(iii)].
I-6

• Outcomes achieved by individuals funded by WIA who participated in 

the programs for which the provider is seeking certification, including:

− The percentage of those participating who completed the training 

and obtained unsubsidized employment.

− The percentage of those who completed the training who were 

found to be in unsubsidized employment six months after being 

placed in employment.

− Wages received by those who completed the training, measured 

six months after the first day of the employment involved.

− If applicable, the percentage of graduates who obtained a license or 

certificate, an academic degree or equivalent, or other measures of 

skills.

The required performance levels on these measures are established by the states, although 

local areas can establish higher levels. In any case, the levels established for a provider 

should take into account characteristics of the local economy and of the persons it serves. 

Performance information on the employment measures should be compiled in a manner 

consistent with the measurement of the states’ own performance on the so-called core 

WIA performance measures. 

According to WIA, the local boards should review all applications, whether for 

initial or subsequent eligibility, and then forward the list of providers they have 

approved to the state. The state, in turn, is to check the performance data for accuracy 

and compile a master state eligible training provider list, which is disseminated to all 

local areas along with the cost and performance data. This full array of vendor 

program and performance information is referred to as the “Consumer Report System,” 

and should include any additional information “necessary for an adult or dislocated 

worker customer to fully understand the options available to him or her in choosing a 

program of training services” (20 CFR 633.570). 

CONCEPTUAL MODEL FOR THE EVALUATION

The use of ITAs potentially represents a dramatic shift in the way in which training 

services are delivered, with possibly important implications for the ways in which OneStop Centers, vendors, and customers relate to each other. However, although the broad 

guidelines described above are prescribed by the WIA legislation and implementing 

regulations, states and local workforce investment boards have substantial discretion in 

how ITAs are developed and used in practice—for example with regard to the 

performance levels required of eligible providers, the types of customers for whom 
I-7

training services are deemed appropriate, the guidance customers are given to assist them 

in making training choices, and the types and duration of training that is considered 

“fundable.” To help us understand this variability, and how it impacts the delivery of 

training services, we have developed a heuristic model of how customers might flow 

through the various levels of the workforce system. To provide a framework for 

evaluating this process, we draw on indicators of high-quality service planning that we 

have developed and applied in many of our previous evaluations for DOL.3

The Prelude to Training

To understand fully the assistance that trainees receive, we need to examine what 

core and intensive services have been provided before the customer accessed training 

services. According to WIA, only those who have been provided at least one core 

service and one intensive service can receive training services. Local areas might have 

very different policies regarding what core and intensive services are provided, and for 

what duration, before access to training services is permitted. For example, in some 

cases, one core and one intensive service might be provided quickly—even in a single 

visit to the One-Stop Center—when it is clear that a customer will need training services 

to be successful in the labor market. In other local areas, a more protracted service 

sequence may be required. Thus, customers may have had very different service 

histories before an ITA is issued.

Regardless of the specifics of this sequence, in general customers undertaking 

training will normally have an individual employment plan (IEP) prepared for them as 

part of intensive services. As defined by the WIA regulations, the IEP should be jointly 

developed by the customer and the case manager and identify “the participant’s 

employment goals, the appropriate achievement objectives, and the appropriate 

combination of services for the participant to achieve the employment goals” [WIA 

Section 134(d)(3(C)].

Under WIA, customers have a substantial new responsibility in developing the IEP 

and carrying it out. Case managers, in turn, have an obligation to provide potential 

trainees with the guidance and information to enable them to make informed choices. As 

part of a high-quality service-planning process, customers should be provided with a 

 

3

 See, for example, A Guide to Well-Developed Services for Dislocated Workers, ETA Research 

and Evaluation Report Series, 1994.
I-8

comprehensive assessment of their basic and occupational skills, interests, and aptitudes, 

leading to a determination of what skills the customer already possesses and which 

additional ones are needed to ensure competitive performance in the labor market. The 

assessment should lead to a determination about whether or not additional training is 

desirable and, if so, for what type of training the customer is best suited. At the same 

time, supportive service needs should be evaluated, so that potential barriers to a 

customer’s successful completion of training can be redressed.

Customers consult with their case managers using the assessment results as a tool in 

developing the IEP. As a first step, case managers can provide career counseling and 

assist the customer in career exploration. A high-quality IEP should then clearly identify 

the customer’s short- and longer-term employment goals and the training objectives for 

achieving those goals, including interim benchmarks. The plan should be tailored to the 

skills, interests, and background of each customer, and should be developed with the 

customer’s direct involvement. Finally, the IEP should establish high yet realistic 

expectations for learning and employment success, and should establish clear and 

appropriate goals that the customer is committed to achieving. With respect to training, 

case managers must balance the need to guide the customer in his or her decision-making 

while allowing for maximum customer choice; how this balance is struck may vary 

across local areas, and, within each area, from case manager to case manager.

Training Content, Duration, and Quality

As part of the evaluation, we also need to understand the limits on customer choice. 

One reason that training choices will vary across sites is because the extent of 

empowerment engendered by ITAs will also vary. For example, in some local areas 

ITAs may present customers with a wide array of choices, while in places with few 

vendors customer choice will be quite limited. These differences will be a function of the 

array of vendors and programs available in local areas to begin with, as well as the 

eligibility rules that states and local areas establish for purposes of building the eligible 

training provider list. Their key consideration must be balancing the need to promote 

high quality within the workforce system, while also ensuring that customers have a wide 

spectrum of training choices. Even in areas with extensive choice, customers must 

negotiate with their case managers to complete a training plan that adheres to state and 

local policy guidelines (e.g., based on what occupations are determined to be in demand, 

with respect to limits on costs or duration of training, etc.). 
I-9

We also need to document how customers make choices, based on what 

information is provided to them. This information can include assessment results, 

discussed above, but also the Consumer Report System, which is designed to provide 

customers with rich information about vendor performance and costs, along with 

additional data that may help them make a training choice. Customers may also 

respond to marketing appeals made by vendors, or have strong prior preferences about 

what training they want to undertake. Accordingly, some individuals may be little 

influenced by either vendor performance, assessment results, or the advice and 

guidance provided by a counselor, and instead rely on more “subjective factors” in 

making their decisions. How these diverse factors interplay in leading to training 

choices and how they vary across local areas are key areas of inquiry for this 

evaluation. 

Coordination of programs and services is one of the basic principles of WIA. 

Thus, according to the WIA legislation [WIA Section 134(d)(4)(B)], case managers will 

also want to make use of funds that might be available from other sources, including 

Pell grants, to support the customer’s training regimen. To ensure that WIA funds are 

used most effectively, case managers should explore these alternative funding sources 

with customers and assist them in accessing these sources whenever feasible. This 

stricture thus imposes the responsibility on program operators to work collaboratively 

with other funding agencies and to leverage other resources when feasible. Thus, it is 

important to document non-ITA funding that customers have secured.

GRANTEES SELECTED FOR THE DEMONSTRATION

In the summer of 1999, DOL issued a Solicitation for Grant Applications for the 

Individual Training Account/Eligible Training Provider Demonstration (SGA/DFA 99-

017). This announcement emphasized that DOL was interested in identifying “a national 

group of vanguard sites” that were committed to implementing ITAs and establishing an 

eligible provider list that was consistent with WIA and “informed by best practice and 

insight from the field.” Chief goals of the demonstration include:

• Support for system-building at the state and local levels.

• Rigorous testing of several key models or approaches to the establishment 

of an eligible training provider process and ITA payment system.

• Identification of key components of effective ITA implementation.

• Support for demonstration “learning laboratory” sites in designing and 

implementing innovative processes and systems.
I-10

• Development of a learning network for information sharing, both across 

demonstration sites and throughout the larger employment and training 

system.

Each grantee was to receive up to $500,000, for a grant period that was to last 18 

months. In support of the demonstration’s goals, the solicitation stipulated that funds 

could be used for a variety of activities, including:

• Designing and delivering in-service training courses that would help case 

managers understand their new role in supporting customer decisionmaking.

• Developing orientation sessions to help customers understand their 

opportunities and responsibilities.

• Developing orientation sessions on ITAs and the eligible provider process 

for vendors.

• Developing a system to track training provider performance.

• Developing an ITA expenditure reporting system to provide individuals 

with information on the status of their account balances.

• Supporting related efforts associated with implementing ITAs and the 

eligible training provider system, including travel for staff.

The solicitation emphasized that funds were not intended to be used for direct 

training expenditures of customers, but rather for “building the ITA/Eligible Provider 

system.” However, grantees were allowed to use up to 20% of their grant award to 

fund such training, to supplement other funds that would also be used for this purpose.

To be consistent with the intent of the WIA legislation, DOL noted that plans for 

ITA systems being proposed by grant applicants would need to adhere to a number of 

guidelines on which substantive variability could not be allowed. Thus, all grantees 

would be required to include all adult ITA customers in the demonstration, inform 

customers of the dollar amount available in their ITA accounts, make available the 

performance and cost information on vendors that WIA requires, pay vendors on a timely 

basis, develop an IEP for each individual undertaking training, and inform staff of the 

role they would be expected to play in the ITA process. Additionally, bidders were 

informed that they would be expected to administer a customer satisfaction survey, both 

to all customers as well as to all staff.

There was nonetheless quite a lot of room for variability in the systems that 

grantees could establish, including with respect to these key features:
I-11

• How the dollar amount of the ITA is determined (e.g., fixed for all 

customers or varying according to the customer’s training needs).

• Who approves the ITA (e.g., a counselor, a committee, etc.).

• What career direction is allowed and how informed choice is provided 

(e.g., how much leeway are customers given in choosing a career 

direction, how are demand occupations identified, what is the role of the 

case manager).

• Who has final authority on selection of the training provider.

• What the ITA can pay for.

Applicants responded to these guidelines, and, on March 27, 2000, DOL 

announced that it had selected thirteen grantees to participate in the demonstration 

project. These are identified in Exhibit I-1. As the exhibit shows, the grantees differ 

with respect to the nature of the partnership that applied for the grant. Thus, six 

grantees are local workforce investment areas that applied individually or on behalf of 

neighboring local areas. The other seven grantees are states. Of these seven, four are 

collaborating with some subset of the state’s local areas, while the other three states are 

developing statewide systems and strategies. 

RESEARCH DESIGN AND DATA COLLECTION

To investigate the issues described earlier in this chapter, the evaluation of the 

ITA/ETP Demonstration entailed a process study that consisted of two rounds of multiday site visits to each of the thirteen grantees. Each site visit entailed interviews at both 

the state and local levels, regardless of whether the grantee was itself a state or local 

area, so that we could examine the interplay between state and local decisions. 

Nonetheless, because the grantees are so different with respect to whether the primary 

activity associated with the grant is at the state or local level, we customized the site 

visit agenda to spend more of the time on site at the grantee’s own level (i.e., by 

spending more time at the state level and less time at the local level for state grantees, 

and vice versa for grantees that are local areas).

The first round of site visits occurred in late summer and fall of 2000; observations 

from these visits were detailed in the evaluation’s Interim Report, issued in the fall of 

2001. The second round of site visits occurred in the fall of 2001 through early 2002. 

This report draws on data gathered from both rounds of data collection, and thus 

represents a comprehensive accounting of the evaluation’s findings.
I-12

Exhibit I-1: List of Grantees and Local Sites Visited

Grantee Area Partners Local Areas Visited

Grantees that are Local Areas

Baltimore Office of Emp. 

Development

A single local area acting alone • Baltimore

Indianapolis PIC Collaborating with the state and 

3 other local areas

• Indianapolis (IPIC)

• North Central

• Tecumseh Area 

Partnership (TAP)

Macomb/St. Clair (MI) 

Workforce Dev. Board

A single local area acting with 

input from nearby areas

• Detroit

• Macomb/St. Clair

• Washtenaw

Metro Portland 

(Worksystems)

A single local area acting alone • Metro Portland

Southeast Los Angeles 

County (SELACO)

A single local area acting alone • SELACO

Southwest Connecticut 

(The Workplace)

A single local area acting alone • Southwest Connecticut

Grantees that are States

Georgia Department of 

Labor

State will collaborate with two 

local areas

• Atlanta Regional 

Commission1

• Northeast Georgia

Missouri Div. of 

Workforce Dev.

All 15 local areas in state will 

provide input

• West Central (Sedalia)

• St. Louis County

Nebraska Department of 

Labor

All 3 local areas in state will 

provide input

• Greater Lincoln

• Greater Nebraska 

• Greater Omaha

North Carolina Dept. of 

Commerce 

State to collaborate with all local 

areas; a few are test sites

• Capital Area (Raleigh)

• Region Q

Ohio Bureau of 

Employment Services

State will collaborate with two 

local areas 

• Cincinnati2

• Ottawa County2

Pennsylvania Department 

of Labor and Industry

State will collaborate with 5 

local areas 

• Allegheny County3

• Pittsburgh3

• Philadelphia

Texas Workforce 

Commission

State will collaborate with 7 

local areas 

• Golden Crescent

• Heart of Texas

• Tarrant County

• Texoma

1 The Atlanta Regional Commission acts on behalf of several Atlanta-area local areas, for ETP purposes.

2 Cincinnati and Ottawa County are part of a single local area that encompasses most of the state.

3 Allegheny County and Pittsburgh, by local agreement, share a common local workforce investment 

board, known as the Three Rivers Workforce Board.
I-13

Overall, the objectives of the site visits were to understand states’ and local areas' 

efforts at implementing the ITA and ETP systems, impediments to implementation that 

were encountered, and variations in key elements of the emerging systems. In 

investigating these issues, we engaged in several days of data collection associated with 

each grantee and each round of site visits that entailed:

• A review of written guidelines and plans regarding the use of ITAs.

• Discussions with the demonstration grant administrator.

• Discussions at the state level with administrators responsible for the 

ITA/ETP system, to learn about the process and procedures that states 

have established.

• Discussions with state and local MIS staff, to learn about the computer 

interface for the consumer report system, tracking, and the like. 

• Discussions with Local Board staff and administrators of the One-Stop 

Center, to learn about local-area ITA policies and ETP procedures.

• Discussions with case managers and counselors, to understand the 

customer flow-through process, including what types of assistance 

customers are given in making their training choices.

• Meetings with several vendors, to discuss the ways in which they have 

responded to the WIA requirements.

A key decision involved deciding which local area or areas should be visited in 

conjunction with the study of state grantees. These areas were selected in consultation 

with the state ITA/ETP grant administrator and generally consisted of local areas that had 

some responsibility for carrying out the state’s grant activities. In general, for state-level 

grantees, we visited different local areas during the first and second rounds of data 

collection. Exhibit I-1, which was previously referred to, identifies these choices. As 

the table shows, altogether we visited 28 local areas across the 13 different states 

represented in this study.

The chapters that follow detail the findings from this data collection. Specifically, 

the report includes, in Chapter II, a discussion of the context for implementation, by 

reviewing the grantees’ grant plans and their prior experience with vouchers and progress 

towards WIA implementation. Chapter III provides a discussion of ITA policies and 

procedures, including the priority given to training services, limits on the use of ITAs, and 

the customer flow-through process, with a focus on how customers become eligible to 

access an ITA and how they make their training choices. Chapter IV discusses the 

infrastructure supporting the ITA system, including how vendors became eligible to 
I-14

provide training services and how vendor performance data are being collected. 

Chapter V provides an overview of the completeness of Consumer Report Systems, from 

the standpoint of documenting the sorts of information that the states’ systems contain. 

The vendors’ response to the ITA/ETP system constitutes the focus for Chapter VI. 

Finally, Chapter VII concludes the report with some summary observations. 
II-1

II. GRANT PLANS AND IMPLEMENTATION PROGRESS

As discussed in the previous chapter, the evaluation focuses on the thirteen 

grantees selected to participate in the ITA/ETP demonstration. At the time these grants 

were awarded, the grantees differed with respect to their prior experience in using 

vouchers and, in general, with their readiness to operate under WIA guidelines. Partly 

for this reason, they articulated different objectives as part of their grant plans and 

experienced varied success in carrying out their grant objectives. To set the stage for 

the report’s subsequent chapters, we now describe these differences.

THE GRANTEES’ PRIOR EXPERIENCE AND READINESS

At least in principle, the use of ITAs, along with the associated obligation for 

states to identify eligible providers and develop a Consumer Report System, could 

represent a substantial transformation in the way training services are delivered. Along 

these lines, and in keeping with the principles of WIA, customers can become 

empowered to take control of their own training choices. In local areas that adopt this 

approach wholeheartedly, case managers will come to adopt a coaching approach in 

working with their customers, rather than making decisions for them. The use of ITAs 

consistent with WIA also necessitates substantial effort towards building and managing 

information systems, such as in processing vendors’ applications for eligibility, 

amassing information required for the Consumer Report System, developing searchable 

databases for customers to use in identifying and comparing eligible training providers, 

and building systems to track the commitment and expenditures of vouchers. These 

tasks are all potentially enormously challenging.

The extent to which they were so in fact, however, was contingent on both the 

grantees’ prior experience with using voucher-like systems for training and their prior 

state of WIA readiness.

Prior Experience with Using Vouchers

A key provision of the system of training for adults and dislocated workers 

mandated by WIA enables prospective trainees to select a program of study and a 

training vendor that best meet their individual needs. This method for delivering 

training services stands in contrast to arrangements once common under JTPA, 

whereby job training agencies contracted with training providers to deliver class-size 

training customized to a cohort of JTPA trainees. Although contracted training is still 
II-2

allowable under WIA, this legislation makes clear that such arrangements should be 

developed only under certain limited circumstances.

However, in actuality most of the ITA/ETP grantees had gradually moved away 

from the widespread use of contracted class-size training during JTPA’s waning years. 

In its place, customers were typically enrolled in occupational skills classroom training 

on an individual-referral basis. Most ITA/ETP sites we visited pointed out that, in the 

years before WIA, this approach either predominated, or at least was very common. 

Indeed, several sites claimed to have predominantly used individual referrals for 

training for many years. For example, the Atlanta Regional Commission first began 

using vouchers as a way of providing training services to the approximately 13,000 

workers who were suddenly dislocated in 1991 when Eastern Airlines went bankrupt, 

fresh on the heels of major dislocations at Lockheed in Marietta. Given its existing 

infrastructure and the size of the dislocation, this local area was facing many more 

prospective trainees than it could readily serve with class-size training. As a result, it 

used a voucher system and largely let dislocated workers choose whatever training they 

wanted. 

Also of direct relevance was the Career Management Account (CMA) 

Demonstration, in which many ITA/ETP grantees participated, either directly or as part 

of a partnership. The CMA Demonstration, which was designed to test the feasibility 

of providing training for dislocated workers through vouchers, was launched in the 

summer of 1995, when DOL awarded grants to 13 local areas. The CMA sites that 

were also awarded ITA/ETP Demonstration grants include Metro Portland (part of the 

Oregon Consortium), Baltimore, the Atlanta Regional Commission, Missouri (Region 

II), Texas (Central Texas Council of Governments), and the city of Cincinnati. All of 

the CMA grantees who became ITA/ETP grantees noted that they were quite pleased 

with their experiences with vouchers under CMA and therefore continued to use 

vouchers at least for dislocated workers when their CMA grant expired. For example, 

Metro Portland was so convinced of the power of voucher-like models that this local 

area recently embraced what it calls Individual Learning Accounts over the last several 

years. Like ITAs, Learning Accounts represent efforts to put money in the hands of 
II-3

customers, who can then use these funds to advance their careers through continuing 

training and education.1 

The prior experience that many of the grantees have had with voucher systems 

and individual referrals for training left them much better prepared for adopting ITAs

as the mechanism for providing training under WIA than would otherwise have been 

the case. For example, respondents at many of our case-study sites remarked that case 

managers had moved years ago to adopt a coaching approach to helping customers 

make training choices. As a result, they felt that, although the transition was difficult 

at the time, the more recent move to ITAs has meant little change to the way they have 

been doing business over the previous several years. For example, Baltimore moved to 

implement a voucher system when it became a CMA demonstration grantee in 1995 and 

has stuck with this approach ever since. As part of its CMA experience, the workforce 

area facilitated a change in the way case managers performed their job, from being 

“paper-pushers” under JTPA, as one respondent described it, to becoming empowered 

to make decisions regarding how to serve customers as individuals with unique needs. 

Some case managers found this transition difficult, and some are still adapting to their 

new role, but the bulk of the turmoil associated with the readjustment occurred before 

WIA implementation.

Additionally, information systems of the kind envisioned by WIA’s call for a 

Consumer Report System were already under development in some cases. For 

example, the Atlanta Regional Commission found that many of the dislocated workers 

whom it served through vouchers as part of the Eastern Airlines bankruptcy had made 

poor training choices (i.e., poor vendor selection in addition to some poor occupational 

choices). Consequently, it began to systematically build a vendor list and carefully 

monitor vendor performance, so that it could provide vendors’ performance information 

to customers. Subsequently, by participating in the CMA demonstration it was able to

further develop and systemize its procedures. 

Similarly, Pittsburgh shifted from class-size training to what the city’s WIA 

manager termed a voucher system in 1996. As part of this process, the local Private 

Industry Council developed a performance review system for training vendors, known 

 

1

 Typically, Learning Account customers make regular contributions to their account, and these 

are matched by contributions made by employers and social service agencies. Funds can then be used to 

pay for education and training expenses.
II-4

as the Singh report. The report was widely accepted by program administrators as a 

powerful tool to aid customers in identifying effective and ineffective programs. 

Elsewhere, North Carolina has had a consumer guide in place for its public 

institutions for a number of years. This tool, the North Carolina Consumer Guide, 

provides employment and earnings information for completers of programs offered by 

the University North Carolina System and the North Carolina Community College 

System. Employment status and earnings data are shown for one year and, where 

available, two years following the quarter of program completion. 

Although grantees that had prior experience with vouchers had a head start 

towards ITA implementation, the transition has not been entirely trouble-free. For 

example, Southwest Connecticut noted that serving exclusively dislocated workers with 

vouchers, as it did under a previous demonstration, turns out to be a very different 

experience than serving low-income or welfare adults with ITAs, as it is doing under 

WIA. Customers in the former group, this grantee feels, are much more self-sufficient 

and need much less assistance and support than those in the latter group. Accordingly, 

the “light-touch” services that it had in place to move dislocated workers through the 

service levels, from self-directed job search to assessment and then referral for training, 

needed to be substantially intensified to meet the needs of low-income adults. 

Other grantees noted that, although case managers may have already come to 

adopt a coaching role to helping customers make training choices, WIA represented a 

new environment in which to do so. Under JTPA, for example, customers and case 

managers shared the presumption that all customers who came in for services would 

undergo training. By contrast, WIA, with its stipulations that training should be 

considered only when core and intensive services were unable to yield satisfactory 

results, changes this mindset completely. Thus, some case managers felt that under 

WIA they were taking much more time with each customer, and thereby came to 

understand the customer’s training needs and pre-existing skills and abilities much more 

completely. On the other hand, they often needed to overcome customers’ mistaken 

assumption that WIA, like JTPA, was “a training program” and that all customers 

would automatically be approved for training.

More generally, even grantees that had prior experience with vouchers, such as 

Baltimore, noted that staff training and capacity building needed to be ongoing efforts if 

the intent of WIA to empower customers was to be fully realized. Moreover, nearly all 
II-5

grantees found themselves far from having in place the informational infrastructure that 

the Consumer Report System requires, as a subsequent chapter will describe.

Overall WIA Readiness

Providing training through ITAs in a manner that is consistent with WIA requires 

more than developing ITA and ETP processes and procedures. On the contrary, ITAs

need to be delivered as part of a comprehensive system for delivering employment and 

training services to a broad spectrum of adults and dislocated workers. This 

requirement entails establishing a One-Stop service delivery system, including the 

articulation of well-developed self-service and staff-assisted core services and intensive 

services. In other words, in order for the ITA/ETP grantees to be effective in carrying 

out their grant obligations, they of necessity need to be operating within the context of a 

much more complex network of partners and services.

The level of WIA readiness varied substantially in the states and local areas that 

we visited, especially during the first round of site visits, and this variation influenced 

to some degree how successful the grantees were in establishing their ITA/ETP 

systems. Some grantees benefited greatly because the local area we studied or the state 

of which it was a part had long anticipated some of the changes required by WIA and 

had moved towards developing a One-Stop delivery system well before WIA was 

enacted. For example, One-Stop Centers had been long established and were 

functioning quite smoothly in Texas quite a few years before our site visits occurred. 

Their development came about as a result of the Texas Workforce and Economic 

Competitiveness Act of 1993, which mandated the establishment of Career Centers that 

were to include the participation of JTPA, Employment Services, Unemployment 

Insurance, and various education programs. According to the legislation, each center 

was to provide labor market information, common intake and eligibility determination 

for all local workforce development programs and services, the independent assessment 

of individuals’ needs and the development of individual service strategies, coordinated 

case management and counseling services, supportive services, and the use of 

individual referral for basic education and classroom occupational skills training. As a 

result of this head start, by the time the grant solicitation was announced, Texas had 

established over one hundred local Career Centers, of which 50 were characterized as 

WIA full-service centers.

In at least this one case, One-Stop service systems were to some degree homegrown and sprang from strong state-level initiatives. In our other case-study sites, 
II-6

however, One-Stop system building generally only began when the state was awarded a 

One-Stop implementation grant expressly for this purpose. These grants—which were 

awarded by DOL to all 50 states over a period of years from the mid- to late-1990s—

provided much needed financial resources that states and local areas used to modify 

infrastructure, purchase new equipment, and develop and support self-service resources 

and tools. Similarly, the national network of support that developed around One-Stop 

system-building provided a fertile ground for the incubation of new ideas and the 

dissemination of information about promising approaches and best practices. Indeed, 

across all the states that we studied, the One-Stop implementation grants, and the DOL 

vision that they represented, constituted a critical formative influence.

That advantage notwithstanding, some states were clearly further along in the 

entire process of WIA implementation than others. A few of the states covered by the 

ITA/ETP demonstration (i.e., Texas, Indiana, North Carolina, and Pennsylvania) had 

applied to be WIA “early implementers,” which means that they were attempting to 

make the transition to WIA before the required start date of July 1, 2000. Some others, 

such as Baltimore, had not applied for early implementation status, but nonetheless 

were also far along in their WIA transition process. As such, they had their state and 

local boards in place, had One-Stop systems in place, and, in general, had established 

necessary and relevant workforce policies and procedures. These states and the local 

areas that they represented had a bit of a head start in establishing their ITA systems, as 

was apparent when we first visited them in the summer and fall of the year 2000.

By contrast, other states were less ready with respect to the essential elements of 

WIA, and it typically slowed their ITA/ETP system development substantially. For 

example, Georgia was going through a somewhat difficult reorganization of its local 

areas at about the time the grant solicitation was announced. The controversy and 

disruption that this caused delayed One-Stop system building in the state such that no 

full-service centers had yet been chartered in the local area that we were going to visit 

during the first round of data collection, and virtually no one had been issued an ITA 

by then. 

In another example, Worksystems in Metro Portland itself was substantially far 

along in making the transition to WIA at the time the ITA/ETP grants were awarded 

and in fact had been designated as an early implementer under WIA. By contrast, 

during this same time, the State of Oregon was undergoing some major revisions to its 

WIA State Plan, including a reconsideration of the definitions of the three service levels 
II-7

and the sequence of services, a development that was hampering the local area’s efforts 

to finalize its ITA policies and procedures. 

GRANT PLANS AND GRANT IMPLEMENTATION

Building off this base, all of the ITA/ETP grantees are ultimately striving to 

obtain the same objectives—well-developed procedures for issuing ITAs, a 

comprehensive and readily accessible Consumer Report System, and procedures for 

determining the eligibility of vendors’ programs. That said, some grantees were further 

along than others in establishing elements of these systems by the time the grants were 

issued, partly as a function of their prior experience with vouchers and overall state of 

WIA readiness. Largely for this reason, they identified different specific objectives in 

their grant plans, consequently planned on using their grant funds in divergent ways, 

and could point to different accomplishments at the grant’s end. 

Grant Plans and Implementation Progress

Exhibit II-1 summarizes the grantees’ grant plans and provides an indication of 

how progress towards grant implementation had proceeded as of the first- and secondround site visits. As a glance at the table suggests, grantees have identified quite a 

wide variety of grant objectives. These range from the very broad, such as to further 

the development of ITA policies and ETP systems (several states), to the very specific, 

such as to develop the NC STARS system (North Carolina) or develop a code of ethics 

for vendors (Pennsylvania). In general, grantees that specified broader goals were not 

as far along in ITA/ETP system development at the time their grant proposals were 

prepared. By contrast, those that specified narrower goals had many elements of their 

systems already in place and were looking to enhance or refine them in some specific 

ways.

Grant objectives can also be categorized with respect to their major area of focus. Along 

these lines, the most common cluster of objectives related to grantees’ efforts to build 

electronic Consumer Report Systems, populated with data on vendors’ performance. It is 

widely recognized that assembling performance information about each program for 

which vendors are applying for eligibility and developing a user-friendly computer 

interface by which prospective trainees can access that information are mammoth and 

costly undertakings. In recognition of this, ten of the thirteen grantees specified grant 

objectives that related in some respect to building or enhancing their Consumer Report 

System. In some cases, this represented an effort on the part of state grantees (or state 

partners of local-area grantees) to build a major part of their statewide CRS. The grants 
II-8

of Indianapolis, Georgia, Missouri, Nebraska, North Carolina, and Pennsylvania can be 

classified in this way, with some being much more specific about the system they were 

building than others were. Additionally, several local-area grantees, including Baltimore, 

Portland Worksystems, SELACO, and Southwest Connecticut, were interested in 

building a local CRS, either as a supplement to the state’s emerging system (Baltimore) 

or as a temporary substitute (SELACO, Portland, Southwest Connecticut) in recognition 

of the fact that the states’ systems would not be operational for quite some time. As a 

further adjunct to the CRS, two grantees (Missouri and a local-area partner working in 

conjunction with Pennsylvania) worked to establish a customer messaging system, 

whereby trainees could post comments about the quality of the training they are 

undertaking.

The next largest cluster of objectives, articulated by eight grantees, related to 

capacity building, typically for staff. As with the group above, the grantees in this 

category varied with respect to the specifics of what they were trying to accomplish. 

For example, Baltimore was interested in hiring experts in assessment and counseling, 

one for each of its several One-Stop Centers, to work with customers and case 

managers to ensure that customers had the best information possible concerning their 

skills and other attributes so that they could make wise decisions about suitable 

programs of study. SELACO had a similar objective but a markedly different 

approach. This grantee was intending to use its grant funds to develop and refine the 

curriculum for a two-week group workshop that all prospective trainees would be 

expected to attend. The objectives of the workshop were to enable customers to 

support and assist each other through the process of making training choices, benefit 

from each other’s experiences and peer support, and develop teamwork skills through 

this “task team” approach. Other grantees focused more traditionally on developing 

and delivering curricula for staff training related to ITA or ETP issues. Although this 

usually was intended to take the form of workshops or conferences for staff, some 

(Missouri and Pennsylvania) were interested in developing web-based staff training 

materials or modules.
Exhibit II-1

Grant Objectives and Progress to Date

Grantee Primary Objectives of Grant Progress at Time of Site Visit Progress at Time of Second Site Visit

Grantees that are Local Areas

Baltimore 

Office of 

Employment 

Development

1) Develop a local supplement to the state 

Consumer Report System, with information 

of local relevance.

2) Strengthen the assessment process by 

hiring an assessment specialist for each 

One-Stop Center.

3) Upgrade fiscal tracking system to track 

voucher obligations and expenditures.

1) Had difficulty hiring staff with the necessary 

evaluation and computer expertise, which delayed 

its development of the local Consumer Report 

System. 

2) Has had difficulty hiring all the assessment 

specialists that it intended, but has compensated by 

having those that were hired rotate across Centers.

3) Had purchased the fiscal tracking system, 

primarily using non-grant funds.

1) Filled the position in January 2001 and hopes to 

launch a local CRS in summer 2002.

2) Made additional hires. Specialists have since 

been researching assessment tools, training case 

managers, and advising clients.

3) System is functioning and is useful, but further 

refinements are planned.

Indianapolis 

PIC

1) Implement an enhanced Consumer 

Report System.

2) Explore alternative local ITA policies 

(each local area will try an alternative).

3) Develop a staff training curriculum.

4) Share results nationally and regionally.

5) Collect information on customer 

satisfaction; share with vendors.

1) Implementation of enhanced CRS will proceed, 

but has been delayed due to contractual issues.

2) Implementation of local policies is proceeding.

3) Staff training tools have been developed.

4) Presentations have been made to share results.

5) Customer satisfaction data are being compiled.

1) Parts of the system were brought on line in 

summer 2001; further refinements are underway.

2) Local areas have established unique ITA 

policies, to suit their local needs.

3) Staff training is ongoing.

4) Presentations have been made to share results.

5) Low volume of ITAs to date has made this not 

worth pursuing for now.

Macomb/

St Clair (MI) 

Workforce 

Development 

Board

1) Establish agreement regarding ITA and 

ETP policies and systems among the seven 

local workforce areas in southeastern MI.

2) Develop a computerized information 

system, for use in the region.

3) Midway through grant, added the 

objective of developing a “debit-card” 

payment system for ITAs

1) Groundwork was laid in identifying regional 

variation, but agreement on common policies had 

not yet been reached.

2) The development of a regional information 

system is in abeyance, because rapid progress on 

the state system may have obviated its need.

3) Had not developed this as a primary grant 

objective yet.

1) Local areas have been made aware of their 

commonalties and differences; no need for greater 

conformity is seen.

2) The development of the local information system 

was suspended.

3) Developed a small pilot program (two vendors 

and six customers) to test the debit-card system.
Exhibit II-1 (continued)

Grantee Primary Objectives of Grant Progress at Time of Site Visit Progress at Time of Second Site Visit

Grantees that are Local Areas

Metro 

Portland 

Worksystems

1) Develop local policies and practices and 

provide TA to Center staff.

2) Develop a cost-benefit matrix for use by 

counselors; conduct ROI analysis.

3) Develop a model local Consumer Report 

System.

4) Incubate Individual Learning Accounts 

with local employers.

1) Has made substantial progress in developing 

local policies and providing TA to Centers.

2) Abandoned plans for cost-benefit matrix and 

ROI analysis.

3) Has made limited progress on this objective.

4) Has been working with local area employers on 

this objective.

1) Has made substantial progress in developing 

local policies and providing TA to Centers.

2) Abandoned plans for cost-benefit matrix and 

ROI analysis.

3) Has developed a small-scale local Consumer 

Report System.

4) Has continued to build strong linkages with local 

employers, some of whom contribute to Learning 

Accounts for employees.

Southeast Los 

Angeles 

County 

(SELACO)

1) Modify a curriculum used previously, in 

which prospective ITA holders participate 

in a 2-week group workshop to provide 

peer-to-peer counseling.

2) Develop a matrix with information on 

local area's vendor performance (to be used 

given that state CRS is delayed).

1) Three group workshops have been conducted; 

however, there have thus far been many fewer 

trainees than expected.

2) Have developed the local area vendor 

performance matrix.

1) Many additional workshops have been held; the 

grantee experimented with the format, and settled 

on one that it feels works best.

2) Have added additional information about 

programs to the local area vendor performance 

matrix.

Southwest CT 

(The 

Workplace)

1) Create local eligible provider list, to be 

used given that state CRS is not yet ready.

2) Develop ITA policies and processes.

3) Market to vendors; review applications.

1) Grappling with practical and logistical 

difficulties in assembling performance data.

2) Have made substantial progress in developing 

ITA policies and processes.

3) Devoted substantial effort to providing TA to 

vendors and reviewing their applications.

1) Have provided funds to support development of 

state CRS, in lieu of developing a local system.

2) Have made substantial progress in developing 

ITA policies and processes.

3) Continues to work with vendors.

Grantees that are States

Georgia 

Department of 

Labor

1) Develop an ETP list and CRS and 

integrate with state MIS.

2) Develop state and model local ITA 

policies based on CMA model used in metro 

Atlanta; adapt and test in a rural area.

1) Development of the state ETP list and CRS is 

underway.

2) State policies, a draft technical assistance guide, 

and model local policies have been prepared.

1) Substantial progress has been made. A module 

to track financial obligations has been added to 

state MIS; further development is underway.

2) No further emphasis on this as a grant objective.
Exhibit II-1 (continued)

Grantee Primary Objectives of Grant Progress at Time of Site Visit Progress at Time of Second Site Visit

Grantees that are States

Missouri 

Division of 

Workforce 

Development

1) Develop and implement consumer 

information to add to CRS.

2) Add a “comparative shopping cart” 

feature to CRS.

3) Create a system to allow customers to 

track ITA balances.

4) Develop a live web-based staff training 

system. 

5) Develop a web-based customer feedback 

system

Work is proceeding on all components of the state’s 

grant plan. 

1) Work is proceeding, but the CRS has limited 

functionality so far. 

2) Work is proceeding, but the CRS has limited 

functionality so far. 

3) The module to track ITA balances has been 

completed.

4) Web-based training is under development.

5) Limited progress on this goal so far.

Nebraska 

Department 

of Labor

1) Build ETP procedures and a 

comprehensive Consumer Report System. 

2) Market to, and recruit, vendors for ETP 

list. 

3) Develop a CRS in Spanish. 

4) Provide staff training.

1) Work is proceeding on establishing ETP 

procedures and building the CRS. 

2) Marketing and recruitment efforts are 

proceeding.

3) Efforts to develop Spanish-language CRS 

systems have been on hold.

4) Some staff training has occurred.

1) Have made substantial progress on establishing 

ETP procedures and building the CRS. 

2) Recruitment efforts were successful.

3) This effort was abandoned.

4) Additional staff training has occurred.

North 

Carolina 

Workforce 

Development 

Commission

The major objective was to create NC 

STARS, a fully automated system with 

modules that include: 

1) An ETP database, allowing for vendors 

to submit online applications and with 

programs crosswalked with occupations, 

2) A provider approval system showing 

programs approved by each local area, 

3) A Consumer Report System w/ vendor 

performance data. 

Additionally, this grantee wants to:

4) Use 3-5 local areas to pilot different ITA 

policies and procedures.

Although refining NC STARS is viewed as an 

ongoing process, substantial progress has been 

made to date in the development of this system

1) The ETP module is largely complete, 

2) The local approval system has been developed 

and is being refined, 

3) The vendor performance subsystem has been 

developed, though performance data are sparse, 

4) The grantee abandoned plans to pilot different 

ITA policies and procedures.

All grant objectives relating to NC STARS have 

been accomplished. System maintenance and 

improvement are ongoing. 

4) The grantee abandoned plans to pilot different 

ITA policies and procedures.
Exhibit II-1 (concluded)

Grantee Primary Objectives of Grant Progress at Time of Site Visit Progress at Time of Second Site Visit

Grantees that are States

Ohio 

Bureau of 

Employment 

Services

1) Develop system to track provider 

performance.

2) Test various ITA payment mechanisms 

and reporting systems.

3) Develop electronic vendor application 

process.

4) Develop and provide staff training.

5) Conduct marketing and public forums for 

vendors.

6) Conduct an environmental scan to poll 

local areas on their policies.

1) Alternative procedures are being explored.

2) Planning for alternative payments are underway 

3) Electronic application has been delayed.

4) Training is underway. 

5) Public forums should be conducted on schedule.

6) Plans for scan are underway but are meeting 

with some local-area resistance.

1) Conducted a survey of vendors to test their 

capability; on this basis, will use UI matching. 

2) “Checkbook” approach, and others, have been 

tested. 

3) Electronic application is still under development.

4) Additional training has been provided. 

5) Public forums have been conducted.

6) The scan is underway, after overcoming 

contractual and other obstacles.

Pennsylvania 

Department of 

Labor and 

Industry

1) Develop subsequent eligibility procedures and Consumer Report System.

2) Provide training to staff.

3) Refine on-line vendor application 

system.

4) Develop a “smart-card” voucher system.

5) Allow trainees to post messages on the 

web about their training.

6) Design an ITA tracking system.

7) Develop on-line system to allow vendors 

to inform LWIAs about students’ progress.

8) Develop an ITA manual for participants 

and a code of ethics for vendors.

9) Upgrade case management software.

10) Develop a system to include employer 

in-house training as part of the ETP system. 

1) Discussions are underway among state 

stakeholders.

2) Training has been provided. 

3) On-line vendor application is functioning.

4) Questioning the value of pursuing this objective, 

due to its high cost.

5) Being considered as an alternative to the 

objective above.

6) Little progress thus far. 

7) Not an original grant objective.

8) Drafts are close to completion

9) Not an original grant objective.

10) Exploring feasibility and alternative 

approaches.

1) Draft subsequent eligibility procedures were 

developed; working out plans for collecting vendor 

performance data.

2) Additional training has been provided. 

3) Ongoing refinements are being made.

4) Dropped this goal in favor of the one below. 

5) Potential liability issues delayed implementation; 

finally hired a contractor to begin work. 

6) Dropped this goal in favor of the one below. 

7) Still being finalized, but a test site was recently 

unveiled. 

8) Customer Choice Guide and Code of Ethics have 

been finalized. 

9) Work is proceeding.

10) Have overcome initial obstacles; have added 

some employer training to ETP list.

Texas 

Workforce 

Commission

1) Enhance and automate the state Training 

Provider Certification System, to allow for 

on-line applications for eligibility.

2) Develop an automated system to track 

ITAs and expenditures.

1) A prototype of TPCS has been developed. 

2) A prototype of the tracking system has been 

developed.

1) TPCS is fully functioning. 

2) Is fully functioning in two LWIAs; will phase in 

elsewhere in the near future.

_____________

Note: Objectives of the grant are as specified in the grant application and updated as of the site visits. Progress in carrying out the grant reflects status at the time site 

visits were conducted, or as updated through subsequent correspondence. 
II-13

The third largest category included six grantees that identified an interest in 

developing or testing ITA policies. Again, the specifics varied. One grantee 

(Indianapolis) was working in conjunction with several other local areas in the state to 

test alternative ITA polices, as a way of gathering information about which ones might 

be most effective. One of the local areas partnering with Ohio administered an 

“environmental scan,” again with an eye to learning about, and learning from, variation 

that is naturally occurring across local areas. 

Macomb-St. Clair had a different approach to regional variation. This 

grantee is concerned (as are many other local areas around the country) about the 

confusion and possible unwanted competition among local areas that could ensue 

if each local area within a regional labor market adopted different ITA policies, 

such as with respect to caps on the duration or costs of training that would be 

funded or occupations considered to be in demand. If there were such variation, 

it is feared, customers could play off one local area against the others, or shop 

around for “the best deal.” To circumvent such difficulties, Macomb-St. Clair 

spearheaded an effort to coordinate regional ITA policies and procedures across 

the seven local areas that make up the Southeast Michigan Workforce Agency 

Coalition (SEMWAC). As part of its grant activities, this grantee hired a 

consultant to facilitate a workgroup of SEMWAC members that discussed: what 

funding streams should use the ITA process, what ITAs can be used to fund, how 

demand occupations should be defined, what funding or duration caps should be 

imposed, payment structures and processes, how ITAs should be coordinated with 

funds from other sources (e.g., Pell grants), and how trainees’ progress while in 

training should be tracked. It also discussed developing a regional approval 

process for vendors submitting eligibility applications, so that a vendor would not 

have one of its programs approved by one local area after being rejected by a 

neighboring one. 

As another way in which grant funds are being used, three grantees attempted to 

develop software to facilitate the tracking of ITAs and ITA expenditures. In some 

cases, the software will enable customers to track their own balances (Missouri) and 

in others it is intended to be of primary benefit to the workforce area’s fiscal staff and 

case managers (Baltimore and Texas). In Texas, one local area (Golden Crescent) 

took the lead in developing ASSET, which will be used by each local area in the state 
II-14

to track ITA budget amounts and expenditures, average costs per student, training 

courses being accessed, and ITA completions. 

Three grantees (Texas, Pennsylvania, and Ohio) used their grant funds to 

automate the vendor application process. These areas have realized that taking and 

processing paper applications is enormously time consuming and burdensome to 

staff. By automating the process, they anticipate realizing substantial resource 

savings.

Finally, a number of grantees have specified objectives that are quite unique and 

do not readily fit into any of the above categories. For example, Metro Portland sees 

that synergies can be realized between ITAs and Individual Learning Accounts and is 

using some of its grant funds to recruit employers to participate in its Learning 

Accounts program. One of the local partners participating with Ohio was testing 

alternative ITA payment mechanisms (e.g., vouchers, smart cards), and a local area 

participating with Pennsylvania developed a code of ethics for vendors. Other areas in 

Pennsylvania explored the feasibility of integrating employers’ in-house training into 

the ETP system, and developed improved case-management software.

This range of variation clearly speaks to the complexity and multi-faceted 

nature of developing ITA procedures and ETP and CRS resources, and the large 

number of distinct activities that must be accomplished and coordinated. In light of 

this, several grantees noted how fortunate they consider themselves to be in having 

been selected to participate in this demonstration, as it has provided them with access 

to special funds for system development that otherwise would have needed to come 

from their regular WIA formula allocation.

Sustainable Elements and Replicable Innovations

In most demonstration projects that DOL has sponsored, the grantees are 

operating ongoing programs, and the demonstration represents an effort to incubate 

novel or innovative features. The value of the demonstration, then, comes from 

learning whether those innovations represent “improvements” when compared with the 

standard program features.

The ITA/ETP Demonstration is unlike this model in substantial ways. To begin 

with, grantees were given wide latitude in deciding what it was they wanted to 

accomplish with their grant funds, rather than needing to adhere to narrow grant 

objectives. Thus, as we have just seen, grantees pursued objectives at the level of both 
II-15

state-wide system design and local-level implementation, and they used their funds for 

objectives as diverse as providing training for staff, supporting hardware purchases 

and software development, hammering out policies and procedures, and collecting data 

on vendor performance. Just as importantly, grant funds were awarded at a time when 

all of the grantees where immersed in the large scale and systemwide transformations 

of their workforce systems associated with the enactment of the WIA legislation. 

Given this context, grant objectives often did not so much represent an effort to tinker 

with an existing program by testing new features, but instead became a part of a 

process of much broader change. 

To this degree, disentangling what exactly the demonstration grantees achieved 

with their funds that they would not have otherwise is in many cases quite difficult to 

discern. In general, though, grantees that used their grant funds for very specific and 

targeted objectives can more easily point to what they accomplished as a grantee, 

specifically tied to their use of grant funds. By contrast, grantees that had established 

more general goals (e.g., developing ITA policies, establishing ETP procedures, 

building a CRS) are less able to do so, because their funds were used for general 

system development that they would have for the most part needed to undertake in 

any case. Both groups put their grant funds to good use and accomplished much, but 

it is harder to identify innovations specifically tied to the grant in the second group of 

grantees than the first.

That said, we highlight in Exhibit II-2 some of the key innovations associated 

with the ITA/ETP Demonstration that are likely to be sustained in some fashion even 

after the demonstration funds have been expended. We highlight some of these 

accomplishments as well in the space below, organized topically rather than by 

grantee.

1. Strengthening support for customers’ training choices. Improving the information 

that customers have available to them in making training choices was an important 

objective of all demonstration grantees in one fashion or another (and, indeed, is a 

critical underpinning of the Consumer Report Systems that are being developed 

throughout the nation). However, several grantees developed special or innovative 

tools or approaches to support customers’ decision-making specifically as part of 

their grant objectives. Among these efforts:

• Although several grantees used their funds for general staff development 

associated with ITA policies and practices, Baltimore’s approach is perhaps 

most likely to have made an important sustainable contribution to capacity 

building. This grantee made a concerted effort to improve staff capacity in 
II-16

providing assessment and counseling support to ITA holders, so as to “set the 

standard for assessment nationally.” As part of its efforts under the ITA/ETP 

Demonstration, then, it hired assessment specialists to review assessment 

instruments and provide training to staff. Although Baltimore may not be able 

to retain the assessment specialists once grants funds run out, the specialists 

will have trained case managers on assessment procedures, thus improving 

overall staff capacity. 

• As a second accomplishment, Baltimore also conducted a survey of local-area 

vendors to provide information that would be entered into a local supplement to 

the state’s CRS, including general information about the provider (e.g., size of 

campus, access via public transportation), course descriptions, teaching 

methods, average class size, and the like. Although work is still underway, 

there is the expectation that this objective will achieve fruition. Portland 

Worksystems and SELACO have been attempting something similar, on a 

smaller scale.

• SELACO refined its STEPS peer-to-peer counseling workshops. These are 

two-week peer-managed workshops in which customers give feedback on each 

other’s training choices and provide general support to one another. 

SELACO has found this approach to be very successful and expects to 

continue to use this model even once the grant ends.

• Philadelphia (a Pennsylvania local-area partner) developed a Customer 

Choice manual that outlines considerations that customers should use in 

making a decision as to whether to undertake training or which program of 

training to select. Components of this manual include advice on 

budgeting, exploring short-term and long-term career goals, choosing the 

vendors, and so on. This local area, and possibly others in the state, now 

use these tools routinely.

• Three Rivers (another Pennsylvania partner) developed an online messaging 

board that students can use to post evaluations and other comments about the 

training vendor and program they have chosen. Although liability issues need 

to be addressed, there is some prospect that this tool will become functional 

within this area beyond the expiration of the grant.

2. Software development. Several grantees made software development or upgrades an 

important grant objective. However, two grantees made this objective a focal point 

of their grant effort. 

• North Carolina used the bulk of its grant funds for refining its NC STARS 

(North Carolina State Training Accountability and Reporting System). This 

multi-faceted tool incorporates a number of component subsystems, 

including:
II-17

− The education and training vendor repository, a relational database with 

relevant data on participating providers and their course offerings, with a 

crosswalk to occupations.

− The training provider application and approval system.

− The vendor/program performance data system, containing performance 

data on which local boards can base certification decisions.

− The consumer guide system, providing customers with access to 

information on ITA-approved vendors.

• Texas also used its grant funds almost exclusively for software development, 

including for these two systems:

− ASSET ITA, which is used to track ITA obligations and expenditures. An 

initial version of this database was developed by one local area (Golden 

Crescent) using its own funds. As part of the ITA/ETP Demonstration, 

the state of Texas took over system refinements, so that it would be 

flexible enough for other local areas to use.

− The Training Provider Certification System (TPCS), which enables 

vendors to submit applications for ETP eligibility electronically.

Several other grantees also developed software innovations as part of their grant:

• Missouri developed a software module that customers can use to track their 

ITA account balances.

• Berks (a Pennsylvania local-area partner) developed a system by which 

vendors can submit comments to the local area’s case managers regarding 

students’ attendance and academic progress.

3. ITA payment mechanisms. As part of their grant activities, several grantees have 

explored alternative ITA payment options. Although none of these has yet achieved 

widespread adoption even within the grantees that tried them, some promising 

approaches include:

• Macomb/St. Clair has been experimenting with a “smart-card” payment 

system. Although it has seen limited use thus far, the grantee is hopeful that 

this technology will see wider adoption within Southeast Michigan. (Three 

Rivers, in Pennsylvania, also experimented with a smart-card system, but 

abandoned the effort because it found its costs to be too high for relatively 

limited use).

• Cincinnati (one of Ohio’s local-area partners) experimented with a 

“checkbook” approach, which it felt empowered customers to manager 

their own ITA accounts and also offered a valuable life skill to those with 

little previous experience with personal bank accounts. Although this 

method has shown promise, it has not yet been adopted very widely within 

this area.
II-18

4. Increasing training choices. Quite a number of grantees used their grant 

funds to hire staff who focused on recruiting vendors to apply for eligibility. 

In this way, many of them can be said to have undertaken efforts to expand 

customers’ training choices (see Chapter IV for more information on some 

of these more general efforts). However, two grantees’ efforts were 

especially innovative.

• Portland Worksystems used some its grant funds as seed money to spur 

the adoption of Individual Learning Accounts among employers and 

workers in key sectors. Worksystems partnered with a variety of 

organizations, including individual firms, professional associations, and 

labor unions, to identify individuals who would benefit from learning 

accounts. These accounts targeted WIA-eligible customers—many of 

whom were underemployed—and were comprised of matched funding 

from employers, individuals, and unions that were partners in the project. 

Individuals used these accounts to undertake training or professional 

development or to work toward certification. Worksystems managed to 

achieve buy-in from a number of employers in the Portland metropolitan 

area and expects to sustain the model in some form once the 

demonstration grant ends.

• Lancaster (a Pennsylvania local-area partner) has been endeavoring to add 

employers’ in-house training to the Lancaster Employment and Training 

web site. This training would then be made accessible to ITA holders, 

specifically to include those who are not employed by the employer who is 

delivering the training. Lancaster views this as a strategy for adding highquality training options to the ETP list, and, thus, as a way of broadening 

customer choice. Thus far, a number of such programs have been added.

Exhibit II-2 is a highly abbreviated list that should in no way be construed as 

suggesting that only these things were accomplished as part of the ITA/ETP 

Demonstration. Indeed, if there is one clear lesson, it is that the sorts of system 

changes associated with ITA and ETP development are enormously complicated and 

costly. For this reason, the extra resources represented by the demonstration grant 

funds were, if not indispensable, certainly very appreciated and were put to good use 

everywhere.
II-19

Exhibit II-2

Sustainable Elements from Grant Activity

Grantees that are Local Areas 

Baltimore Office of 

Employment 

Development

1) Although nothing concrete is available yet, there is the 

expectation that data about local vendors’ programs will be 

available to supplement what is provided by the state's 

Consumer Report System.

2) May be unable to retain the assessment specialists they 

hired with grant funds, unless additional funds can be found. 

However, the specialists will have trained case managers on 

assessment procedures, thus improving overall staff capacity. 

Indianapolis PIC Most of this grantee’s funds, as well as those of its state and 

local-area partners, were used for general ITA/ETP system 

development.

Macomb/St Clair 

(MI) Workforce 

Development Board

1) Grant funds helped strengthen regional collaboration 

within southeast MI (although local areas within the region 

have largely agreed to maintain their own separate policies).

2) There has been some exploration of a debit card system 

for paying ITAs, although its value and permanence have yet 

to be established.

Metro Portland 

Worksystems

Has had success in having a few employers adopt Individual 

Learning Accounts, in which employers, public agencies, 

and others contribute to accounts that employees can use for 

skill building. May be able to expand on this base in future 

years.

Southeast Los 

Angeles County 

(SELACO)

Has gained extensive experience with its STEPS model, a 

peer-to-peer group workshop in which ITA holders critique 

each other’s training choices and provide mutual support.

Southwest CT (The 

Workplace)

Most of this grantee’s funds were used for general ITA/ETP 

system development, including funding for the development 

of the state’s CRS.

Grantees that are States 

Georgia Department 

of Labor

Most of this grantee’s funds, as well as those of its local-area 

partners, were used for general ITA/ETP system 

development.

Missouri Division of 

Workforce 

Development

Created a software module that allows customers to track 

their ITA balances. Other funds were used for general 

system development and staff training.
II-20

Exhibit II-2 (concluded)

Grantees that are States (continued)

Nebraska 

Department of Labor

Most of this grantee’s funds were used for general ITA/ETP 

system development.

North Carolina 

Workforce 

Development 

Commission

Developed and refined NC STARS, a software package that 

can be used for, among other things, electronic vendor 

applications for eligibility and displaying data for customers 

about vendors’ programs.

Ohio 

Bureau of 

Employment 

Services

1) Experimented with a “checkbook” approach for paying 

ITAs, which may be adopted more widely. 

2) A statewide local-area survey, still underway, might 

provide useful information to spur greater regional 

collaboration.

Pennsylvania 

Department of Labor 

and Industry

The state used its grant funds for general system 

development. Additionally, each of five local areas 

experimented with various innovations that may have lasting 

value:

1) Three Rivers. Developed an on-line messaging board. Is 

close to having a prototype, although concerns about liability 

issues may limit utility.

2) Berks. Developed a system by which vendors can submit 

information concerning students’ attendance and progress. A 

prototype has been developed and was found to be useful.

3) Philadelphia. Developed a Customer Choice manual, 

outlining factors for customers to consider in making training 

choices. Developed a “code of ethics” for vendors. Both 

are routinely used in this area and possibly in some others.

4) North Central. Developed a case management software 

for local-area use.

5) Lancaster. Has been adding employers’ in-house training 

programs to ETP list, as a way of broadening customer 

choice. Has successfully recruited a few employers thus far.

Texas Workforce 

Commission

Developed and refined two software packages:

1) ASSETS, is a system for local areas to use in tracking 

ITA expenditures and obligations. Has been piloted in a few 

local areas with good results; may be phased in elsewhere.

2) TPCS, is an electronic system for vendors to submit ETP 

applications and local areas to process them. Is being used 

successfully statewide. 
II-21

Concretely, for many grantees they meant the difference in being able to hire an 

additional staff member or two specifically devoted to some facet of ITA or ETP issues, 

and thereby enabled grantees to devote much more attention to particular implementation 

challenges than might have been possible otherwise. For other grantees, they represented 

an opportunity to undertake system refinements or innovations that would otherwise have 

been difficult to afford or might not have been contemplated. 

Moreover, as another tangible benefit of the demonstration, grantees greatly 

benefited from the exchange of ideas and information among each other (e.g., through 

the grantee conferences and other mechanisms). Indeed, several respondents mentioned 

that hearing about what others were doing—policies they were developing, software 

they were creating, etc.—represented a very real benefit to being an ITA/ETP 

Demonstration grantee. Further, they shared their insights widely with other state and 

local areas in formal and informal state and regional conferences and other forums.

For these reasons, the ITA/ETP Demonstration can be said to have accomplished 

its chief objectives of spurring system development, innovation, and the exchange of 

ideas.

SUMMARY

The above review suggests that the grantees that were chosen to participate in the 

ITA/ETP Demonstration generally had moved firmly away from contracted class-size 

training even before WIA was enacted, and many had substantial prior experience with 

using vouchers, either as a CMA Demonstration grantee or otherwise. Due to their 

having received DOL One-Stop implementation grants during the mid- to late-1990s, 

almost all also had made substantial progress in having built the One-Stop systems that 

WIA requires. 

These facts made their transition to the use of ITAs much easier than it otherwise 

might have been. Thus, some of our respondents noted that case managers had 

previously adopted a “coaching” approach to working with their customers and that 

customers were encouraged to exercise individual choice in choosing programs of study 

and vendors. Similarly, even before WIA, some sites had even begun to assemble lists 

of vendors, with information about programs offered and the duration of training and 

its costs, as a resource for potential trainees to use in making their decisions.

This prior progress notwithstanding, the transition to the use of ITAs and the 

development of ETP lists entailed substantial additional work that funds from the 
II-22

ITA/ETP Demonstration grant helped support. When we examined their grant plans, 

grantees identified a wide variety of ways in which they would be using their grant 

funds to help facilitate these changes. These plans varied in their specificity, with some 

grantees focusing on general system development, while others identified more narrow 

and targeted objectives and innovations. Most common were plans to use funds to help 

build Consumer Report Systems. Other key objectives articulated by the grantees were 

to use funds for staff training or other capacity building, developing or testing ITA 

policies, and building automated vendor application processes, among others. 

Although all grantees found their grant funds to be enormously valuable in 

accomplishing key objectives associated with ITA/ETP implementation, we identified 

some innovative practices that were developed specifically with grant funds and that are 

likely to be sustained even once grant funds expire. Some of these innovations involve 

tools or procedures to help customers make training choices, software refinements, 

alternative ITA payment vehicles, and efforts to expand customers’ training choices. 
III-1

III. CUSTOMERS’ USE OF THE ITA

WIA has set forth a new vision for services to customers. Instead of strictly 

targeting dislocated and disadvantaged workers, as JTPA did, the new law seeks to 

make employment and training services available to the entire labor force. In so doing, 

it establishes a new service design, in that, unlike in JTPA, where training was the 

expected service for most customers, WIA permits training only if job seekers are 

unsuccessful in the core and intensive services they undertake. Further, the law 

attempts to establish a delicate balance between a regulatory environment, through 

which Congress sought to assure achievement of national policy goals, and a marketbased environment, in which customers make key decisions about the services they 

receive after drawing on information and other resources provided by the One-Stop 

system. 

We observed considerable variation in the ways that the ITA/ETP Demonstration 

grantees implemented these varying objectives. This variation is the subject of this 

chapter. The chapter begins with a discussion of the emphasis that sites placed on 

training in general and ITAs in particular. We next delineate the path that customers 

take through core and intensive services before being permitted to undertake training, 

and discuss how local areas design flexibility into their procedures to deal with a 

variety of customers and their specific needs. The chapter then moves on to detail the 

ITA decision-making process of choosing training occupations and vendors. The

chapter concludes with a discussion of the services that One-Stop centers provide to 

ITA holders during and after training and the ways that One-Stop centers coordinate the 

ITA with funding from other sources. In addition, to illustrate the points we make, an 

appendix to this chapter draws a composite profile of a dislocated worker customer. 

This profile is based on the interviews and file reviews we conducted for 22 customers 

during the second round of site visiting.

In examining customer issues in this fashion, we note the important caveat that 

the sample of local areas we visited is likely heavily skewed towards local areas that 

emphasize the importance of training. This is so because states and local areas 

applying for the ITA/ETP grant presumably had a strong interest in training to start 

with. Moreover, where we had a choice (as with many of the state grantees), we 

typically selected local areas to visit that had experience with using ITAs, because an 

important objective of the evaluation is to examine ITA systems in operation.
III-2

EMPHASIS ON TRAINING AND ITAS

Customer access to and use of ITAs are determined by a number of important 

systematic threshold factors. These include the emphasis that Local Boards in the 

demonstration sites place on training, a policy decision that takes on significance in the 

WIA context where training is only one form of service that customers might receive. 

Within their broad commitment to training, boards must decide as well what constitutes 

training as opposed to intensive services, an issue that has provoked considerable 

controversy within the workforce system in the first few years of WIA implementation. 

Then, once a training decision is taken, boards can use ITAs or provide training 

through a contract, which can include OJT, customized training, or traditional classsize training for individuals with serious barriers to employment. We look at these 

issues through the prism of the local policies and funding levels for training that local 

boards have established. Exhibit III-1 summarizes some of our findings.

Emphasis on Training in General

Whether a local site has a strong interest in raising the skill levels of a small 

number of people through training or assisting a large number through job search and 

other less costly services reflects a long-running debate in the employment and training 

community. While JTPA clearly established training as the default service, WIA shifts 

the debate considerably by opening services for adults to the entire labor force. In 

expanding the range of eligible customers, Congress recognized that the WIA service 

strategy had to incorporate cheaper services if the workforce development system were 

actually to expand its target populations. In this context, states and local areas need to 

decide the emphasis they will place on training as opposed to the less costly core and 

intensive services.

The sites we studied varied across this continuum. As Exhibit III-1 shows, some 

states and local areas sought to continue the high levels of funding for training that they 

had experienced under JTPA, and expected to spend 50% or more of their total WIA 

allocation for adults and dislocated workers for this purpose.1 Others adopted a strategy 

of emphasizing core and intensive services, and as a consequence cut back on 

investments in training considerably. Still others were more balanced in their approach. 

 

1

 JTPA required that 50% of Title II funds be spent on training, so local boards that established 

such a policy were essentially attempting to maintain the status quo. 
III-3

The numbers of local areas that fell into these three categories changed from the 

first-round to second-round site visits, reflecting an evolving policy context. In the first 

round of site visits, we found a greater number of local areas that were limiting funding 

for training in favor of emphasizing pre-training services. In fact, some anticipated 

spending less than 20% of their funds for training. These local areas were especially 

concentrated in four states of the thirteen we visited (Missouri, North Carolina, 

Pennsylvania and Texas). In each case, state officials were suggesting to their local 

areas that WIA required an emphasis on core and intensive services, along with a 

reduction in emphasis on training services. In keeping with this guidance, several areas 

indicated that they were intending to adopt a genuine “work-first” approach. 

However, by the second round of site visits, the interest in “work first” and a 

corresponding de-emphasis on training had clearly waned. Thus, all but one of the 

local areas we visited in the four states we mentioned above reported that their states 

had shifted away from de-emphasizing training. This evolving policy context seemed to 

be due to their realization that a customer-focused approach was fully consistent with 

the intent of WIA. For example, staff in Ottawa County (Ohio) acknowledged that its 

local board promoted a work-first policy in PY 2000, WIA’s first full year of 

implementation. However, later in that program year, they determined that WIA 

allowed a more flexible approach, and on this basis attempted to develop a service 

strategy that was more tailored to each customer’s needs. Local areas in Missouri, 

North Carolina, Pennsylvania, and Texas echoed this sentiment.

Shifts in the availability of funds for training also accounted for changes in the 

training emphasis as well. A number of the local sites we visited in the first round of 

data collection reported that they had much less money to spend on training than they 

did under JTPA, in part because of the WIA requirement to establish a One-Stop 

system and three levels of service, including core services for the entire labor force. 

Given this requirement, respondents in several states noted that there was not enough 

money to provide good quality core and intensive services and still have funds available 

for training. This funding limitation often disappointed local officials who felt that the 

higher level of training under JTPA made their program successful. For example, 

Golden Crescent (Texas) staff indicated that, after paying for a One-Stop infrastructure, 

little was left for training. Similarly, officials in Pennsylvania pointed out that 

establishing the One-Stop system imposed a substantial financial burden on the local 

areas. The Three Rivers Local Board’s executive director confirmed the state’s 

observation, noting that maintaining the One-Stop system drained WIA funds because 
III-4

Exhibit III-1

Emphasis on ITA and Other Training

State Local Area Emphasis on Training Use of Non-ITA Training

Areas Visited During Round-One Site Visits (or During Both Rounds)

CA Southeast Los Angeles 

County (SELACO)* 

Training expected to be lower 

than under JTPA because of 

strong labor market

Some use of OJT and 

customized training

CT Southwest 

Connecticut*

Strong commitment to training, 

although fears resources will be 

limited

Very little use of contract or 

customized training

GA Atlanta Regional 

Commission

Each One-Stop center has a 

training budget

Some class-size contracts for 

certain high volume occupations, retaining customer 

choice 

IN Indianapolis* Wants to maintain a strong 

emphasis on training

Very little contract or 

customized training

Tecumseh Area 

Partnership 

Planned 40% of WIA funds to 

be spent on training

Very little contract or 

customized training

MD Baltimore* Planned 50% of WIA funds to 

be spent on training

Strong interest in OJT, classsize, and customized training

MI Macomb/St. Clair Planned one-third of WIA funds 

to be spent on training

Some customized training and 

OJTs are planned

MO West Central Strong emphasis on core and 

intensive

Considering some use of OJT 

and customized training

NC Capital Area/Raleigh Strong emphasis on core and 

intensive

No plans to use OJT or 

customized training. 

NE Greater Omaha* Community perceives One-Stop 

center as place for training

No customized or contract 

training, but considering OJT.

Greater Lincoln Planned 50% of WIA funds to 

be spent on training

ITA is sole source of training

OH Cincinnati Less WIA-funded training 

expected than under JTPA, but 

more training overall because of 

One-Stop partners

Reduced use of OJT under WIA

Emphasis on literacy and 

computer skills in intensive 

services 

OR Metro Portland* At least initially, followed state 

guidance in greatly expanding 

intensive services to encompass 

much short-term skills training 

Little use of OJT or customized 

training. Emphasis on literacy 

and computer skills in intensive 

services

PA Three Rivers 

Workforce Board 

Statewide work-first emphasis 

and limited resources; reduced 

emphasis on training

Some class-size contracts for 

certain high volume occupations 

while retaining customer choice
III-5

Exhibit III-1 (concluded)

State Local Area Emphasis on Training Use of Non-ITA Training

Areas Visited During Round-One Site Visits (continued)

TX Golden Crescent Limited resources to support 

training

Very little OJT; emphasis on 

literacy and computer skills in 

intensive services

Heart of Texas Limited resources to support 

training

Very little OJT; emphasis on 

literacy and computer skills in 

intensive services

Areas Visited Only during Round-Two Site Visits

GA Northeast Georgia Very heavy emphasis on 

training

No contract training

IN North Central Indiana Planned 40% of WIA funds to 

be spent on training; however, 

encountered start-up problems

No use of contract or 

customized training

MI Detroit Firmly committed to training. OJT and customized training 

used extensively

MO St. Louis County Initially adopted work-first 

approach, but are now doing 

much more training

No use of contract or 

customized training

NE Greater Nebraska 70% of customers enroll in 

training.

No use of contract or 

customized training

NC Region Q North 

Carolina

Strong emphasis on core and 

intensive services during first 

WIA year, but changing 

subsequently

Some budget for OJT in one 

county.

OH Ottawa County Strives for balanced approach 

between job search and training.

OJT used on limited basis; 

customized training under 

consideration

PA Philadelphia Emphasis on training, despite 

initial work-first guidance from 

state

Extensive customized training, 

but very little OJT

TX Tarrant County Training is considered as a last 

resort

Little use of contract or 

customized training

Texoma Strong work-first emphasis at 

first but have since shifted when 

state issued new guidance

Little use of contract or 

customized training

_______________

Note: We visited local areas marked with an asterisk during both the first and second rounds of data 

collection.
III-6

other partners were not contributing a fair share of the infrastructure costs. Notably, 

more sites made this observation in the first-round than the second-round site visits, 

suggesting that the concern may have dissipated as local areas made one-time 

investments in establishing their One-Stop systems and subsequently could devote more 

funds for other purposes.

By contrast, other sites were able to maintain ample funds for training from the 

outset, partly because of deliberate decisions about their local priorities and their prior 

progress in One-Stop system building (e.g., using DOL One-Stop implementation 

funds), and because other partners (such as the Employment Service) contributed 

substantially to funding core services. Thus, Baltimore and the Greater Lincoln local 

areas planned to spend 50% of their funds on training, while Indianapolis earmarked 

40% for this purpose. 

The general state of the economy naturally also heavily influenced the emphasis 

on training by affecting supply and demand for such services. During tight labor 

markets, the demand for training generally declines as workers can find jobs readily 

and employers who need workers immediately may be willing to accept less-skilled job 

applicants. The tight labor markets that characterized the first round of site visiting 

thus tended to dampen demand for training. SELACO (CA), for example, which 

intended to spend as much on training in its first year of WIA operations as it did under 

JTPA, found that it was spending much less than expected because of low demand for 

training by job seekers. By the same token, a number of local areas made a strong 

commitment to employers as their primary customers, and these local areas endeavored 

to fill the employers’ immediate demand for labor. For example, the demand for 

workers was so strong in its labor market during our first round of data collection that 

Three Rivers (PA), even when it provided training, was looking for very short-term 

interventions. In addition, local areas frequently could get jobseekers employed 

through minimal skill building that could be conducted through the short-term, prevocational services of the intensive service tier. Several local areas noted that a variety 

of office jobs could be filled with brush-up-type classes on some basic business 

software applications through intensive services. 

Conversely, slack labor markets signal workers that it may be a good idea to 

increase employability through training, and employers can be more patient and 

selective in their hiring as well. Most of our local respondents in the second round of 

site visits noted that their labor markets were still fairly tight, but that demand had 
III-7

clearly slackened. This observation is consistent with the general increase in the extent 

of training that we observed in the second round.

Use of Pre-Vocational Services

In considering the emphasis that sites placed on training, it is worth bearing in 

mind the use they made of pre-vocational services, identified by the WIA legislation as 

an intensive service. According to WIA [Section 134(d)(3)(C)(vi)], this activity 

consists of the:

development of learning skills, communication skills, interview skills, 

punctuality, personal maintenance skills, and professional conduct, to 

prepare individuals for unsubsidized employment or training.

DOL expressed concern that customers’ desire for quick employment was leading 

some local areas to circumvent the ITA system and customer choice by providing 

occupational skills training in the guise of pre-vocational services. Sites, indeed, 

struggled with deciding how the line between pre-vocational services and training 

services should be drawn. Most commonly, they made a distinction based on whether 

the activity provided training for a specific occupation or a broad family of occupations. 

If the training was for a specific occupation, they thought it should clearly be classified 

as a training service; by contrast, if it provided skills useful in a broad range of 

occupations, it was often classified as a pre-vocational service. In keeping with this, 

short courses in computer literacy and basic computer applications were thus often 

treated as pre-vocational services. The local areas that adopted this approach contended 

that learning the basics of a desktop computer and acquiring some minimum level of 

proficiency with Microsoft Windows or Microsoft Office are common and basic 

requirements associated with many occupations.

Most sites also implicitly or explicitly operationalized the distinction between 

intensive and training services based on a course’s duration. For example, Baltimore 

did so implicitly by offering training in Microsoft Windows on all three service levels, 

with the extent of instruction varying across the levels depending on the customer’s 

objectives. Thus, as a core service, its One-Stop centers offered a self-service software 

package designed to provide an introduction to this software application; in intensive 

services, a customer might learn the subject in slightly greater depth in a staff-assisted 

computer-learning lab or short Microsoft Office classes; and, as a training service, 

customers could be issued an ITA to pursue much more intensive vocational 

applications in computing. 
III-8

Two states were more explicit in using duration as a distinguishing criterion in 

how courses should be classified. Georgia, whose ITA coordinator participated on a 

DOL ITA Work Group, established the following definition based on DOL guidance:

• Programs less than 40 hours are intensive.

• Programs between 40 and 120 hours are intensive, if there is no specific 

job outcome intended.

• Programs over 120 hours are training.

Oregon provides another example of a state that established explicit distinctions 

between the service levels based on the length of the instruction that was provided. 

Officials in this state needed to accommodate a state law that defined training as a 

course of study that culminated in a degree or certificate for successful completers. In 

keeping with this, in its initial implementation of WIA, courses of instruction of less 

than 600 hours were generally classified as intensive services. However, this state’s 

incidence of training was, as a consequence, quite low, leading DOL to prod the state 

to adopt a much less limiting policy. Accordingly, it has since abandoned the effort to 

adopt distinctions between service levels based solely on a program’s duration and has 

instead begun to base classifications according to whether the content is specifically 

vocational.

These examples suggest that states and local areas ultimately adopted decision 

rules that they found to be sensible and that seemed to work quite well. However, 

many did so only after protracted debate and uncertainty. Accordingly, several 

respondents mentioned that they would have liked clearer DOL guidance on this issue 

at the outset.

ITA versus Non-ITA Training

Once local areas make a commitment to training as a service, they still have the 

option to use other forms of training besides the ITA [WIA Section 134(d)(4)(G)]. 

These options include customized training and on-the-job-training (OJT), as well as 

contracts for training that can be established if there is an insufficient number of 

providers to establish a competitive training market or to fund community-based or 

other private organizations that have proven themselves to be successful in serving 

customers with multiple barriers to employment.2 

 

2

 However, even providers funded through contracts must meet performance requirements 

established by the Local Board.
III-9

The overwhelming bulk of the demonstration sites made little use of these nonITA training alternatives, but there were important exceptions.3

 Among these 

exceptions, three local sites (Baltimore, Detroit, and Philadelphia), all very large cities 

with declining industrial bases and large numbers of low-skilled adults, indicated that 

they used contract training to a substantial degree. Additionally, a number of smaller 

local areas (including Ottawa County in Ohio, Region Q in North Carolina, West 

Central Missouri, and SELACO, among others) indicated that they used at least modest 

amounts of OJT or customized contracts. 

These sites used non-ITA training for a variety of reasons. To begin with, they 

cited its perceived advantages in serving targeted customer groups. For example, 

Baltimore used contracts to provide vocational training, combined with English-as-asecond-language instruction, to a group of dislocated garment workers, because the site 

believed that the mutual support afforded by a single class would be more effective in 

achieving positive outcomes than would individual choices. Among the other 

advantages local areas noted for using non-ITA training was that customized training 

was virtually assured of leading to job placements for the training customers and 

meanwhile typically provided them with an income stream while they underwent 

training. Thus, this approach serves customers’ interests quite well. At the same time, 

it also meets the needs of employer customers for a trained workforce, and, thus, very 

directly advances the areas’ economic development objectives. Moreover, contract 

training was generally found to yield much lower unit costs than ITAs.

ESTABLISHING CUSTOMER ACCESS TO ITAS

WIA limits training to those who are unable to meet their employment objectives 

through core and intensive services. Accordingly, local areas must establish a set of 

services and processes that provide an adequate test of skills against the labor market in 

order to ensure that only eligible individuals will have access to the scarcer, more 

expensive training services. We found that local sites vary a good deal in the way that 

they do so, including in their methods for delivering core and intensive services and the 

way customers pass through them. This variation has a direct bearing on the course of 

the individual’s ITA, because these earlier service levels serve as the forum for 

customers to make a decision on whether to train. Accordingly, we spend some time in 

 

3

 We should note that the use of non-ITA alternatives might be substantially greater in the nation 

as a whole, because ITA/ETP Demonstration grantees can be expected to be more committed to the use 

of the ITA than other local areas might be.
III-10

this section discussing how customers flow through these lower service levels to arrive 

at a training decision.

Content of Core and Intensive Services

All the sites follow the statutory requirement that a customer must receive at least 

one core and one intensive service before moving on to training. Most of the sites 

established their pre-training tiers with processes that clearly required customers to test 

the transferability of their skills through some form of job searching. In core services, 

that requirement was typically met through registration for the Wagner-Peyser labor 

exchange, preliminary or self-service assessments, or workshops. In all sites, the job 

search in intensive services was aided by greater staff involvement through some level 

of assessment and completion of an individual employment plan, but sites varied 

considerably on their use of workshops, counseling sessions, basic skills training, and 

brush-up skills on computers and basic office software.4

Core Services. Core services, with their emphasis on labor exchange, labormarket information, and self-service or light-touch staff assistance are designed to open 

the workforce development system to anyone, regardless of employment status, income 

or degree of self-sufficiency. Most of the ITA sites reflected this emphasis by 

designing a set of services for the universal customer that were also suitable for 

customers—including the traditional workforce development customers of low-income 

adults and dislocated worker—who were likely to move on to the more extensive and 

costly intensive and training services. These core services enabled customers to test 

their skills against the labor market, assess their skills, and learn about potential 

training opportunities. 

Core services in many WIA One-Stop systems draw heavily on the WagnerPeyser labor exchange, and the ITA sites were not an exception. Baltimore’s 

organization of core services is fairly typical of the ITA sites in this regard. In this 

site, a customer would typically begin by registering with the Employment Service job 

matching system. If a match occurs, the customer would receive a referral, but 

otherwise is scheduled to meet with a case manager who would help the customer refine 

the job matching criteria and use self-assessment tools. 

 

4

 Local areas differed substantially on the classification of services between core and intensive, 

but these differences are not significant for assessing for the path to training.
III-11

However, there are important differences among the sites that are generally 

related to the overall emphasis that they place on training. In the sites with less of an 

emphasis on training, job search is the focal point. These sites tend to send only a 

small portion of their intensive-service customers on to training and then only after a 

period of concerted job searching. Among the common reasons given for this approach 

are to meet the demands of local employers, limited funding for training, and the 

statutory requirement that training occur only after core and intensive services are 

found to be unsuccessful in helping the customer to find a job. An example is the 

Pittsburgh One-Stop center, which was emphasizing immediate employment during a 

very tight labor market to better serve their employer customers.5 This site had the 

most exacting requirements for core services built around its labor exchange:

• Attend the One-Stop orientation.

• Take the core assessment.

• Register with the Employment Service job matching system

• Attend two job search workshops.

• Apply for six jobs without an interview or attend three interviews 

without receiving a job offer.

At the other end of the spectrum, in Greater Lincoln, where ITAs are quite 

accessible, the core service could be the One-Stop orientation before an individual 

moved to intensive services where nearly all were preparing for training. This 

contraction of pre-training services was echoed in other sites, such as Northeast 

Georgia, where the focus of WIA is nearly exclusively on training.

Despite the general thrust of core services towards job searching in most sites, 

many job seekers are still able to explore training as a service and even to begin some 

of the preparatory services for making the decision to train and the secondary decisions 

about occupation and vendor. They typically have full access to labor-market 

information and have varying degrees of access to assessment and staff assistance in 

making career decisions. In Southwest Connecticut, for example, the One-Stop center 

uses extensive self-assessment tools during core workshops that allow customers to 

explore future training decisions. In all sites, the Consumer Report System is also 

available to provide information about training programs, although these systems vary 

 

5

 We conducted our site visit to this local area during the first round of data collection. The site 

might have since changed its approach.
III-12

considerably in their stage of development and completeness (as we discuss in 

Chapter V).

Intensive Services. We observed two general objectives for the basic design of 

intensive services that were also related to the local areas’ overall emphasis on training: 

(1) an emphasis on intensive services as a stand-alone activity, or (2) use of intensive 

services to prepare customers for training. 

Most local areas designed their intensive-service tier with the first objective in 

mind: that customers would be continuing the job searching that they had begun in core 

services, but with staff help and a more extensive set of tools and staff assistance. 

Typically, they had rich service offerings as part of intensive services and fully 

expected that many customers would receive intensive services and nothing further. 

These offerings often include short workshops on a variety of topics, including various 

computer applications and on life skills and job readiness. In one site, for example, the 

first intensive service is a conversation between a case manager and customer to 

analyze why the job search in core services was unsuccessful. The customer then 

receives assessment and is routed into particular job search or other workshops to 

address skill gaps and any shortcomings of the earlier job search strategy. 

In contrast to these examples, about one-third of our study sites reported that they 

essentially designed their intensive services around the second objective: preparing 

individuals for training and ITA planning. Under this model, the local area would 

expect that all, or virtually all, customers who reach intensive services would also 

receive training services. Consequently, in these sites the principal intensive services 

of case management, assessment, and developing an IEP tend to revolve around making 

the ITA decisions of selecting a training occupation and a vendor. 

For example, in Greater Omaha staff have typically already identified those 

people who needed and wanted training (the majority of all customers) before moving 

them into intensive services. For such individuals, the assessment and individual 

employment plan developed as part of intensive services focus on training decisions. 

Similarly, in Allegheny County the individual employment plan identifies the skills and 

courses the customer needs to achieve a career goal. 

Sites also vary in the thoroughness with which they conduct assessment as part of 

developing the IEP. Some sites seem quite strong in this regard. For example, St. 

Louis County contracts with its local community college for professional assessments 
III-13

using a wide variety of aptitude, career interest and inventory, and personality tools. 

The IEP includes establishing short- and long-term goals and identifying any barriers to 

employment that may require remediation of basic skills or supportive services. 

Baltimore also makes a very strong commitment to conducting careful and 

thorough assessments. In fact, this site used its ITA/ETP grant funds to hire 

Assessment Facilitators, with specialized expertise in assessment and testing. As part 

of their duties, these new staff were to research the adequacy of different assessment 

instruments for various purposes (e.g., for assessing interests and aptitudes for jobs of 

different types) and different types of customers (e.g., limited English speakers, those 

with limited literacy skills, etc.) and build the capacity of the centers’ case managers in 

interpreting assessment results.

At the other extreme, other sites made minimal use of formal assessment 

instruments and thus needed to rely to a much greater degree on informal diagnoses 

conducted by case managers through the course of their discussions with customers. 

For example, Philadelphia conducted no formal assessment of customers’ occupational 

interests and aptitudes (although it did routinely administer the TABE, a test of literacy 

skills). It felt, through prior experience, that the tests of occupational interests and 

aptitudes that it had been using did not consistently yield reliable results and hence were 

not worth continuing. It was, however, in the midst of reconsidering this decision. 

Customer Flow through Pre-Training Services

Systems that strive for responsiveness to the needs of individual customers should 

not typically establish a specific duration for their core and intensive services, and none 

of our sites currently do so (though a few did experiment with set durations when they 

first implemented tiered services). And sites were uniformly quick to point out that 

they are able to move customers with an evident need for training through the lower 

levels very quickly. That said, however, in nearly all of the sites it generally took 

customers several weeks to complete a core and an intensive service, make a decision 

to train, and then make the ITA decisions on selecting an occupation and a vendor. 

The shortest typical period that any site reported was about 21⁄2 weeks, while the longest 

period was about 9 weeks.6 

 

6

 This finding is based on a review of all sites that were willing to provide an estimate of duration 

and includes the time elapsed from the onset of staff-assisted core services to the point of selecting a 
III-14

Durations generally followed the rhythm of case management appointments and 

the specific job search and research and information-gathering requirements that 

occurred in-between appointments. Where these requirements were longer, so too were 

typical durations before a customer would be issued an ITA. Thus, in sites such as 

Baltimore and St. Louis, both of which used multiple assessment instruments and a 

generally more rigorous assessment process, longer average durations were necessary 

to accommodate this rigor. In other sites, such as Philadelphia, where assessment was 

less rigorous, customers would typically move through more quickly.

Typical was the requirement in Greater Nebraska:

• Appointment #1: WIA registration, including a preliminary assessment 

by which staff determine the customer’s suitability for WIA services.

• Appointment #2. Resume preparation, review of labor-market 

information, job matching in the labor exchange, and, if appropriate, an 

introduction to training opportunities.

• Appointment #3. Prepare IEP and review research on training and ITA 

decisions.

• Appointment #4. Issue an ITA.

Staff at this site indicated that they try to move customers through at a reasonable pace 

that commonly results in a one-month period between registration and approval of an 

ITA. 

Although, as we noted, there was some across-site variation in average durations 

in core and intensive services depending on how appointments such as these were 

structured, we found that within-site variation in durations was much greater. This 

variation was due to a number of factors, among which was the customer’s own 

diligence in filling out the necessary paperwork and scheduling appointments with the 

case manager. Thus, customers who missed appointments or took longer to schedule 

them could undergo protracted periods in core and intensive services before being 

awarded an ITA.

Additionally, case managers with whom we spoke were quick to note that 

customers who were evidently in need of training could move through the service levels 

very quickly, potentially moving from core to intensive services within a single day. 

 

training program and vendor. The actual onset of training might not occur for an additional few weeks, 

or longer, depending on the vendor’s training schedule.
III-15

Typical of such accommodations were for dislocated workers whose skills were useful 

only in a contracting industry or welfare recipients who may have been referred from a 

TANF program to get training.

Similarly, customers who had clear expectations for training that were recognized 

by One-Stop staff as being reasonable tended to navigate the early service levels 

quickly. Along these lines, nearly all sites indicated that a very significant proportion 

of their customers came into their One-Stop centers with a strong commitment to 

training and a clear idea of what training they wanted to pursue. In Detroit, for 

example, a front-line staff person estimated that 95% of dislocated workers came in 

because they wanted training, while 65% of adults did so. In Omaha, where the OneStop center was widely known in the community from its role under JTPA as the place 

to get training, those proportions were estimated at 90% and 80%. For these 

customers, the One-Stop center’s main task was to try to accommodate their wishes to 

shortcut the rigor of the pre-training processes, while ensuring that the decisions they 

make are based on sound information and effective decision-making. By contrast, 

others whose need for training was less clear or who were less certain at the outset 

might spend much longer before advancing to training.

Establishing Eligibility for Training

As the above discussion suggests, much of the assessment and service planning 

associated with core and intensive services involves identifying those customers who 

need training services to attain their employment objectives. This objective is 

motivated by WIA’s stipulation that limits the higher-cost intensive and training 

services to customers who are unemployed and unable to obtain employment through a 

lower tier of services or are employed but need such services to attain self-sufficiency 

[WIA Section 134(d)(3)(A)]. All the One-Stop centers we visited follow these statutory 

requirements. Still, while not modifying the qualifying requirements, some sites 

interpret them very narrowly, making a very careful review for evidence that a 

customer has any transferable skills. This was especially the case during the first round 

of site visits, when some sites indicated that they were cautious in approving training in 

order to avoid mistakes.

With respect to those already employed, the law does not set a specific means test 

for self-sufficiency, theoretically allowing local areas to provide intensive and training 

services to any adult. However, most Local Boards set their own definitions at fairly 

low levels, effectively limiting intensive and training services for employed adults to 
III-16

those whose earnings are very modest. Exhibit III-2 shows these levels for some of the 

local areas in our sample. Thus, as the exhibit shows, some areas (e.g., Indianapolis, 

Detroit, the Atlanta Regional Commission) establish self-sufficiency at the poverty level 

or 70% of the Lower Living Standard Income Level (LLSIL), which is the federal 

means test for low-income adult status. A few local areas peg their self-sufficiency 

level somewhat higher, however, such as at the LLSIL (West Central, Texoma), 200% 

of the LLSIL (Baltimore), or up to 200% of the poverty level (Greater Lincoln, Ottawa 

County), allowing more adults to qualify for these services. Other areas used a formula 

that takes into account family composition and other circumstances. For example, 

Philadelphia’s Self-Sufficiency Budget Worksheet was developed by the Women’s 

Association for Women’s Alternatives and provides estimated monthly living expenses 

for households of different compositions. These range from a wage of $7.10 per hour 

for a single adult living alone to $16.61 per hour for an adult with a preschool child. In 

some areas, separate self-sufficiency standards were established for dislocated workers 

who had obtained new employment before seeking services, for whom self-sufficiency 

was defined as being just below (Greater Lincoln, Ottawa County, Dallas) or at (Three 

Rivers) the layoff wage.

Closely linked to self-sufficiency is the legislation’s stipulation [WIA Section 

134(d)(4)(E)] that, within the adult program, priority for intensive and training services 

should be given to those who are low-income adults or are welfare recipients when 

funds are limited. In keeping with this, all but two sites formally declared a funding 

shortage at the beginning of the program year to ensure priority for the economically 

disadvantaged. Several also adopted additional priority criteria. For example, the 

Missouri West Central local area gives higher priority to those with the greatest number 

of barriers to employment. One site uses a geographic emphasis, giving priority to 

residents of its Enterprise Zone. Four local sites make veterans a priority group. 

Another site establishes priority for those who lack any work experience, lack relevant 

work experience, or have the least education. 
III-17

Exhibit III-2:

Definitions of Self-Sufficiency for Selected Local Areas

State Local Area Definition

MD Baltimore 200% of the Lower Living Standard Income Level (LLSIL). This 

applies for adults and dislocated workers. 

GA Atlanta Regional 

Commission

The poverty level or 70% of the LLSIL, whichever is lower. For 

dislocated workers, lacking self-sufficiency can be demonstrated if 

the person is at a wage or skill level significantly lower than job 

of dislocation.

IN Indianapolis 100% of poverty level; raised to 150% of the poverty level

MI Detroit 100% of poverty level

MO West Central 100% of the Lower Living Standard

NE Greater Lincoln 170% of the poverty level; for dislocated workers, earnings that 

are 80% of the layoff wage

OH Ottawa County Individuals are not self-sufficient if they receive food stamps, 

housing assistance, medical benefits or cash assistance, or income 

is below 200% of poverty level. 

For dislocated workers, threshold is 82% of layoff wage. 

PA Three Rivers For welfare recipients: no longer in need of government aid, or 

total family income is no longer below the poverty level.

For dislocated workers: have returned to employment at 93% of 

former wage

Is considering raising threshold to $8-$10 for adults, and 105% of 

layoff wage or $10 (whichever higher) for dislocated workers

Philadelphia Uses a matrix that shows projected budgeted amounts for housing, 

child care, food, and other expenses, for families of different 

sizes. Self-sufficient hourly wages for:

-- a single adult are $7.10

– an adult and infant are: $12.39

– an adult and preschooler are: $16.61

– an adult and school-age teen are: $11.23

– two adults and preschooler are: $9.22 per adult

TX Dallas Full-time employment at $7.50/hr. For a dislocated worker, 

defined as 75% of prior wages or satisfaction with wages and 

benefits of new employer.

East Texas Annual earnings of $13,539; for dislocated workers, 85% of prelayoff wage.

Texoma 100% of LLSIL
III-18

These guidelines notwithstanding, many local boards provide exemptions to their 

priority policies to permit intensive and training services to adults who are not lowincome and may not fall into any of the other priority groups that were established. 

Thus, even where funding shortages were declared, case-managers were often 

authorized to exercise their discretion to approve intensive or training services for some 

customers outside the priority groups. Nevertheless, in actuality these income 

eligibility issues were largely academic because sites reported that relatively few 

higher-income adults were seeking training services. Thus, the overwhelming majority 

of WIA registrants in the adult program were low-income individuals.7

MAKING ITA DECISIONS

Once customers have been determined eligible for training services, they need to 

select a training occupation and a training vendor. Doing so requires that they consider 

a series of factors relating to the suitability of alternatives. These factors, along with 

the information customers need to make decisions and the One-Stop tools or procedures 

available to them, are schematically displayed in Exhibit III-3. In the section of this 

chapter that follows, we build on this diagram to discuss the resources and processes 

that local areas use to effectuate choice. We look at several models of customer choice, 

the processes used to make selections and approvals, and the impact that vendors have 

on decision-making.

Types of Customer Choice

The essence of customer choice is present in all the sites we visited, in that 

customers are typically choosing occupations and vendors that they want. We saw no 

evidence that customers are being placed in classes contrary to their preferences 

because the local area had negotiated a contract and needed to fill training slots.8 These 

findings suggest that there has been substantial progress in the sites we visited in 

reaching one of the key statutory objectives of the ITA system. 

 

7

 Data from the PY 2000 Workforce Investment Act Standardized Record Data, WIA’s clientlevel data system, confirm this conclusion. This source shows that in nearly all the local areas we 

studied more than 80% of the WIA adult registrants were low-income individuals. 

8

 Three sites in large metropolitan areas reported that they retained class-size contracts for certain 

occupations for use by ITA holders. Staff indicated that these contracts were for high volume training 

occupations such as certain computer network engineers or various medical specialists and technicians, 

and were only used for customers who clearly expressed a preference.
III-19

Exhibit III-3

Information and Tools to Make Training Decisions

Key Training Questions Information Needed One-Stop Tools

What occupations are in 

demand that meet my 

income and other needs?

Occupational descriptions and 

requirements, supply and demand 

data, and data on wage and other 

compensation 

State labor-market 

information systems

Staff knowledge 

Do I have the skills, interest, 

and general capability (for 

example, financial capability 

to defer income) to complete 

a particular training program 

successfully and obtain a 

target job?

Knowledge of personal skills, 

interests, and capacity.

Vocational 

assessment tools 

What are the best programs 

to provide the skills 

required by the labor 

market and help me to 

obtain the job that meets my 

needs?

Program information and vendor 

quality 

Consumer report 

system

How will the combination 

of my new and existing 

skills prepare me for the 

labor market?

Testing the labor market through 

job searching

Job searching and 

placement activity

The real variation among sites occurs in the way choices are made and in the 

rigor of the decisions rather than whether there is choice at all. To help explain these 

variations we have posited three models of customer choice that reflect the broad 

middle and the outer bounds of how decisions are made in the ITA system at the sites 

we visited.9 We call these variants informed choice, directed choice, and free choice.

Informed Choice. In the informed choice model, which constitutes the broad 

middle ground, the One-Stop centers provide information and assistance and develop 

procedures that lead customers to make wise choices on their own. We found an 

overwhelming preference for this approach among the One-Stop centers we visited.

 

9

 Our colleagues at Mathematica Policy Research developed these three models for the 

experimental evaluation of ITA outcomes described in Chapter I. 
III-20

At these sites, our respondents consistently described customers as making 

choices based on good labor market information, a realistic assessment of their abilities, 

and some knowledge of the training programs and their performance. Most of the key 

system components to support this type of decision-making are already present. All the 

states have labor market information systems available, typically allowing online access 

in searchable formats. Assessments are required and conducted using the vocational 

counseling industry’s standard tools. Some local areas use certified counselors, either 

in-house or under contract. In addition, some information about the training programs 

and vendors comes from the Consumer Report Systems that are mostly now available 

online. Case managers also frequently supplement the information available from the 

Consumer Report System with their personal knowledge of vendor capabilities. In 

general, we found that, when customers articulated training plans grounded in these 

resources, staff were likely to provide a wide scope for their choice. Overwhelmingly, 

then, our interviews and observations of counseling sessions suggested that the 

customer is the real decision-maker. As one case manager in Southwest Connecticut 

noted, “We have to respect the customers’ knowledge of themselves (because, 

ultimately) they are the ones who have to show up (for the training).” 

Nonetheless, front-line staff play a key role in guiding or facilitating choices. 

Case managers consistently describe themselves as guides, facilitators, or information 

brokers, and one site even includes the term “facilitator” in all the job titles for its 

front-line staff. Case managers thus clearly play a pivotal role in ensuring that the 

customer’s choice is actually informed by good information. The role generally 

appears to be a real compromise between expertise and facilitation. One case manager 

in the Atlanta area describes this balancing role in working with a customer who may 

be making a poor decision as follows (below is a paraphrase): 

We work extensively with customers to persuade them about which vendor 

would be suitable, in terms of the curriculum, teaching style, and the types 

of services available at the school. However, if a customer insists on 

choosing a particular vendor—even if we think there is a more suitable 

vendor—we normally approve the request, but remind the customer that 

there will be no second chance (at further training).

Consistent with the facilitative role, several sites substantially increased the level 

of responsibility for front-line staff when compared with their roles before a voucher 

system was introduced. Thus, for example, administrators in Baltimore noted that their 

front-line staff were no longer the “paper pushers” they had been under JTPA, but are 

now empowered to work with customers to respond to their unique needs.
III-21

Directed Choice. Directed choice provides the same types of labor market and 

vendor information to the customer, but in this model staff work especially hard to 

ensure that the customer makes a decision that is heavily influenced by the staff’s 

professional judgment. Only three sites appear to be practicing directed choice for all 

their customers. In these sites, case managers try to control the terms of the decisionmaking to a greater degree than in the informed choice sites. For example, some staff 

we spoke with at one site felt that their professional judgment was often preferable to 

the vagaries of the customer’s less informed judgment. They may be very blunt in 

suggesting to their customers what program to select and where they should go to 

school. By way of justification, an administrator noted that “people do not always 

know what is best for them.”

Under some circumstances a directed choice model is also used in centers that 

generally practice informed choice. For example, one case manager who worked 

mostly with current or former welfare recipients indicated that many of her customers 

did not readily understand how to use the decision-making tools or how to interpret 

their output, so she has to take a much more directive role than she otherwise would. 

Elsewhere, case managers often play a key role in dissuading customers from their 

choices when they come into the center with ill-informed training ideas (e.g., training 

programs for which the customer lacks a proper background or those with an approach 

to teaching ill-suited to the customer’s needs). The high level of staff intervention with 

such customers effectively reduces the scope of independent decision-making that is 

available to most others.

Free Choice. In a free choice model, customers can access labor market data, 

vendor information, and assessment tools at their discretion, but are generally left free 

to make their own training choices (so long as the program of study is for a demand 

occupation and the vendor is on the eligible provider list). While no sites use a 

standard free-choice model exclusively, most sites appear to have used a free choice 

model for customers who know exactly what they want before they come into the OneStop center. These customers often have very specific training plans with an 

occupation and a vendor already picked out at the outset. Where the One-Stop center 

staff can confirm the soundness of the plan based on labor market information, the 

vendor’s suitability, and a realistic assessment of the individual’s own capabilities, 

these customers typically get their choice with a minimum of staff intervention and tend 

to move very quickly through the early service tiers and through the ITA decision-
III-22

making process. These patterns were confirmed by both customer and staff 

respondents in all the sites we visited.

Processes Used in Making Choices

In operationalizing the choice models, all the sites established one or more 

processes that organized the use of basic information tools at the appropriate time to 

help their customers make the key training decisions. These tools and resources 

include the assessment and service planning that are conducted as part of core and 

intensive services, as we discussed earlier in this chapter. Case management and 

counseling were also critical in helping the customer make training decisions. Indeed, 

many of the respondents we spoke with—both case managers and training customers—

indicated that customers respect their case manager’s judgement and come to rely on it 

very heavily in many cases. However, many sites made use of additional decisionmaking processes, including workshops, customer field research, the Consumer Report 

System, and formal ITA approval mechanisms.

Workshops. Several sites use workshops or orientation sessions as an important 

tool in assisting customers in making choices. For example, both Baltimore and 

Philadelphia require that customers approved for training attend an orientation session, 

during which they learn about their training alternatives such as an ITA, OJT, or 

customized training. If they choose the ITA method, they attend an ITA workshop, 

which helps them to prepare their application for an ITA that identifies and justifies 

their selection. Similarly, Southwest Connecticut also uses a workshop to prepare 

customers to conduct active research and teach them how to interpret performance 

information. 

In a very different approach, SELACO uses a two-week, peer-managed workshop 

where customers work together in a team to help each other to select appropriate 

programs of study and vendors. In its view, this approach allows customers to support 

and assist each other during career planning, helps customers benefit from each other’s 

experience, creates a sense of peer support, and helps develop teamwork and critical 

thinking skills.

Customer-Field Research. One of the most common and powerful methods of 

informing choice was to either require or strongly encourage customers to conduct their 

own field research. Typically, the research process requires customers to visit vendors 

to talk with staff and observe the facilities. Several also add a requirement that 
III-23

customers must either talk with current or former students of the program, interview 

employers who would potentially hire trainees in that occupation, or both. 

For example, the Pittsburgh One-Stop center requires its ITA candidates to 

interview at least two schools, two students who had completed the program, and two 

potential employers. Similarly, in SELACO prospective trainees are required to 

participate in informational interviews with employers and current employees in the 

occupation for which they are seeking training, and those in Detroit must visit the 

vendors they are considering. In Baltimore customers must:

• Interview at least one employer that hires people in the customer’s 

intended training field, to obtain information on starting salaries, 

training requirements, working conditions, and the like.

• Interview at least two persons who formerly took the training that the 

customer is interested in pursuing, to learn where they are currently 

employed, at what wage, what they like/dislike about their job, where 

they got their training, and their assessment of the adequacy of the 

training they received.

• Conduct field research on at least two training providers who offer the 

training the customer is interested in pursuing, to inspect the vendors’ 

facilities, assess the adequacy of equipment, and discuss teaching 

methods and schedules. 

Where customer field research was used, case managers found it very effective in 

making customers comfortable with their choices. Thus, for customers who had come 

in with poorly thought out ideas, a case manager in Atlanta noted that the local area’s 

research requirement was very effective in “bringing them back to earth.” Some case 

managers also felt customer research increased the customer’s commitment to the 

eventual choice, and also enhanced the case manager’s knowledge base. 

Approval Process. In most local areas, case managers or One-Stop managers are 

authorized to approve a customer’s ITA request. In a few sites, by contrast, customers 

must make formal presentations that required them to justify their training decisions. 

Such a process is used to ensure that customers have thought through all the elements 

required for effective decision making and that their decisions are justifiable. In sites 

that have such a process, potential trainees must submit a formal application or make a 

presentation before a review panel with the authority to approve the ITA. For example, 

Baltimore’s approval process is as follows:

Prospective trainees must prepare a written justification for their choice of a 

training program that should include their reasons for wanting the training, 
III-24

evidence that the training is for a demand occupation and that the individual has 

the skills needed to successfully complete the training, and a record of research 

that the customer is required to undertake, including from interviews with 

vendors, former trainees, and employers. Prospective trainees then present this 

justification to a panel of the city’s One-Stop center directors who are authorized 

to approve the ITA. Those whose application is deemed incomplete or 

unsatisfactory must conduct additional research and prepare a new application, 

with the assistance of the case manager, until the review panel is convinced that 

the choice is a wise one. 

Similarly, Golden Crescent has its customers present an oral proposal to an ITA 

review committee, which includes a former ITA holder, justifying the career and 

vendor choices, identifying expected employment outcomes and barriers to completing 

the training, and presenting a financial plan that demonstrates that training and living 

expenses can be met. 

Local areas that require approval by a formal review committee believe that this 

approach promotes system accountability and ensures that the customer has a justifiable 

training plan. It can, however, typically delay ITA approval by up to an additional few 

weeks. For example, in Baltimore the review committee meets just twice a month. 

Accordingly, customers are advised to submit their ITA application to the committee at 

least four weeks before the start of the proposed training to allow time for the 

committee’s review.

Consumer Report System. The legislation clearly lays out a very important role 

for Consumer Report Systems to provide high quality information about programs and 

vendors so that staff can dispense advice and ITA holders can make an informed 

decision. However, at the time of our site visits, these systems were still incomplete. 

As we discuss in detail in Chapter V, most states had online systems with information 

about vendors, including critical information about costs, program duration, location, 

and facilities. But what was missing from nearly all these systems was information on 

the performance of former trainees, with respect to their program completion rates and 

post-training employment success. 

Only in Texas, which had a fully functioning system, were customers able to 

make substantial use of the performance information. Case managers at the four local 

areas we visited in that state indicated that there was substantial use of the system by 

customers and staff. In several other sites, such as Baltimore and St. Louis County, 

staff respondents suggested that their Consumer Report System was somewhat helpful 
III-25

already and would be very useful to both staff and customers once the system recorded 

performance information. 

In the absence of completed state systems with performance information, several 

sites created their own systems to fill the gap. For example, in Detroit, SELACO, and 

Pittsburgh, the local board had been measuring vendor performance for some time, and 

the local leadership was confident that their own measures were providing staff and 

customers with high quality information. Portland, which was unhappy with the delay 

at the state level, also developed a local variant of a vendor information system, and 

Baltimore was using grant funds to supplement the state’s Consumer Report System 

with a database of much richer information about local training vendors. Elsewhere, 

Three Rivers had plans to support an Internet messaging site on which trainees could 

post comments on their training providers, including prerequisites, curriculum, and 

instructor quality.

STAFF TRAINING AND ORGANIZATION TO SUPPORT DECISION-MAKING

Case managers played a very substantial role in helping the customer make both 

the decision to train and the key decisions about the ITA. This is true in all sites and is 

pertinent regardless of any emphasis on training, organization of process, or choice 

models used. In keeping with this, the customers we interviewed provided very 

favorable comments about the role their case managers played.

Given the importance of the facilitative role played by case manager throughout 

our study sites, it is hardly surprising that staff competency in working with customers 

was widely considered to be critical to the success of the ITA process as the 

employment and training system moved to increase the emphasis on customer choice. 

Reflecting this emphasis, the grant solicitation identified staff training as one of the

common elements of the demonstration, and most of the grantees duly proposed some 

type of staff training. 

However, we found the overall volume and intensity of staff training to be 

relatively modest. Respondents suggested that one reason for this was that, because 

many of the sites were already using some form of vouchering under JTPA, the ITA 

system did not change the case manager’s role that much. Indeed, no site indicated that 

it had to retrain its front-line staff systematically in order to promote the facilitative 

staff role that we found to be very widespread.
III-26

Nonetheless, all sites provided staff training of some sort, and some devoted 

substantial resources to doing so. The training that occurred has been a mix of statesponsored and local in-house training. For example, a number of sites, such as 

Indianapolis and Metro Portland, provided in-house training on their own ITA policies 

and procedures. Several other sites reported that ITA rules and procedures were 

covered in their general WIA training, either provided in-house, directly by state staff, 

or by the state training institute. Several sites additionally provided training to help 

case managers play a facilitative role, because the sites were trying to embed 

facilitation in the entire range of customer relationships. Along these lines, roleplaying exercises were very common. 

Among the sites with very deliberate plans to build staff capacity, Baltimore’s 

efforts are noteworthy. This local area used some its grant funds to hire assessment

specialists who would not only work with customers, but provide guidance and training 

to other staff at the center. Similarly, the Three Rivers Local Board paid for facilitator 

training at a local university for any interested case managers. 

Apart from capacity building, another key staffing issue as sites moved to 

implement ITAs was how they organized staff to assist customers. We found two 

general approaches. Some centers used a single point of contact, while others relied on 

specialization. In the former approach, a single case manager stays with the customer 

throughout training, intensive, and staff-assisted core services. The advantages of this 

approach are that the case manager builds up knowledge of the customer and establishes 

a personal relationship. Region Q and Capital Area (both in North Carolina) and 

Golden Crescent (Texas) are examples of One-Stop centers where the case managers 

build a personal rapport to aid in the decision-making process, even though these three 

sites differ in the way in which case managers provide information and the degree of 

influence over customer choice. 

In the second approach, One-Stop centers use specialists on staff in working with 

customers in different capacities. This approach enables case managers to develop 

greater expertise in the subjects to which they are dedicated. Baltimore has the clearest 

division of labor, with case managers who specialize in job search during core and 

intensive services, and others who focus on assessment and career planning. Similarly, 

Detroit uses the specialization approach, because it contracts out all its One-Stop 

operations (for core and intensive services) and ITA processes to separate organizations. 

Customers there have access to different case managers during their training period and 
III-27

might change back and forth to ensure that they are receiving guidance from the staff 

member who was best qualified to help with any particular issue.

LIMITS ON CUSTOMER CHOICE THROUGH LOCAL POLICIES

Regardless of the customer-choice models that local sites selected, nearly all 

implicitly limit choice by exercising their authority to set limits on the ITA dollar 

amount and duration, and they also restrict how ITA funds can be used. All sites also 

implicitly narrow choice by identifying occupations in demand in the local economy for 

which training will be supported. Other types of limitations such as residency 

preferences or formal linkages to local economic development are uncommon, but did 

occur in a few areas.

Dollar and Time Limits

Dollar and time limits for the case-study sites are shown in Exhibit III-4. As the 

exhibit shows, of the 26 local areas we visited, all but two have established a dollar cap 

limiting the amount of the ITA they will fund. The average dollar limit among the 24 

areas with caps is about $5,000, ranging from a low of under $2,000 (West Central 

Missouri) to a high of $10,000 in Tarrant County and Three Rivers. A number of sites 

provide authority for staff to authorize ITAs that are higher than the local cap, if special 

circumstances warrant, while two other sites (Portland and Indianapolis) allow their 

One-Stop centers to establish their own caps. As with most other aspects of the ITA 

system, local areas generally do not distinguish between dislocated workers and adults 

by setting different caps. Only one center in Metro Portland was an exception, in that 

it caps its ITA for adults at $1,200 per year while dislocated workers could be awarded 

up to $1,500 per year. 

In setting their dollar caps, all Local Boards strive to maintain some financial 

discipline, both for customers who otherwise have no incentive to economize in their 

choices and for vendors that might price their programs at whatever cap the Board 

sets.10 But in establishing a specific amount, each Board must make a trade-off 

between maximizing customer choice and maximizing the number of customers who 

can get training, subject to their allocations and general emphasis on training. The 

Boards that established a relatively high cap (say, over $6,000) are to some extent 

maximizing customer choice of vendors by excluding only what they consider to be the 

 

10 We found some evidence that vendor prices were sensitive to the cap. See Chapter V for a 

discussion of this issue.
III-28

Exhibit III-4

Local Limits on Amount or Duration of ITA

State Local Area Dollar Limit Time Limit

CA SELACO $5,500* None

CT Southwest Connecticut $3,000 1 year*

GA Atlanta Regional Commission $8,000 2 years

Northeast Georgia $5,000 2 years

IN Indianapolis $2,500* 2 years

North Central Indiana $2,500* None

Tecumseh Area Partnership $2,500 None

MD Baltimore $7,000 1 year

MI Detroit $5,000* 1 year

Macomb/St. Clair $5,000 2 years

MO St. Louis County $5,000* None

West Central $1,700 2 years

NC Capital Area/Raleigh $5,600 2 years

Region Q $8,600 2 years

NE Greater Lincoln $5,000* 2 years*

Greater Nebraska $6,000 2 years

Greater Omaha $4,000 2 years

OH Cincinnati $5,000* None

Ottawa County $5,000 Variable

OR Metro Portland None** None

PA Philadelphia $6,000 2 years*

Three Rivers Wrkforce Board $10,000 2 years

TX Golden Crescent None 2 years

Heart of Texas $5,000 2 years

Tarrant County $10,000 2 years

Texoma $5,000 2 years

________________

* Limit can be waived with approval.

** The Portland LWIB allows each center to establish its own caps. The center 

we visited used a cap of $1,200 per year for adults and $1,500 per year for 

dislocated workers.
III-29

most costly programs. The consequence of a high cap, however, is that there are likely to 

be fewer ITAs available if customers do elect these high-cost alternatives. By contrast, 

the larger number of the demonstration sites that set their caps around the mid-point of 

$5,000 are effectively optimizing the two competing objectives of increasing choice and 

the numbers of ITAs available. And finally, a few sites set their caps at quite low levels 

so that customers are virtually precluded from using proprietary vendors, except for very 

short-term training programs, but presumably have a larger number of ITAs to issue.

In deciding on the cap amount, a few sites had a specific reference point in mind. 

For example, Philadelphia adjusted its cap annually, based on the median cost of all 

programs that it had approved for ETP eligibility during the prior year, while MacombSt. Clair pegged its cap at the cost that the local community college charges. Similarly, 

North Carolina required any local cap to be set above the cost of training at the 

community college in order to encourage meaningful competition between proprietary 

vendors and community colleges, which dominated training throughout the state.

The other type of limitation that was nearly universal was on the duration of an 

ITA, which was established by all but five sites. With respect to the limits on duration, 

the local areas reached a fairly high degree of uniformity. Nearly all the local sites, 

including those with no dollar cap, set a two-year duration on the ITA, although a few 

(Baltimore, Southwest Connecticut) established one-year limits. 

Local respondents, both staff and ITA holders, suggested that the duration 

limitation had little impact on the typical length of training, because most trainees were 

interested in short training periods. 

Permitted Uses for the ITA

Nearly all the sites permit ITAs to be used for tuition, fees, books, uniforms, and 

equipment. Only one site limits the ITA to the tuition, while one other prohibited its 

use for fees. Only two sites include some supportive services within the ITA, such as 

reimbursement for transportation costs and child care, and thus make it subject to the 

cap. One of the sites that keeps its supportive services separate still linked the amount 

of those services to the size of the ITA. In that site, supportive services could not 

exceed 20 percent of the ITA amount.

The inclusion of supportive services within the ITA amount is potentially a strong 

effort to introduce market discipline into the ITA process, at least to the extent that a 

customer is considering selecting a vendor at or near the cap. Under those 
III-30

circumstances, a customer might have to consider trading off tuition costs against the 

availability of financial support, a trade-off that consumers make every day in the 

commercial marketplace. 

Demand Occupations

While the law requires that all training occur for demand occupations, sites define 

this requirement in different ways. A few local areas (for example, Macomb-St. Clair, 

SELACO, Southwest CT) rely predominantly on a list of demand occupations 

developed by the state, while a few others (e.g., Indianapolis, North Central, Detroit, 

and Northeast Georgia) rely entirely on locally developed lists. Most, however, use 

state labor market information but rely on the discretion of front-line staff whose 

knowledge of their respective local labor markets is considered quite good. A number 

of local areas (Baltimore, Philadelphia, St. Louis County, Portland) will accommodate 

customers who want to train in an occupation not on the demand occupation list, if they 

can demonstrate (e.g., through a letter from an employer) that a job is available in the 

field for which they want to seek training; by contrast, some other local areas (e.g., 

Macomb-St. Clair) allow such waivers only with Local Board approval.

Several local areas sought to achieve certain local policy objectives by placing 

additional restrictions on demand occupations to support high quality jobs or to enhance 

the economic development potential of their ITA system. For example, Texoma limited 

its ITAs to occupations that paid benefits, were linked to career ladders, and showed 

lower-than-average turnover. Three Rivers was considering linking ITA amounts to 

important growth sectors in the local economy, as follows:

The Three Rivers Local Board was considering a new policy in which the 

amount of the ITA would vary with the occupation chosen. Under the 

policy, the Local Board would reserve ITAs with the highest permissible 

funding amounts for training in jobs that are considered critical for regional 

economic growth. The absolute maximum would apply for the occupations 

that the Local Board believes are most important for the local economy and 

that pay the highest wage. Other occupations would be capped at a 

stipulated percentage of the maximum amount, varying with the relative 

importance of the occupation to economic growth in Pittsburgh and 

surrounding Allegheny County and its ability to lead a customer towards a 

living wage, which is expected to be in the $8-$10 range.

Residency Preferences

A few sites also establish a residency preference for those who wish to access an 

ITA. Even though WIA removed JTPA’s residency requirement for service to 
III-31

economically disadvantaged adults, several sites within the demonstration have decided 

to impose residency preferences on the issuance of ITAs. For example, two local areas 

(Greater Lincoln and Detroit) adopted policies to limit service to state residents. 

Several sites also limited service based on local residency. For example, the Atlanta 

Regional local areas give priority when funds are short to residents of their respective 

local areas or to dislocated workers formerly employed in those local areas, on the 

grounds that serving those outside the area comes with no additional funding. 

Philadelphia and Detroit provided similar priorities. 

SERVICES PROVIDED TO TRAINEES DURING AND AFTER TRAINING

The extensive investment in the ITA by both the customer (time and effort and 

opportunity costs) and the local area (staff time and training outlays) raises the stakes 

for training outcomes. Thus, both parties have a strong interest in ensuring the 

successful completion of training and achieving a good placement. In addition, given 

WIA’s emphasize on performance outcomes measured over the six-month period 

following the customer’s exit from services, local areas might be expected to 

demonstrate increased emphasis on providing follow-up services.

Services during Training

For the customer, a variety of counseling services may be beneficial or necessary 

to assure training completion. The One-Stop center staff, in turn, will want assurance 

that the customer is attending classes and otherwise making progress in the training, 

and will want the opportunity to provide additional services if they are needed. Thus, 

sites find it important to maintain periodic contact with customers while they are in 

training. All the sites carried out these functions in some way, but they differed 

considerably in the types of services offered or required and in who carried out these 

various responsibilities and services.

Virtually all sites want trainees to keep in periodic contact with their case 

managers during their training period to verify attendance or to meet with the case 

manager. The most common period used is a monthly meeting or report. However, 

sites differ in whether the responsibility for contact lies with the trainee or the case 

manager. In some sites, the trainees need to check in to file an attendance or progress 

report and could then elect to see a case manager to discuss any concerns that they have 

with their training. Trainees are also supposed to report any problems that might 

interfere with the completion of their training program, such as health or family 

emergencies. The Greater Omaha site wants contact on a more frequent basis. 
III-32

Trainees there have to bring in attendance reports signed by the training instructor at 

least every two weeks. This visit coincides with the time customers pick up bus passes, 

an important supportive service for many of them. When the customers come in, the 

case manager gets a chance to talk with them informally, review their progress, and 

discuss any concerns that they have with their training.

Case managers have parallel responsibilities to monitor a trainee’s progress. All 

sites have their case managers monitor progress in the training in some form. In those 

sites that require a personal visit, case managers meet with their customers to solve 

personal problems that might interfere with completion of the program or to review 

supportive services needs. In some sites, such as Southwest Connecticut, where the 

trainee is not required to come in, the case manager checks monthly with a trainee to 

learn about attendance and any concerns with the training. The case managers in some 

sites, such as Capital Area, may even make a home visit, if there was a concern about a 

customer’s attendance. Another site sends trainees periodic surveys mailed to their 

home; this approach was developed partly because case managers have very heavy 

caseloads, which make routine personal contacts infeasible. 

Finally, most of our sites required their vendors to report progress or to identify 

when students have particular problems. Five sites specifically require the vendors to 

provide periodic progress reports, most often on a monthly basis. One site (Berks 

County, a local-area partner on the ITA/ETP Demonstration grant with Pennsylvania) 

used grant funds to develop a system whereby vendors could electronically submit 

information about students’ attendance and progress to One-Stop center staff.

Nearly all vendors also provide important services during training. For example, 

they have financial aid case managers to assist with applications for Pell grants and 

other forms of financial support, and a counseling staff to provide training in study 

skills or help their students with general problems. 

However, many of the front-line staff we spoke with noted that active counseling 

is much more prevalent among the proprietary schools than public institutions, which 

are predominantly community colleges. The community colleges usually have similar 

counseling available, but usually the student must take the initiative to use such 

services. The proprietary schools, on the other hand, are very active in bringing those 

services to their students. The proprietary schools are also generally better at 

monitoring student progress and reporting to the One-Stop centers than the colleges. 

Baltimore staff, for example, noted that their proprietary schools provide greater levels 
III-33

of support because they are much more concerned with demonstrating the institution’s 

high performance. A case manager in Jonesboro (who herself is an employee of a 

college that operates the One-Stop center and is a training vendor) spoke about how 

much better the proprietary schools are at reporting to the case manager information 

about students who are experiencing difficulties. 

Despite these general patterns, in some places the One-Stop staff reported that the 

community college also does a good job of communicating progress and supporting 

student progress. For example, in Greater Omaha and Greater Lincoln, both of which 

have few private training schools, the community colleges provide good counseling 

services and a host of other supportive services, such as on-site child care and support 

groups for people entering non-traditional employment.

Placement Services

WIA’s performance management requirements make good placement services 

extremely important for all One-Stop customers, including ITA holders. The sites are 

fairly evenly divided as to whether to make the One-Stop center, the vendor, or the 

customer responsible for placement. Sites that make the One-Stop centers responsible 

typically want the customer whose training is ending to come in for job search 

workshops, counseling sessions, meetings with job developers, and help in preparing a 

resume, and to use other core and intensive services. For example, Metro Portland has 

the case manager refer the job seeker to an in-house job developer. Capital Area, 

which makes its case managers responsible for outcomes, asks its trainees to come to 

the One-Stop center two weeks before completion of their training to start using job 

search resources, and in Greater Nebraska customers are to come into the center three 

months before their training is to end to begin developing a placement strategy. 

A slightly larger group of sites rely on placement by vendors (including 

Southwest Connecticut, Northeast Georgia, the Atlanta Regional Commission, 

Baltimore, Detroit, Greater Lincoln, Greater Omaha, and Philadelphia). In these areas, 

the local boards require the vendors to assist program graduates with getting jobs. 

Providing a strong incentive for them to do so, Southwest Connecticut pays vendors 

using a benchmark system, according to which 25% of the vendor’s payment is 

withheld until the customer obtains a training-related placement (an additional 25% is 

also contingent on the customer’s completing the training). Similarly, a technical 

training school in the Pittsburgh area has full-time job developers who get bonuses for 

exceeding placement targets. The vendors in the Atlanta Regional local areas 
III-34

predominantly also have formal placement programs, some of which we describe 

below:

The vendors in the Atlanta area use a variety of means to provide placement 

assistance. One vendor forms job clubs and has job developers on staff. 

One computer training school that serves predominantly low-income, 

minority students integrates job search into its curriculum and makes all 

staff responsible for placement outcomes. On the other hand, another 

computer training school that serves predominantly upper income dislocated 

workers among its WIA customers facilitates self-directed job search using 

an informal network of the school’s graduates. As an adjunct to this vendor 

activity, an Atlanta Regional local area case manager brings his customers 

to the One-Stop center 60 days before completion to begin job search. 

As with counseling services, proprietary schools generally seem more proactive in 

helping trainees with job placement than public colleges are.

Finally, several sites give the customer the choice of whether to rely on vendor or 

One-Stop placement services, taking advantage of the customer’s obvious interest in 

prompt and remunerative employment. SELACO notes that most of its trainees take 

advantage of vendor services, but that case managers encourage their customers to 

come in if they think that they need extra help. 

Regardless of who is primarily responsible, all the One-Stop centers indicate that 

their full job search facilities, from core and intensive services, are available to those 

who complete training. 

COORDINATING THE ITA WITH OTHER SERVICES

Seamless service to customers is an expected feature of the integrated system 

intended by WIA, as is the leveraging of funds from multiple sources. Thus, in order 

to understand the full array of resources and services that trainees are accessing, we 

must understand 1) how service is integrated across programs and 2) how WIA funds 

are used in conjunction with funds from other sources. In fact, in most of the 

demonstration sites, the ITA itself is only one part of a complex mosaic of funding

sources that contributes to training of ITA holders. These sources include Pell grants 

from the U.S. Department of Education, trade-related dislocation programs through the 

Unemployment Insurance system, vocational rehabilitation, state training or higher 

education grants, and welfare sources, to name the most common. Further, WIA 

requires that none of its appropriated funds supplant other federal funds otherwise 

available to support employment and training services. 
III-35

With respect to service integration, we found that co-enrollment and other forms 

of collaboration are highly developed in some sites but weak in others. At the one 

extreme, the two sites in Indiana are notable for an especially high degree of 

collaboration across the various One-Stop partners. Thus, Indianapolis has about onethird of its ITA customers co-enrolled in welfare-to-work, vocational rehabilitation, or 

Trade Adjustment Assistance. Similarly, Tecumseh Area Partnership, also in Indiana, 

has quite a few ITA holders enrolled in vocational rehabilitation. At the other extreme, 

many sites had little or no co-enrollment or active collaboration. The predominant 

explanation for lack of collaboration was that their One-Stop systems were still being 

built, and they expected greater collaboration in the future. However, we did not 

observe significant increases in coordination by the second round of site visits.

Of the One-Stop programs, the trade-related dislocation programs, including the 

Trade Adjustment Assistance and the North American Free Trade AgreementTransitional Adjustment Assistance programs, are perhaps the most successful in 

obtaining coordinated enrollments. These programs provide training, job search, and 

extended Unemployment Insurance benefits for dislocated workers who suffer a job loss 

because of trade impacts. However, worker eligibility for trade benefits, which are 

administered in conjunction with the Unemployment Insurance system and apply 

generally only to producers of manufactured goods, are contingent on a determination 

by DOL that a plant closing or layoff was due to foreign trade. Thus, in many of the 

local areas we visited, there is little applicable dislocation and consequently little use of 

the trade programs altogether, so collaboration is a moot issue.

In those local areas that sustained trade-impacted dislocations, there is a mixed 

record of coordination. Most of the local sites we visited reported that coordination 

occurs infrequently or is limited in scope because of limited programmatic demand for 

coordination. Respondents offered a variety of reasons for the low level of 

coordination, but the most common reasons were funding and organizational 

difficulties. Several sites indicated that the trade programs are well funded and do not 

require ITAs to supplement the trade program training allowances. Several others cited 

organizational difficulties. For example, Northeast Georgia, whose dislocated workers 

in the textile and clothing industries are predominantly eligible for trade benefits, found 

little basis for coordination, largely because the trade programs are entirely 

administered in the Unemployment Insurance central office with which coordination has 

been difficult. 
III-36

On the positive side, seven local areas (West Central Missouri, Philadelphia, 

Pittsburgh, and Allegheny County in Pennsylvania, Indianapolis, North Central, and 

Tecumseh Area Partnership in Indiana, and Golden Crescent in Texas) reported 

excellent coordination. In general, while the ITA was a secondary payer, the ITA 

could be committed before the trade stipend was approved. Thus, usually the ITA is 

used to get a trainee enrolled in a program, and the local area receives reimbursement 

once the trade amount is approved. Thus, coordination between the major trade 

programs with the general training programs, which DOL has been promoting over the 

past several years, appears to be having at least some success. 11

State efforts to foster such coordination have also been important in some cases. 

For example, in Pennsylvania, state officials adopted a coordination policy that requires 

mandatory co-enrollment of customers. The policy also requires the trade programs to 

use the eligible provider list even if there were no ITA to supplement the trade benefits. 

Case managers at Allegheny County pointed out that the incentive to train was so strong 

and knowledge of the program was so widespread in the community that nearly all 

eligible dislocated workers came in specifically for training services and that it was 

essential to coordinate the two programs to ensure as much money as possible is 

available to meet their needs. 

Among other efforts at coordination, in keeping with the terms of the WIA 

legislation, coordination with the Pell grant occurs at virtually all sites.12 In managing 

this coordination, these sites generally adhere to WIA’s requirement that Pell funds, 

among other sources of funds, should be spent before obligating an ITA. However, 

because the ITA amount could be determined and committed immediately, in general 

ITA funds are typically used to begin a training program and then Pell grants are used 

to provide reimbursement from the training vendor once the Pell grant is awarded. 

Sites either considered an estimated Pell award in preparing the ITA, as in Indianapolis 

and Capital Area, or the eventual grant effectively reduced the amount of the ITA, as in 

 

11 The Trade Adjustment Assistance Reform Act of 2002 amends the TAA program to 

substantially increase the duration of allowable benefits for Trade Readjustment Allowances (see ETA’s 

Training and Employment Guidance Letter No. 11-02). This provision may further enhance the 

attractiveness of coordinating WIA with TAA resources for customers undergoing training.

12 The Pell legislation contains a provision requiring that other federal financial aid programs not 

consider an individual’s eligibility for Pell when computing available resources for that person. At the 

time of the site visits, DOL and the Education Department have issued regulatory guidance that 

coordination between the two funding sources should occur. 
III-37

Cincinnati, Baltimore, and West Central. In a departure from this pattern, several 

other sites pay for tuition through the ITA but use the subsequent Pell grant to defray 

the cost of supportive services or provide living expenses for the customer.

State programs were another important source of funds in some cases. For 

example, in the Atlanta Regional local areas, the presence of a state scholarship 

program funded through lottery revenue is central to their ITA strategies. These 

scholarships have no income restriction and are thus available to virtually all residents 

who meet minimum academic requirements. They must be considered the first source 

of payment for training, so residents must apply for these scholarships as well as for 

Pell grants. The combination of these funding sources makes the ITA a relatively small 

player in the training marketplace. Pennsylvania also has a state higher education 

assistance program that could offset the ITA. 

Personal financial contributions are also a common form of additional support. 

Many of the sites indicated that they expected a personal financial contribution from the 

trainee wherever the ITA amount or other financial resources did not cover the cost of 

training, supportive services, and other related costs. These amounts are figured into a 

financial plan that the sites required customers to prepare as part of the Individual 

Employment Plan. Southwest Connecticut is perhaps the most vigorous site in 

obtaining contributions as a matter of policy. The local board believes that a personal 

financial contribution will increase a customer’s personal stake in the training and the 

outcomes. Although not widely used, that local area also offered low-cost loans 

sponsored by private companies and corporate-funded scholarships to ensure that there 

would be additional resources if they were needed. 

SUMMARY

One of the key reform principles that DOL has developed for the implementation 

of WIA is increased customer choice. This is clearly evidenced in the ITA system, 

which explicitly guarantees customers the right to choose eligible vendors. Yet, WIA 

also sets that choice within a regulatory framework to assure that Congressional goals 

are furthered and ensure that, when funds are limited, resources are available to serve 

those (e.g., low-income workers and welfare recipients) whose needs have long been 

recognized. This regulatory framework effectively constrains choice to some degree. 

We observed that the policies, practices, and procedures that local areas used in 

implementing the ITA system were quite effective in creating a balance between these 

two objectives. We observed that each tendency is moderated by a major factor that 
III-38

brings it closer to its counterpart. For example, customer choice in the ITA system is 

not the unrestricted choice of the commercial marketplace. Rather, it is constrained by 

the training provider eligibility requirements, limits on training occupations and ITA 

amounts, and requirements that customers demonstrate their need for training and 

participate in pre-training activities designed to promote reasonable guidance by 

professional staff. Conversely, the implementation of these potential constraints on 

choice are marked by considerable flexibility. For example, cap and duration limits are 

often quite generous and can sometimes be waived in any case, and our findings on the 

path that customers follow through pre-ITA services clearly show a high degree of 

flexibility and responsiveness to customers.

At the same time, the scope of our study does not cover very important questions 

about the effectiveness of these processes in promoting high-quality customer outcomes. 

Overall, our qualitative work suggests that most of the ITA/ETP Demonstration sites 

have tools and processes for customers to use in making well-informed decisions. 

However, the evidence is strong that many, perhaps most, customers have a preexisting commitment to training and to particular vendors. In the interests of being 

responsive to customers’ needs, local areas consequently telescope the customer’s 

decisionmaking process considerably, potentially yielding poor quality decisions. 

Whether the workforce system, and customers, would be better served by 

mandating explicit decisionmaking processes using established tools—even at the 

sacrifice of flexibility—can only be adequately addressed through the experimental 

evaluation of alternative customer choice models that is currently underway. As 

described in Chapter I of this report, this experiment is testing three different 

approaches that vary with respect to the amount of guidance that customers are required 

to receive before being awarded an ITA and in the discretion they are permitted in 

making a decision that the case manager may view as inappropriate. On the one 

extreme, one approach requires that the customer carefully research at least three 

alternative training programs to identify the one that is believed to yield the greatest 

return on investment; at the other extreme, customers need not undertake any research 

beyond what is required through intensive services and will generally be free to select 

any training program on the eligible training provider list. Results of the experiment 

will thus yield important information about optimal decision-making approaches from 

the standpoint of return-on-investment calculations. 
III-39

Attachment to Chapter III

Customer Profile

Below we have developed a composite profile of a dislocated worker customer to 

illustrate some of the key themes in the services provided and the methods of choice. 

This composite profile is based on interviews and file reviews of 22 customers during 

the second round of site visiting. All of these customers were ITA holders and most had 

not yet completed their training. Some details of their experience, however, derive from 

our other findings about the course that customers take in obtaining an ITA.

Our prototypical dislocated worker is a white man, aged 41, with slightly more 

than a high school education. He is an Unemployment Insurance claimant who is still 

in his benefit period and using his benefits to finance his training. 

He came into the One-Stop center specifically to obtain training and had some 

idea of both the occupation for which he wanted to train and the vendor to provide that 

training (it is possible that he may have received a reverse referral). Although he was 

initially upset at having to test his skills against the labor market in core services, he, 

nevertheless, acquiesced. However, he made only cursory use of labor-market 

information on the Employment Service’s integrated system because he was quite 

confident that his expected training occupation was in strong demand with high wages. 

To nobody’s surprise, he moved on to intensive services after a finding by staff that he 

would not be able to replace his pre-layoff wage through Employment Service job leads 

and other core services. 

He continued to make good progress in intensive services, where he developed an 

IEP that focused entirely on training. The IEP, however, was developed without the 

benefit of formal assessment instruments. Our dislocated worker felt that there was no 

question about his ability to complete the training or about his real interest in the 

occupation, and the case manager agreed. He scheduled his meetings with his case 

manager at regular intervals and was reasonably diligent in conducting field research in 

between those appointments. For example, he visited two vendors to look at their 

facilities and find out about outcomes of their completers. He also talked with trainees 

at the schools as well as with two potential employers. 

Overall, our dislocated worker spent almost five weeks in core and intensive 

services and another five weeks making the final decisions about the occupation and 

vendor. The latter included a presentation to a local-area ITA-review committee that 

had final approval authority. Most of the latter period is attributable to a wait for the 

training program to begin. 

This customer decided to train in an information technology occupation that he 

had wanted from the time he first came into the One-Stop center. However, based on 

the participant research, he decided to select a different proprietary vendor than the one 

he had in mind when he first came to the center. The actual training vendor had a 
III-40

somewhat shorter duration program, was closer to his home, and had more up-to-date 

equipment than the vendor he initially had in mind. 

There were no problems with any of the local policy limitations on the ITA. The 

tuition, books, and fees for this program were well within the local area’s cap, and the

program’s short duration did not raise any conflict with the area’s two-year limit. His 

selection of an information technology program was for an occupation in great demand.

Overall, he felt that he received good service from the One-Stop center, even 

though he would have preferred to move to training immediately. The advice from his 

case manager, especially in comparing vendors, was perhaps the most important service 

he received.
IV-1

IV. DEVELOPING THE ELIGIBLE TRAINING PROVIDER LIST

Customer choice and system accountability are two important elements embodied 

in the WIA legislation that to some degree could be viewed as being in tension with 

each other. With respect to customer choice in training services for adults and 

dislocated workers, WIA empowers customers, through the issuance of ITAs, to make 

their own decisions regarding the training program and vendor that suit them best. At 

the same time, WIA’s emphasis on system accountability works to promote high 

standards of system performance as measured by customers’ post-program employment 

success and other outcomes—an objective that some program operators might feel is 

jeopardized by allowing customers too much leeway in making ill-considered training 

choices.

To a large degree, the tension between these objectives is mitigated by ensuring 

that customer choice is supported by ample and good quality information from the OneStop system. Thus, as we have discussed in the previous chapter, One-Stop centers are 

actively working to help customers make informed choices, by making labor market 

data available, providing assessment and counseling services, conducting workshops, 

and having ITA holders conduct informational interviews with vendors. Also critical, 

according to the WIA legislation [e.g., Section 134(d)(4)(F)], is that customer choice 

will be supported through an Eligible Training Provider (ETP) list. Thus, ITAs can be 

redeemed only by vendors whose programs are “eligible”—certified by states and local 

workforce areas as meeting acceptable levels of performance. Similarly, performance 

and other relevant information about the eligible programs should be assembled in a 

Consumer Report System (CRS), to aid customers in comparing programs to identify 

the one that is right for them. According to the vision embedded in the legislation, the 

ETP list and CRS are thus linchpins of the ITA system, serving as key resources to 

promote the full information that is necessary to support customer decision-making. As 

such, they help local workforce areas and states provide choice to customers while still 

adhering to standards of high system performance.

In this chapter, we provide a discussion of provider eligibility by detailing the 

considerations that sites are using in building their ETP list, including what they define 

as a program, how they established initial eligibility, and their plans for subsequent 

eligibility and performance measurement. The next chapter will provide an assessment 

of the quality and completeness of the Consumer Report Systems.
IV-2

WHAT COUNTS AS A “PROGRAM”?

Although called an eligible training provider list, the ETP list should more 

properly be thought of as a list of eligible training programs. In keeping with this, 

vendors need to apply for eligibility for each of the training programs for which they 

are seeking ITA eligibility. This requirement poses the question of what should count 

as a training “program” for eligibility purposes.

The Final Rule for the WIA legislation, issued by DOL in August of 2000, 

provides some guidance by defining a program of training services as

One or more courses or classes, or a structured regimen, that, upon 

successful completion, leads to:

a) A certificate, an associate degree, baccalaureate degree, or

b) The skills or competencies needed for a specific job or jobs, an 

occupation, occupational group, or generally, for many types of jobs or 

occupations, as recognized by employers and determined prior to 

training (20 CRF 663.508).

In defining a training program for purposes of assembling their ETP lists, states 

are generally closely adhering to this definition—or, in actuality, the variant of it that 

was a part of the Interim Final Rule.1 Thus, in their policy issuances, Georgia, 

Nebraska, North Carolina, and Pennsylvania, among others, explicitly use the language 

in WIA’s Interim Final Rule.

Some states provide additional clarifications or include other limiting factors. 

Thus, a few states, such as Connecticut and Oregon, require that a program lead to the 

acquisition of a certificate or some other credential before it can be considered for ETP 

eligibility. Typically, this policy was adopted explicitly or implicitly to ensure that 

persons who undertake WIA-funded training would have the reasonable prospect of 

attaining a credential, for purposes of ensuring the state’s good progress towards 

attaining its negotiated performance levels on the credentialing rate. The credentialing 

rate is one of the core measures of performance on which states and local areas are 

 

1

 DOL issued WIA’s Interim Final Rule on April 15, 1999. This rule defined a program of 

training services as “(a) one or more courses or classes that, upon successful completion, leads to: (1) a 

certificate, an associate degree, or baccalaureate degree, or (2) a competency or skill recognized by 

employers, or (b) a training regimen that provides individuals with additional skills or competencies 

generally recognized by employers.” State policy guidance that we saw during our site visits typically 

used the language from the Interim Final Rule, rather than that of the Final Rule.
IV-3

evaluated, and includes in its calculations those adults and dislocated workers who 

undertake training services. Thus, by limiting the definition of training to programs 

that award a certificate or other credential, these states ensure that their measured 

performance on the credentialing rate will not be undermined by persons undertaking 

training that holds no prospect of leading to a credential of any sort. 

Vendors argue that the exclusion of such non-credential course sequences as a 

consequence limits customer choice and makes it difficult to tailor programs of 

instruction to meet students’ needs. Partly for this reason, states that restrict the 

definition of training only to credential programs have encouraged vendors to develop a 

certificate specifically for course offerings that would otherwise not result in one.

Oregon’s experience best illustrates this scenario. This state’s definition of 

training is bound by a state law that references the awarding of a credential. According 

to this law, training is defined to be:

Any organized teaching or learning activity with open enrollment of which 

successful completion qualifies a student for a degree, a certificate of substantial 

academic or vocational learning short of a degree, a certificate of preparation 

related to new or modified occupational licensure, or other academic or 

vocational certificate that represents a shorter period of activity but has value as a 

public credential.

By adhering to this definition, the state was unable to award ETP eligibility for 

non-credential programs of study and, consequently, decided to classify many of them 

as intensive services. Thus, as described in the previous chapter, the state defined 

intensive services to include both Professional Development courses, consisting of 

programs of instruction of less than 210 hours, and Individual Employment courses, 

consisting of programs of instruction of between 210 and 600 hours; the former was 

designed to accommodate the service needs of customers who wanted to undertake very 

short-term courses of instruction, and the latter was aimed at those who needed 

somewhat more skills training but wanted to pursue an “individualized program/course 

of study based on their specific needs for employment.”2 

Due to its low expenditures of WIA training dollars during the first year of WIA, 

however, Oregon faced scrutiny from DOL and was asked to modify its approach. It 

 

2

 CCWD Policy “Design to Accommodate a Continuum of WIA Intensive and Training 

Services,” December 28, 2000: p.4.
IV-4

did so within the constraints of state law by allowing special credentials to be awarded 

for individualized programs of study, so long as satisfactory completion could be 

documented. In developing this policy, state administrators identified that most 

customers undertaking the shorter programs of study that were being classified as 

intensive services were doing so at community colleges. They realized further that they 

could track completion for individual community college courses through OCCURS, 

the state’s community college data system. Given this context, the state decided that 

customers could henceforth be issued an ITA for these otherwise non-credential 

sequences of courses, so long as the sequence was identified in the customer’s IEP. 

Moreover, the community colleges were authorized to award a specially designed 

Employment Skills Training certificate for successful completers of these individually 

tailored courses of instruction. Through this solution, ITA-funded training could 

always potentially result in a degree or certificate, consistent with the definition of 

training embedded in state law and in furtherance of the state’s measured performance 

on the credentialing rate, while still ensuring that customers could undertake a wide 

range of training choices.

Oregon’s experience highlights the difficulty states encountered in classifying, for 

ETP purposes, non-credential programs or sequences of courses that are in fact quite 

common in community college settings. Although the specific approaches vary, states 

generally handle this dilemma by allowing vendors to self-define what constitutes a 

program for which they want to seek eligibility. Vendors, thus, specify the course or 

course sequence and completion requirements as part of the application packet. For 

example, Michigan allows vendors to submit eligibility for individual courses or 

sequences of courses, whatever they feel is most appropriate. For non-credential 

programs in Maryland, vendors have the discretion to “block” related courses into a 

“bundle” that they want to get approved on the ETP list. Taking a different tack, 

Pennsylvania clarifies that non-credential courses should be certified separately. Thus, 

according to Pennsylvania’s approach, if a single course is all that is needed by 

individuals to meet their employment goals, that course must be separately certified.

Finally, with respect to what counts as a program of training, we investigated 

state policies regarding whether identical course sequences offered by the same vendor 

but at separate locations needed to be separately certified. Although state policies were 

not often very clear on this point, we identified some states that adopt each approach. 

For example, California stipulates that “identical programs offered in different locations 
IV-5

by the same training provider shall be considered as one program, and will not require 

separate applications unless the regulatory agency uses location as a factor in defining a 

unique program.”3 Georgia and North Carolina have a similar policy. By contrast, 

both Pennsylvania and Texas clarify that training providers offering the same program 

of instruction at multiple sites around the state are to treat the program at each locale 

separately for purposes of the ETP application process. This policy was viewed as 

extremely burdensome by some colleges and universities with identical programs at 

multiple locations throughout the state, and was a factor that caused some of them, such 

as Pennsylvania State University, to think twice about applying for ETP eligibility. 

Similar concerns were voiced by vendors in other states with similar policies. 

GENERAL APPLICATION AND INITIAL ELIGIBILITY PROCEDURES

Developing the initial eligibility application process was an important first step in 

building the ETP list. Several key components of this process were marketing to elicit 

applications from vendors and developing the actual application and approval 

procedures.

Marketing Efforts by States and Local Areas 

States in our study generally adopted the approach that they wanted their initial 

ETP list to be as inclusive as possible. They felt that the first order of business should 

be to develop an extensive list of ITA-approved vendors for initial eligibility—a Yellow 

Pages, as one respondent called it—and then worry about developing performance 

criteria to winnow the list for subsequent eligibility. As such, they generally worked 

aggressively to market the ETP system and induce vendors to apply for initial 

eligibility. 

With respect to marketing and disseminating applications to vendors, both states 

and local areas played important roles, though the balance between them varied 

somewhat. In Nebraska, for example, ITA/ETP grant funds were used to hire a staff 

member at the state level whose duties included contacting vendors to encourage them 

to apply for ETP eligibility. In Georgia and Pennsylvania, by contrast, the state 

identified local areas as predominantly responsible for the solicitation of providers’ 

applications. 

 

3

 EDD Policy “Workforce Investment Act Eligible Training Provider List Policy and 

Procedures,” April 2002.
IV-6

In other states, there was greater balance between state and local roles. Often, 

for example, relevant state agencies helped identify potential vendors for inclusion on 

the state list by accessing existing lists of training providers that were licensed or state 

certified. These states or their local areas then sent these vendors applications or 

information on the ETP application process and encouraged them to apply. For 

example, Maryland identified all vendors in the state that were certified by the 

Maryland Higher Education Commission and who were thereby authorized to market 

training services in the state. Local areas were then asked to follow up, to encourage 

those vendors to submit an application for ETP eligibility. Connecticut followed a 

similar approach. Local Boards in this state sent a notification and pre-application 

packet to all known and interested providers in the state, including those approved by 

the state’s Department of Higher Education and others that the local areas had worked 

with in the past. Additional marketing efforts included posting public notices or 

announcements, running newspaper advertisements, and the like. 

States and/or local areas also organized periodic regional informational 

workshops for vendors to attend. For example, Southwest Connecticut and SELACO 

had periodic meetings with vendors in their areas to explain the ITA system and the 

application procedures, to complement informational sessions conducted by the states. 

Cincinnati was notably aggressive in its marketing efforts, holding quarterly meetings 

with vendors that continued beyond initial eligibility. Similarly, Ohio organized 

periodic workshops in various locations around the state to which vendors were invited 

to attend. These sessions were viewed as both vehicles for disseminating information 

about the ETP application process and as a way of eliciting input from vendors as to 

how they would like to see the eligibility process operate.

Some states undertook special marketing efforts for rural areas, where customers 

might have few choices without special outreach efforts. For example, Georgia worked 

hard to identify vendors who might be able to serve the state’s rural areas, in which 

customers might otherwise encounter severely constrained choices. The state also 

provided technical assistance to vendors to adjust their programs to help them meet 

training needs in particular areas.

Another special focus of marketing was on community colleges. As we will 

discuss in Chapter VI, community colleges in many states were very reluctant to apply 

for eligibility because they viewed the prospect of needing to submit performance data 

as overly burdensome and not worth the college’s effort in light of the small number of 
IV-7

ITA-holders that they anticipated serving. But, although community colleges did not 

feel themselves to be dependent on ITAs for much of their business, WIA workforce 

agencies by contrast felt themselves very reliant on community colleges as a resource 

for good quality and relatively inexpensive training. Without community colleges on 

the ETP list, in fact, workforce administrators felt that their customers’ training choices 

would be sharply curtailed, especially in less urbanized areas, where few alternatives 

exist. Accordingly, state and local officials sometimes undertook vigorous recruitment 

efforts to encourage community colleges to apply for ETP eligibility, often by 

appealing to their sense of community responsibility rather than leaving them to make a 

strict business decision. 

In Texas, for example, the Texas State Technical College, one of the largest 

training providers in the state, initially refused to go through the eligibility process and 

sent the Texas Workforce Commission and the local boards a letter explaining that the 

application process was too burdensome for the few ITA customers they were expecting 

to serve. After much convincing by TWC staff, the college agreed to apply, but 

warned that it would do so for only one year.4 In a different approach, the North 

Carolina Community College System is actually the manager for the state’s ETP 

system, which has obviously helped foster positive working relationships.

Application Procedures

With respect to application procedures, all of the states in this demonstration 

followed a fairly standard model based on the legislative requirements for the initial 

eligibility process. Thus:

1. The vendor completes an application to the ETP list for each of its 

programs.

2. The application is submitted to the local area board.

3. The local board reviews the application and forwards it to the state with 

a recommendation for entry on the ETP list. 

4. The state reviews the application and either denies or accepts the request 

for eligibility.

 

4

 This example occurred during Texas’s first year of subsequent eligibility. The Texas Workforce 

Commission has subsequently had some success in winning the longer-term cooperation of the state’s 

community colleges, as we discuss in Chapter VI.
IV-8

5. The provider and local board receive notification of acceptance or 

denial.

6. Providers who are denied entry to the list may choose to appeal to the 

state or local board following set appeal procedures. 

This commonalty notwithstanding, there were some administrative differences in 

the way the application process worked. First, states differed somewhat in the guidance 

they gave as to which local area the application should be submitted. Most states 

suggested that vendors should submit to the local area in which they intended to do 

business, or where their central offices were located, and further stipulated that vendors 

doing business in more than one area needed to submit only once, to a single area. For 

example, California’s policy directive notes that: “applications...must be initiated by the 

training provider and submitted to a local board where they wish to provide services. If 

services are provided in multiple areas, the provider may submit an application to just 

one local board to provide services in all of the areas.”5 California’s policy goes on to 

note that out-of-state providers are also invited to apply. Other states, such as 

Pennsylvania, were less prescriptive, and thereby implicitly allowed vendors to apply to 

any local area that they wished. Finally, a few others, such as North Carolina and 

Texas, suggested that vendors might want to submit applications to multiple local areas 

simultaneously, as a way of accommodating the separate eligibility requirements that 

each local area might have established.

Another difference was whether the vendors were to submit directly to the local 

area or to a central state clearinghouse, which in turn forwarded the application to the 

relevant area. The latter process occurred in Georgia, Michigan, Nebraska, and 

Pennsylvania, among others. For example, in Nebraska and Pennsylvania, vendors can 

access state web pages to enter their application information and submit it 

electronically, indicating in a drop down menu the local area to which they are 

applying. The state ETP administrator then forwards the applications to the relevant 

locals periodically, or in advance of upcoming Local Board meetings, so that the 

applications can be considered. The advantage of this approach, it was felt, was to 

streamline the process, so that vendors would not need to deal with separate and 

possibly idiosyncratic local application procedures or forms. This more streamlined 

process was viewed as removing a potential major impediment to vendors’ wanting to 

 

5

 EDD, “Workforce Investment Act Eligible Training Provider List Policy and Procedures,” 

April 2002: p. 6.
IV-9

apply, as well as substantially easing the burden on the local areas themselves. 

However, even in the states that were recipients of applications first, the local board 

still made the initial decision about whether the vendor’s application should be 

recommended for inclusion on the state list. For example, in Michigan, the state 

receives the application and then forwards it to the appropriate local boards for 

approval. The local boards in this example still retain the authority to give, or 

withhold, their approval. 

Also as a way of streamlining the application process, over half of the states we 

visited (including California, Michigan, Missouri, Nebraska, North Carolina, Oregon, 

Pennsylvania, and Texas) made it possible for vendors to submit applications 

electronically. Although the specifics vary, where electronic application is an option 

vendors submit a completed application over the Internet directly to a local board or 

state clearinghouse. Several of the states are allowing both paper and electronic 

applications as acceptable forms of submission, though at least one of the states will 

only allow electronically submitted applications once its electronic system is fully 

functioning. Finally, a few states (e.g., Indiana, Ohio) are still solely using a paperapplication process, although most are hoping to switch to electronic submissions in the 

future.

States that accepted electronic applications felt that doing so expedited the 

application and review process in a major way. In fact, some of the demonstration 

states (e.g., North Carolina, Pennsylvania, Texas) started out with a paper application 

and later moved to electronic submissions, and they remarked how much easier the 

latter made the entire process. For example, the burden of having to keypunch data 

entries could be entirely eliminated. Additionally, a preset electronic application 

ensures that all necessary information will be collected from the vendor, obviating the 

need for call-backs or otherwise dealing with incomplete applications. For example, in 

Texas, the electronic system will not allow an application to be submitted if certain data 

fields are incomplete or missing. With an electronic process in place, approval of 

applications also proved to be a much easier task, in that the software could check 

automatically that vendors met minimum requirements for approval to the ETP list, 

such as being state licensed. Thus, these electronic systems made the application 

process easier for vendors, local boards, and the state alike. 

As we discussed in an earlier chapter of this report, several states specifically 

used some of their ITA/ETP grant funds to develop their electronic submission 
IV-10

procedures. North Carolina, for example, used funds to upgrade its NC STARS. This 

software package has multiple functionality, but with one component that consists of an 

online vendor application process. Similarly, Texas used grant funds to develop the 

Training Provider Certification System (TPCS). This package allows local areas to 

customize information elicited from the vendor as part of the application (i.e., to 

impose additional requirements beyond the state’s core requirements).

Information Required for Application

All of the states require the same basic vendor and program information on the 

application form, though some states’ requirements go beyond these basics. The basic 

information includes vendor/institution name, contact person, address, and information 

on accreditation. Typical program data include name of program, whether certificates 

or degrees are awarded, hours of instruction, credits to be earned, and costs. 

Some application forms are extremely simple and streamlined, asking for just this 

basic information. Other states have somewhat more elaborate information requests. 

Among the other kinds of information that some states required were:

• Other attributes of the institution, including:

− Whether it provides job search assistance, placement assistance, 

assessment and counseling services, or on-site child care

− The institution type (e.g., public, for-profit, non-profit, other)

− Whether financial aid is available for students

− Whether the institution is eligible to accept receipt of Pell grants

− An attestation that the organization is EEO-compliant and is 

financially stable

• Other program information, including the:

− Program schedule (e.g., start dates, frequency of offerings, days 

per week, hours per day) 

− Various locations where the training is provided

− Criteria for admission or target audience

− Teaching delivery options (e.g., classroom, Internet, 

correspondence)

− Qualifications of instructors and adequacy of equipment

− Average class size

− Program goal (e.g., certificate, skill attainment, license, degree) 
IV-11

− Curriculum (e.g., titles of the courses that make up the program)

− Accessibility (e.g., whether parking is available, whether the 

location is convenient to public transportation, whether there are 

provisions for sign language)

Texas provides an example of an application form that is particularly rich in the 

kinds of information it asks vendors to supply. This state’s application is 13-pages long 

and asks the vendor to report on the program’s teaching delivery options, minimum 

entry requirements, entry recommendations but not requirements, average class size, 

instructor-to-student ratio, instructor qualifications, and equipment-to-student ratio. 

Moreover, local areas in Texas are able to add to these requirements by adding more of 

their own.6

Texas’s approach also highlights the tension between wanting to streamline the 

application process by developing a common application form that all local areas would 

use vs. accommodating local areas that wanted to add their own unique informational or 

other requirements. Allowing local areas discretion was viewed as their right, 

especially as the state moved to subsequent eligibility, pursuant to WIA’s regulations 

(see 20 CFR 663.575 and 20 CFR 663.510). Thus, as noted above, many states 

centralized the vendor application process to facilitate initial eligibility for both vendors 

and the local areas; they did so by developing a common application form and 

automating the application process through a state-supported web page. However, 

some states, such as Texas and California, explicitly allowed local areas to add their 

own informational requirements on top of the core requirements that the state had 

developed. We do not have information on how frequently local areas made use of 

these opportunities. However, having local areas with discrete reporting requirements 

raised the possibility that a vendor would need to submit separate applications for the 

same program to different local areas, so that the requirements of each area could be 

met.7

 

6

 Because Texas was an early implementer under WIA, in actuality the application form we 

reviewed was for subsequent eligibility, not initial eligibility. The form the state used for initial 

eligibility might have been different from what is described here.

7

 Some states are very explicit in noting that a customer should be free to access any program on 

the state list, so long as it provides training for an occupation in demand. In actuality, though, as was 

discussed in the previous chapter, many local areas are reluctant to spend ITA funds on training 

programs that do not meet their own requirements. To ensure that customers have full choice, and to 

allow vendors full marketing opportunities, Texas encouraged vendors to electronically forward their 
IV-12

Another way in which states’ applications for initial eligibility varied was in 

whether information on program performance was elicited. As we will discuss below, 

a few states imposed eligibility requirements relating to program performance for initial 

eligibility. However, a few others asked vendors to supply performance information 

about their programs if it was available. For example, applications for initial eligibility 

asked vendors to supply information on:

• In Georgia, their programs’ completion rate (along with a definition of 

completion), placement rate, training-related placement rate, and wage 

at placement,

• In Michigan, their programs’ completion rate,

• In Missouri, their programs’ number of participants, exiters, 

completers, and completers who are employed,

• In North Carolina, their programs’ completion rate, placement rate, 

wage at placement, and credentials awarded.

• In Texas, their programs’ completion rate, placement rate, and wage at 

placement, 

Often, however, the requirement was waived if the provider asserted that it was not 

able to provide such performance data at the time of application for initial eligibility. 

Often, too, as in California, providers were invited to supply performance data, but the 

Consumer Report System included a disclaimer that the information supplied had been 

“self-reported by the provider and has not been verified.” Regardless of what the states 

requested or required, in actuality performance data submitted by vendors as part of 

initial eligibility was sparse.

Obviously imposing information requirements beyond basic core information can 

be of substantial benefit to ITA-holders in helping them make their training choices. 

But it does so by adding appreciably to the burden experienced by vendors. 

Overwhelmingly, the vendors we spoke with reported that the application process for 

initial eligibility “went smoothly” or was “easy” or “straightforward,” and local area 

staff were described as being very helpful. However, where application requirements 

required more of the vendors—such as where application forms were lengthy and 

separate applications needed to be submitted for each program location—vendors were 

 

ETP applications (using the state’s automated TPCS) to each local area in which they intended to do 

business.
IV-13

less pleased with the process. For example, an administrator at a community college 

with whom we spoke reported needing to submit separate lengthy applications for each 

program the community college offered at each of its several different locations. This 

official complained that the process was “excruciating and a waste of resources...a 

bureaucratic nightmare.” In Ohio, public and state colleges were so unhappy about 

needing to supply even the core information required by the state for exempt providers 

for initial eligibility that the state agreed to gather this data on its own from other 

sources. California acted similarly, accessing program data on the state’s community 

colleges from the California Chancellor’s Office, obviating the colleges’ need to supply 

information themselves. (Further details about vendors’ reactions to the ITA/ETP 

system are described in a later chapter to this report.)

Performance and Other Requirements for Initial Eligibility

In keeping with WIA’s requirements, degree-granting post-secondary institutions 

and apprenticeship programs are granted automatic initial eligibility so long as they 

apply. However, other (i.e., “non-exempt”) training vendors may be asked to meet 

minimum performance requirements. In fact, all but two of the states we visited as part 

of this demonstration, Missouri and Texas, waived performance requirements for initial 

eligibility for all vendors. Of these two, Missouri required non-exempt vendors to 

record a minimum 50% placement rate among completers during the 12 months prior to 

application. In Texas, non-exempt providers needed to record:

• A program completion rate of 60%

• A placement rate of 60%

• An average wage at placement of $6.18 (calculated as 120% of the 

federal minimum wage), or an average quarterly earnings of $2,410.20 

($6.18/hour X 30 hours X 13 weeks)

In contrast to these two states, all others explicitly waived performance 

requirements for initial eligibility. The overwhelming reason that states gave for doing 

so was that they were not sure what requirements they should adopt and needed to 

confront complicated issues about how key measures should be defined and 

operationalized. Moreover, in most of these states the information systems needed to 

collect, process, and display the volume of information that performance requirements 

would imply were not operational at the time initial eligibility began in July 2000. 

Clearly, states expected that establishing performance requirements for eligibility would 
IV-14

give rise to extraordinarily complicated measurement and data issues, and they wanted 

to give themselves time to address them in a thoughtful and deliberate way.

Another reason for states’ waiving performance measures under initial eligibility 

was to address the concerns of many vendors, especially public institutions, who were 

anxious about what might be required and how the new system would work. Some 

vendors admitted that they were not nearly ready to provide performance information 

about their programs and were very uncomfortable with the levels of performance they 

might show if they did so. For example, some community colleges worried that their 

open-admissions policies would make them appear to be very poor performers. In 

recognition of these very legitimate concerns, and as a sign to vendors of their 

importance, states made the decision to waive performance requirements.

States also felt that waiving performance requirements would help to ease the 

transition to WIA for vendors. Providers, like local areas, case managers and 

customers, need time to get used to the new way of conducting business that an ITA 

and ETP system implies—for example, by taking more care to support students through 

training and afterwards to see that they complete their training and obtain employment. 

Moreover, on a more practical note, waiving performance measures allowed the states 

to retain the large numbers of providers they felt they would lose if performance 

requirements were imposed. During initial eligibility, states hoped, they would be able 

to work out many of the system difficulties, develop an initial ETP list that was as 

inclusive as possible, and have the opportunity to demonstrate to providers that 

attaining ETP eligibility was valuable and worth their added effort. 

Although few states imposed formal performance requirements for initial 

eligibility, they commonly imposed some other basic requirements on vendors’ 

programs before they would be accepted for initial eligibility. To begin with, most of 

the demonstration states (for example, California, Connecticut, Georgia, Maryland, 

Nebraska, Oregon, and Pennsylvania, among others) explicitly required that providers 

be licensed or certified by the state’s Higher Education Commission or equivalent body 

or bodies. Often different licensing agencies would be involved, depending on the type 

of provider. For example, in California non-exempt vendors would only be granted 

initial eligibility for the ETP list if they were: a) accredited by an institution recognized 

by the federal Department of Education, b) approved by the California Department of 

Education, c) approved by the Chancellor’s Office of the California Community 

Colleges, or d) were approved, registered, or exempted by the Bureau of Private 
IV-15

Postsecondary and Vocational Education. Similarly, in Oregon providers needed to be 

state licensed, with the licensing body varying by type of program, but including the 

State Board of Education, the Student Assistance Commission, the State Board of 

Higher Education, the State Apprenticeship Council, or the state Bureau of Labor and 

Industry.

States gave three reasons for requiring this basic certification or licensure before 

an application for ETP eligibility would be approved. First, such certification was 

typically mandated by state law well before the enactment of WIA, and, as such, was 

required for all training vendors doing business in the state. Second, especially because 

states generally did not impose performance requirements relating to placement or 

outcomes for initial eligibility, they used the state certification as a way of ensuring that 

vendors met minimum state standards for quality. Finally, reliance on a pre-existing 

list of state-certified vendors was very expedient, in that it provided an obvious starting 

point for developing the ETP list and served as a ready means for identifying vendors 

from whom applications should be sought. For example, in developing its ETP process 

Connecticut used information about programs already available from certification 

requirements of the state’s Department of Higher Education. As a result, the ETP 

application form itself could be greatly simplified.

States’ certification processes vary somewhat in scope and content, but in general 

they focus on process rather than outcomes. For example, Maryland’s Higher 

Education Commission (MHEC) certifies this state’s vendors through a site visit that 

examines whether curricula and equipment are up-to-date, whether facilities are 

adequate, and whether student-to-staff ratios are appropriate. Every public or 

proprietary post-secondary school offering training to the general public must have 

certification from MHEC, so it seemed obvious to use this list as the starting point for 

ETP eligibility. 

One complication is that states requiring state certification for providers have in 

some cases needed to set up special processes to deal with the large influx of 

applications from vendors applying for inclusion to the ETP list. In Oregon, for 

example, while reviewing applications to the ETP list, state representatives discovered 

that some providers were offering training courses without proper certification. To 

ensure that applications from new providers and those not certified will be sped through 

the certification process, the state set up a quick certification process so that providers 

could get on the ETP list in a timely way. 
IV-16

An additional difficulty is that some states are finding that state certification is 

serving to exclude from the ETP list vendors that might have been used frequently 

under JTPA, sometimes with good results. For example, in Connecticut, providers 

without certification from the Department of Higher Education cannot offer training 

services to the public or advertise. However, under JTPA, non-certified vendors in this 

state could forge agreements with a local job-training agency to conduct class-size 

training. Given WIA’s reduced emphasis on contract training, providers lacking 

certification will be little used and might have difficulty finding enough other business 

to stay afloat. Now, some of these training providers are seeking state certification to 

be eligible to accept ITA customers. 

Georgia’s experience is similar. This state requires that private vendors be 

certified by the Georgia Nonpublic Post-secondary Education Commission (GNPEC). 

However, vendors submitting ETP applications that are not GNPEC approved, such as 

some CBOs that might have been relying on JTPA contract training, have been 

accepted for initial eligibility and given a one-year waiver of GNPEC certification, so 

long as they attest that they have initiated the GNPEC approval process.

Apart from state licensure or certification, some states also imposed an additional 

requirement for initial eligibility that the vendor agree to begin collecting the 

performance data that would be necessary for subsequent eligibility. This requirement 

was explicitly mentioned in California, Georgia, Maryland, and Nebraska. Thus, these 

states are attempting to ease the burden on vendors in the short-run, while endeavoring 

to get them to commit to be players in the system once performance requirements are 

imposed.

In general, then, nearly every state that we studied endeavored to develop a very 

inclusive list for initial eligibility, and, consequently, imposed none but very basic 

requirements, and sought to make accommodations with other postsecondary school 

requirements.

State, Local, and Regional Approval Systems

As we have discussed, local boards review vendors’ applications for eligibility 

and decide whether or not they should be approved and forwarded to the state for 

inclusion on the state list. Although we did not learn a lot about the decision-making 

process itself, Georgia’s approach provides an example of what these procedures can 
IV-17

look like. In this state, local areas review vendors’ applications using a scoring sheet 

that includes three major categories, each with various criteria to be considered. 

• The appropriateness of the training, including:

− Whether the proposed training is in a growth occupation or 

otherwise can be demonstrated to be in demand,

− Whether the provider fosters a positive learning atmosphere and 

has up-to-date equipment,

− Whether all relevant costs are included,

− Whether job search assistance is provided,

− Whether references indicated that completers attain marketable 

skills,

− Whether training leads to a certificate, diploma, or degree, and

− Whether training schedules are on-going

• Prior experience and performance, including:

− Whether the provider’s performance on completion rates is 

satisfactory,

− Whether the provider has proven experience in placing 

individuals in training-related jobs,

− Whether the program is likely to result in employment at a wage 

necessary to attain self-sufficiency,

− Whether placements will occur in full-time positions with 

potential growth and benefits,

− Whether the student loan default rate is below 25%, and

− Whether the provider is listed on current federal, state, or local 

debarment or suspension lists

• The capability of the training organization, including:

− Whether the organization has been in operation for at least six 

months and has submitted evidence of financial stability,

− Whether the curriculum has been certified by an appropriate 

accrediting agency, and

− Whether the organization’s programs have been certified by 

GNPEC (if applicable).

Raters use the vendor’s application in assessing these factors, and are also asked to 

make a brief visit to the provider’s training venue. On the basis of this evidence, raters 
IV-18

score each of the three major categories using a 5-point scale (0=not acceptable, to 

4=excellent), and can award bonus points if the provider has experience with providing 

training to special populations, such as the disabled, homeless, or those with language 

or other barriers. Programs meeting a minimum score are then considered approved.

Regardless of what process was used, though, in actuality nearly all applications 

were approved virtually everywhere. For example, state respondents in Michigan told 

us that about 95% of all applications were approved. Respondents in other states, such 

as Nebraska and Pennsylvania, reported similarly high approval rates. These very high 

approval rates are an obvious corollary of most states’ decisions to impose no 

performance requirements for vendors for initial eligibility. Given that decision, 

applications would be denied for a small number of reasons that included:

• The vendor’s application was incomplete. 

• One of the key requirements for eligibility, such as state licensing or 

certification, was not met. 

• The program for which eligibility was being sought was not deemed to 

be training (e.g., it was classified by the local area as an intensive 

service rather than a training service).

• The training was not for a demand occupation.

Even given these fairly loose criteria, however, local areas often made every 

effort to figure out how to “fix” an application, rather than summarily dismiss it. For 

example, if staff encountered an incomplete application, they often would phone the 

vendor to retrieve the missing information. Similarly, vendors whose program was not 

state licensed might be given an expedited review by the state licensing agency or 

granted a one-year waiver from this requirement. Thus, in keeping with what was 

described earlier in this chapter, states were very intent on developing a very inclusive 

list of training choices for initial eligibility, and thus worked hard both to market to 

vendors to elicit applications and see them through to approval.

Perhaps because of the extra effort that locals put forth, some local-area 

respondents reported that the review process was extremely tedious and timeconsuming. For example, Macomb-St Clair (Michigan) used grant funds to hire a staff 

member whose duties included reviewing vendor applications for eligibility; this area 

wondered how other local areas in the state were managing this responsibility without 

having the funds to hire someone especially for this purpose. Similarly, respondents in 

Southwest Connecticut reported that they received a high volume of phone calls from 
IV-19

vendors requesting information or clarifications about the area’s application process; 

grant funds enabled this area to hire someone to field these inquiries, which they 

reported was a huge help. Some local areas in Texas found the vendor review process 

so time-consuming that they joined together to hire a full-time staff member whose 

duties included only that; based on their experience and that of other areas, the state 

was considering recommending that each local area might need to hire a staff member 

devoted to vendor review and approval.

Apart from the amount of effort involved, some local areas were also concerned 

about what might happen if different areas set different local standards for approval. In 

principle, ITA holders are free to choose any vendor on the state list, so long as the 

training program is for an occupation in demand (20 CFR 663.585). This circumstance 

might provide a reason for vendors to “shop around” to find a local area with the most 

lenient review process, and could put local areas in the position of issuing an ITA that 

would be used to procure training services from a vendor that had been denied 

eligibility by that local area but approved by an adjacent one. Compounding the 

difficulty, several states allow (or at least do not prohibit) vendors whose application is 

denied eligibility from one local area to resubmit to another. 

Several of the case-study sites were disturbed by this possibility and stated that 

they would be very reluctant to expend their training funds for a program that they had 

denied for eligibility. States helped resolve their dilemma by issuing policy that 

emphasized that local areas need not—and should not—award an ITA for training that 

was not deemed by that local area to be for a demand occupation or that exceeded the 

area’s dollar caps, even if the program was on the ETP list. A few other states went a 

bit further by authorizing local areas to deny an ITA award if the program selected by 

the customer failed to meet other of the local area’s criteria. Still another issued a 

policy that explicitly prohibited a vendor whose application was denied eligibility by 

one area from resubmitting elsewhere, and another stipulates that a vendor can apply 

only to the local area in which its main administrative offices are located.

Elsewhere, local areas were attempting to coordinate the vendor approval process 

across adjacent areas, at least to some degree. Doing so was in fact a chief focus of 

Macomb-St. Clair’s ITA/ETP Demonstration grant plan. This grantee used its grant 

funds to hire a consultant who was serving as a regional coordinator responsible for 

identifying existing variation across local areas in the Detroit metropolitan area, not 

only with respect to their vendor approval criteria, but also regarding caps on ITA 
IV-20

costs, the maximum allowable duration of training, how demand occupations were 

defined, and so on. The consultant attempted to facilitate the coordination of policy 

formulations across the local areas to achieve greater consistency. 

The several local areas that make up metropolitan Atlanta developed a different 

approach, specifically focused on the problem of ensuring regional agreement with 

respect to vendor suitability. These areas empowered the Atlanta Regional 

Commission, which was the administrative entity for several local areas in the Atlanta 

area, to act on their behalf in reviewing vendor applications and deciding which ones 

should be forwarded to the state for approval. California also explicitly allows local 

boards to designate another local board to act on their behalf in making determinations 

for eligibility, although we did not hear of instances in which this happened.

Other strategies were also in evidence. North Carolina was including separate 

data fields in its Consumer Report System to identify specifically which programs had 

been approved by which local areas. With this information readily at hand, potential 

trainees and their case managers could at least quickly identify vendors that the local 

area had deemed as unsatisfactory. Similarly, Georgia and Ohio were attempting to 

work out a procedure whereby a local area reviewing a vendor’s application would be 

notified if that same vendor’s application had been denied eligibility elsewhere.

In actuality, though, nearly all vendors who applied for initial ETP eligibility had 

their applications approved, no matter to which local area they applied. Thus, the 

concern that motivated the regional approach to vendor approval turned out not to 

materialize to any great degree in most states, at least during the initial eligibility 

period.

SUBSEQUENT ELIGIBILITY PROCEDURES AND REQUIREMENTS

The procedures and requirements that many states are establishing for subsequent 

eligibility have been in a profound state of flux and uncertainty that is often still far 

from complete resolution. Thus, in some states we were given copies of policies 

regarding subsequent eligibility during our first round of data collection that turned out 

to have changed in important ways when we visited the states again during the second 

round of data collection, approximately one-year later. In other states, even as of the 

time of the second round of data collection, subsequent eligibility procedures still had 

not been established in even draft form (at least not in a form that sites were willing to 

share with us). Similarly, in every state but one the date when subsequent eligibility 
IV-21

was to have begun kept getting pushed back, and still has not begun in some instances. 

Finally, there is a lack of consistency across states as to what establishing subsequent 

eligibility means in practice. For example, in some states subsequent eligibility is what 

happens when initial eligibility ends and data on vendor performance starts to populate 

the Consumer Report System. In other states, subsequent eligibility means that 

performance requirements or thresholds on some or all of the seven performance 

measures have been imposed.

When Subsequent Eligibility Begins

With this fluidity in the establishment of subsequent eligibility as a backdrop, 

Table IV-1 presents the dates when subsequent eligibility was slated to begin in the 

ITA/ETP Demonstration states, as of the time of the second round of site visits 

(generally late fall 2001 to early 2002). 

Table IV-1

Target Start Dates for 

Subsequent Eligibility

Target Date

California First quarter 2002

Connecticut January 2002, or later

Georgia January 2002

Indiana Date not established

Maryland January 2002

Michigan January 2002

Missouri Date not established

Nebraska January 2002, or later

North Carolina June 2002

Ohio January 2004

Oregon January 2005

Pennsylvania June 2002

Texas Already underway

As the table shows, only one state, Texas, had begun subsequent eligibility at the 

time of the site visit. In fact, Texas was so much ahead of the other states in 

establishing its ETP procedures that it had actually begun subsequent eligibility as early 

as July 2001. All other states were planning on using at least the full 18 months 

allotted by the WIA regulations and legislation for initial eligibility (e.g., 20 CFR 

663.530) or even longer, and none were planning on beginning subsequent eligibility 
IV-22

before January 2002. These include the several other states (besides Texas) that were 

early implementers under WIA (Indiana, North Carolina, and Pennsylvania), each of 

whom began implementing WIA by January 2000. Note also that two states, Indiana 

and Missouri, had not established a firm date to begin subsequent eligibility, as of the 

fall of 2001. Two others, Ohio and Oregon, specifically requested a waiver from DOL 

to defer the start of subsequent eligibility for at least another few years.8 

Reasons why the start of subsequent eligibility had been delayed so long can be 

attributed to the difficulty that states encountered in establishing the data systems that 

were needed to support the reporting of performance data about individual programs. 

As we will discuss later in this chapter, either vendors were being asked to bear the 

burden of assembling and reporting performance data about their programs, or states 

were assuming the responsibility of collecting the bulk of these data through 

Unemployment Insurance wage matching. In either case, difficult and costly data 

collection, transmission, and reporting protocols and procedures needed to be 

developed and executed. Additionally, thorny conceptual issues needed to be worked 

out, including how key terms should be defined and what performance guidelines 

should be established. Making these final determinations has been extraordinarily timeconsuming, as states attempt to consult with all entities involved, including local 

boards, community colleges, private training vendors, One-Stop center operators, case 

managers, and the customers themselves. 

Given these difficulties, many of the states admitted to purposefully postponing 

subsequent eligibility for as long as possible. For example, in submitting a waiver 

request to defer the start of subsequent eligibility to June 2005, Oregon drew attention 

to its efforts to assume much of the data collection burden on behalf of vendors, so as 

to minimize impediments to vendors’ applying for subsequent eligibility and thereby 

promote the development of an extensive ETP list that would maximize customer 

choice. However, in assuming this burden, the state noted that it needed to work out 

confidentiality provisions for the release of students’ Social Security Numbers and 

coordinate independent and differently-configured data systems. It also pointed to the 

time lags inherent in UI data reporting for post-training measures that would make the 

compilation of data for WIA-funded students infeasible for quite some time. By 

deferring subsequent eligibility, this state argued that it would be able to improve and 

 

8

 Both of these waiver requests were pending at the time we prepared this report.
IV-23

coordinate the data collection and reporting procedures, assure that reported data are 

accurate and verifiable, and allow adequate time to gather and analyze data to set 

reasonable criteria for subsequent eligibility.

Performance Requirements as Part of Subsequent Eligibility

In keeping with the WIA legislation [Section 122(d)(1)(A)], providers whose 

programs are determined to be subsequently eligible should submit information relating 

to the performance of all students as well as specifically WIA-funded students. The 

measures on which they are to report include:

• Measures of the performance of all students in the program of study for 

which the vendor is seeking certification, including with respect to the:

− Program completion rate. This rate is to be calculated for all 

individuals participating in the applicable program.

− Employment rate. The percentage of all individuals participating 

in the applicable program who obtain unsubsidized employment.

− Average wage at placement. 

• Measures for WIA customers (i.e., individuals funded by WIA) who 

participated in the programs for which the provider is seeking 

certification), including with respect to the:

− Program completion-employment rate. The percentage of 

participants who have completed the applicable program and who 

are placed in unsubsidized employment.

− Retention rate in employment after six months. The retention 

rate in unsubsidized employment of participants who have 

completed the applicable program, six months after the first day 

of employment

− Average wages after six months. The wages received by 

participants who have completed the applicable program, six 

months after the first day of employment.

− Rate of licensure, degree attainment, or certification. Where 

appropriate, the rates of licensure or certification, attainment of 

academic degrees or equivalents, or attainment of other measures 

of skills, of the graduates of the applicable program.

Providers must annually meet minimum performance benchmarks on these 

measures, as established by the states [WIA Section 122(c)(5)(B)]. Local areas, in 

turn, may set higher performance levels (but not lower ones), and states and areas may 

establish additional performance criteria as well.
IV-24

In most but not all states, the establishment of subsequent eligibility has meant 

that thresholds relating to vendors’ performance have been established on some or all of 

these measures. Even in some states where subsequent eligibility is not slated to begin 

for some time, preliminary plans for what performance thresholds should look like have 

been discussed. Table IV-2 provides a glimpse of the range of performance levels that 

states have established, or that they are planning to establish. As the table shows, four 

states (Connecticut, Indiana, Nebraska, and North Carolina) had not yet made decisions 

regarding performance benchmarks for subsequent eligibility (although Nebraska’s 

workgroup has recently made preliminary recommendations). The remaining states 

span a range of levels and approaches.

For example, three states (California, Maryland, and Missouri) set standards for 

subsequent eligibility on some of the measures but not all of them, reasoning that their 

data systems were not sufficiently developed to do otherwise. For example, Maryland 

officials argued that they simply had no reasonable basis for setting performance 

thresholds on most of the measures and hence would not attempt to do so until they had 

better data. In this state, vendors, in collaboration with the state, are expected to 

collect performance information on all measures, but, as of now, the only performance 

benchmarks required for subsequent eligibility will pertain to employment rates. 

Three states (Missouri, Georgia, and Pennsylvania) set performance levels for all 

or most of the measures but did not require vendors to meet all of them. For example, 

Georgia set levels on all seven measures but required that vendors meet only four of 

them (2 of the 3 all-student measures and 2 of the 4 WIA-student measures). Nebraska 

was contemplating achieving the same objective using a different and novel weighting 

methodology, but details of its approach were still being worked out. Conversely, only 

Texas and Michigan established performance levels on all seven measures and expected 

vendors to meet each of them.

Finally, two states (Ohio and Oregon) requested of DOL a waiver of several 

years before implementing subsequent eligibility, but nonetheless implemented “draft” 

or “placeholder” goals as a way of setting performance targets for the workforce 

system. Members of Oregon’s Performance and Accountability Committee were 

especially adamant that some minimum targets should be established to promote quality 

training, while recognizing that it would be imprudent to set hard standards without 

better data on which to base them.
Table IV-2: Minimum Performance Thresholds for Subsequent Eligibility

All-Student Measures WIA-Student Measures

State Requirement CR ER Wage CER RR RWR LCR

CA Requirements linked to those of vendors’ 

certifying body. 

48%-64% 58%-67% none none none none none

CT No decisions as yet --- --- --- --- --- --- ---

GA Must meet 2 of 3 of ALL student 

measures and 2 of 4 of WIA measures

50% 50% $184 per 

week

60% 60% $184 per 

week

60%

IN No decisions as yet --- --- --- --- --- --- ---

MD May establish goals on other measures in 

later years

61% none none 61% none none

MI Must meet all measures, unless LCR does 

not apply or an exception is granted

50% 40% $7.50 70% 75% $3,100 per 

quarter

50%

MO These are draft plans. Must meet 1 of 3 

of ALL student measures and 2 of 3 of 

WIA measures.

25% 60% $1 above 

min. wage

75% 50% $1.25 above 

min. wage

none

NE No decisions yet. Is contemplating a 

weighted-average approach. Preliminary 

workgroup recommendations are shown

pending 33% $5.15 --- --- --- ---

NC No decisions as yet --- --- --- --- --- --- ---

OH These are draft goals. Have requested a 

waiver to delay SE until 2004

75% 75% To be set 

by LWIAs

75% 75% To be set 

by LWIAs

60%

OR These are placeholder goals. Have 

requested waiver to delay SE until 2005

80% 77% $9.00 75% 85% $9.87 6%

PA Must meet 2 of 3 of ALL student 

measures and 2 of 4 of WIA measures

70% 74% $7.88 70% 82% $5,682 per 

2 quarters

50%

TX Performance levels are the same for PY 

’01 and PY ‘02

60% 60% $6.18 

(120% of 

min wage)

65% 70% $2,511 per 

quarter

45%

______________

Note: Column headings are as follows: CR, completion rate; ER, employment rate; wage, average wage at placement; CER, completion-employment rate; RR, retention 

rate; RWR, retention wage rate; LCR, licensing-credentialing rate. Procedures are as of the spring of 2002.
IV-26

Not only did the general approach vary across states, but the absolute levels of 

performance that vendors were expected to meet for their programs on each of the 

measures varied widely as well. For example, excluding from consideration the 

placeholder goals established by Ohio and Oregon (which were goals rather than 

standards, and, probably for this reason, were consistently high), performance 

thresholds on the all-student performance measures ranged:

• On the all-student completion rate, from 25% (Missouri) to 70% 

(Pennsylvania),

• On the all-student employment rate, from 40% (Michigan) or less 

(Nebraska’s preliminary threshold) to 74% (Pennsylvania),

• On the all-student wage at placement, from the federal minimum wage 

(Nebraska’s preliminary threshold) or just above it (Georgia’s earnings 

threshold of $184 per week divided by 35 hours per week equals $5.25) 

to $7.50 (Michigan) or $7.88 (Pennsylvania).

Thresholds for the WIA-student measures on the completion-placement rate were 

somewhat less dispersed, but those for the remaining WIA-student measures varied 

widely as well.

Some states also made provisions for how to apply WIA-student credentialing 

standards to programs that might not award credentials. As we discussed earlier in this 

chapter, a few (Oregon and Connecticut) handle this problem primarily by encouraging 

vendors to develop certificates of completion for courses or programs for which they 

want to attain ETP approval. Others, such as Ohio, broaden the definition of the WIAstudent credentialing measure to include not only licenses, certificates, and degrees, but 

also the “attainment of other measures of skills of the WIA graduates from the 

program.” Texas effectively accomplishes something similar, but goes about it 

differently. This state has introduced four separate measures—a licensing rate, a 

certification rate, a degree rate, and a skill attainment rate. The performance threshold 

for each of these measures is set equivalently at 45%, and the vendor is expected to 

meet the performance level on the one of the four measures that applies. Finally, 

Michigan exempts vendors from meeting the WIA-student credentialing rate standard if 

the vendor’s program does not lead to a certificate.

Rationales for the decisions that states made with respect to their approaches 

reflected some similar themes, even if they did often lead to different decisions. 

Among the most common considerations we heard mentioned was the states’ effort to 

establish performance criteria that are rigorous enough to ensure a high enough level of 
IV-27

quality among vendors who are certified as being eligible, while not establishing them 

so high that so many vendors will be excluded from the ETP list as to unduly restrict 

customer choice. Thus, on the one hand, respondents spoke of wanting to establish 

high criteria for excellence in the workforce system, to encourage vendors to strive for 

high performance, avoid conveying the message that mediocre performance is 

satisfactory, and to assure themselves that vendors could be meaningful contributors to 

the state’s own efforts to meet its negotiated performance levels on the WIA core 

measures. On the other hand, they recognized that standards that were too high would 

serve to exclude from eligibility programs that were perceived to be worthy but whose 

measured performance might fall short. Paradoxically, there was a fear that 

community colleges, among the local areas’ most favored providers because of their 

perceived effectiveness and low cost, might be put most at risk, because their openenrollment policies and diverse service populations could cause their measured 

performance to be rather low. In any case, respondents feared that restricted customer 

choice—in clear contradiction to one of WIA’s key tenets—might be the result, if 

performance thresholds were too high. 

Building on existing performance measurement systems also emerged as a theme 

in a few states. For example, officials in Pennsylvania determined that a 70% 

completion and employment rate is used for approving a program’s Pell eligibility, so 

the standards it set for subsequent eligibility on these two measures are approximately 

the same. Similarly, Texas accepts program-specific data and performance thresholds 

adopted by the Texas Higher Education Coordinating Board as a proxy for the allstudent completion rate that would otherwise apply. 

The availability of data was another consideration. Although it has not entered 

subsequent eligibility yet, North Carolina expected to establish performance thresholds 

for the WIA-student measures but none for the all-student measures for at least the first 

year of subsequent eligibility, because existing data systems—the WIA MIS—could 

support the first set of measures but not the second. California, too, plans on 

calculating measures for WIA-funded students using its WIA MIS fairly soon, though it 

has not done so yet. By contrast, Ohio considered but abandoned the idea of using its 

WIA MIS to compute vendors’ performance measures for WIA-funded students, 

because the MIS client records do not contain data on key information that would be 

required, including the student’s exit date from training and vendor and program name.
IV-28

California’s efforts to build on existing data and performance measurement 

systems went the furthest. In the interests of expediency and in the absence of good 

data to do otherwise, officials in this state decided to link subsequent eligibility 

standards for its vendors directly to those already being applied by the vendors’ 

oversight bodies for other purposes. It determined that all or most of the state’s eligible 

training providers were already implicitly or explicitly subject to the regulatory 

authority of one of three types:

• CDE Perkins. Adult Education and Regional Occupational are 

administered by the California Department of Education (CDE) and are 

subject to the performance and reporting requirements of the Carl D. 

Perkins Act as required by the U.S. Department of Education.

• COCCC Perkins. Post-secondary Vocational and Technical Education 

programs administered by the Chancellor’s Office of California 

Community Colleges (COCCC) are similarly subject to the Perkins Act, 

but with somewhat different requirements. These requirements cover 

credit courses offered by the state’s community colleges. Non-credit 

courses are not covered by these requirements, but the majority of them 

have credit-course analogues whose requirements can be used.

• BPPVE programs. Private training programs on the ETP list must be 

approved, registered, or exempted by the state’s Bureau of Private 

Postsecondary and Vocational Education (BPPVE). This agency 

requires the annual reporting of a completion rate and an employment 

rate for “approved” programs. Programs covered by BPPVE that are 

classified as “registered” or “exempted” are not required to report on 

their performance under current law and will be given the option of 

using the performance system defined for the public programs under 

Perkins or those for BPPVE for approved programs. 

While acknowledging that the operational definitions and computations of the 

performance and reporting requirements covered by these various agencies are very 

different, California officials built on this existing base. As such, they decided that 

subsequent eligibility performance requirements would be set at 80% of the goals 

established by Perkins and BPPVE, except that no employment goal would be less than 

60% unless the responsible regulatory agency has negotiated a rate of less than that. 

Thus:

• For CDE Perkins, 80% of the Perkins completion goal of 80.28% is 

64.21%, and the Perkins employment goal was negotiated to be 

58.18%. Thus, the subsequent eligibility performance levels for 
IV-29

providers covered under CDE Perkins is 64.21% for the completion rate 

and 58.18% for the employment rate.

• For COCCC Perkins, 80% of the Perkins completion goal of 59.82% is 

47.86%, and 80% of the Perkins employment goal of 83.19% is 

66.55%. Thus, the subsequent eligibility performance levels for 

providers covered under COCCC Perkins is 47.86% for the completion 

rate and 66.55% for the employment rate.

• For BPPVE programs, 80% of the BPPVE completion goal of 60% is 

48%, and 80% of the BPPVE employment goal of 70%, subject to the 

60% employment cap, is 60%. Thus, the subsequent eligibility 

performance levels for providers covered under BPPVE is 48% for the 

completion rate and 60% for the employment rate.

Because neither CDE Perkins, COCCC Perkins, nor BPPVE sets goals for the students’ 

wage at placement, no standard has been set for this measure for subsequent eligibility. 

Similarly, obviously none sets goals for WIA students, so these are similarly left 

inoperative for determining ETP subsequent eligibility, at least for now. California 

intends to revisit its approach when data systems to support something different are 

further along.

California’s approach thus demonstrates the consideration of setting subsequent 

eligibility performance guidelines in a way that were aligned with existing performance 

or reporting requirements to which vendors were already subject. Expounding on this 

principle, some vendors in California and elsewhere indeed wondered why they needed 

to be subjected to additional requirements for eligibility under WIA when they had 

already demonstrated compliance with requirements established by state or other 

regulatory bodies as required for certification or licensing, including for Pell eligibility. 

Another consideration that we heard mentioned was setting standards for vendors 

in a way that could take into account differences in likely success rates for vendors 

operating in very different local economic conditions. For example, respondents in 

Nebraska pointed to the extreme gradient in employment opportunities and average 

wages as one moved across the state from east to west. Standards that were high 

enough to be at all meaningful in the eastern part of the state would thus serve to 

exclude from eligibility the vast bulk of vendors operating in the state’s western half. 

Respondents in Georgia voiced very similar concerns, fearful that standards that were 

appropriate for vendors in the state’s urban areas would leave rural areas with no 

vendors on the eligible provider list whatsoever. Preferring to favor vendor 

inclusiveness in rural areas, Georgia deliberately set its performance thresholds on 
IV-30

employment-related measures relatively low (at least in comparison to other states) for 

this reason.9 

Local Area Exceptions and Options

In keeping with the terms of the WIA legislation, local areas at their option can 

impose additional performance requirements or higher thresholds than those established 

by the state [WIA Section 122(c)(6)(B)]. States explicitly made provisions for this in 

their policies, but they way they did so varied somewhat. 

Texas’s policies regarding local discretion are perhaps the most explicit. This 

state allows local boards to establish adjustments or exceptions to the minimum 

thresholds established by the state itself. For example, the state’s guidance gives an 

example of a weighting methodology that might be used, to take into account a local 

area’s exceptionally high unemployment rate or other local economic or demographic 

factors. Similarly, special exceptions can be granted, such as for vendors whose 

program completion rate is below the state threshold but whose performance on other 

measures are well above the state standards. These provisions are viewed as in keeping 

with WIA’s authorization that local areas can set higher, but not lower, levels of 

performance and that require Local Boards to take into consideration “the specific 

economic, geographic and demographic factors in the local areas in which providers 

seeking eligibility are located, and...the characteristics of the populations served” (20 

CFR 663.535).

Examples of related policies that we uncovered in other states include these:

• Pennsylvania noted that local areas could establish more stringent, but 

not more lenient, standards for subsequent eligibility than those imposed 

by the state itself. 

• For the first year of subsequent eligibility, California is allowing local 

areas to approve a vendor for the ETP list, even if the vendor has failed 

to meet the state’s minimum performance thresholds. Acceptable 

reasons for local areas to grant such a waiver include doing so for 

programs that target the hardest-to-serve populations.

 

9

 As we discuss below, however, the Atlanta Regional Commission exercised its local authority to 

set higher performance requirements for the Atlanta area.
IV-31

• Michigan, too, allows local areas to retain as ITA-eligible programs that 

fail to meet the state’s standards, if there are special factors related to 

the student populations served.

• Ohio deferred entirely to the locals’ authority for establishing 

performance goals on the average wage measures, in recognition of 

area-to-area variation in earnings opportunities. 

Despite this flexibility, we encountered very few instances of local areas 

exercising such discretion. The Atlanta Regional Commission, in Georgia, provides 

one of the few exceptions. This area established a WIA-student completionemployment rate of 70%, ten percentage points above the state imposed minimum. It 

had also added a training-related placement rate as an additional performance measure 

on which it expected vendors in its area to report.

In actuality, though, many more local areas exercised discretion, but they did so 

in more subtle ways. Thus, local area officials in some states told us that they in 

essence established an informal “preferred” provider list. As such, they elected not to 

use ITA funds to support training provided by certain vendors, even though the 

vendors’ programs might be on the ETP list, because they felt the vendors’ 

performance was low or their services were of poor quality. Conversely, they would 

steer their customers towards those programs whose providers they trusted. Others 

mentioned that they would “test” vendors they had not used before by limiting the 

number of ITAs they would issue for the vendors’ programs, until the vendor could 

demonstrate its worthiness to the local area’s satisfaction.

Additional Problems and Concerns

A clear issue that runs through the above discussion is states’ efforts to establish 

policies that at once ensure high quality standards but that do not undermine customer 

choice. A key difficulty was doing so in the absence of good historical data on which 

to base their decisions. For example, states that in principle would like to establish 

performance levels at a certain percentile of performance find that they have no way of 

knowing what absolute score that percentile would correspond to. The small number of 

trainees in some programs, especially students that are WIA-funded, is a related 

practicality that makes the identification of appropriate performance levels difficult. 

For example, Ohio points out that it had thus far issued only about 7,000 ITAs but has 

about 4,600 programs on its ETP list, for a per program average of less than 2. Given 

this calculation, it points out, it will be a long while before performance levels on the 

WIA-student measures can be meaningfully established or measured. On this same 
IV-32

basis, Texas has effectively waived performance requirements relating to WIA-funded 

students for vendors’ first year of subsequent eligibility.

Apart from these difficulties, respondents expressed the fear that, almost no 

matter what policies they set, the eligible training provider list would shrink 

dramatically when subsequent eligibility began. Only one state (Texas) had actually 

begun subsequent eligibility at the time of our site visit, and its experience lends some 

credence to these fears. One respondent in Texas estimated that the ETP list had at 

first fallen by 80% from initial to the first year of subsequent eligibility. The reason 

given for this drop-off was that vendors, mostly community colleges, were unwilling to

undertake the difficulty and expenditure of time and resources necessary to gather and 

submit performance information, when they expect that only a trickle of ITA customers 

will result. Fortunately, this state also reports that its list has since rebounded to nearly 

its original size, as its aggressive marketing efforts and appeals to public interest appear 

to have caused some vendors that dropped off to reconsider.

Because subsequent eligibility had yet to begin elsewhere, respondents in other 

states were basing their concerns on conjecture rather than their actual experience. 

Nonetheless, their warnings were dire. Respondents in Indiana, Maryland, Michigan, 

North Carolina, and Pennsylvania were all anticipating dramatic cuts in their ETP 

lists—of the magnitude of a 50% cut or more—when subsequent eligibility began. Of 

the states about to embark on subsequent eligibility, Missouri was one of the few that 

voiced only limited concerns along these lines.

Respondents in some states also voiced the opinion that the performance 

information that was a part of the ETP and CRS procedures was next to meaningless. 

Differences in the ways vendors measure and report various data items as well as 

differences in the needs and abilities of the incoming students, service emphases, and 

local area characteristics, they felt, made comparisons across programs misleading at 

best. Others worried that vendors would “game” the system, by (for example) 

changing their admissions policies or completion requirements.

Additional key concerns included developing the data systems necessary to 

measure vendors’ performance and establishing common operational definitions of key 

terms that are integral to performance calculations. We take these issues up in the next 

section.
IV-33

DATA CAPTURE AND DEFINITIONAL ISSUES

By far the greatest challenge that states confronted in establishing subsequent 

eligibility was in developing procedures to measure the indicators of performance 

specified in the WIA legislation. The two key components of this task are defining key 

terms in a clear and consistent way and gathering the necessary data to measure them. 

All states expended an extraordinary amount of effort and resources in grappling with 

these issues, but they developed resolutions that vary greatly in their adequacy.

Systems for Data Capture

States’ strategies for measuring the performance information required for 

subsequent eligibility and the Consumer Report System vary along the dimension of 

whether the onus for data collection was placed on the state or on the vendor. Table 

IV-3 highlights some of these differences. A look at the table suggests that the states 

can be roughly clustered into two groups. In the former group, at least employmentrelated performance information will be gathered from matches that states conduct with 

Unemployment Insurance (UI) wage files; among the latter group of states, vendors are 

responsible for generating their own data, and do so primarily through customer 

surveys.

UI and Other Matching. The majority of states—nine of the thirteen in our 

sample (Connecticut, Georgia, Maryland, Missouri, Nebraska, North Carolina, Ohio, 

Oregon, and Texas)—fall into the first group. These states have taken—or plan on 

taking—primary responsibility for measuring employment-related performance data for 

vendors through UI wage matching. In most cases, the state will do so without charge, 

although Texas has worked out a vendor payment schedule of $50 for each 150 student 

records that it is asked to match (e.g., $50 for 1-150 records, $100 for 151-300 

records, and so on). Nebraska has so far been using ITA/ETP Demonstration grant 

funds to support its matching, but may institute a small charge to vendors as well, when 

its grant funds run out.

Texas’s procedures illustrate the general approach. In this state, vendors can 

submit “seed data” (i.e., students’ Social Security Numbers) at any time to the Texas 

Workforce Commission (TWC), which takes responsibility for carrying out the 

matching. Vendors do so by submitting two Excel spreadsheet files copied onto a 

single floppy diskette for each of their programs; i.e., a separate diskette must be 

submitted for each program for which the vendor is seeking eligibility. Of the two files 

on each diskette, the first contains identifying information about the vendor (e.g., 
IV-34

provider name, location, and program name and the Classification of Instructional 

Programs, or CIP, code). The second file contains the seed records, including the 

students’ Social Security numbers (exactly nine characters, with no hyphens, formatted 

as a text field) and limited other identifying information. Given the time lags in when 

UI data become available and other factors, TWC spells out very clearly for which 

periods seed records are to be submitted. For example, for the all-student performance 

measures, a diskette submitted for matching in August 2002 should include customers 

who actively participated but were scheduled to complete the course or program during 

the 12-month period from April 1, 2001 through March 31, 2002.

Table IV-3

Planned or Actual Procedures for Data Capture

California

Eventually plans on using UI matching. In the interim, will arrange to 

have all-student completion and placement rates measured by oversight 

agencies or be self-reported by vendors.

Connecticut Plans on using UI matching for employment-related measures. Vendors 

must self-report completion and credentialing.

Georgia Is using UI matching for employment-related measures. Vendors must 

self-report completion and credentialing.

Indiana Will expect vendors to self-report all measures. May allow sampling.

Maryland Is using UI matching for employment-related measures. Vendors must 

self-report completion and credentialing.

Michigan Vendors are expected to self-report all-student measures, based on a 

customer or employer survey. WIA-student measures can be calculated 

for vendors from the WIA MIS, with UI matching.

Missouri Plans on using UI matching for employment-related measures. Vendors 

must self-report completion and credentialing.

Nebraska Is using UI matching for employment-related measures. Vendors must 

self-report completion and credentialing.

North Carolina Plans on using UI matching for employment-related measures. Vendors 

must self-report completion and credentialing.

Ohio Plans on using UI matching for employment-related measures. Vendors 

must self-report completion and credentialing.

Oregon

Plans on using UI matching for employment-related measures. May do 

matching with community college enrollment files for completion and 

credentialing rates; other vendors must self-report.

Pennsylvania Vendors are expected to self-report all measures.

Texas Is using UI matching for employment-related measures. Vendors must 

self-report completion and credentialing.
IV-35

States cite multiple reasons for relying on UI wage matching. First, doing so is 

arguably required by the legislation [WIA Section 122(c)(5)(B)]. An additional 

imperative includes the perceived advantages of relying on UI matching in comparison 

to the alternatives, such as ensuring that data are collected from a reliable source and in 

a uniform manner.10 Moreover, using this source substantially relieves the burden on 

vendors, which is a key consideration if one is concerned about inducing them to apply 

for eligibility, as officials in Oregon noted. In short, the state has the resources and 

technology to collect employment information in a reliable manner, whereas many 

providers do not.

Although UI matching clearly seems like it is the least burdensome from the 

vendors’ point of view, state respondents cited the logistical and other problems they 

have been encountering. Developing data sharing agreements and the protocols for data 

transfers among state agencies, for example, has taken some time to work out. 

Respondents in several states (e.g., Nebraska, Oregon) additionally expressed a serious 

concern about confidentiality issues that needed to be overcome. In Oregon, for 

example, the state’s Attorney General has ruled that specific language must be used in 

individuals’ voluntary release forms before SSNs can be used for data matching for 

WIA purposes; this language has only recently been used on a consistent basis by 

virtually all the state’s vendors. Moreover, in Oregon and some other states, it is 

recognized that not all vendors have been routinely collecting students’ Social Security 

Numbers to begin with. For these reasons, implementation has been delayed to some 

degree.

Our state respondents mentioned additional drawbacks to using UI data that have 

also been widely discussed in the context of the WIA performance measurement 

 

10 The advantages and disadvantages of UI wage matching for measuring performance have been 

extensively considered by DOL and various work groups that it has convened. Among the advantages, it 

has been pointed out that UI matching is not subject to recall or non-response bias as survey results might 

be, defines and measures earnings comparably, can yield data for large numbers of individuals at a much 

lower cost than a survey would, and, accordingly, makes a full enumeration an attractive alternative to 

sampling, and easily permits the measurement of long-term outcomes or outcomes measured at multiple 

points in time. On the other hand, not all jobs are covered by the UI system (though well over 90% of 

them are included in most states), some employers might accidentally or deliberately fail to report some 

earnings of their employees, and those employed out-of-state will not be captured without the inter-state 

exchange of data (such as through the DOL-sponsored Wage Record Interchange System). For a 

summary of the advantages and disadvantages of UI wage matching for purposes of measuring program 

performance, see Robert Kornfeld and Howard Bloom, “Measuring Program Impacts on Earnings and 

Employment: Do UI Wage Reports from Employers Agree with Surveys of Individuals?” (July 1997).
IV-36

system. These include the long time lag between when an individual begins 

employment and when the associated employment information is available from the 

appropriate state agency. Incomplete coverage was also mentioned, including the fact 

that some types of employment will not show up in UI files (e.g., self-employment), 

nor will employment in another state (without an inter-state agreement, along the lines 

of the Wage Record Interchange System).11 For these reasons, many of these states 

(e.g., Georgia, Maryland, Nebraska) have already decided that vendors could 

supplement UI data on employment with information that the vendors gather from their 

own sources (e.g., surveys), if they choose. Other states are still weighing this 

possibility.

Because of these procedural and other problems, in actuality in almost all of these 

nine states UI data matching has not yet occurred or just got underway recently. Only 

Texas has been routinely conducting data matching for vendors for quite some time.

At least two states among the nine using UI matching (Georgia and Oregon) 

additionally are planning on conducting matches with postsecondary enrollment data 

files maintained by state education agencies, so that program completion and 

certification rates can be calculated by the state on the vendors’ behalf (at least for 

community colleges) as well. By contrast, the other states will do UI matching to 

generate employment-related statistics, but hold vendors responsible for self-reporting 

completion and credentialing rates.

Customer Surveys by Vendors. In contrast to the above strategy for data 

capture, three states (Indiana, Michigan, and Pennsylvania) rely primarily on having 

vendors self-report data, which they typically are expected to gather through customer 

or employer surveys. Pennsylvania made clear that all vendors without exception were 

expected to self-report on all seven of the performance measures required for 

subsequent eligibility. It has thus far provided little direction on required data 

collection methodologies, as it is still working out these procedures. 

Michigan’s guidance is further along. This state found that many vendors were 

already conducting surveys of their graduates to meet other state and federal 

requirements, so adopting this approach for ETP-approval purposes seemed expedient 

 

11 The Wage Record Interchange System has been created to allow states to conduct matches with 

other states’ UI files, but participation in this system is voluntary.
IV-37

and efficient. Thus, this state requires that vendors must self-report data for the allstudent measures (WIA-student measures can be computed for vendors through the 

WIA MIS). In recognition of the fact that vendors have ongoing survey efforts, the 

state tries to be minimally restrictive as to questionnaire content or survey method. 

Thus, for the two all-student measures that are employment related, vendors are 

advised that they may conduct either mail or telephone surveys of either customers or 

employers, though they are expected to make multiple attempts to contact each intended 

respondent. Large programs (i.e., those with a large number of completers) can use 

sampling, which eases the vendors’ burden somewhat. However, when the state’s 

guidance was first issued, programs were expected to show at least a 50% response rate 

before even submitting an application for subsequent eligibility, a requirement that 

vendors in this state reported to be very challenging and costly to meet. As a result, 

the state relaxed this requirement. According to its recent guidance, if the number of 

completed surveys constitutes less than a 50% response rate, the local area to which the 

vendor submits its application can decide whether or not to issue approval.12

Where vendors are being asked to self-report, data verification—one of the state’s 

presumed responsibilities (20 CFR 663.510)--becomes a key concern. Michigan has 

contracted with a private firm to conduct “spot checks,” whose duties include visiting 

providers and requesting copies of and reviewing relevant documentation (e.g., copies 

of surveys and the like). Pennsylvania has taken a more hands-off approach. This state 

requires that vendors develop and submit a written methodology for data collection and 

verification that provides reasonable assurances of accuracy. Indiana, the third state 

that relies on vendors’ self-reported data, has not yet specified its intended approach to 

verification. Finally, states relying on UI matching generally assume that no additional 

data verification is required.

Hybrid Approach. Finally, one additional state, California, also plans on 

assuming much of the data collection burden for as many of its providers as it is able 

to, but not through UI matching (at least not at the outset). As noted earlier in this 

chapter, this state is using the reporting and performance guidelines developed by the 

vendors’ oversight agencies, including COCCC Perkins, CDE Perkins, and BPPVE. 

 

12 Of course, response rates of even 50% can yield data subject to serious non-response bias. 

Although there is no agreed-upon standard for a minimally acceptable response rate, the U.S. Office of 

Management and Budget generally expects that survey procedures to which it gives approval will yield at 

least a 75% response rate.
IV-38

This state expects that the COCCC will be able to compute performance outcomes 

on program completion and employment rates (the two measures the state is currently 

using for subsequent eligibility) for community colleges, as part of COCCC’s need to 

fulfill Perkins requirements. CDE Perkins programs should be able to have similar 

calculations performed for them by local education agencies, again in accordance with 

federal requirements. However, private providers, as well as programs operated by the 

University of California and California State University systems, will need to selfreport on these two measures. California views its approach as a stopgap, as eventually 

(i.e., once it has the necessary data protocols fully functioning) it plans on taking on the 

calculation of employment-related performance measures for all vendors through UI 

matching, first for the WIA-student measures and later for the all-student measures. 

Defining Key Terms

As part of the process of establishing performance criteria, many states are 

finding that one important conceptual hurdle is deciding how to define key terms. The 

most fundamental decision is defining what counts as a “program” in the first place, as 

was discussed earlier in this chapter. However, to ensure that performance measures 

are defined and interpreted uniformly and meaningfully, other key terms must also be 

given clear definition, such as who counts as a participant or a completer. In general, 

the WIA legislation and implementing regulations provide little guidance as to how 

these terms should be defined, thus giving states wide leeway to establish definitions for 

themselves.

Exhibit IV-4 presents summary information on some of the definitions that states 

have established on some of the key measures used in performance calculations. A 

quick look at this table shows that there is substantial state-to-state variation in how 

terms are defined. We present a discussion of these differences below.

Participants. One critical term that enters into many of the performance 

calculations is defining who is a program participant; for example, in calculating the 

completion rate, the number of completers is divided by some measure of the number 

of participants. 

In defining what constitutes a participant for purposes of this calculation, some 

states have devised solutions that address a variety of concerns. First, their all-student 

measures sometimes acknowledge that many persons who undertake training, especially 

from community colleges, do so for a variety of reasons and may not have employment 
IV-39

as their ultimate objective (e.g., persons who take courses as part of an avocation) or 

may not intend to take all courses in the sequence that make up the “program” that is 

certified by the vendor. Under such circumstances, vendors worry that their 

completion and employment rates might look quite dismal if such persons are counted 

as participants, without it necessarily reflecting anything meaningful about the vendor’s 

performance. At least a few states (Maryland and Nebraska) have decided to resolve 

this difficulty by counting as enrolled only those students who make a formal 

declaration that they intend to complete the course sequence. Maryland additionally 

requires participants to declare that employment is their ultimate objective. Thus, in 

Maryland, persons who enroll in the first course in a sequence that makes up a vendor’s 

program of training are asked to attest that they intend to complete the entire sequence 

with employment-related objectives in mind. Those who do not give such assurances 

are not counted as participants for performance-measurement calculations.

A related concern is accounting for the fact that some enrollees have interrupted 

sequences of participation, rather than enrolling in a program of training and continuing 

straight through until they complete. Maryland handles such individuals by counting as 

participants (for completion rate calculations) those persons for whom 1.5 times their 

scheduled completion date falls within the report period. In other words, in this state 

those not enrolled who did not complete are not automatically counted as a noncompleter until some period of time has elapsed, to ensure that they do not soon reenroll. Michigan, by contrast, counts as participants those who were enrolled during 

the report period but are no longer enrolled. Other states (e.g., Georgia, Pennsylvania, 

Texas) count as participants those who were expected to complete during the reporting 

period.

Finally, some states exclude from the calculations those enrollees who dropped 

out of training (short of completion) for various reasons, such as due to death or illness 

(California, Michigan, Ohio, Pennsylvania), or who drop out very quickly after 

enrolling (Georgia, Michigan, Ohio), or who obtain employment before completing 

(Michigan) or miscellaneous other reasons (e.g., Missouri, Pennsylvania).

Completion. The states in our sample provide relatively little guidance as to how 

program completion should be defined. Likely, this hands-off approach reflects the 

recognition that different programs of training will have different definitions of 

completion, consistent with their own licensing or certification requirements. For 

example, California advises that completion should be defined consistent with the
Table IV-4: Definitions of Key Terms

State Participants Completion Employment

CA Varies across Perkins and BPPVE programs. 

Perkins includes completers and those who 

left without completing but did not enroll in 

any CA community college for a full year. 

The latter includes those enrolled excluding 

those who completed less than 50% of their 

program or dropped out due to death, illness, 

or certain other reasons. 

Varies across Perkins and BPPVE 

programs. For the former, earned at 

least 12 vocational units or earned a 

degree or certificate. For BPPVE, 

receives a certificate, degree, or 

other document certifying 

completion.

Varies across Perkins and BPPVE 

programs. For Perkins, leavers and 

completers found in UI, federal 

employment, the military, or a fouryear public institution. For BPPVE, 

must be a minimum of 32 hrs/wk for 

at least 60 days within 6 mo. Of 

completion.

CT No decisions as yet No decisions as yet No decisions as yet

GA Number exiting or scheduled to exit. Those 

who remain enrolled for less than 2 wks are 

excluded

Defined as “meeting the program’s 

completion requisites;” thus, varies 

by vendor. 

As measured by UI

IN Varies by vendor Varies by vendor Those who obtain employment within 

6 months of completion. 

MD Those who complete or fail to complete 

within 1.5 times their planned completion 

date. Must also have declared that their 

purpose in enrolling is employment related 

and that they intend to take all courses that 

make up the program. Students must declare 

their intent before 20% of program has been 

completed; those who drop out before then 

are excluded. Also exclude part-time 

students. 

For certificate programs, completion 

means attainment of the certificate. 

For non-certificate programs, it 

means the student is eligible to 

receive a Letter of Recognition or a 

Continuing Education Unit.

Any employment as measured by UI. 

Calculation excludes those who 

remain with the same employer that 

employed them prior to training. 

Employment rate will be calculated 

from among either those who 

complete or those who exit, at the 

vendor’s option.

MI Students who exited a program during the 

report period, excluding those who drop out 

while they are entitled to a full refund and 

those who leave training (before completion) 

to take a job or due to death or disability.

Students who earned a degree, 

diploma, journeyman’s card, 

certificate, license, or other 

recognized credential.

Self-reported by vendors through a 

survey. Allows 180 days for 

placement to occur. Includes those 

who continue with education, join 

military, or become self-employed.
Table IV-4: Definitions (continued)

State Participants Completion Employment

MO Still being developed. Want to exclude 

students who continue their education and 

foreign students who return to their 

homeland.

For all students, those who obtain a 

certificate; for WIA-funded students, 

those who complete their training 

objectives, as defined by the LWIA.

As measured by UI. Details 

pending.

NE Total number who participated, or, for 

programs longer than one year, those who 

declare an intention to complete.

For now, will use institutional (not 

program) completion rate, since this 

is all that is available.

As measured by UI. For all-student 

measure, divides by completers.

NC No decisions as yet An individual who has completed a 

course of study, culminating in a 

degree, certificate, or diploma.

Requires a minimum of 24 hours of 

work per wk.

OH All those who exit during a reporting period 

and are not expected to return, but excluding 

those who leave or are expelled early enough 

to qualify for a 100% refund or who depart 

due to death, illness, injury, other any other 

situation that renders the student unable to 

complete.

Those who attain the planned 

program objectives or leave the 

program for unsubsidized 

employment.

As measured by UI, for the quarter 

after the exit quarter. For WIA, 

divides by all WIA participants

OR Those who exit during the report period. No specific definition. As measured by UI, for the quarter 

after the exit quarter. For all-student 

and WIA-student measures, divides 

by completers.

PA Individuals who actively participated and 

were expected to complete during the period, 

excluding those who exit due to death, 

incarceration, institutionalization, health, etc. 

Also excludes those employed at entry.

No specific definition. Self-reported by vendor. Status is 

assessed within 90 days of 

completion or last receipt of training. 

Excludes those already employed at 

enrollment. For WIA measures, 

divides by completers.

TX Individuals who actively participated and 

were expected to complete during the period.

No specific definition. As measured by UI.
IV-42

oversight bodies’ standard definitions. Thus, for vocational and technical education

programs covered under the Perkins Act, a completer “...is a student who completes 

the capstone or final course in a sequence of courses or units established for (the 

program)...or who successfully completes an industry-validated examination.”

Among the states that go further, Ohio advises that completers should be assumed 

to include students who leave the training program for unsubsidized employment. 

Thus, its solution to the problem of persons who obtain employment before completing 

the program is to count them as a completer, while other states (Michigan) exclude such 

individuals from the base through their definition of what constitutes a participant (see 

above).

Employment. Since most states use UI matching to measure employment, they 

are constrained by the definitions implicit in using this source (e.g., not counting as 

employed those who are self-employed or are in non-covered employment). California 

attempts to circumvent these limitations through matching with federal employment 

files and military files. Elsewhere, North Carolina plans on requiring a minimum of 24 

hours of work per week before an individual will be counted as employed. Some states 

also are—or plan on—excluding from the calculation persons who were employed at 

enrollment. 

Another element that is variable across states is when employment must occur 

before a placement can be counted. Because they are not constrained by the UI system 

(since they are relying on self-reported data), Indiana and Michigan use a window of 

six-months and Pennsylvania of 90 days. Another difference is in how the all-student 

employment rate itself is calculated. Most states compute it as the number of those who 

obtained employment divided by all participants, while several (e.g., Michigan, 

Nebraska, Oregon) divide by completers, and another (Maryland) will calculate it either 

way, at the vendor’s option.

State-to-state disparity in the base used in the calculation of the all-student 

employment rate emerges as well in some of the WIA-student calculations. For 

example, some states calculate the retention rate at six months with a base of WIA 

completers (e.g., Georgia, Nebraska, Pennsylvania), while others use WIA completers 

who were employed at exit (e.g., North Carolina, Ohio, Oregon).

Small Cells. Finally, many states recognize that small cell sizes will impair the 

meaningfulness of performance-rate calculations, and provide varying guidance as to 
IV-43

how this problem should be handled. Among other rules we encountered, some waive 

performance accountability for those measures that are calculated based on fewer than 

six enrollees (Maryland), eight enrollees (Missouri), ten enrollees (Pennsylvania), or 25 

enrollees (North Carolina). 

In short, state differences in key definitional issues and in the degree of guidance 

that is provided are pronounced. These differences doubtless will make state-to-state 

comparisons in vendors’ performance very difficult, and in some cases compromise 

within-state comparisons as well.

SUMMARY

Across all states, establishing procedures for vendor eligibility as well as 

developing the necessary data systems to support them have been time-consuming 

exercises, as states have endeavored to make decisions regarding complex issues while 

balancing conflicting objectives. In sorting out these issues, the majority of the states 

convened work groups made up of representatives from local boards, proprietary 

schools, community colleges, and state officials. In one of the states the work group 

met every other week for six consecutive months to determine what performance data 

vendors should report for subsequent eligibility and what minimal levels of performance 

would be required. 

In general, states adopted the strategy of casting a very wide net for initial 

eligibility, and winnowing the list once subsequent eligibility began. Thus, they (or 

their local areas) aggressively marketed the ETP list to vendors to encourage them to 

apply for initial eligibility. Similarly, they imposed minimal informational or 

performance requirements, and they tried to streamline the vendor application process 

as much as possible, by allowing for electronic applications and using centralized 

application procedures. These approaches seem to have been largely successful, as 

most states reported receiving ample applications for initial eligibility, accepted 

virtually all applications they received, and consequently could boast of a wide range of 

training choices from which ITA holders could choose.

Challenges associated with subsequent eligibility were much more daunting, 

however. Among the decisions that gave states pause was deciding what performance 

thresholds to establish and on which measures. Based on a variety of conflicting 

constraints and objectives—the nature of existing data systems, promoting high quality 

while not undermining customer choice, taking into consideration regional variation in 
IV-44

labor market conditions—states made greatly varying decisions. Thus, some 

emphasized vendor performance on some measures but not others, and they established 

minimum performance thresholds that varied greatly from one state to the next. 

Another challenge that states faced with regard to establishing eligibility for 

vendors was deciding how performance should be defined and measured. States were 

able to take advantage of the flexibility that WIA allows to craft definitions and develop 

procedures that made the most sense to them; by the same token, though, this flexibility 

often led to protracted discussions and negotiations whose outcomes reflected a balance 

between the competing needs for data rigor, convenience and practicality. Thus, key 

concepts—such as what constitutes a “program” for ETP purposes, and who should 

count as a program participant, a completer, or employed—needed to be hashed out to 

ensure some degree of uniformity within each state. Again, state-to-state variations in 

the nuances of the decisions that were made are pronounced. Moreover, states made 

different decisions as to whether the onus of data collection should be on the vendor or 

the state. Thus, in many states, the state itself bore the burden of measuring the 

vendors’ performance, through UI wage matching; in other states, by contrast, vendors 

needed to self-report and would collect the information they needed through participant 

and employer surveys, with or without substantial state guidance regarding survey 

parameters.

As a consequence of the complexity of these issues, most states pushed off the 

start of subsequent eligibility for the full 18 months allowed by the law, or delayed it 

even beyond then.
V-1

V. NATURE AND QUALITY OF THE 

CONSUMER REPORT SYSTEM

According to the vision underlying the WIA legislation, the Consumer Report 

System (CRS) is critical for promoting informed customer choice and system 

accountability. For their part, armed with important and useful information about 

different training programs—their costs, class schedules, their performance, and other 

relevant facts—customers can be well equipped to make prudent choices that maximize 

their own interests. At the same time, faced with the requirement to display facts about 

their programs (including performance) in a public and easily accessible format, 

vendors will strive for high performance and responsiveness to customers’ needs. For 

these reasons, the CRS is an instrumental part of the WIA training system.

In keeping with the program regulations, the CRS “must contain the information 

necessary for an adult or dislocated worker to fully understand the options available to 

him or her in choosing a program of training services” (20 CFR 663.570). Such 

information might include program performance on the three all-student and four WIAstudent measures, described in the previous chapter, and the costs of training, among 

other things. In this chapter, we present some information on the quality and 

completeness of the Consumer Report Systems that are available to customers with 

respect to providing access to this sort of information. In doing so, we draw on data 

we collected through site visits to the ITA/ETP Demonstration grantees, supplemented 

with Internet explorations for additional states to provide a comprehensive look at 

Consumer Report Systems throughout the nation as a whole.

DESIGN OF INFORMATION SYSTEMS

At the time of our site visits, all states had developed an ETP list that had been 

distributed to One-Stop centers for their use. These lists took different forms, 

reflecting states’ unique visions for their systems, and they varied in terms of their 

formats and functionality.

Vision for the Consumer Report System

A primary question that states had grappled with was determining what purposes 

and audiences the ETP list should serve. According to the WIA legislation, the list is 

to serve primarily two functions. First, as the name implies, it constitutes a list of 

those providers whose programs may be considered by adults and dislocated workers 
V-2

who are undertaking training funded by WIA [WIA Section 134(d)(4)(C)]. Second, the 

list is to serve as a resource for the universal customer who, through core services, is 

interested in conducting research on training providers in the state, whether or not they 

will be issued an ITA [WIA Section 134(d)(2)(F)]. 

There is potentially some tension between these two objectives. To meet the first 

objective as efficiently as possible, the ETP list might be restricted to only those 

providers eligible to redeem an ITA. In this way, ITA holders can examine the 

universe of providers from whom they can undertake training with the least confusion 

or distraction. By contrast, the universal customer might be interested in researching a 

more comprehensive list of vendors, including those that may not be ITA eligible. 

How programs managed these varying goals differed among the sites we visited. 

A number of the demonstration states (e.g., California, Connecticut, Maryland, 

Pennsylvania, and Texas) gave primacy to the first of these objectives, by developing a 

Consumer Report System intended primarily to meet the needs of WIA training 

customers. In these states, the CRS consists exclusively of ITA-eligible training 

programs and providers. The logic that these states followed was that similar lists 

containing information on other programs and providers already existed to meet the 

needs of customers in core services. For example, for the sake of expediency 

Pennsylvania was focusing on creating a CRS that would include information only for 

ITA-approved programs, on the grounds that doing otherwise would be duplicative with 

a more inclusive list already available from the state’s Department of Education. 

Sometime in the future, this state reasoned, these two lists could be merged to create a 

comprehensive list that identified which vendors were WIA approved, but in the shortterm it felt that its efforts would best be focused on the more limited objectives at hand. 

Another reason it and other states gave for developing a restricted list was to provide a 

strong incentive for vendors to apply for eligibility. According to this logic, vendors 

would want to apply for eligibility for their programs to take advantage of the 

marketing opportunities that the CRS offered.

By contrast, other states in our study (Indiana, Michigan, and North Carolina) 

developed a broader CRS that included non-ITA eligible providers. For example, 

North Carolina’s NC STARS provides users with a searchable database of training 

programs statewide, whether or not the programs have been ITA approved. Once a 

program has been selected, the user can view “ITA Decisions,” which presents 
V-3

information on the ETP decision made by each of the local areas to which that vendor 

submitted its program for approval. These decisions include:

• Approved for the use of an ITA

• Pending (the application is under review)

• No Decision

• Special Approval (an ITA will only be awarded for this training by the 

specified local area based on special circumstances)

• The training is for an occupation deemed not to be in demand in that 

local area

• Declined for WIA voucher

NC STARS thus provides case managers and customers with a comprehensive training 

provider list, with information on the separate ETP eligibility decisions made by each 

of the local areas to which the vendor applied.

States that opted to open the CRS to include non-ITA providers did so for several 

reasons. First, they wanted to ease the workload for their cross-trained staff, by 

creating more efficient search tools that were easy to use. Rather than different 

agencies having a unique list for its own purposes or having case managers search 

multiple lists with different levels of certification or types of approval, a single list 

(with indicators denoting which vendors had been approved for an ITA) would enable 

staff members to access all the training options for their customers quickly and with 

ease. 

A second consideration for those developing an inclusive list was to maximize the 

use of the WIA administrative funds that were expended in building the list to begin 

with. Faced with limited funding for the development of their One-Stop infrastructure, 

and the high costs of developing the computer interface for the ETP and CRS, states 

want to leverage their investment in compiling data and developing software by having 

the resultant tools serve multiple purposes. This consideration becomes especially 

important in states that are anticipating a relatively low flow of WIA customers into 

training, because in this circumstance a relatively small number of persons would 

benefit if the CRS list were narrowly restricted. By contrast, including a larger number 

of providers and programs in the CRS justifies the expense of creating this new tool. 

Related to this, states developing an inclusive list wanted a tool that was intended 

to be a multi-purpose resource. In recognition of the needs of the universal customer 
V-4

who may have funds for training from other sources, these states viewed their CRS as 

an important resource for all individuals looking for training. Moreover, such a list 

could serve as a resource for employers as well. In Georgia, for example, state 

representatives sent out notices to small and mid-sized employers encouraging them to 

use the list for their incumbent-worker training. These customers would not necessarily 

have had a use for the CRS if non-WIA eligible training programs were not included on 

the list. 

Finally, another advantage of a broader list, it was thought, was to provide an 

incentive for vendors to seek ITA approval. This explanation was paradoxical given 

that states adopting a CRS restricted only to ITA-approved vendors also provided this 

as a rationale for their strategy. Both groups recognized how difficult it might be to 

encourage vendors to go through the trouble of filling out the required paperwork and 

collecting the necessary performance data to seek eligibility for a program, given that 

they could generally expect only a small customer flow of ITA holders. Thus, some 

states wanted to emphasize the advertising advantages that the vendors’ inclusion on the 

list would offer, and felt that restricting the list only to ITA-eligible providers would 

best serve this purpose. By contrast, others took the opposite approach. For example, 

respondents in Michigan noted that any vendor could be included on the CRS, whether 

or not it was ITA approved, but each must submit essentially the same paperwork and 

information about its programs, including some performance data. Given this, the 

marginal burden of applying for ITA eligibility was expected to be very modest, which, 

it was hoped, would induce vendors to apply. 

Formats, Searches, and Information Displays

During the several years of our study, the states’ Consumer Report Systems have 

been evolving dramatically. From nascent or non-existent systems at first, all of the 

thirteen demonstration states now have databases with at least basic information about 

training programs.1 Exhibit V-1 displays basic features of these states’ systems. 

As the table shows, all sites but one are searchable, in that a customer can 

identify search criteria to generate a list of matching programs. Key search criteria 

available in most or all of the states include:

 

1

 Maryland has a searchable database, but it has been temporarily taken off line. All of the 

remaining twelve states have Internet-accessible ETP lists, with some consumer information, as of the 

summer of 2002, when this research was conducted.
V-5

• Program. Users can specify a search string relating to the program to 

generate a list of matching programs. Some of these search algorithms 

are more limiting than others. For example, some search for a 

matching text string in only the program title. Thus, if a user entered 

“computer” as the search string, only programs that had this word in its 

title would produce a match (programs like Microsoft Certified would 

therefore not be selected). Others are more versatile, and match on the 

basis of program name or description.

• Location. In various states, one can select programs by city, county, or 

zip code. 

• Provider. One generally can also search for specific providers, usually 

by their name.

• Occupation. About half of the states we studied allow one to search by 

occupational area or career cluster (e.g., manufacturing and processing, 

health services, educational services).

Less common are engines that allow searching by program cost, whether financial aid 

might be available, accessibility (e.g., near public transportation, has ample parking), 

school type (e.g., public or private institution), and training delivery mode (e.g., 

classroom format, correspondence, Internet, etc.). Finally, some of the states whose 

CRS includes non-ITA approved programs allow searching by whether the vendor is on 

the eligible training provider list for WIA purposes.

Once one has specified the search criteria, the customer is presented with a list of 

those vendors and programs that result from the match. By selecting one of these one 

is able to access information that varies in its detail across the various states’ systems. 

Except for Oregon’s (whose system is still undergoing development), every CRS 

provides information on the vendor’s location, program costs (usually separately 

specifying tuition and other costs, such as books and supplies and registration fees), and 

program duration (e.g., hours of instruction). Some systems additionally provide 

information on:

• Course schedule, such as whether day, evening, or weekend schedules 

are available.

• Occupational training areas, including CIP codes.

• Program description, such as a few sentences describing the program 

content and training objectives, and sometimes the specific course 

sequence that makes up the program.
Exhibit V-1: Characteristics of Consumer Report Systems

Search Criteria Information Available about Programs and Providers

ProgramLocationProviderOccupation

Other

CostDurationScheduleOccupationsDescriptionAll PerfWIA Perf

Other

CA X X

Delivery mode, cost, 

financial aid, accessibility, 

additional services 

X X X X X X

Prerequisites, target audience, additional 

services offered, accessibility, delivery mode, 

financial aid

CT X X X X X X X

Certificate

GA X X X X X X L

School setting, student body size and 

characteristics, application procedures, financial 

aid, student activities, accreditation,

IN X X X X ITA-approved or not X X X L L Degree offered

MI X X X X ITA-approved or not X X X X Waiting period, financial aid, credentials, 

admission requirements, date of last update

MO X X X Cost X X X

NE X X X X L Year established, accreditation, waiting list, 

NC X X X X ITA-approved or not X X X X L Accessibility, additional services offered, 

financial aid, certificates

OH X X X X School type, accredited, 

services provided, ITAapproved or not

X X X X X L L Type, degrees awarded, financial aid, refund 

policy

OR Degree/certificates

PA X X X X X X X X X L Financial aid, additional services available

TX X X X X X X X X X ITA-approval date, certificates, minimum entry 

requirements, entry recommendations, 

completion criteria

_________________

Note: X=denotes the item is generally available; L denotes that some information is available but that many programs have missing data. Information on 

vendor location is available everywhere. At the time this table was prepared (mid-2002), Maryland’s CRS was down for renovation.
V-7

• Performance information on the all-student and WIA-student measures. 

At this point, CRSs are more likely to have performance information 

computed based on all-students than WIA students, and even for the 

former the data are often spotty, available for some programs but not 

others.

Less common are systems that provide information on:

• Entrance prerequisites and target audience.

• Additional services offered, such as whether the school provides 

placement assistance, career counseling, or on-site child care.

• Accessibility, including whether parking is available, whether the 

program is accessible to public transportation, whether ESL is provided, 

and whether training can be provided in languages other than English, 

including sign language.

• Delivery mode, such as whether the forum for instruction is a classroom 

or is accessed via the Internet or correspondence.

• Which degrees or certificates are awarded, if any.

Additional information available in a few systems include characteristics of the student 

body (e.g., racial and gender composition and student-body size), application 

procedures, whether student activities (such as clubs) are supported, and the ITAapproval status (include date of ETP certification). 

Thus far, Texas’s system is the most complete, from the standpoint of having rich 

information about programs and vendors, including performance data. However, other 

states’ systems—such as California’s, Pennsylvania’s, North Carolina’s, Ohio’s, and 

others—are quickly catching up. Indeed, compared to the state of their Consumer 

Report Systems when we undertook a similar inspection about a year ago, the evolution 

of these data systems has happened quickly.

VENDOR CHOICES AND THE QUALITY AND COMPLETENESS OF INFORMATION

The above description reflects how the Consumer Report Systems have been 

designed by the 13 states in our sample. We additionally undertook an examination of 

the composition of the lists in a number of our case-study states, and also conducted a 

uniform set of searches to assess the quality and completeness of these systems across 

all 50 states, as of the summer of 2002.
V-8

Composition of ETP Lists

As we mentioned in the previous chapter (and will discuss still further in the next 

chapter), community colleges and other public educational institutions have been among 

the most vocal critics of the eligibility requirements for listing on the eligible training 

provider list. They argue, among other things, that their need to report employmentrelated performance outcomes is unduly burdensome given the few ITA customers they 

serve, that their mission is to serve as educational and not job-training (or jobplacement) institutions, and that their measured performance could misrepresent their 

programs’ actual quality given their open-admissions policies. For these reasons, 

public educational institutions in many states have balked at applying for ETP status 

under subsequent eligibility, despite the fact that WIA program operators view them as 

critical partners in their workforce development systems.

Although we were told that some important public educational institutions would 

not be applying for subsequent eligibility in many states, we have no way of quantifying 

the extent to which this was so. Nonetheless, Exhibit V-2 attempts to characterize the 

composition of eligible training provider lists in selected case-study states by showing 

the number of programs and providers that were listed, and the percent of each that are 

public educational institutions, private and for-profit providers, and other (i.e., noneducational) public and non-profit agencies. As the table shows, the absolute size of 

the lists varies dramatically, from a low of just over 100 vendors and 876 programs in 

Oregon, to a high of over 9,000 in California. These differences are of course roughly 

what one would expect based on the size of the states’ populations.

The composition of these lists also varies dramatically across states. For 

example, the average number of programs per vendor varies from 16.9 in Georgia 

(i.e., 2,723 programs divided by 161 vendors) to a low of just 5.4 in Connecticut (i.e., 

887 divided by 162).2 Elsewhere in the table, we show that public educational 

institutions make up just 10% of all institutions on the list in Indiana, but 45% of all 

institutions in Georgia. Conversely, programs offered by private vendors make up 

about one-half of all programs in Connecticut, but just 10% of all programs in Georgia.

 

2

 To ensure some level of comparability across states, programs offered at separate locations 

within a state but that are otherwise identical are counted as a single program for purposes of this table, 

even though in some states the program at each separate location is listed separately on the state’s ETP 

list and each may require a separate eligibility application. 
Exhibit V-2:

Composition of Eligible Training Provider Lists

Number of: Pct of Vendors That Are: Pct of Programs Provided by:

Vendors Programs Public Ed Private

NonProfits Public Ed PrivateNonProfits

California na 9,206 na na na na na na

Connecticut 162 887 24.1% 55.6% 20.4% 36.7% 51.3% 12.0%

Georgia 161 2,723 44.7 42.9 12.4 89.3 9.8 0.9

Indiana 229 1,676 9.6 72.1 18.3 53.2 38.7 8.1

Missouri 344 8,022 na na na na na na

Michigan 247 2,384 29.6 60.7 9.7 58.2 33.4 8.4

Nebraska 98 2,520 23.5 55.1 21.4 na na na

Ohio 457 4,586 27.1 53.8 19.0 62.0 29.3 8.7

Oregon 115 876 20.0 60.9 19.1 56.9 33.5 9.7

_____________________

Note: We have this information only for the nine demonstration states that are shown. Columns show the percentage of eligible training 

provider vendors listed on the CRS that are: public educational institutions, private and for-profit institutions, and other public and non-profit 

institutions. Percent of programs represent the percentage of all ITA-approved programs on the ETP list that are offered by institutions of 

these three types. To ensure some level of comparability across states, programs offered at separate locations within a state but that are 

otherwise identical are treated as a single program for purposes of this table, even though in some states the program at each separate location 

is listed separately on the state’s ETP list and each may require a separate eligibility application. Statistics are as of mid-2002.
V-10

Yet, despite these differences, there are important commonalties as well. For 

example, in all states, but one, public educational institutions account for at least half of 

all program offerings, suggesting that these institutions continue to constitute a rich 

source of training opportunities for ITA holders, despite fears to the contrary. The 

average number of programs per vendor is also much higher for public educational 

institutions than it is for others, as in every state the percentage of programs offered by 

such institutions is higher than their percentage composition of the vendor pool. Put 

differently, private for-profit vendors make up the majority of all vendors on most 

states’ lists, but programs offered by public educational institutions make up the 

majority of all program offerings. Finally, non-profit and other public agencies 

constitute a small but still appreciable percentage of both vendor and program lists in 

most states.

Overall, these findings suggest that states’ eligible training provider lists offer a 

substantial number of training choices and a rich array of offerings by institutions of a 

variety of different types. In other words, customer choice seems well supported by 

these figures, at least when one looks within each state as a whole.

Quality and Completeness of Performance Data

As an additional way of gauging the adequacy of choice, and of information 

available in Consumer Report Systems to support that choice, we identified five training 

fields that, based on our site visits, appeared to represent frequently made training 

choices. These training fields are: 

• Heating and Air-conditioning Repair

• Medical Assistant

• Microsoft Certified Solutions Developer

• Paralegal and Legal Assistant

• Commercial Vehicle Operator, including truck and bus driver.

For each of these training fields, we visited the CRS web sites of all 50 states to 

tabulate, as of the summer of 2002, the number of eligible programs that were listed, 

and, of these, the number of listings that provided:

• Basic information about the program, such as

− The number of participants enrolled

− Program costs

− Program duration
V-11

• Performance information measured for all students, including

− Starting wages

− Program completion rate

− Employment rate

− Employment rate in a training-related field

• Performance information measured for WIA-funded students, including

− Employment rate

− Six-month retention rate

− Six-month wage rate

− License or certification rate

These tabulations appear at the end of this chapter as an Attachment; each of the 

five occupations we are profiling appears on successive tables in this attachment, in the 

order listed above.3 This information is summarized in Exhibit V-3, which shows, for 

each occupation, the number of states with ETP lists of various sizes and the number of 

states whose CRS provides information for various percentages of those programs. In 

conducting this research, we were able to access the Consumer Report Systems for 42 

states.

This exhibit reaffirms the notion that Consumer Report Systems in these states in 

general offer customers ample choices. For example, for each of the training fields we 

have profiled except one (Microsoft Certified Solutions Developer), the majority of 

states offer 11 or more training choices—i.e., separately listed programs providing 

training of this type. Indeed, some states list more than 50 training programs in some 

of these fields, with the number of options for some fields ranging beyond 100 separate 

programs in a handful of states. Thus, at least for these common training occupations, 

customers have an array of training programs from which to pick, at least within the 

state as a whole.

The exhibit further demonstrates that the array of information available about 

these programs on the states’ Consumer Report Systems varies substantially. 

Encouragingly, most states’ systems provide information on the costs and program 

duration for at least 75% of the program listings identified by the column heading. 

 

3

 These tabulations were performed in early 2002. Some information may have changed since then.
V-12

Exhibit V-3

States’ ETP Listings for Five Selected Occupations Heating/ AC Repair Medical Assistant Microsoft Certified Paralegal/ 

Legal AsstCommercial Driver

Number of States with Various Sizes 

of ETP Lists for 5 Training Fields

None (no program listings) 3 2 11 2 3

1-10 listings 13 12 26 20 15

11-25 listings 14 16 5 16 14

26-50 listings 8 5 0 2 5

51 or more listings 4 7 0 2 5

TOTAL NUMBER OF STATES 42 42 42 42 42

No. of States Whose Eligible 

Programs Have Information on:

Program Costs

None of the programs on the 

state’s list has this info 8 10 7 8 6

1% to 25% of programs have it 2 1 0 1 2

26% to 50% of programs 3 1 0 0 1

51% to 75% of programs 3 1 0 1 1

76% to 100% of programs 23 27 24 30 29

Program Duration

None of the programs on the 

state’s list has this info 13 12 7 9 9

1% to 25% of programs have it 0 1 0 1 0

26% to 50% of programs 2 0 3 1 2

51% to 75% of programs 3 1 1 3 1

76% to 100% of programs 21 26 20 26 27

All-Student Starting Wages

None of the programs on the 

state’s list has this info 25 23 22 25 22

1% to 25% of programs have it 3 3 2 4 1

26% to 50% of programs 6 3 3 4 4

51% to 75% of programs 4 8 2 4 6

76% to 100% of programs 1 3 2 3 6
V-13

Exhibit V-3 (concluded) Heating/ AC Repair Medical 

AssistantMicrosoft CertifiedParalegal/ Legal AsstCommercial Driver

All-Student Completion Rate

None of the programs on the 

state’s list has this info 23 23 22 25 21

1% to 25% of programs have it 3 3 3 4 2

26% to 50% of programs 6 2 0 3 1

51% to 75% of programs 5 7 3 4 9

76% to 100% of programs 2 4 3 4 6

All-Student Employment Rate

None of the programs on the 

state’s list has this info 27 25 23 30 24

1% to 25% of programs have it 4 3 3 2 2

26% to 50% of programs 3 2 1 4 1

51% to 75% of programs 3 7 2 1 8

76% to 100% of programs 2 3 2 3 4

WIA-Student Employment Rate

None of the programs on the 

state’s list has this info 37 34 29 38 32

1% to 25% of programs have it 1 4 2 2 4

26% to 50% of programs 0 1 0 0 2

51% to 75% of programs 0 1 0 0 1

76% to 100% of programs 1 0 0 0 0

WIA-Student Retention Rate

None of the programs on the 

state’s list has this info 38 38 30 39 34

1% to 25% of programs have it 1 2 1 1 2

26% to 50% of programs 0 0 0 0 2

51% to 75% of programs 0 0 0 0 1

76% to 100% of programs 0 0 0 0 0

WIA-Student License Rate

None of the programs on the 

state’s list has this info 37 37 30 40 33

1% to 25% of programs have it 2 2 1 0 3

26% to 50% of programs 0 0 0 0 2

51% to 75% of programs 0 1 0 0 1

76% to 100% of programs 0 0 0 0 0
V-14

Nonetheless, appreciable numbers of states (10 for Medical Assistants) provide 

information on costs for none of the programs they list in these training fields, and 

slightly more provide no information on program duration. Clearly, most states’ CRS 

generally provide cost and duration data, but a few states’ systems lag badly behind.

Data are much sparser for the all-student performance measures. Looking at 

Heating and Air Conditioning Repair as an illustrative example, 25 states’ systems 

provide information on starting wages for none of the programs listed in this training 

field, while only 1 state provides this information for more than 75% of their programs. 

Statistics are comparable for the all-student Completion Rate and Employment Rate, 

and across each of the five training fields we sampled.

Finally, data for the WIA-student measures are even much sparser. No state 

routinely provides information for most of their programs on the WIA-student 

employment rate, retention rate, or license/certification rate.

In short, some basic program information appears to be available in most states’ 

systems relating to the cost and duration of training, but information on performancerelated measures is much less frequently available.

SUMMARY

The development of Consumer Report Systems, with rich information about 

vendors’ programs, has represented a long and arduous process in the ITA/ETP 

Demonstration states and, indeed, in the nation as a whole. On the one hand, as of the 

summer of 2002, these systems are still far from the vision embedded in the WIA 

legislation, in that few states’ systems have extensive information about vendor 

performance, and almost none do so on the WIA-student measures. On the other hand, 

nearly every state has a web-accessible system containing extensive information—

program descriptions, costs, schedules, accessibility features, and so on—that is bound 

to be very useful not only to ITA holders but to the universal customer as well. 

Moreover, ETP lists typically represent hundreds of eligible providers from a diverse 

mix of provider types—including community colleges, proprietary schools, and 

others—and sometimes thousands of eligible programs from which ITA customers can 

choose. In this sense, the ETP and CRS implementation since the start of WIA 

represents a truly remarkable mobilization of effort.
Appendix to Chapter V

Part 1

Occupation: Heating/AC Repair

Participants Cost DurationStarting 

Wages

Completion 

Rate

Employment 

Rate

Related 

Field Rate

Employme

nt Rate

6 Month Retention Rate6 Month 

Wage Rate

License/Certificate Rate

Alabama

Alaska

Arizona 1 0 1 1 0 0 0 0 0 0 0 0

Arkansas 22 5 16 6 3 5 5 0 0 0 0 0

California 25 0 25 25 0 0 0 0 0 0 0 0

Colorado 1 1 1 1 1 1 1 0 1 0 0 0

Connecticut 16 0 16 16 0 0 0 0 0 0 0 0

Delaware 5 0 5 5 0 0 0 0 0 0 0 0

D.C. 0 0 0 0 0 0 0 0 0 0 0 0

Florida 56 46 4 46 34 46 45 43 0 0 0 0

Georgia 41 0 0 0 0 0 0 0 0 0 0 0

Hawaii 0 0 0 0 0 0 0 0 0 0 0 0

Idaho 4 0 4 0 0 0 0 0 0 0 0 0

Illinois 33 0 0 0 0 0 0 0 0 0 0 0

Indiana 18 0 14 0 7 0 0 0 0 0 0 0

Iowa

Kansas 11 0 1 0 0 0 0 0 0 0 0 0

Kentucky 26 0 23 14 13 14 14 0 0 0 0 0

Louisiana 29 0 0 0 0 0 0 0 0 0 0 0

Maine 4 4 0 0 0 0 0 0 0 0 0 0

Maryland 14 0 14 14 0 0 0 0 0 0 0 0

Massachusetts 12 0 0 0 0 0 0 0 0 0 0 0

Michigan 9 3 9 0 4 6 4 0 0 0 0 0

Minnesota 37 0 10 37 0 1 0 0 0 0 0 0

Missouri 37 11 31 30 0 11 0 0 0 0 0 0

Montana

Nebraska 15 0 15 15 0 0 0 0 0 0 0 0

Nevada 2 0 2 2 0 0 0 0 0 0 0 0

New Hampshire 1 0 0 0 0 0 0 0 0 0 0 0

New Jersey 39 0 39 39 0 0 0 0 0 0 0 0

New Mexico 7 4 7 7 4 4 4 0 0 0 0 0

Number Providing Information for WIA-funded 

Number Students On:

of Eligible 

Programs

Number Providing 

Information On:

Number Providing Information for

All Participants On:
Appendix to Chapter V

Part 1

Occupation: Heating/AC Repair

Participants Cost DurationStarting 

Wages

Completion 

Rate

Employment 

Rate

Related 

Field Rate

Employme

nt Rate

6 Month Retention Rate6 Month 

Wage Rate

License/Certificate Rate

Alabama

Alaska

Arizona 1 0 1 1 0 0 0 0 0 0 0 0

Arkansas 22 5 16 6 3 5 5 0 0 0 0 0

California 25 0 25 25 0 0 0 0 0 0 0 0

Colorado 1 1 1 1 1 1 1 0 1 0 0 0

Connecticut 16 0 16 16 0 0 0 0 0 0 0 0

Delaware 5 0 5 5 0 0 0 0 0 0 0 0

D.C. 0 0 0 0 0 0 0 0 0 0 0 0

Florida 56 46 4 46 34 46 45 43 0 0 0 0

Georgia 41 0 0 0 0 0 0 0 0 0 0 0

Hawaii 0 0 0 0 0 0 0 0 0 0 0 0

Idaho 4 0 4 0 0 0 0 0 0 0 0 0

Illinois 33 0 0 0 0 0 0 0 0 0 0 0

Indiana 18 0 14 0 7 0 0 0 0 0 0 0

Iowa

Kansas 11 0 1 0 0 0 0 0 0 0 0 0

Kentucky 26 0 23 14 13 14 14 0 0 0 0 0

Louisiana 29 0 0 0 0 0 0 0 0 0 0 0

Maine 4 4 0 0 0 0 0 0 0 0 0 0

Maryland 14 0 14 14 0 0 0 0 0 0 0 0

Massachusetts 12 0 0 0 0 0 0 0 0 0 0 0

Michigan 9 3 9 0 4 6 4 0 0 0 0 0

Minnesota 37 0 10 37 0 1 0 0 0 0 0 0

Missouri 37 11 31 30 0 11 0 0 0 0 0 0

Montana

Nebraska 15 0 15 15 0 0 0 0 0 0 0 0

Nevada 2 0 2 2 0 0 0 0 0 0 0 0

New Hampshire 1 0 0 0 0 0 0 0 0 0 0 0

New Jersey 39 0 39 39 0 0 0 0 0 0 0 0

New Mexico 7 4 7 7 4 4 4 0 0 0 0 0

Number Providing Information for WIA-funded 

Number Students On:

of Eligible 

Programs

Number Providing 

Information On:

Number Providing Information for

All Participants On:
Appendix to Chapter V

Part 1

Participants Cost DurationStarting 

Wages

Completion 

Rate

Employment 

Rate

Related 

Field Rate

Employme

nt Rate

6 Month Retention Rate6 Month 

Wage Rate

License/Certificate Rate

Number Providing Information for WIA-funded 

Number Students On:

of Eligible 

Programs

Number Providing 

Information On:

Number Providing Information for

All Participants On:

New York 51 0 51 51 0 0 0 0 0 0 0 0

North Carolina 29 0 0 0 12 12 12 0 0 0 0 0

North Dakota 6 0 6 6 0 0 0 0 0 0 0 0

Ohio 11 0 0 0 0 0 0 0 0 0 0 0

Oklahoma 18 0 16 12 5 6 3 0 0 0 0 0

Oregon

Pennsylvania 54 0 54 54 2 2 2 0 0 0 0 1

Puerto Rico

Rhode Island

South Carolina 2 0 1 2 0 1 1 1 0 0 0 0

South Dakota 4 0 3 0 0 0 0 0 0 0 0 0

Tennessee 25 0 25 25 0 0 0 0 0 0 0 0

Texas 100 69 72 72 69 69 69 0 7 6 5 6

Utah

Vermont 0 0 0 0 0 0 0 0 0 0 0 0

Virginia 3 0 3 3 0 0 0 0 0 0 0 0

Washington 12 9 6 6 6 9 0 0 0 0 0 0

West Virginia 11 2 11 11 2 3 2 0 0 0 0 0

Wisconsin 15 6 15 15 9 5 0 5 0 0 0 0

Wyoming
Attachment to Chapter V

Part 2

Occupation: Medical Assistant

Participants Cost DurationStarting 

Wages

Completion 

Rate

Employment 

Rate

Related 

Field Rate

Employme

nt Rate

6 Month Retention Rate6 Month 

Wage Rate

License/Certificate Rate

Alabama

Alaska

Arizona 20 7 15 20 3 5 5 1 0 0 0 0

Arkansas 6 0 5 0 0 0 0 0 0 0 0 0

California 230 0 230 230 0 0 0 0 0 0 0 0

Colorado 14 10 14 14 8 8 8 0 0 0 0 0

Connecticut 24 0 24 24 0 0 0 0 0 0 0 0

Delaware 6 0 6 6 0 0 0 0 0 0 0 0

D.C. 0 0 0 0 0 0 0 0 0 0 0 0

Florida 49 35 17 41 29 31 33 21 0 0 0 0

Georgia 45 0 0 0 0 0 0 0 0 0 0 0

Hawaii 3 0 0 0 2 2 2 0 0 0 0 0

Idaho 5 0 5 0 0 0 0 0 0 0 0 0

Illinois 17 0 0 0 0 0 0 0 0 0 0 0

Indiana 66 54 6 34 4 4 4 2

Iowa

Kansas 14 0 0 0 0 0 0 0 0 0 0 0

Kentucky 26 0 26 17 18 17 18 0 0 0 0 0

Louisiana 7 0 0 0 0 0 0 0 0 0 0 0

Maine 13 10 0 0 0 0 0 0 0 0 0 0

Maryland 12 0 12 12 0 0 0 0 0 0 0 0

Massachusetts 24 0 0 0 0 0 0 0 0 0 0 0

Michigan 57 45 54 46 34 41 41 39 0 0 0 0

Minnesota 15 0 3 15 0 0 0 0 0 0 0 0

Missouri 29 0 29 29 0 0 0 0 0 0 0 0

Montana

Nebraska 7 0 7 7 0 0 0 0 0 0 0 0

Nevada 1 0 1 1 0 0 0 0 0 0 0 0

New Hampshire 2 0 0 0 0 0 0 0 0 0 0 0

Number Providing Information for WIA-funded 

Number Students On:

of Eligible 

Programs

Number Providing Information for

All Participants On:

Number Providing 

Information On:
Attachment to Chapter V

Part 2

Participants Cost DurationStarting 

Wages

Completion 

Rate

Employment 

Rate

Related 

Field Rate

Employme

nt Rate

6 Month Retention Rate6 Month 

Wage Rate

License/Certificate Rate

Number Providing Information for WIA-funded 

Number Students On:

of Eligible 

Programs

Number Providing Information for

All Participants On:

Number Providing 

Information On:

New Jersey 72 0 72 72 0 0 0 0 0 0 0 0

New Mexico 2 1 2 2 1 1 1 0 0 0 0 0

New York 54 0 54 54 0 0 0 0 0 0 0 0

North Carolina 24 0 0 0 17 17 17 0 0 0 0 0

North Dakota 3 0 3 3 0 0 0 0 0 0 0 0

Ohio 47 0 0 0 0 0 0 0 0 0 0 0

Oklahoma 5 0 5 4 4 4 4 0 0 0 0 0

Oregon

Pennsylvania 88 0 88 88 8 5 6 4 6 4 6 4

Puerto Rico

Rhode Island 11 0 11 11 11 11 10 10 7 0 7 7

South Carolina 6 0 6 6 6 6 6 6 3 0 0 0

South Dakota 3 0 0 0 0 0 0 0 0 0 0 0

Tennessee 18 0 18 18 0 0 0 0 0 0 0 0

Texas 85 24 85 85 23 24 24 0 10 6 4 3

Utah

Vermont 0 0 0 0 0 0 0 0 0 0 0 0

Virginia 15 0 15 14 0 0 0 0 0 0 0 0

Washington 17 16 15 15 12 16 0 0 0 0 0 0

West Virginia 22 14 22 22 11 14 14 0 0 0 0 0

Wisconsin 21 5 21 21 5 5 0 5 5 0 0 0

Wyoming
Attachment to Chapter V

Part 3

Occupation: Microsoft Certified Solutions Developer

Participants Cost DurationStarting 

Wages

Completion 

Rate

Employment 

Rate

Related 

Field Rate

Employme

nt Rate

6 Month Retention Rate6 Month 

Wage Rate

License/Certificate Rate

Alabama

Alaska

Arizona 3 0 3 3 0 0 0 0 0 0 0 0

Arkansas 0 0 0 0 0 0 0 0 0 0 0 0

California 11 0 11 11 0 0 0 0 0 0 0 0

Colorado 1 0 1 1 0 0 0 0 0 0 0 0

Connecticut 3 0 3 3 0 0 0 0 0 0 0 0

Delaware

D.C. 0 0 0 0 0 0 0 0 0 0 0 0

Florida 0 0 0 0 0 0 0 0 0 0 0 0

Georgia 22 0 0 0 0 0 0 0 0 0 0 0

Hawaii 0 0 0 0 0 0 0 0 0 0 0 0

Idaho 1 0 1 0 0 0 0 0 0 0 0 0

Illinois 13 0 0 0 0 0 0 0 0 0 0 0

Indiana 4 0 4 2 2 1 1 0 0 0 0 0

Iowa

Kansas

Kentucky 7 0 7 3 3 4 2 0 0 0 0 0

Louisiana 0 0 0 0 0 0 0 0 0 0 0 0

Maine 3 0 3 3 0 0 0 0 0 0 0 0

Maryland 4 0 4 4 0 0 0 0 0 0 0 0

Massachusetts 4 0 3 3 2 3 3 0 0 0 0 0

Michigan

Minnesota 3 0 0 1 0 0 0 0 0 0 0 0

Missouri 8 0 7 7 0 0 0 0 0 0 0 0

Montana

Nebraska 3 0 3 3 0 0 0 0 0 0 0 0

Nevada 1 0 1 1 0 0 0 0 0 0 0 0

New Hampshire 6 0 0 0 0 0 0 0 0 0 0 0

New Jersey 6 0 6 6 0 0 0 0 0 0 0 0

New Mexico 3 3 3 3 2 3 3 0 0 0 0 0

Number Providing Information for WIA-funded 

Students On:

Number 

of Eligible 

Programs

Number Providing Information for

All Participants On:

Number Providing 

Information On:
Attachment to Chapter V

Part 3

Participants Cost DurationStarting 

Wages

Completion 

Rate

Employment 

Rate

Related 

Field Rate

Employme

nt Rate

6 Month Retention Rate6 Month 

Wage Rate

License/Certificate Rate

Number Providing Information for WIA-funded 

Students On:

Number 

of Eligible 

Programs

Number Providing Information for

All Participants On:

Number Providing 

Information On:

New York 9 0 9 9 0 0 0 0 0 0 0 0

North Carolina 4 0 4 3 0 0 0 0 0 0 0 0

North Dakota 0 0 0 0 0 0 0 0 0 0 0 0

Ohio 0 0 0 0 0 0 0 0 0 0 0 0

Oklahoma 3 0 3 3 3 3 3 0 0 0 0 0

Oregon

Pennsylvania 8 0 8 8 1 1 1 1 1 1 1 1

Puerto Rico

Rhode Island

South Carolina 3 0 3 3 2 2 2 2 0 0 0 0

South Dakota 2 0 0 0 0 0 0 0 0 0 0 0

Tennessee 6 0 6 6 0 0 0 0 0 0 0 0

Texas 13 8 13 13 3 1 1 0 1 0 0 0

Utah

Vermont 0 0 0 0 0 0 0 0 0 0 0 0

Virginia 6 0 5 4 0 0 0 0 0 0 0 0

Washington 1 0 0 0 0 0 0 0 0 0 0 0

West Virginia 0 0 0 0 0 0 0 0 0 0 0 0

Wisconsin 16 14 16 16 16 14 0 13 0 0 0 0

Wyoming
Attachment to Chapter V

Part 4

Occupation: Paralegal / Legal Assistant

Participants Cost DurationStarting 

Wages

Completion 

Rate

Employment 

Rate

Related 

Field Rate

Employme

nt Rate

6 Month Retention Rate6 Month 

Wage Rate

License/Certificate Rate

Alabama

Alaska

Arizona 3 0 3 3 1 0 0 0 0 0 0 0

Arkansas 2 1 2 1 0 1 0 0 0 0 0 0

California 41 0 41 41 0 0 0 0 0 0 0 0

Colorado 9 6 9 9 1 5 4 0 1 1 1 0

Connecticut 15 0 15 15 0 0 0 0 0 0 0 0

Delaware 1 0 1 1 0 0 0 0 0 0 0 0

D.C. 0 0 0 0 0 0 0 0 0 0 0 0

Florida 20 19 19 19 18 19 19 18 0 0 0 0

Georgia 20 0 0 0 0 0 0 0 0 0 0 0

Hawaii 0 0 0 0 0 0 0 0 0 0 0 0

Idaho 9 0 9 0 0 0 0 0 0 0 0 0

Illinois 7 0 0 0 0 0 0 0 0 0 0 0

Indiana 17 0 17 3 9 2 2 2 1 0 0 0

Iowa

Kansas 14 0 0 0 0 0 0 0 0 0 0 0

Kentucky 11 0 8 6 7 7 7 0 0 0 0 0

Louisiana 9 0 0 0 0 0 0 0 0 0 0 0

Maine 6 5 0 0 0 0 0 0 0 0 0 0

Maryland 4 0 4 4 0 0 0 0 0 0 0 0

Massachusetts 13 3 12 13 2 3 0 0 0 0 0 0

Michigan 15 8 15 15 3 5 5 2 0 0 0 0

Minnesota 4 0 1 4 0 0 0 0 0 0 0 0

Missouri 17 5 13 12 0 4 0 0 0 0 0 0

Montana

Nebraska 14 0 14 14 0 0 0 0 0 0 0 0

Nevada 4 0 4 4 0 0 0 0 0 0 0 0

New Hampshire 6 0 0 0 0 0 0 0 0 0 0 0

New Jersey 22 0 22 22 0 0 0 0 0 0 0 0

New Mexico 13 9 13 13 5 8 5 0 0 0 0 0

Number Providing Information for WIA-funded 

Students On:

Number Providing 

Information On:

Number Providing Information for

Number All Participants On:

of Eligible 

Programs
Attachment to Chapter V

Part 4

Participants Cost DurationStarting 

Wages

Completion 

Rate

Employment 

Rate

Related 

Field Rate

Employme

nt Rate

6 Month Retention Rate6 Month 

Wage Rate

License/Certificate Rate

Number Providing Information for WIA-funded 

Students On:

Number Providing 

Information On:

Number Providing Information for

Number All Participants On:

of Eligible 

Programs

New York 51 0 51 51 0 0 0 0 0 0 0 0

North Carolina 20 0 0 0 17 17 17 0 0 0 0 0

North Dakota 3 0 3 3 0 0 0 0 0 0 0 0

Ohio 32 0 0 0 0 0 0 0 0 0 0 0

Oklahoma 5 0 5 5 3 5 4 0 0 0 0 0

Oregon

Pennsylvania 52 0 52 52 3 0 0 0 0 0 0 0

Puerto Rico

Rhode Island

South Carolina 8 0 8 8 3 6 4 4 0 0 0 0

South Dakota 2 0 2 0 0 0 0 0 0 0 0 0

Tennessee 14 0 14 14 0 0 0 0 0 0 0 0

Texas 16 7 16 16 6 1 1 0 0 0 0 0

Utah

Vermont 2 0 2 2 0 0 0 0 0 0 0 0

Virginia 4 0 4 3 0 0 0 0 0 0 0 0

Washington 11 11 11 11 10 11 0 0 0 0 0 0

West Virginia 3 0 3 3 0 0 0 0 0 0 0 0

Wisconsin 7 1 7 7 4 2 0 3 0 0 0 0

Wyoming
Attachment to Chapter V

Part 5

Occupation: Truck, Bus, and Other Commercial Vehicle Operator

Participants Cost DurationStarting 

Wages

Completion RateEmployment RateRelated 

Field Rate

Employment Rate6 Month Retention Rate6 Month 

Wage Rate

License/Certificate Rate

Alabama

Alaska

Arizona 2 1 1 1 0 1 1 1 0 0 0 0

Arkansas 23 13 20 11 10 13 13 1 1 0 0 1

California 148 0 148 148 0 0 0 0 0 0 0 0

Colorado 17 9 17 17 9 9 9 0 6 6 7 6

Connecticut 9 0 9 9 0 0 0 0 0 0 0 0

Delaware 1 0 1 1 0 0 0 0 0 0 0 0

D.C. 0 0 0 0 0 0 0 0 0 0 0 0

Florida 47 34 2 36 19 33 33 31 0 0 0 0

Georgia 27 0 0 0 0 0 0 0 0 0 0 0

Hawaii 0 0 0 0 0 0 0 0 0 0 0 0

Idaho 1 0 1 0 0 0 0 0 0 0 0 0

Illinois 30 0 0 0 0 0 0 0 0 0 0 0

Indiana 11 0 11 0 8 0 0 0 0 0 0 0

Iowa

Kansas

Kentucky 36 0 36 36 30 27 26 0 0 0 0 0

Louisiana 21 0 0 0 0 0 0 0 0 0 0 0

Maine 18 7 0 0 0 0 0 0 0 0 0 0

Maryland 10 0 10 10 0 0 0 0 0 0 0 0

Massachusetts 20 12 19 20 10 12 12 0 0 0 0 0

Michigan 46 29 46 46 26 26 26 20 0 0 0 0

Minnesota 9 0 1 9 0 0 0 0 0 0 0 0

Missouri 17 2 16 15 0 1 0 0 0 0 0 0

Montana

Nebraska 8 0 8 8 0 0 0 0 0 0 0 0

Nevada 10 0 10 10 0 0 0 0 0 0 0 0

New Hampshire 6 0 0 0 0 0 0 0 0 0 0 0

New Jersey 61 0 61 61 0 0 0 0 0 0 0 0

New Mexico 23 15 23 23 15 15 15 9 9 9 9 9

Number Providing Information for WIA-funded 

Students On:

Number 

of Eligible 

Programs

Number Providing 

Information On:

Number Providing Information for

All Participants On:
Attachment to Chapter V

Part 5

Participants Cost DurationStarting 

Wages

Completion RateEmployment RateRelated 

Field Rate

Employment Rate6 Month Retention Rate6 Month 

Wage Rate

License/Certificate Rate

Number Providing Information for WIA-funded 

Students On:

Number 

of Eligible 

Programs

Number Providing 

Information On:

Number Providing Information for

All Participants On:

New York 453 0 453 453 0 0 0 0 0 0 0 0

North Carolina 1 0 0 0 1 1 1 0 0 0 0 0

North Dakota 2 0 2 2 0 0 0 0 0 0 0 0

Ohio 0 0 0 0 0 0 0 0 0 0 0 0

Oklahoma 17 0 17 17 14 14 14 0 0 0 0 0

Oregon

Pennsylvania 51 0 51 51 2 0 2 0 2 0 0 0

Puerto Rico

Rhode Island 3 2 3 3 1 2 2 0 2 2 2 2

South Carolina 12 0 12 12 11 11 10 10 3 2 3 3

South Dakota 4 0 4 0 0 0 0 0 0 0 0 0

Tennessee 19 0 19 19 0 0 0 0 0 0 0 0

Texas 159 115 159 159 95 30 30 20 20 15 4

Utah

Vermont 2 0 2 2 0 2 0 0 0 0 0 0

Virginia 5 0 5 5 0 0 0 0 0 0 0 0

Washington 12 8 8 9 7 8 0 0 0 0 0 0

West Virginia 19 18 19 19 18 17 18 0 0 0 0 0

Wisconsin 21 17 21 21 17 18 0 16 0 0 0 0

Wyoming
This page intentionally left blank.

Insert blank page here when making double-sided copies
VI-1

VI. VENDORS: 

INVOLVEMENT AND PERSPECTIVES

This chapter discusses the ITA system from the point of view of the diverse group 

of institutions that provide training to ITA holders. We use this diversity as a departure 

point to understand the characteristics of vendors available to any particular local 

demonstration site and examine how those vendors serve the needs of the ITA 

customers and the workforce development system. After describing the types of 

vendors available, we discuss vendor interactions with customers, focusing on several 

key factors affecting customer choice, including how vendors recruit students and 

structure their programs to accommodate student needs. We then discuss vendor 

responses to the eligible training provider requirements and to ITA administration. The 

chapter concludes with an analysis of vendor pricing and the general structure of the 

training marketplace.

Information in this chapter is based on interviews with representatives of 39 

eligible training vendors, and supplemented by our conversations with other 

respondents. Of the 39 vendors, 14 are public institutions (mostly community 

colleges), 21 are proprietary schools, 3 are community-based organizations, and 1 is an 

apprenticeship program.

VENDOR NETWORKS

The ITA system is intended to create a competitive marketplace for training to 

promote real choice for ITA customers. Two major dimensions reflecting the vigor of 

an effective marketplace are that 1) it supports a variety of types of vendors to suit the 

interests and choices from a diverse group of training customers; and 2) there are a 

sufficient number of vendors to provide realistic choice and stimulate competition.

Types of Vendors

As we noted in Chapter V, there are three principal types of training vendors on 

the eligible provider lists:

• Public institutions include both two- and four-year schools that are the 

largest providers of training to the ITA system. We use the term 

“public college” throughout this chapter to describe the publicly-funded 

institutions that educate and vocationally train students. 
VI-2

• Proprietary schools are typically privately-owned, for-profit, 

vocational schools. They are important providers, especially in urban 

areas.

• Community-based organizations are non-profit organizations that 

provide training to their communities, most often specializing in 

assisting low-income or minority individuals. These organizations 

typically grew out of, or together with, other organizations with a social 

service mission. Their role under WIA, however, appears smaller than 

it was under JTPA.

Although there are sub-groups within each of the broad types and hybrids that share 

characteristics of more than one type, each type is reasonably distinct, and the training 

vendors within each group have fairly similar responses to the ITA system. 

Public Institutions. The public institutions are ubiquitous and represent an 

important training resource in all the sites. As a group they were in nearly all the sites 

the most important providers of training under JTPA, and their continued participation 

in the ITA/eligible training provider system is considered by most respondents to be 

critical to the success of that system. As we noted in Chapter V, they provide the 

majority of eligible programs in most of the demonstration states, and nearly all of the 

programs in at least one of them. Training at the colleges generally has several 

common characteristics: 1) numerous degree and certificate programs; 2) adherence to 

a regular academic calendar, with courses beginning at only a few regularly scheduled 

dates; 3) large size, with enrollments running as high as 25,000 among our sample of 

colleges1; 4) large service area, although some maintain a network of satellite 

campuses; and 5) generally low tuition costs because of tax subsidies. 

Within the broad group of public institutions, however, are several important 

types of schools. The most important type for the ITA program is the community 

college, which is typically a comprehensive institution with diverse missions. These 

colleges provide a full range of associate degrees and the ability to transfer credits to a 

four-year college in addition to offering occupational training. For many colleges, their 

mission includes contributing to local economic development and providing education 

and training opportunities for second-chance workers such as the WIA adult and 

dislocated worker groups. But, given the breadth and diversity of the mission, WIAfunded students comprise a small part of their student body and ITAs represent a 

 

1

 As reported by respondents.
VI-3

miniscule portion of their revenue. In Indiana, for example, the public post-secondary 

training system has revenues of almost $1 billion, while the entire statewide 

commitment of ITA and other WIA training funds is only a few million dollars at most. 

Thus, even if all WIA training occurred through ITAs at public post-secondary 

institutions, the WIA training revenue would represent much less than 1% of the 

institutions’ total. Similarly, Macomb Community College estimated that WIA (and 

JTPA before it) contributes about 0.3% of its revenues. 

But, in some states, the community colleges may focus mostly on academic 

programs, with occupational training predominantly being carried on by the technical 

colleges. These schools also grant associate degrees, but additionally have many nondegree certificate programs that typically focus on the needs of the local economy. 

Demonstration states with extensive technical college networks include Georgia, North 

Carolina, Texas, and Indiana. For example, in both North Carolina and Georgia, all 

types of public colleges make available 90% of all eligible programs, but the technical 

colleges represent the more typical choice for ITA holders. 

A number of the demonstration sites also reported that public four-year colleges 

are important purveyors of training. In most cases these are non-degree programs since 

few of the demonstration sites had caps or durations that would permit an ITA holder to 

attain a four-year degree. The four-year schools do not typically have an explicit role 

in local economic development, and their continuing participation in the eligible 

training provider process in several demonstration states is somewhat tenuous, as we 

discuss later in this chapter. 

Proprietary Schools. In contrast to the public system, proprietary schools are 

for-profit institutions and, in most cases, their sole general mission is vocational 

training. They suggest that they are very market oriented and thus responsive to the 

needs of their customers, including ITA customers and local areas.

Also, in contrast to the comprehensive public schools, with their broad missions 

and large service areas, the proprietary vendors tend to serve more sharply defined 

market niches. For example, it is quite common for a school to serve a geographic 

sector of a labor market, because so many customers consider convenient location as a 

major decision factor in selecting a training vendor. Also by way of carving out a 

market niche, these schools also typically specialize in programs in specific areas, such 

as medical technology or information technology or truck driving. Others also focus on 
VI-4

particular customer groups. In the Atlanta Regional Commission local areas, for 

example, where there is a strong emphasis on computer training, some schools 

specialize in serving dislocated workers while another serves low-income adults with 

limited basic skills. The latter school offers a “boot camp” approach through which 

instructors help students with a range of basic, life, and labor market skills in addition 

to teaching them occupational skills. Other schools use different curricula and 

pedagogical methods designed to appeal to different types of students. For example, 

one computer school uses an intensive approach in the classroom and expects students 

to complete practice exercises on their own in the computer labs. Others offer a more 

structured classroom environment with greater reliance on having the instructors 

demonstrate techniques to solve relevant computer and network problems. 

There is some important variation among the proprietary group along the 

following dimensions:

• Basic organizational structure. Some schools are branches of large, 

national training companies or are locally franchised, while others are 

small and locally owned. 

• Dependence on WIA funding. Some schools are only minimally 

dependent on WIA funding. For example, one of the Atlanta-area 

computer training organizations has fewer than 10% of its students 

funded by workforce development programs (but still finds that 

individual referrals under WIA are a reasonably steady source of 

income). Others, such as another Atlanta-area computer training vendor 

and one in Philadelphia, are much more reliant on public funds. All 

recognize that the flow of students and funds under WIA is much less 

reliable than it was under JTPA, but most are confident that, if they 

provide a good program that is responsive to customer needs and have a 

good outcomes, they will continue to draw students. 

• Comprehensiveness. As noted above, most of the schools focus on a 

fairly narrow range of programs, usually in one occupational area. 

However, many larger local areas have comprehensive vocational 

schools like Triangle Tech in the Three Rivers local areas. That 

vocational school is long-established with a wide variety of programs in 

the manual trades to support construction and manufacturing. This 

school has credential programs and awards associate degrees in many of 

its programs. And, like a few other proprietary schools that we studied, 

it has linkages to degree-granting four-year colleges so students can earn 

bachelor’s degrees. 

• Size and capitalization. Some proprietary schools are well established 

and exist to serve major training markets. These are large, very stable 
VI-5

organizations that are well capitalized and thus less vulnerable to swings 

in enrollment because of competition under an ITA system. Others, 

however, are small, serve narrower groups, and are thinly capitalized. 

The latter, which often focus on low-income and minority 

communities—and thus come to resemble community-based 

organizations—are especially vulnerable to competitive pressures under 

WIA, because their customer flow is less predictable and they have 

fewer resources to purchase new equipment and keep up with the latest 

pedagogical techniques. 

• Entrepreneurial. All proprietary schools profess to be market driven, 

but some are much more entrepreneurial than others. An example of a 

very entrepreneurial school is one in Philadelphia that is training 

workers to respond to environmental problems, such as handling 

hazardous materials. This school is very aggressive in seeking 

customers and adding new occupations of interest to ITA holders. 

Other schools, although not aggressively adding programs, would still 

change a program’s schedule in response to strong demand from the 

local labor market and specific instructions from the local workforce 

board.

Community-Based Organizations. Community-based organizations are nonprofit organizations that provide training services as part of their mission to serve a 

particular community. 2 Typically, they are heavily, or entirely, dependent on public 

funds, whether from WIA or other programs, such as TANF. Many of these vendors 

provided training to economically disadvantaged adults under JTPA through contracts,

which tended to provide a stable flow of customers and income. While contractual 

training is still available as a limited exception to the use of ITAs, few communitybased organizations expect that local areas will use much contract training, so many of 

them are attempting to qualify as eligible training providers. 

However, the community-based organizations are among the most vulnerable of 

vendors to the ITA/eligible training provider system in two ways. First, their 

traditionally narrower customer base in particular low-income or minority communities 

makes them much more vulnerable in a competitive, ITA marketplace. For example, 

the Metropolitan Career Center in Philadelphia, which offers eight eligible programs, 

serves mostly low-income people with 75% of its funding from public sources including 

 

2

 We include in our classification the Detroit branch of the American Red Cross, a non-profit 

which is providing several medical programs, even though it is a part of a much larger organization that 

transcends both the training function and the local area.
VI-6

WIA. But it has seen a dramatic reduction in the number of students since JTPA, from 

50 per year to 2 in first 20 months of WIA, despite the fact that the Philadelphia 

workforce professionals are quite pleased with this vendor’s performance. Second, the 

community-based organizations are usually thinly capitalized and are likely to have 

difficulty coping with an irregular flow of ITA students. For example, Career 

Resources in Southwest Connecticut was providing a number of computer-related 

training, among other programs, since JTPA days. However, the school receives too 

few students from WIA to justify investment in new computer equipment to ensure that 

trainees will have marketable skills. Thus, this community-based vendor is finding it 

increasingly difficult to maintain parity with proprietary vendors and is likely to give up 

the computer programs altogether. As an alternative, the school is also examining the 

recruitment of more private-pay students.

Number of Vendors

Having a large enough group of vendors complements variety to make choice real 

and introduce competition into the training system. Chapter V clearly suggests that 

states have large numbers of eligible providers and programs, but the adequacy of the 

vendor network in local areas in another important issue for us to consider. In doing 

so, we found, as one might expect, that the number of training vendors generally is 

larger in the demonstration sites located in large metropolitan areas, so customers in 

these areas will generally have a greater degree of choice. Thus, not surprisingly, our 

urban local area respondents felt that they had a large enough network from which 

customers could choose. For example, at the time of the site visits, Baltimore reported 

70-80 vendors, each with an average of about 5 or 6 programs. Macomb-St. Clair and 

Indianapolis had 349 and 240 programs respectively. SELACO has a very large 

number of regional vendors and programs that were first approved by all the Southern 

California local areas as a result of regional collaboration under JTPA. By contrast, 

smaller states such as Nebraska and Oregon have only about 100 vendors each, 

statewide. 

Yet, even our respondents in rural areas and smaller urban areas considered their 

vendor lists to be sufficient to provide ITA customers with good quality training even 

though there were fewer choices. For example, Greater Nebraska staff, who serve the 

rural areas of the state, felt that even their small list was adequate. They felt that as 

long as they have multiple programs in the key areas of office skills, medical 
VI-7

technologies, and information technology—the programs that prepare students for the 

most common occupations—they can provide students with real choices. 

Even though most of our local-area respondents feel that their lists are large 

enough to support effective choice, they recognize the considerable risk that subsequent 

eligibility might reduce the vendor network’s adequacy. In fact, as we discussed in 

Chapter IV, some of our respondents predicted, or feared, that subsequent eligibility 

might reduce the number of eligible programs by one-half, or more.

Predictions notwithstanding, the only evidence from a full cycle of subsequent 

eligibility comes from Texas, and its results are mixed. The state had 8,000 programs 

statewide for initial eligibility, but that number fell substantially when the state first 

instituted subsequent eligibility, due to a sharp drop-off in the number of public 

institutions that applied for eligibility. However, state and local workforce 

development and education officials were successful in reversing the decline. In 2001, 

at the time of the second-round site visit, state officials indicated that they regained 

their earlier level of 8,000 programs through substantial efforts.

Regardless of the future impact of subsequent eligibility on the number of vendors 

and programs, there is some potential for increases through economic growth and 

through active steps to increase the numbers of programs. First, some states and local 

areas are reporting some growth in the number of vendors and programs from PY 2000 

to PY 2001. For example, both the Detroit and Atlanta sites indicated that there were 

small increases in the number of eligible vendors.3 However, they noted that all of this 

growth was located in the suburban areas that were growing anyway in labor force and 

population. Southwest Connecticut also reported an increase in the number of its 

statewide vendors. 

Second, some local areas are actively trying to expand their vendor base through 

innovative methods to tap new sources or traditional methods of outreach and 

persuasion to extend the reach of existing vendors. The Lancaster County local area in 

Pennsylvania, for example, is using demonstration grant funds to increase the scope of 

 

3

 Detroit officials found that growth in the number of vendors presented some risk at the same 

time that it might increase choice. Local-area officials there felt that they had already identified and 

selected all the good vendors in the area, and these are certainly sufficient to provide their customers 

with choice. Adding vendors, they contend, could only dilute the overall quality of the existing list and 

increase the risk of customers selecting poor-quality vendors. 
VI-8

its vendor group by linking the ITA to the training resources of local employers, as 

follows:

This local area is working to include training provided by the area’s 

businesses for its own employees as part of the ETP system, with the 

expectation that the training would thereby become accessible to ITA 

holders not employed by the firm. Such an arrangement is believed to be 

beneficial to employers, as it enables them to share their training resources, 

to some degree defray the costs of their own in-house training, and 

participate as part of a larger training marketplace. At the same time, it 

benefits the WIA system by increasing the training choices for ITA and 

other WIA customers. Thus, Lancaster expects that this innovation will 

create a more efficient local market for training. However, it is still 

working on methods to measure performance for the employer-based, inhouse training.4

Elsewhere, Georgia’s ITA coordinator is trying to increase the supply of vendors 

in rural areas by having Atlanta-area vendors expand their reach by setting up programs 

in the outer ring of suburbs, and he is trying to persuade some out-of-state distancelearning vendors to bring their programs to Georgia’s list. Beyond these especially 

aggressive outreach efforts, all local areas and states engaged in some level of 

marketing to vendors to get them to apply for eligibility, as we discussed in more detail 

in Chapter IV.

VENDORS’ RESPONSES TO CUSTOMERS

A broad goal of the ITA system is to promote training programs and vendors that 

are responsive to the needs of customers. Thus, vendors need to be cognizant of their 

customers, through their efforts at recruiting through to the design and structuring of 

their programs.

Recruiting 

Some training vendors, especially those who may be heavily dependent on ITA 

customers, are not passive players in the ITA process, simply waiting for customers to 

choose them from the eligible provider list. Instead, they may try to actively market 

their services to customers and staff within the One-Stop system.

 

4

 We have recently learned that this effort may be abandoned, as a new state law is requiring that 

all training vendors must be state certified.
VI-9

Perhaps the most common technique that vendors use to recruit potential ITA 

holders is the reverse referral, whereby customers come into the One-Stop center 

seeking an ITA because they were referred there directly by a training vendor. This 

practice was common under JTPA, when most customers were slated to receive 

training. In this model, a vendor would recruit an individual who was interested in 

training, through advertising, word-of-mouth, or other marketing techniques.5 The 

vendor then referred the person to the JTPA program with the expectation that the 

individual would be determined eligible for JTPA and sent back to the school for 

training.

The ITA process under WIA introduces considerable uncertainty into this 

practice. If an individual were referred back to a One-Stop center, that person would 

first have to progress through core and intensive services and be determined eligible for 

training, a less likely prospect in some areas than under JTPA. Second, the individuals 

would go through the ITA decision-making process, during which they might choose 

another vendor from the eligible provider list. To address the risks of low refer-back 

rates, some vendors expressed interest in placing a staff member at the One-Stop center 

to help shepherd their reverse-referred people through the ITA process, but center 

administrators will not usually allow this. 

While respondents in one local area think that these uncertainties in whether 

vendors will get back their referred customers will tend to diminish the use of reverse 

referral, we found strong evidence that the practice is flourishing in many of the sites. 

For example, staff in Greater Omaha suggested that the majority of all its ITA holders 

come into the system through reverse referrals. Similarly, staff at the Capital Area 

indicated that reverse referrals were widespread, although they are trying to control the 

practice through staff guidance. Vendors in other local areas also admitted that most of 

their WIA students were enrolled through reverse referrals. In Detroit, for example, 

the program director at a private four-year college that operates a proprietary computertraining program stated that most of its WIA students come in through reverse referral 

(he estimates that the school gets about 70% of its referrals back, compared to 90% 

under JTPA). One of Philadelphia’s largest proprietary vendors also admits to using 

reverse referrals (though with a disclaimer to the customer that issuance of an ITA 

 

5

 In some areas, vendors held outreach contracts from the Service Delivery Areas to find 

customers, determine their eligibility, and enroll them in their own training services.
VI-10

cannot be guaranteed), even though the practice is frowned on by the Philadelphia 

workforce administrator. 

There is also some use of reverse referrals by community colleges in two sites. A 

low-income customer who was attending courses at Baltimore City College got a 

recommendation from a counselor to obtain ITA support for her program. In Northeast 

Georgia, the training program is set up to encourage reverse referrals, because the local 

community college holds a contract from the local area to support WIA-sponsored 

trainees.

Most vendors defend reverse referrals as a necessity and some local area officials 

accept it. There may even be benefits to the workforce system, as one Indianapolis 

local area official contends that reverse referrals provide free marketing for a public 

program that is not widely known among some segments of the workforce. On the 

other hand, we did hear of clear abuses, such as one vendor that enrolled customers and 

charged their personal credit cards for tuition, while assuring them that they would be 

reimbursed once they went to the One-Stop center to get an ITA. Needless to say, the 

affected local area took strong action.

Vendors, especially proprietary vendors, also rely on other marketing techniques 

besides reverse referrals, such as advertising. But the use of the eligible provider list 

seems to have different effects on the extent of advertising in different places. In 

Georgia, a truck driving school was able to reduce the size of its advertisement in the 

Yellow Pages phone books throughout the state, a very large savings, because it was 

able to get a reasonable flow of students through the ITA system. Indianapolis localarea staff expected their vendors to ease spending on advertising also. On the other 

hand, the state staff in Michigan and local staff in Detroit both expected to see vendors 

increase their advertising to make up for the increased risks of reverse referrals, and 

some community-based organizations are desperately exploring advertising to attract 

ITA customers, as a way of making up for the customers they have lost from the 

general abandonment of contract training.

Some vendors have also established an on-site presence at the One-Stop centers, 

though not for direct client marketing. At least two local sites invite vendors into their 

One-Stop centers to provide One-Stop customers in core and intensive services with 

labor market information about the occupations for which they provide training. In one 

other site, training vendors also provide direct services such as career exploration or 
VI-11

job search workshops. Some center staff, however, find that these workshops are often 

thinly disguised recruiting efforts and they make efforts to ensure that the content 

remains balanced. Elsewhere, the Atlanta Regional local areas invite vendors to their 

centers semi-annually to make formal presentations to the counseling staff about their 

services.

Structuring Programs 

Vendors can structure their programs to make sure that they are responsive to 

customers’ needs in several ways. Most importantly, they can offer successful 

programs that prepare ITA holders for good quality jobs and work to ensure that their 

trainees complete programs and get placed at high wages. Subsequent eligibility is 

intended in the future to organize and validate that performance. In the meantime 

vendors may respond to customers’ needs by improving accessibility, adjusting the 

timing, location, or duration of programs, or adding auxiliary services. All these are 

very important factors in the ITA decision-making process for customers.

The timing of programs (e.g., when courses are scheduled) plays an important 

role in responding to customer needs and has great immediacy to prospective trainees. 

Proprietary schools are widely considered to be quite responsive in this regard. Some 

schools, especially those providing computer training which is in great demand at all 

sites, offer open entry so that a student can begin the training at almost any point. One 

such school in the Atlanta Regional local areas, for example, accepts students at any 

time, although it suggests to prospective students that the best starting times are on 

Mondays and Wednesdays. Although it is less common, community colleges can also 

be flexible. A community college in the Macomb-St. Clair area, for example, 

increased the frequency with which it offers its courses, from three times per year to 

five times per year, to be more responsive to its prospective ITA students.

A short duration of the training period is another dimension that is important to 

students. Most students are unemployed and thus eager to improve their skills and 

return to the labor market as soon as possible. In addition, those who are receiving 

welfare, unemployment insurance, trade readjustment allowances, or another form of 

temporary income support have a strong incentive to conclude their training before the 

income support expires. Further, in tight labor markets, trainees are eager to complete 

training as soon as possible to take advantage of favorable employment conditions. 

Several respondents noted that proprietary schools often provide shorter, more intensive 

programs than their community college counterparts. Nevertheless, we found at least 
VI-12

one community college in Metro Portland that is working with the One-Stop center to 

develop shorter-term training to meet the needs of ITA holders. 

The wider availability of auxiliary services for ITA holders, such as counseling, 

study skills workshops, and placement assistance, which we discussed in Chapter III, is 

another feature that distinguishes proprietary schools from community colleges. The 

proprietary schools commonly deliver such services directly to most of their students as 

a matter of course. Although some community colleges do so as well, in general we 

found that students typically must seek out those services on their own to take 

advantage of them.

VENDORS’ RESPONSES TO THE ITA SYSTEM

In the ways discussed above, the ITA system can be said to enhance relationship 

between the vendor and the ITA holder. At the same time, vendors still have a direct 

relationship with the workforce development system through the submission of an 

application for eligibility and the submission of performance data for subsequent 

eligibility. In addition, they must maintain ongoing administrative relationships for 

payment or because the local areas may have other requirements for their vendors. In 

this section, we discuss this relationship from the vendor’s perspective.

Application Process

Most vendors felt that the application process to get on the initial eligible provider 

list was fair and reasonable, regardless of the length of the application or the method of 

filing the applications. Nevertheless, a few vendors found the application process to be 

burdensome. For example, some vendors in Texas cited three elements of the state’s 

application process that were found to be burdensome: 1) an unusually lengthy 

application form that, at the beginning of WIA implementation, was entirely on paper; 

2) unlike some states, a requirement that vendors submit a separate application for the 

same program delivered in each of multiple sites; and 3) additional information or 

requirements imposed by individual local areas beyond the standard items required by 

the state. This substantial paperwork burden may account for the initially strenuous 

reaction of Texas vendors to the eligible training provider system. 

In response to the new requirements and expected paperwork burden, most of the 

states and local areas took several steps to ease those burdens. First, most of the states 

made the application process available electronically either on a web site or through 

email. Second, the states and local areas held information sessions to explain the new 
VI-13

system to vendors. States also provided help through on-line help screens, email or 

toll-free telephone service, and individualized assistance to vendors. For example, 

Pennsylvania’s local areas designated a single point of contact for all ITA matters, 

including vendor eligibility. 

Subsequent Eligibility

The burden will increase under subsequent eligibility, when performance 

information is required. Reactions of the public colleges to this requirement are 

generally quite different from those of the proprietary schools. We found the former 

group generally quite distressed with the idea that they would need to provide 

performance data, while the latter group was much less so. For example, in SELACO, 

the local community college did not want to apply for initial eligibility because of the 

performance requirements, even though the school did not have to submit performance 

information for that first round. In Texas, the number of programs dropped statewide 

by about 80% from initial eligibility, when no performance information was required, 

to the first year of subsequent eligibility. Nearly this entire drop came as a result of the 

public colleges’ refusing to participate. 

Three types of objections were raised by the public colleges in Texas and other 

states:

• Performance measures would unfairly rate the performance of colleges 

with open enrollment policies.

• Performance measurement is inconsistent with their educational mission.

• The costs of data collection far exceed the benefits of performance 

measurement.

First, several colleges indicated that their open admission policies would unfairly 

characterize their performance compared to proprietary vendors with more selective 

admission policies. For example, in Baltimore, state law requires the Baltimore City 

Community College to admit all applicants regardless of their preparation. 

Consequently, the school experiences a 50% dropout rate. Similarly, the University of 

Cincinnati, a hybrid two- and four-year school, also has an open admissions policy and 

a high dropout rate. Under these circumstances, these institutions felt that any reported 

low performance might well be caused by student characteristics rather than by the 

quality of their programs. However, both of these colleges decided to participate in the 

eligible training provider system regardless of any future disparate impact on their 
VI-14

programs, because the schools believe they have an obligation to train second-chance 

students through WIA and participate in local economic development. 

Second, some public colleges contend that performance measurement focused on 

employment outcomes, regardless of its cost, accuracy, or any disparate impact, is 

inconsistent with their educational role. This argument was advanced by some 

comprehensive community colleges that view their primary mission as providing 

education, often leading to an associates degree or preparing students to transfer to 

four-year institutions, rather than providing job training. As such, they feel that a focus 

on employment outcomes could distort their basic educational mission. They also point 

to the diversity of students who enroll, many of whom do so without employment 

objectives in mind. Nevertheless, each of the colleges in our sample that advanced this 

argument also ultimately decided to participate in the system, also because of their

obligation to train WIA students and participate in local economic development.

The argument for educational purity is also used with even greater force by fouryear colleges and universities, and they reached a different result. Four-year colleges 

were active training providers in Pennsylvania, Indiana, and Nebraska under JTPA 

through satellite campuses. But they decided not to participate in the eligible training 

provider system under subsequent eligibility. These schools are more remote from the 

local communities and tend to have little or no explicit responsibility for local economic 

development, even if they provide substantial amounts of local worker training and 

their education and research mission may be considered critical to the state’s economy. 

The third objection they raise is about the cost of complying, and this is the one 

that community college officials, with very limited resources, feel is most important. 

Nearly all public institutions pointed out that the costs of obtaining the performance 

information required of them by WIA is too high for the relatively few customers they 

might receive through ITAs. For example, in the first-round site visit, a representative 

of the Texas State Technical College in Waco (Heart of Texas local area) noted that the 

performance requirement was a “bureaucratic nightmare” that required her to spend 

substantial amounts of time filling out forms obtaining the data for 40 programs at 3 

campuses. This activity took her away from her regular duties, which include grantwriting, so she felt that working on subsequent eligibility cost the college even more 

money. These costs were viewed as unacceptable even more because WIA students and 

funds represent such a small part of their college’s enrollment and revenue. For 

example, St. Louis Community College, with 25,000 students, has only 57 ITA 
VI-15

holders. The college’s planning director argued that it would take 5,000 ITAs to justify 

the cost of gathering the data. 

However, it is turning out that the direst predictions that public colleges would 

abandon the ITA system wholesale are not occurring, at least at the beginning of 

subsequent eligibility. Although all the public colleges acknowledged that the cost of 

compliance is excessive, a number of colleges in the local areas we visited indicated 

that they would participate despite the burden and cost. In general, respondents from 

these colleges felt an obligation to support local training and economic development, 

and they indicated that the benefit to the community of providing this training 

outweighed the college’s own cost. 

In some local areas, persuasion from the Local Board or local elected officials 

helped in getting community colleges to participate in the eligible training provider 

system. For example, in Michigan, most local boards were quite effective in 

persuading the community colleges (including three hybrid two- and four-year schools) 

to participate, based on their long-standing connections with the schools, some of 

whose representatives serve on the Board or Board-advisory panel. The only areas in 

Michigan in which local persuasion was not successful are the ones in which the 

community colleges are not important vendors. We found a similar example of local 

persuasion in the Three Rivers and SELACO local areas. There, local area staff 

reported that their community colleges are also reluctant to provide performance 

information. But these schools are locally controlled, and the workforce development 

staff believe that the elected city and county leaders will exert sufficient political 

pressure on the elected trustees who govern the community colleges to get the colleges 

to participate.

In some cases, state intervention also was key, in two major ways: 1) bringing to 

bear the power of persuasion, and 2) agreeing to take over the data collection and 

measurement function, thus relieving the local schools of major compliance duties. The 

clearest example of the importance of persuasion was found in Texas, where state-level 

WIA officials worked extensively with the state’s higher education coordinating board 

to persuade these public institutions to participate. Beyond this, however, states could 

substantially ease the vendors’ burden in tangible ways by taking on data collection 

functions. As we discussed in the previous chapter, nine states have taken some steps 

to centralize the performance measurement function by using either state-level college 

administrative data for program completion rates or Unemployment Insurance wage 
VI-16

records to measure entered employment rates, or both. For example, Georgia early on 

worked out an agreement with the governing agencies of its two postsecondary systems 

for those agencies to handle all the measurement issues through a cooperative 

agreement between the education and workforce agencies, and, partly for this reason, 

has substantial participation rates by community colleges in the eligibility system. By 

contrast, Indiana has no such state-level role and also has substantially lower 

participation by their community colleges.

On the other hand, although there were exceptions, proprietary and communitybased institutions expressed many fewer reservations about supplying performance 

information. Those institutions that are heavily dependent on public funds were 

tracking performance information under JTPA and other programs, so that compliance 

with ITA requirements imposes no additional burden. Other proprietary schools that 

are less dependent on workforce development funds also indicated that they are already 

tracking performance for eligibility for Pell grants, accreditation, or state licensing, or 

because they are strongly market-driven. Others felt that the business from ITA was 

worth the cost of tracking performance. For example, in the Atlanta Regional local 

areas, the New Horizons computer school, a unit of the world’s largest independent 

computer-training school, pursues a corporate strategy to include ITA and other 

publicly-funded students. School officials believe that JTPA was a good source of 

revenue and the students who were referred to the school performed very well. They 

expect similar results under the ITA system.

Administrative Relationships with Local Areas

Although their primary relationship is with the ITA holder, vendors can also 

benefit by maintaining a favorable relationship with the local areas. This is so for 

several reasons. To begin with, vendors want to make themselves known to case 

managers, whom they recognize as having an important influence in customers’ 

decision-making. Indeed, a number of local areas acknowledged that they limit the 

number of ITAs that they will permit with vendors with whom they are unfamiliar in 

order to “test them out” and ensure that they are producing high quality outcomes for 

WIA students. Thus, vendors appreciate the opportunity to meet with center staff, to 

inform them about the training they offer and establish a personal relationship.

Second, local areas and vendors must establish on-going administrative 

relationships to assure a smooth payment process. Typically, each WIA grant recipient 

(the political subdivision) creates a master, non-financial agreement with each eligible 
VI-17

vendor, to which a specific financial addendum is added for each ITA holder. This was 

the same financial process that most areas used for individual referral under JTPA. 

RESHAPING THE TRAINING MARKETPLACE FROM JTPA

It is evident that Congress intended to affect a thorough reshaping of the way 

training is conducted in the public workforce development system. We examine how 

two elements of that re-shaping process affect vendors: market structure and 

competition and tuition cost.

Market Structure and Competition

The prototypical ITA system seeks to restructure the training marketplace in two 

major ways. To begin with, it shifts the vendor’s fundamental market relationship from 

the workforce agency to the individual customer. Under JTPA, agencies were required 

to procure training services through competitive, arms-length transactions, although 

there was often a close and ongoing relationship between agencies and vendors. By 

contrast, under an ITA system, the local area functions largely as a neutral broker, 

qualifying credentials from vendors who then establish a customer relationship directly 

with the end customer. Second, the ITA system seeks to dramatically increase the 

number of vendors from which customers can choose, creating a viable competitive 

marketplace that should drive the system as a whole to better performance outcomes 

and help to reduce costs. 

The record thus far in achieving these broad goals is mixed. The structural 

transition in the customer relationship has occurred, although to some extent this 

already had occurred under JTPA, as most of the demonstration sites had already 

shifted from class-size contracts to individual referrals from an approved list of 

vendors. Nonetheless, the evidence we have presented thus far clearly speaks to 

vendors’ efforts to be responsive to their customers’ needs.

Regarding increased competition and expansion of the market for WIA-funded 

trainees, effects appear to be mixed, although it is too soon to make definitive 

judgements. On the one hand, the ETP list gives customers a wide array of programs 

from which to choose, and thus may expand the number of vendors with which local 

areas have ongoing relationships. On the other hand, a likely adverse effect will be on 

the small, thinly capitalized community-based organizations that were heavily 

dependent on contract training under JTPA who may not be able to survive in an ITA 

environment. Because of their expertise in working with special populations, their 
VI-18

demise could potentially represent an important loss to the workforce system. 

Additionally, to the extent that some public institutions and other vendors do not seek 

subsequent eligibility, customer choice could be sharply curtailed in some local areas. 

Tuition Costs

Although competitive markets are normally supposed to affect costs, the 

ITA/eligible training provider system seems to have had little impact either on the price 

that individual vendors charge or on narrowing prices between vendors, including the 

very wide price differences that generally exist between proprietary and public vendors. 

Several respondents indicated that their community colleges had always charged much 

lower tuition than the proprietary schools, and the differential remains under the ITA 

system. There are several reasons for this outcome. First, the structure of the market 

is such that ITA holders have no incentive to bring down the price.6 Second, although 

it is possible for local areas to set restrictive caps, we found that most local areas set 

their caps at the very high end, generally to control excessive costs of both vendors and 

customers rather than to influence the prices of the larger number of vendors that would 

cluster around the mid-point (See Chapter III). Only four sites that used mid-range 

caps indicated that some vendors had to reduce their tuition because of the ITA cap.7 A 

few other sites found, to the contrary, that several low-priced vendors took advantage 

of the cap to bring their prices up to that level. 

Only one area was explicitly hoping to reduce the cost of training in the universal 

marketplace through its demonstration grant, but this effort sputtered. Early in the 

grant period, the Three Rivers Board was using grant funds in an effort to increase 

competition among training vendors and drive costs down. Specifically, the Board 

planned to support a for-profit, Internet start-up company that was trying to create a 

 

6

 The only ITA holders who do have an incentive to look for the best pricing are those in the 

handful of local areas that include supportive services within the ITA amount. If the sum of tuition and 

supportive services exceeds the cap, it may be in the trainee’s interest to seek a vendor with a lower 

tuition price.

7

 One of the vendors that did reduce its tuition price indicated that its catalog price of $6,000 is 

somewhat inflated because of the lending system that finances its private pay students. This truck driving 

school arranges for third-party loans for most private-pay students to finance their training. The student 

borrows $6,000, of which the school gets $5,000, the same price that it charges the local area. The 

third-party lender retains the difference as its income and then sells the loans to yet another party, which 

then aggregates loans and securitizes them. This is essentially the same system that underlies other 

consumer and mortgage loans. 
VI-19

bidding process on a web site so that public and private sector organizations seeking 

training services for their incumbent employees could aggregate their requests to help 

reduce the average cost of training. For example, a One-Stop center might need 

computer network training paid for by an ITA at the same time that a small business 

needs a similar course for one of its employees. The web site would bring the two 

requests together and seek a bid from a training provider that they expect would be 

lower than either organization could obtain on its own. However, this effort fell victim 

to the collapse of the Internet bubble and has been abandoned.

SUMMARY

Congress intended that the ITA system would substantially revamp the training 

system and create a new relationship between vendors and the workforce development 

system and its customers. The new system should be based on a marketplace model, 

albeit with considerable regulatory content, because of eligibility requirements for 

customer access and policy limits that the local areas could impose on ITAs. However, 

our review of the impact of this system on vendors and the vendor response suggests 

that there have been relatively few changes that have occurred. Some of the anticipated 

changes already had occurred under JTPA and others await full implementation of the 

system.
This page intentionally left blank.

Insert blank page here when making double-sided copies
VII-1

VII. SUMMARY OBSERVATIONS AND CONCLUSIONS

The sorts of ITA and ETP systems called for in the WIA legislation clearly 

represent important innovations that, just as clearly, will take substantial time to reach 

maturity and be properly assessed. What is certain at this juncture is that states and 

local areas around the country have been engaged in a tremendous mobilization of 

effort to revamp their workforce development systems that is truly impressive in its 

scale and scope. 

The experiences of the ITA/ETP Demonstration grantees reflect this progress. 

The evaluation’s two rounds of data collection span a period of tremendous growth and 

maturation of their ITA and ETP systems. During the first round of site visits, in the 

summer and fall of 2000, ITA policies were just being formulated, as sites were still 

feeling their way and not quite sure to whom they could issue an ITA and when. 

Similarly, although ETP lists had been developed, states had barely begun to think 

about their subsequent eligibility policies, and fully functioning Consumer Report 

Systems were virtually nonexistent. 

A little more than a year later, by contrast, during our second round of site visits, 

these systems had evolved considerably. Thus, sites were much more comfortable in 

authorizing training for their customers, the groundwork for subsequent eligibility had 

been laid (at least in most states), and virtually every state had a searchable, web-based 

Consumer Report System with at least basic information about vendors’ programs.

This system development was carried out in ways that seemed clearly consistent 

with the key tenets of the WIA legislation. Thus:

• Local areas displayed a high degree of attention to customers’ unique 

needs, and an emphasis on allowing wide latitude for customer choice 

seemed paramount. 

• Efforts to improve accountability were evident in the performance 

requirements that states imposed for subsequent eligibility and their 

efforts to gather and display performance information about vendors in 

Consumer Report Systems that were accessible and user-friendly to 

customers.

• States and local areas took advantage of the flexibility offered by WIA 

by designing unique ITA and ETP systems that they felt best met their 

communities’ and customers’ needs.
VII-2

Based on our review of this remarkable trajectory, we can offer some final 

thoughts about remaining questions and potential next steps. We have divided these 

into issues relating to grant (and other) funding, ITA issues, and ETP and CRS issues.

Grant and Funding Issues

1. The development of ITA and ETP systems in the demonstration states 

represented a tremendous mobilization of effort and resources—for example, 

in reviewing and processing vendors’ applications for eligibility, establishing 

procedures to measure vendor performance and track ITA expenditures, and 

developing software to display Consumer Report Systems. For this reason, 

the extra resources represented by the demonstration grant funds were, if not 

indispensable, certainly very appreciated and were put to good use 

everywhere. 

2. The grantees endeavored to design their systems such that they would be 

sustainable once grant funds ran out, with minimal support from WIA 

formula allocations. The extent to which they were successful in doing so, 

however, remains unclear. Thus, some investigation of the ongoing costs 

associated with maintaining these systems would be helpful, so that an 

assessment of how costs compare to expected benefits can begin.

3. For the very reason that grantees used much of their funding for critical 

system development, the timing of the awards made this less like a 

demonstration project—in the sense of testing departures from standard 

practice—than it otherwise would have been. Nonetheless, some grantees 

were developing some innovative features of their ITA/ETP systems, such as 

experimenting with alternative ITA payment mechanisms, online customer 

messaging systems, and the like. Few of these were fully functioning at the 

time the demonstration ended, and thus none could be properly assessed. 

Nonetheless, some of them seem to hold substantial promise, and their 

efficacy will be worth exploring further.

4. State and local-area flexibility is a key tenet of WIA, and this principle 

seemed to be clearly on display in the systems that we studied. Thus, 

although there are obvious broad similarities in the systems that are 

developing, states and local areas are making unique decisions regarding key 

features of ITA policies and ETP systems.

5. That flexibility, although clearly embraced by states and local areas as a good 

thing, caused some confusion and uncertainty at early stages of system 

development. Thus, some local areas were unsure exactly what their policies 

and systems should look like, and were eager to compare their experiences 

with those of other states and local areas. Although much of this uncertainty 

has since been resolved, states and local areas are likely to highly value 

forums for the continued exchange of information across the workforce 

development system.
VII-3

ITA Issues

6. The demonstration grantees, just as the workforce system as a whole, 

realized a substantial drop-off in the number of persons undertaking training 

in PY 2000, WIA’s first year of implementation. This occurred for a variety 

of reasons, including a strong economy that made job opportunities plentiful, 

competing priorities for using scarce WIA funds, and case managers’ 

uncertainty regarding for whom training could and should be authorized. 

However, training levels rebounded sharply upward subsequently, reflective 

of the changing economic climate, one-time expenses in establishing a OneStop infrastructure that were behind them, and local areas’ greater degree of 

comfort in authorizing training in response to customers’ needs.

7. Most grantees have strongly embraced ITAs as the preferred method of 

delivering training services to adults and dislocated workers under WIA. 

Some, however, point to the strong role that customized training can still 

play in meeting the workforce system’s diverse needs (e.g., in directly 

meeting an area’s needs for economic development targeted to employers, 

and providing trainees with an income to support themselves through training 

and a nearly guaranteed job afterwards). Similarly, other grantees noted the 

important role that contract training plays in meeting the needs of their hardto-serve customers, for whom an ITA might be less effective. Yet neither of 

these alternatives to the ITA was used very widely in the local sites we 

visited. It is perhaps worth considering in a broad way the proper role for 

customized and contract training in the WIA context, and how these 

alternatives can be developed while still promoting customer choice.

8. As is their prerogative, local areas have established caps on the amount of 

the ITAs they would issue. These vary widely from a low of about $1,700 

per participant in some local areas to as much as $10,000 in others. In 

setting these caps, local areas struggle with the tradeoff between ensuring 

that a diverse array of high quality training choices are available to 

customers, while spreading their available training dollars over as many 

customers as possible. A quantitative analysis of how these different 

decisions impact the choices that customers make, and how the workforce 

system’s return on investment is impacted, would be very informative.

9. In keeping with one of WIA’s major principles, sites maintained a strong 

customer focus in the way they approach training and pre-training services. 

Thus, although sites have guidelines for how customers should move through 

the service levels, it was apparent that those guidelines were not meant to be 

followed rigidly and that customers’ obvious needs were taking precedence.

10. Similarly, customer choice is clearly apparent in the way that sites are 

working with customers to help them select training programs and vendors. 

This choice is structured within a framework that requires that customers 

undertake a careful assessment of their skills and abilities and conduct labor 
VII-4

market and other research. To this degree, customers are making choices 

only after being exposed to a range of good information. Notwithstanding 

this fact, although all local areas make some provision for structured decision 

making, in actuality the degree of assessment and counseling that customers 

are likely to undergo varies greatly from area to area. Further, there are 

wide disparities in the way in which the local areas respond to the very large 

percentage of customers who come into the system with already established 

training choices. Thus, additional information on optimal approaches for 

dealing with customers of different types would be helpful.

11. Front-line staff are generally playing roles that support informed customer 

choice. In most local areas we visited, case managers were playing the role 

of “facilitators,” and were lending the benefit of their expertise without being 

overly directive. To this degree, customer empowerment was being 

promoted. At the same time, the exacting role that case managers are 

expected to play by providing assessment and counseling to a diverse mix of 

customers, while still respecting customer choice, places great demands on 

their expertise. For this reason, opportunities for capacity building among 

front-line staff must remain a high system priority.

12. The quality of the choices that customers are ultimately making is unknown. 

Nor is it known whether customers would benefit from more or less 

structured progressions through pre-training services and greater guidance 

and direction from their case managers. The ITA Experiment, being 

undertaken by Mathematica Policy Research, should help resolve these 

important questions. This experiment is using experimental methods to test 

three alternative procedures for advising customers about their training 

choices:

• Structured Customer Choice: The counseling provided under this 

approach is intensive and mandatory. Case managers are expected to 

steer customers to training programs with the highest net benefit.

• Guided Customer Choice: This approach is similar to the one most in 

evidence in the sites we visited as part of the ITA/ETP Demonstration, 

in that case managers guide customers without being overly directive.

• Maximum Customer Choice: Under this regimen, customers are not 

required to undertake any research or counseling as aids in making 

training choices (though counseling is made available to them if they 

request it), and they are free to select any program on the state’s ETP 

list.

ETP and CRS Issues

13. Training providers play an obvious and critical role in the workforce 

development system. The demonstration states clearly recognized this, and 

thus most made every effort to recruit them to the eligible training provider 
VII-5

list and ease the burden they bear in applying for eligibility. Their strategies 

for doing so included automating the vendor application process and 

conducting data capture on their behalf for performance information 

associated with the Consumer Report System. Such efforts remain critical if 

a rich array of training choices is to remain available to ITA holders. These 

efforts are especially important to ensure the continued engagement of 

community colleges, which are at once important providers of low-cost highquality training for WIA customers, yet who often view the ETP 

requirements as very burdensome. 

14. Also as a way of easing the burden for training vendors, efforts should be 

explored to align as much as possible the ETP eligibility requirements to 

those of other licensing bodies to which training vendors are subject. These 

efforts might include adopting similar definitions of common terms and 

building on existing reporting and performance requirements where possible. 

15. At this point, little is known about how states should best establish 

performance levels for vendors in the eligible provider system that promote 

high quality without undermining customer choice unduly by curtailing the 

size of the ETP list. The demonstration states have largely been making 

their decisions in an information vacuum, but they clearly display a broad 

range of approaches to this issue. Understanding how very different 

performance thresholds affect the size and composition of the eligible 

provider lists and how vendors react to the different requirements constitute 

high priorities for research once subsequent eligibility takes hold.

16. The underdeveloped state of most Consumer Report Systems has meant that 

concrete and comparable information on vendor performance has not been 

one of the sources that most customers have been using in making their 

training decisions. When these systems are mature, additional information 

on how customers use them and how they can be configured to best meet 

customers’ needs would be desirable. 

17. Along these same lines, to ensure that performance data for different vendors 

can be interpreted meaningfully, ways must be explored for systematically 

considering the characteristics of customers that vendors serve and of their 

local labor markets, both at the stage of reviewing vendors for eligibility to 

the ETP list and helping customers interpret performance data in making 

their training choices. Although WIA clearly allows for this, we found little 

evidence that such efforts are thus far occurring in a systematic way.

18. Similarly, states vary in the way they define key terms related to 

performance measurement (e.g., participant, completer) and in the clarity 

and completeness of the guidance they give their vendors about them. This 

variability will strongly impair the comparability of performance data once it 

becomes available in Consumer Report Systems, even within individual 

states. DOL and the states must confront the dilemma of needing to impose 
VII-6

greater uniformity to promote comparability, without being unduly 

prescriptive. Without such comparability, it remains to be seen how useful 

the performance data in the Consumer Report Systems will be or how they 

can best be used to help guide customers’ choices.

19. The ITA/ETP process is presenting substantial challenges to training vendors 

that relied heavily on workforce development funding under JTPA. Reliant 

for so long on contract training for serving special populations, these 

organizations are finding that their customer flow has been gravely 

interrupted since the enactment of WIA. This challenge has fallen especially 

heavily on community-based organizations. Without an alternative customer 

base, many of these institutions are facing insolvency. To the extent they 

provide an array of valued social services, their absence can represent a 

substantial loss to the high-poverty communities they serve and could reduce 

the number of appropriate training options for customers with special needs. 

Notwithstanding these remaining research issues and concerns, the ITA/ETP 

Demonstration project has been very valuable in shedding light on key issues related to 

ITA and ETP implementation. Moreover, whatever problems they have encountered or 

challenges that remain, the demonstration grantees have all made important 

contributions to our knowledge of these systems. They have demonstrated as well the 

extraordinarily mobilization of effort that has been entailed in developing their systems. 

Indeed, their progress in building their systems in the period encompassed by our site 

visits has been truly remarkable.