32564 Federal Register / Vol. 63, No. 113 / Friday, June 12, 1998 / Notices

DEPARTMENT OF LABOR

Employment and Training

Administration

United States Employment Service;

Labor Exchange Performance

Measures

AGENCY: Employment and Training

Administration.

ACTION: Notice and request for

comments.

SUMMARY: In response to the

requirements of the Government

Performance and Results Act (GPRA) of

1993, the national call for government

programs to be more accountable and

results-oriented, the Department of

Labor (DOL) Employment and Training

Administration (ETA) launched a

project called the Workforce

Development Performance Measures

Initiative (WDPMI) to develop a menu of

key performance measures for use in the

workforce development system. Due to

the absence of established key

performance indicators for the public

labor exchange program that measures

both the self-service and staff-assisted

service options now available, the

United States Employment Service

(USES) initiated a project at about the

same time to work cooperatively with

States in developing program-specific

performance measures. This FederalState workgroup also was charged with

ensuring that the labor exchange

performance measures it proposed

would be compatible with and

complementary to the overall WDPMI

performance measures being developed.

The Labor Exchange Performance

Measures workgroup prepared a

discussion draft issues paper entitled

‘‘America’s Labor Exchange

Performance Measures’’ that identified a

list of potential measures. This paper

was shared informally among the State

Employment Security Agencies during a

period beginning on September 10,

1997, until October 31, 1997. More than

20 States provided reactions and

feedback on the proposed measures. The

Labor Exchange Performance Measures

workgroup considered these comments

and in subsequent discussions refined

the potential program measures.

This Federal Register Notice (FRN)

provides a description of the conceptual

framework within which public labor

exchange services are delivered, and

requests comment from interested

parties and USES’ stakeholders on

proposed performance measures for

labor exchange services. The

Department of Labor also is interested in

comments on the appropriate number of

measures, and whether the proposed

measures take into account the full

range of services and service options in

the modern labor exchange. In addition,

the Department is interested in learning

about other measures that State labor

exchange agencies have found useful for

management and continuous

improvement purposes.

There currently are two bills being

considered by the Congress (H.R. 1385

and S. 1186) which will provide the

framework for the Nation’s workforce

development system. Among the

provisions of the bills is a requirement

that the Wagner-Peyser funded labor

exchange program functions are

provided through the One-Stop system

(called ‘‘full-service’’ in H.R. 1385).

These proposed measures are a

starting point for development of

comprehensive measures for the labor

exchange function of the emerging

workforce development system. This

FRN does not address the data elements

needed to produce the performance

measures nor proposes specific changes

to the ETA reporting requirements. That

will be the subject of a subsequent

notice.

DATES: Comments on the proposed

Labor Exchange function performance

measures must be received by the U.S.

Department of Labor on or before July

27, 1998. Late-filed comments will be

considered to the extent possible.

ADDRESSES: Comments must be filed in

Room N4470, U.S. Department of Labor,

Employment and Training

Administration, United States

Employment Service, 200 Constitution

Avenue NW., Washington, DC 20210.

FOR FURTHER INFORMATION CONTACT: John

R. Beverly, III, United States

Employment Service, U.S. Department

of Labor, 200 Constitution Avenue NW.,

Room N4470, Washington, DC 20210,

Tel. 202–219–5257, Fax 202–219–6643,

E-mail jbeverly@doleta.gov

SUPPLEMENTARY INFORMATION:

Contents

I. Authority

II. Introduction

A. Mission of the US Department of Labor

B. Role of the Public Labor Exchange

III. Performance Measures Development

Process

A. Labor Exchange Performance Measures

Workgroup

B. Models for Service in the Modern Labor

Exchange

1. Self-Service strategy

2. Facilitated Self-Help strategy

3. Staff-Assisted strategy

C. Labor Exchange Performance

Measurement Strategy

D. Initial Comments on the Draft for

Discussion Issues Paper

IV. Definitions

A. Job Search Assistance

B. Facilitated Self-help

C. Self-help Tools

D. Staff Assisted Labor Exchange Services

E. Job Listing Services

F. Referral Related Action

G. Business Assistance Services

H. Entered Employment

I. Entered Employment Rate

J. Cost per Entered Employment

V. Proposed Labor Exchange Performance

Measures

A. For Self-Service Strategies

1. Holdings

2. Users

3. Transactions

B. For Facilitated Self-help Strategies

1. Number of Job Seekers Using Self-Help

Service

2. Customer Satisfaction

C. For Staff-assisted Service Strategies

Measures of Job Seeker Customer Services

1. Entered Employment Rate

[Option] Job Development Entered

Employment Rate

Measures of Employer Customer Services

2. Job Listing Return Business Rate

3. Business Assistance Service Return

Business

4. Referral Response Time

5. Average Time Lapse to Successful

Referral

6. Job Order Fill Rate

D. System Measures

1. Cost Per Entered Employment

2. Duration of Benefits Compensated

I. Authority

Labor exchange performance

measures will be implemented under

the following authority:

A. Wagner-Peyser Act section 10 (c), 29

U.S.C. 49i(c)

(c) Reports

Each State receiving funds under this

Act shall (1) make such reports

concerning its operations and

expenditures in such form and

containing such information as shall be

prescribed by the Secretary, and (2)

establish and maintain a management

system in accordance with guidelines

established by the Secretary designed to

facilitate the compilation and analysis

of programmatic and financial data

necessary for reporting, monitoring and

evaluating purposes.

B. Government Performance and Results

Act of 1993

Purposes—The purposes of this Act

are to improve the confidence of the

American people in the capability of the

Federal Government, by systematically

holding Federal agencies accountable

for achieving program results;

—improve Federal program

effectiveness and public accountability

by promoting a new focus on results,
Federal Register / Vol. 63, No. 113 / Friday, June 12, 1998 / Notices 32565

service quality, and customer

satisfaction.

II. Introduction

A. Mission of the US Department of

Labor

The 1998 USDOL Strategic

Performance Plan states that it is the

mission of the Department of Labor to

foster and promote the welfare of the job

seekers, wage earners and retirees of the

United States by improving their

working conditions, advancing their

opportunities for profitable

employment, and protecting their

retirement, health, and other benefits.

In carrying out a part of this mission,

the Department, through the

Employment and Training

Administration, administers a variety of

Federal programs to help employers

find workers, to help job seekers find

employment, to provide unemployment

insurance benefits and reemployment

services to jobless workers, to track

changes in employment and to provide

job training to meet the changing

workplace skill needs.

B. Role of the Public Labor Exchange in

the USDOL Mission

In a May 1, 1933, report to the Senate

from the Education and Labor

Committee, and a May 22, 1933, report

to the House of Representatives from the

Labor Committee, the stated purpose of

the public labor exchange is:

(1) To foster, promote and develop the

welfare of the wage earners of the

United States including juniors (youth)

regularly employed;

(2) To improve their working

conditions;

(3) To advance their opportunities for

profitable employment by regularly

collecting, furnishing, and publishing

employment information as to

opportunities for employment;

(4) For maintaining a system for

clearing labor between the several

States; and

(5) For cooperating and coordinating

the public employment offices

throughout the country.

On June 6, 1933, the Wagner-Peyser

Act (29 U.S.C. 49 et seq.) became law

and has since provided the basic

funding support for the public labor

exchange system.

Over the 64 years of its history, the

role of the public labor exchange has

changed and evolved to meet the needs

of the nation, employers and job

seekers. Dedicated and hard working

employees have labored in service to the

United States from its earliest role in

assisting the nation to recover from the

depths of the Great Depression as part

of the national recovery effort, through

the efforts in support of the nation’s

economy by assisting private sector

employers as their need for workers was

revived. In the years that followed, the

public labor exchange was an important

part of the mobilization in support of

war industries during World War II and

the return to a post war economy.

During that time the federal/State

partnership was restructured with a

redirection of effort to support private

sector employers’ workforce needs.

Through the 1960’s and 70’s the

national system of Wagner-Peyser

funded labor exchange offices was

instrumental in advancing a variety of

social programs designed to expand

participation and conditions of

employment for several target groups

including jobless workers, the

economically disadvantaged and those

who had particular difficulty in gaining

access to employment opportunities.

In 1986, the Secretary of Labor

conducted a series of four public

hearings at which 138 individuals

testified and 562 submitted written

testimony representing a cross section of

the stakeholders of the nation’s public

labor exchange, including business and

industry, the unemployed, target group

advocates (e.g. veterans etc.), labor

unions, State governments, employees

and the training and education

communities. The intent was to assess

the current role of the labor exchange

and to secure input on the appropriate

role for the Wagner-Peyser Act funded

labor exchange in the 21st century.

The vast majority of respondents

(97.6%) indicated that the nationwide

public labor exchange system was

viable, and should be maintained and

improved. Many of the respondents

highlighted the need to streamline the

system and to return to providing basic

labor exchange activities. The list which

follows represents the most frequently

identified services in their order of

priority:

• Intake, general assessment, and

referral to jobs, training or re-training

opportunities, and/ or support services;

• Labor market and occupational

information;

• Preliminary screening of applicants

for employers’ job orders;

• Basic skill and/or aptitude testing;

• Direct placement (includes job

referral);

• Job Search Assistance, including

training and tips on interviewing, etc.;

and

• Specialized recruitment for

employers with large staffing

requirements and workforce needs.

The majority of commentators

indicated that the public labor exchange

services should be available free to all

employers and job seekers, and that the

role of the federal, State and local

governments and the private sector

should be maintained as they were at

the time of the review and are today.

In addition to responding to specific

questions posed in the FRN that

announced the hearings, the

commentators also offered other

recommendations including:

• The public image of the labor

exchange should be improved to

promote use of the available services,

• There be an increased use of

automation and technology to provide

linkage among states and localities and

related programs to assist job matching,

and to provide more timely and accurate

labor force projections,

• DOL develop better standards to

evaluate performance and incentive

systems,

• Staffing levels be increased,

• Data collection and reporting

systems be enhanced,

• Staff training be increased,

• Better job skills assessment tools be

developed,

• Labor market information be

improved,

• Federal support for State-level

program integration efforts be increased,

and

• Alternative funding sources to

finance operations be found.

In the 12 years since the reexamination, the public labor exchange

system has been streamlined.

Revitalized operations now provide

service using flexible tiered service

delivery strategies. The public labor

exchange has significantly increased the

use of automation and technology to

provide service and information, and

implemented many of the changes and

improvements which were suggested. In

cooperation with the Unemployment

Insurance Service, the public labor

exchange also developed a system of

worker profiling and re-employment

services for Unemployment Insurance

(UI) claimants to speed their return to

productive employment.

The Wagner-Peyser funded public

labor exchange has become a core

component of the new Workforce

Development System and is the

universal access component of the

nation’s One-Stop Career Centers. For

many business and job seeking

customers, their only experience and

contact with the Workforce

Development System will be WagnerPeyser funded services.

The lack of additional financial

investment in the public labor exchange

may be traced, to some degree, to its

failure to develop national labor
32566 Federal Register / Vol. 63, No. 113 / Friday, June 12, 1998 / Notices

exchange performance measures.

Performance data that have been

collected and used provides an

inaccurate picture of the overall success

and value of the services provided.

Moreover, these data fail to address the

employer customer who is taxed to pay

for the public labor exchange services.

In this fiscal environment where labor

exchange funding has either remained

the same or declined, a good

performance measurement system will

help States and agencies manage

programs and decide on the most

appropriate mix of labor exchange

services to offer.

For the most part, previous

measurement schemes were simple

counts of service outputs, e.g.,

individuals referred to jobs, with the

effectiveness of those outputs gauged

through ratios made by comparing those

counts to the total pool of persons

registering with the labor exchange, e.g.,

24% of all labor exchange applicants

received a job referral. Moreover,

measurement of the effectiveness of

employer services were confined to a

count of the number of job openings

received by the public labor exchange as

compared to the number of openings on

which a placement action occurred, i.e.,

the job openings filled rate.

The ETA 9002 reporting and data

collection system has been the source of

these program data. The authority to

collect ETA 9002 data will expire on

August 31, 1999. This sunset date offers

an opportunity to re-examine how the

labor exchange function is measured. It

is DOL’s hope to secure comment from

all of the stakeholders including

employers, wage earners, unions, and

partner agencies in the emerging

workforce development system,

including the education, training, and

public assistance communities on both

the performance measures proposed

herein and the conceptual framework

within which those measures would

operate. The comments received will be

reviewed and adjustments will be made

to the proposed measures as

appropriate. Following this

participatory process, another FRN will

be issued to promulgate the key

performance measures, to identify the

data elements needed to produce those

measures, and to establish State data

reporting requirements.

ETA recognizes its responsibility to

continue to collect information on the

race, age, sex, religion, ethnicity,

disability status, and veteran status of

the job seekers who register to use its

services.

III. Performance Measurement

Development Process

A. Labor Exchange Performance

Measures Workgroup

The USES Labor Exchange

Performance Measures Initiative

(LEPMI) Workgroup was composed of

fifteen representatives drawn from State

employment service agencies, USDOL

regional and national offices, the

Veterans’ Employment and Training

Service, and the Interstate Conference of

Employment Security Agencies (ICESA).

The members of the Labor Exchange

Performance Measures Initiative

(LEPMI) possessed on average over 20

years of experience in a variety of

program areas and disciplines including

tracking and monitoring program

performance.

The workgroup was convened in June

1997 to develop a menu of conceptual

performance measures which could be

adopted for the Labor Exchange

function of the Workforce Development

System. The Workgroup used guidelines

similar to those used by the ETA

Workforce Development System

Performance Measures Initiative Policy

Committee to assess alternative

measures, and maintained a close

coordination between that effort and its

work. Several persons served on both

workgroups. This close working

relationship between the two

complementary performance measures

initiatives has continued.

The Workgroup began by agreeing

that the new performance measurement

system should be directed toward

measuring the outcomes of service

delivery, whenever possible, and not the

inputs or outputs of the processes used

to provide service. The first task the

LEPMI work group tackled was to

identify the core services being

provided in today’s labor exchange. The

workgroup identified the following as

the core service categories of the public

labor exchange: for Employers the core

services identified were Job Listing,

Business Assistance, and Job Matching

and Initial Screening of Candidates for

Employment; for Job Seekers the core

services identified were Job Search

Assistance, Job Development, and

Facilitating Access to Self-Help Tools.

There are a wide variety of activities

which are included within the broad

categories of service, examples of which

can be seen in Figure #1 which follows

this FRN.

To develop performance measures,

the workgroup benchmarked a variety of

measurement strategies including the

way in which the results of the public

labor exchange has been measured in

the past, workload-based measures

similar to measures used in UI

programs, measures of the effectiveness

of training programs, approaches used

to measure private sector business

success and indicators of success in the

public library system.

The workgroup also agreed early in its

deliberations to a set of four guiding

principles:

1. The Performance Measures

proposed are intended to apply to the

labor exchange function without regard

to the administrative or program

structure in which labor exchange

services are delivered. Traditionally, the

Labor Exchange function has been

within the purview of State

Employment Security Agencies

(SESA’s) nationwide. With the

emergence of the Electronic Labor

Exchange, worker profiling and

intensive job search assistance activities

under the Dislocated Worker programs,

Welfare Reform, and One-Stop Career

Center service delivery approaches, the

Labor Exchange function is now being

performed in a variety of agency and

program models. These measures focus

on the function not the program.

2. The customer universe against

which staff-assisted service outcomes

are to be measured would be limited to

those persons who have received staffassisted services and not all persons

registered with the public labor

exchange, whether or not they are

seeking work.

There are several customer groups

who use the public labor exchange for

purposes other than immediate

employment. These include but are not

limited to applicants and recipients of

Food Stamps, Home Relief or State

General Assistance, Medicaid,

temporary housing, and day care

programs. Many of these programs

require registration with the public

employment service as part of the

program application process.

Other customers have an intermittent

interest in pursuing employment. These

can include applicants who are not in

the labor market and those who are

currently employed but interested in

new job opportunities. Many of these

individuals will periodically register for

employment with the labor exchange. A

segment of this group includes

candidates for employment with

prominent local firms that have

developed exclusive recruitment

agreements with the public labor

exchange office. Some employers, such

as auto industry manufacturing

companies and major retail outlets, post

signs outside of their plants and

personnel offices which inform

candidates that they hire candidates for

employment who are referred by the
Federal Register / Vol. 63, No. 113 / Friday, June 12, 1998 / Notices 32567

public labor exchange only, directing

the job seekers to complete applications

at a local labor exchange office.

Mass recruitment control services are

valued by many employers and result in

registration of thousands of job seekers

who are interested in working for only

one of these major area employers. If

any of the members of these customer

groups fail to respond to other employer

recruitments or job referral call-ins, and

never return to the offices for job search

assistance, they will ultimately be

inactivated with no service. These

customers are excluded from

consideration when calculating the

outcomes for labor exchange service

because they did not receive and were

not interested in staff-assisted service.

3. Although the performance

measures presented in this FRN are not

segmented by target group, e.g., Welfareto-Work, Migrant and Seasonal Farm

Workers, etc., ETA encourages

segmenting performance measures to

allow a comparison among groups of

customers. Measuring program

performance based simply on overall

entered employment outcomes, without

taking into account the labor market

characteristics of the various target

groups of job seekers being serviced,

may under-represent the true

performance of the public labor

exchange system. Because public labor

exchange services are available to all job

seekers and all employers (universal

access), its performance outcomes

routinely include hard-to-place

customer groups, e.g., potential UI

Exhaustees, Public Assistance

Recipients, Migrant/Seasonal Farm

Workers, or Youth. However, current

methodology does not account for the

difficulty factors these groups may

present. Segmentation of performance

measures may offer insight into these

overall outcomes. For example, an

entered employment rate might be

developed for Welfare-to-Work job

seekers which could be compared to the

entered employment rate for all job

seekers.

ETA believes segmentation of

outcomes for specific groups of

customers can provide valuable insights

into overall performance outcomes. This

FRN does not address the data elements

needed to produce the segmented

outcome measures nor does it propose

specific changes to ETA reporting.

These will be the subject of a

subsequent notice.

4. Data gathering, to the extent

possible, should be directly related to

service delivery and the cost of data

collection and reporting should be

proportionate to the cost of the service

provided. A performance measurement

system developed in the current

environment that has been shaped by

the national effort to reduce the

administrative paperwork burden and to

focus on the results of government

programs should not propose measures

that cost substantially more to collect

and report than the unit price for

delivering the service.

B. Service Delivery Strategies in the

Modern Labor Exchange

To continue to meet customer needs

for labor exchange services in a period

of diminishing resources, States

operating Employment Security, OneStop, and Workforce Development

programs have devised a variety of

approaches and service delivery models

which are significantly different from

the traditional model of an

‘‘Unemployment’’ office that provided

one-on-one staff-intensive interviews

and individualized assistance to job

seekers in finding jobs. This model, as

the sole means delivery of labor

exchange, is now largely extinct.

Today, labor exchange services are

typically provided using a tiered

delivery system composed of three

flexible and adaptive service strategies:

‘‘Self-Service’’; ‘‘Facilitated Self-Help’’;

and ‘‘Staff-Assisted Service.’’ Figure #2,

which follows this FRN, shows this

service model. Each strategy is designed

to respond to differing service needs

and differing service populations. It is

the Department’s expectation that State

agencies providing labor exchange

service will use each of the three service

delivery strategies.

This FRN represents the first effort to

describe the tiered service delivery

continuum, particularly the electronic

Self-Service and Facilitated Self-Help

strategies. Accordingly, more

descriptive text has been devoted to the

descriptions of the Self-Service and

Facilitated Self-Help service strategies

compared to the description of the

traditional Staff-Assisted service

strategy. This should not be interpreted

as a preference for one strategy over

another. Each has a necessary and

appropriate role in labor exchange

service delivery.

1. In a Self-Service strategy, States

make labor exchange resources available

which customers can utilize

independently, i.e., without staff

intervention. Three current trends have

helped drive the development of selfservice strategies in today’s public labor

exchange.

First, government is increasingly

required to do more with less, and so

must find ways to deliver its products

and services more efficiently.

Second, the labor market itself has

changed, with unemployment being less

associated with cyclical pressures and

more with structural causes where

increasing numbers of job seekers are

considered dislocated workers. Studies

have suggested that in the future job

market most workers will change

careers (not just jobs) several times

during their work lives.

Third, customer expectations have

changed, and the public labor exchange

system faces new demands from its

customers regarding the quality of the

services that are provided. Customers

also expect to participate in the

decisions involving what services are

provided and how they are provided.

This delivery mode places the

customers in charge of the services they

receive. The acceptance of self-service

modes of service delivery is

demonstrated by the increased number

and availability of public access

computers, automated teller machines,

debit cards, and library swipe cards

with bar-coded data, etc. There has also

been a general movement within the

United States toward placing more

responsibility on, and expecting greater

participation from, an individual who

wishes to receive governmental services

or benefits. In line with these trends,

customers of the public labor exchange

increasingly have made clear their

preference for exercising informed

choices in determining which products

and services they receive and how they

receive them.

An additional and significant benefit

of the self-service mode of service

delivery is that self-service expands the

capacity of the system beyond the limits

of the available staff resources, thereby

efficiently handling a wide variety of

customer labor market and employment

information needs.

2. In the Facilitated Self-Help strategy,

customers are provided access to selfhelp resources at the One-Stop Career

Center or local labor exchange office,

generally through a dedicated Resource

Room. For many customers this is their

first exposure to self-service tools and

computer-based systems. In addition,

one or more staff are assigned to assist

customers who need help in using those

resources. These staff interact with the

customer, as needed, to facilitate the

customer’s job search using governmentprovided resources, e.g., personal

computers, word processing and/or

resume writing software, fax and copy

machines, etc., and online access to the

DOL-funded Internet-based tools, e.g.,

America’s Job Bank (AJB), America’s

Talent Bank (ATB), America’s Career

InfoNet, etc. After being introduced to

and assisted to use these Internet-based
32568 Federal Register / Vol. 63, No. 113 / Friday, June 12, 1998 / Notices

self-service systems many customers

will be able to use them in the future

without assistance both in the One-Stop

Center and in other locations with

access to the Internet, e.g., schools,

libraries, and at home. As funding for

the traditional labor exchange or

employment service has remained flat

or declined, use of this mode of service

delivery has increased. It continues to

place the customers in charge of the

services they wish to access, but

provides the staff facilitation needed to

ensure that the customers can obtain the

service or information they require.

Again, the cost for providing services

using this strategy is less than under a

Staff-Assisted service strategy, since the

ratio of staff to customers is much

higher.

3. The Staff-Assisted strategy ranges

from intensive one-on-one services

where a staff person is assigned to a job

seeker as a case manager, to those where

a customer interacts with service staff in

a small group setting. This is the more

traditional mode of operation for

delivery of public labor exchange

services. The services most frequently

delivered in this manner are assessment,

intensive job search assistance,

employment plan development, case

management, counseling and vocational

guidance, and job development. The

expected output is a job referral and the

expected outcome is entry to

employment. Group services, such as

job clubs and workshops on

interviewing techniques, the world of

work, and use of labor market

information are also classified as staffassisted services, although they are

provided on a one-to-many service

basis.

C. Discussion of Labor Exchange

Performance Measurement Approaches

DOL is proposing a different approach

for measuring performance under each

of these service strategies. The

Department does this for two primary

reasons. The first reason is that each

strategy results in significantly different

relative costs. Costs can be measured

per unit of service provided or as an

overall cost for providing access to

resources. The costs to provide a service

should be the primary determinant in

deciding how much to invest to measure

that service and how much customer

information to collect. The more

significant the service intervention, e.g.,

the higher the unit cost of that service,

the greater the rationale for collecting

data to justify the expenditure of public

funds. For example it would not appear

to be prudent to spend $10.00 pursuing

a follow-up action to determine the

outcome of a service that costs less than

25 cents to deliver. However, one

should be willing to expend significant

resources in determining whether a

$5000 investment in training results in

a long term payoff to the taxpayer.

The following are examples of labor

exchange services and resources that

might routinely be provided in a selfservice or facilitated self-help mode and

for which it might be difficult to

document an employment outcome

without a substantial data collection

and follow up effort.

• Labor Market Information.

• Self-help pamphlets, magazines,

newspapers, and reference books.

• Internet access for use of self-help

job finding tools, e.g., AJB/ATB.

• Access to computer hardware and

software, fax and copy machines,

telephones, and other equipment in

resource centers.

Gaining Internet access to resources

and tools such as these from a job

seeker’s home or from a community

resource, such as a library or

community center, may cost the public

labor exchange system only a few

pennies per transaction. First, because

economies of scale are achieved in the

development and maintenance of

Internet-based systems and resources.

Second, because the cost of the

computer equipment, Internet service

provider access, telecommunications

software and hardware, supplies,

electric power, and facilities are borne

by the user or by another organization.

This cost per transaction rises slightly

when these latter costs are borne

directly by the public employment

service, such as when offered through a

resource room. When these self-service

resources are further augmented by

making staff resources available to assist

and/or instruct customers in using these

tools and resources, the unit cost again

rises.

A similar case might be made for

employer services. An employer may

post an unsuppressed (broadcast) job

opening directly to AJB over the Internet

instructing interested parties to contact

that employer directly. This is a very

low cost self-service option.

Employers may also request

assistance from public labor exchange

staff in placing their job openings on

America’s Job Bank. The cost of this

service can increase further when the

employer also asks the Labor Exchange

to screen and refer applicants to that job

order. The cost to provide this level of

staff-assisted service may rise to the

level where tracking outcome data is

important.

The second reason for pursuing

different kinds of performance measures

when using different service delivery

strategies is that, for program outcome

measures, one must be able to make a

rational argument that there is a nexus

between the service(s) provided and the

outcome measured. For example, it

might be difficult to argue that a single

user session of browsing through labor

market information in a self-service

mode over the Internet can be directly

linked to a person obtaining a job 30

days later. However, should that same

person come into the local public labor

exchange office several times over the

course of a few weeks and attend

workshops, obtain help in his or her job

search from the local office staff, and

receive one or more job referrals, such

a nexus can be reasonably concluded.

Clearly, finding a way to gauge the

results of self-service and facilitated

self-help strategies presents a challenge

to the system. This is particularly true

since the Government Performance and

Results Act (GPRA) became law. Under

GPRA, government agencies must

account to their investors (Congress and

the American people) as to both how

their programs perform and what results

they achieved. The challenge is to find

a means to satisfy this requirement

where the service costs only a few

pennies per transaction to deliver and

the relationship to traditional program

results is difficult to draw. As the level

of service intervention increases, staff

involvement grows, and dollar

investment rises, the use of outcome

and impact performance measures may

become necessary to justify and explain

the expenditure of public resources.

DOL hopes that it has addressed these

concerns in the performance measures

which follow.

1. Measurement Approach for SelfService Strategies

A small ad-hoc task team was

convened to come up with an approach

to measuring the self-service

components of the workforce

development system. This team

included several persons who were

members of both the Workforce

Development Performance Measure

Initiative (WDPMI) and the Labor

Exchange Performance Measures

Initiative (LEPMI) workgroups. The

team concluded that the following

factors should be used to determine

when outcomes measures would be

appropriate for gauging the performance

of a particular self-service or facilitated

self-help service:

• There is a significant value-added

provided by the service, i.e., the service

is more than just an information

exchange.
Federal Register / Vol. 63, No. 113 / Friday, June 12, 1998 / Notices 32569

• The cost to collect and track

outcomes is less than the cost per unit

to provide the service.

• The service itself is intended to lead

to an employment or other measurable

outcome.

• The cost of providing the service

rises to a point which indicates a need

to justify the expenditure of public

resources.

• The time lapse between receiving

the service and the measurement of the

outcome is short enough that the service

can be considered to be causative of

and/or contributory to that outcome.

The Ad-hoc Task Team used these

factors to help determine whether selfservice and facilitated self-help

strategies should be included in

outcome measurement calculations. The

first group of services considered were

those which could be provided in a

completely self-service mode, requiring

no staff assistance. These services

included:

• accessing and using electronic labor

exchange information services,

• completing online self-assessments,

• finding information, such as labor

market information (LMI) or community

services,

• online training, self improvement,

and skill enhancement, and

• applying for services and benefits

(e.g., using telephones for original and

continued UI claims).

The Ad-hoc Task Team agreed that

customers receiving services in this

fashion should not be included in

outcome measurements, but rather

services should be measured using

output measures.

In general, the Ad-hoc Task Team

agreed with the performance

measurement strategy which had been

developed by the Labor Exchange

Performance Measures workgroup.

Under this strategy, measuring the value

of self-service strategies, such as using

DOL and State-developed Internet-based

tools (America’s Job Bank, America’s

Talent Bank, and America’s Career

InfoNet) should be analogous to

measuring the value of services

measured in public library systems.

Using this concept, performance would

be measured by counting the growth

over time of:

• Holdings: the numbers of jobs and

resumes that are maintained in its

inventory;

• Usage: the number of times

customers use its services; and

• Transactions: how many times

specific kinds of services are used, e.g.,

referral information on job listings are

accessed or downloads of resumes are

requested.

Figure #3, which follows this FRN,

shows a model of this approach to

measure program performance.

2. Measurement Approach for

Facilitated Self-Help Strategies

Under the facilitated self-help service

strategy, staff assist customers to use

self-help tools and resources. The

services provided include:

• the self-service activities noted

above;

• access to computer hardware,

software, telephones, office equipment

and other physical resources in a Labor

Exchange office; and

• staff assistance in using any of the

above services or resources.

The approach to measurement of

these services would be a combination

of usage and customers’ satisfaction.

Usage would be measured as a simple

count of the number of persons using

resources rooms. This could be

collected using swipe cards, tallies,

sign-in sheets, and/or automated session

counts. Customers using self-help

services should not be included in

outcome measurement until or unless

they are provided more intensive staffassisted service. Customer satisfaction

would be used to look at the dimensions

of service: accessibility, quality,

timeliness, and security. This could be

done by surveys that would not be

reported nationally. These surveys

would be locally developed, controlled,

and used for continuous improvement

projects at the local level.

3. Measurement Approach for StaffAssisted Strategies

Services delivered using a staffassisted service strategy represent a

value-added service. Measuring the

performance of these value-added

services requires addressing their

employment-related outcomes for job

seekers and employers. Additionally,

measures should be adopted which can

be used to assess the overall

effectiveness and impact of the public

labor exchange system.

D. Initial Comments on the Draft for

Discussion Issues Paper

The United States Employment

Service in cooperation with the

Interstate Conference of Employment

Security Agencies (ICESA) released the

issues paper, entitled, ‘‘Draft for

Discussion—America’s Labor Exchange

Performance Measures,’’ in September,

1997. It provided a review of the

rationale and a suggested conceptual

framework for establishing a set of key

performance measures of labor exchange

services. Included was a discussion of a

variety of alternative measures and the

identification of 10 potential

performance measures of staff-assisted

job seeker and employer services. In

addition, performance measures were

suggested for the electronic labor

exchange which counts holdings, usage,

and transactions. Also included were

two system measures. One was a costto-benefit measure that looks at unit cost

per outcome, and the other was a

measure of the impact of service on

Unemployment Insurance benefits

compensated.

State Employment Security Agency

Administrators were asked to provide

written reactions to the suggested

measures by October 31, 1997. The

‘‘Draft for Discussion’’ issue paper was

also made available on the Internet on

the USDOL-supported ICESA Workforce

ATM website (http://www.icesa.org/

national/docs/LABOREX.HTML). This

approach provided an opportunity for

the public labor exchange system to

have direct input to the initial

development of the performance

measures that are the subject of this

FRN.

Based on the comments received from

States, and subsequent additional

discussion and consideration by the

workgroup, the number of job seeker

measures has been reduced and several

improvements have been made to other

measures. In some cases, where

different approaches were suggested and

there was no clear-cut decision as to

which approach was the better choice,

more than one option has been included

in this FRN for comment.

Current system redesign efforts to

enhance and integrate the features of

America’s Job Bank and America’s

Talent Bank systems will make it

possible to collect and aggregate some

data electronically without additional

State data collection or transmission

effort. It is the Department’s intention to

actively pursue electronic data

collection approaches to the extent

feasible.

IV. Definitions

A. Job Search Assistance (JSA):

Includes services currently defined in

the ETA 9002 Data Preparation

Handbook NO. 406 under the ‘‘Received

Some Reportable Service’’ categories

and in items 12 through 42. The service

activities included are: job search

workshops, job finding clubs, resume

assistance, providing specific labor

market information, job search plan

development, job matching, job

development, referral to jobs, vocational

guidance, assessment interviews,

testing, vocational counseling, federal

bonding, and referral to other services
32570 Federal Register / Vol. 63, No. 113 / Friday, June 12, 1998 / Notices

(including skills training, educational

services and supportive services).

B. Facilitated Self-help: Is a staffassisted service in which staff facilitates

customer access to a variety of in-office

self-help job finding tools and resources.

Support staff provides limited

assistance on an as needed basis. Most

job seekers require limited assistance.

Normally, job-seeking customers are not

required to register for service before

they are allowed access to self-help

tools or resource areas, rooms or centers.

C. Self-help Tools: Include, but are

not limited to, resource rooms,

automated labor exchange system

devices, Internet access, fax machines,

telephones, photocopiers, personal

computers, word processors, career and

labor market information, and reference

materials.

D. Staff Assisted Labor Exchange

Services: Include job search assistance,

job development assistance and job

referrals, and can be characterized as

service which is conducted one-on-one

or in small groups.

E. Job Listing Services (JLS): Include

activities performed on behalf of

employers to assist them in filling their

workforce needs. JLS includes but is not

limited to services and activities such as

job order taking, job order maintenance,

referral follow-up and interview

support, together with connecting

activities including searching job seeker

databases, transmitting resumes, and

marketing job orders to the job seeker

(applicant) pool, etc.

F. Referral Related Action: Includes

the following services and activities:

(1) Referral of qualified candidates, or

(2) Contact with the employer to

review potential referrals, to develop a

candidate recruitment strategy or adjust

features of the job order which prevent

identification of candidates for referral,

i.e., salary, experience requirements,

etc.

G. Business Assistance Service (BAS):

Includes, but is not limited to, providing

employer education seminars, job and

task analysis, providing local labor

market information, and referral to other

workforce development or economic

development services or agencies.

There are a number of employer

service activities which are excluded

from the definition of BAS. They are:

periodic mass mailings and routine

promotional mailings to increase

employer job listings, providing labor

market information via the Internet and

employer contacts for the provision of

other core services, e.g., job listing, job

matching, initial screening or referral

services.

H. Entered Employment: The

unduplicated count of job seekers

(applicants) who enter employment by

job placement or obtained employment.

See page II–12 of ETA 9002 Data

Preparation Handbook, ET Handbook

NO. 406.

(1) Job Placement: The hiring of a job

seeker by a private or public employer

after referral to a job by the Agency staff,

or collocated or out-stationed staff in

cooperation with the Agency, provided

that the following conditions were

fulfilled:

(a) Prepared a job order prior to

referral, except in the case of a job

development contact on behalf of a

specific job seeker,

(b) Made prior referral arrangement

with the employer,

(c) Referred an individual who was

not designated by the employer, and

(d) Verified from a reliable source that

the job seeker had entered work, and the

placement was recorded in the agency

data base.

See page II–12 of ETA 9002 Data

Preparation Handbook ET Handbook

NO. 406.

(2) Obtained Employment:

Individuals who secure employment

within the current quarter or the next

completed quarter following the last

staff-assisted job search assistance

service that was partially funded by the

Agency, such as:

(a) Participating in job search

activities,

(b) Accepting a position resulting

from the use of an agency-sponsored

automated labor exchange,

(c) Receiving employment counseling

or testing or development of an

employability plan,

(d) Receiving bonding assistance,

(e) Terminating from a skill training

program to which a job seeker was

referred by the Agency, or

(f) Receiving tax credit voucher, and

receipt of verification from a reliable

source, preferably the employer.

See page II–13 of ETA 9002 Data

Preparation Handbook ETA NO. 406.

Entered Employment can be counted in

the current quarter or the next

completed quarter following the receipt

of the last staff-assisted service (a

maximum of 180 days).

I. Entered Employment Rate (EER): Is

the percentage of job-seekers securing

employment after receiving StaffAssisted Services.

J. Cost per Entered Employment

(CPEE): Is the cost of achieving the

positive outcome of entry to

employment following the provision of

service. The CPEE is determined by

dividing the total of those who entered

employment by the total funding

received for the federal Wagner-Peyser

Act, and Veteran Services grants and

State appropriations used for the Labor

Exchange function during a program

year (July 1–June 30).

V. Proposed Labor Exchange

Performance Measures

Specific questions have been

developed regarding these performance

measures for which public comment is

sought. Questions can be found at the

end of sections A., C., and D. below, and

they are numbered sequentially, 1

through 16. Commentators need not

repeat questions as part of their

response, but addressing the specific

questions by number would be most

helpful to this effort.

A. For Self-Service Strategies

Performance measures will be output

measures based on holdings, usage and

transactions (much like a library) until

customer-friendly, non-invasive sign-on

and low cost follow-up approaches are

developed to identify job-seekers in the

electronic labor exchange.

In developing these performance

measures, the needs and interests of

legislators, policy makers, program

managers, budget planners, analysts,

employers and other investors were

considered. These output measures may

be used for strategic planning, program

management, continuous improvement

and research. This information

combined with customer satisfaction

surveys and feedback could assist in the

design of system improvements to meet

customer needs.

It is DOL’s expectation that national

data reporting will be produced by the

AJB/ATB system, and will be aggregated

and reported electronically. It is also

DOL’s expectation that these

performance measures will not require

additional state data collection.

1. Holdings: This measure is a count

of the number of employer job orders

and the number of job seeker resumes in

the AJB/ATB system. Continued growth

in the number of job orders and resumes

in the AJB/ATB system provides an

indication of customer satisfaction and

perceived value by the customers.

2. Usage: This measure is a count of

user sessions on the AJB/ATB system. A

user is an individual who accesses the

AJB/ATB system for any purpose. Some

users of the system supply personal

identification numbers and passwords;

others are anonymous. A user session

represents each single continuous

access. This is different than a ‘‘hit’’,

which measures the number of Internet

server accesses (ie. computer to

computer communications). Under this

measure, we will count registered

employers, registered job seekers and

anonymous sessions. Enhancements to
Federal Register / Vol. 63, No. 113 / Friday, June 12, 1998 / Notices 32571

the AJB/ATB system are underway to

provide better information on

anonymous sessions. This output

measure will provide a gauge of how

many customers are using these

electronic labor exchange services over

some defined period of time and

whether the usage is growing, remaining

static or declining.

3. Transactions: This is an output

measure of a higher level of interaction

with the AJB/ATB system beyond a

review of the information resources. For

example, among the variety of

transactions which could be collected

electronically, two stand out. They are

Job Seeker Referral Requests and

Employer Resume Downloads.

‘‘Job Seeker Referral Requests’’

measures the number of job order

referral instruction screens that are

viewed by job seekers on AJB. This

output measure is a count of the number

of times job order referral instruction

screens are viewed by job seekers on

AJB after viewing a job order

description. This measure identifies the

number of users who have requested

specific job referral information and are

likely to respond to the employer’s job

opportunity. There are two ways in

which this occurs: unsuppressed job

orders and suppressed job orders. In

unsuppressed job orders the employer

contact information is broadcast and not

hidden from the job seeker. In this case,

the job seeker self refers. In suppressed

job orders the employer contact

information is not viewable or

accessible through computers and the

job seeker must be referred to the

employer by the local labor exchange

office holding the job order.

‘‘Resumes Downloaded’’ is a count of

job seeker resumes that employers have

selected from a list of job seekers, who

met an employer defined search criteria,

to obtain job seeker contact information.

Questions

The Department specifically invites

interested persons to provide comments,

data, information and views concerning

the following:

Q1. Do these proposed measures

adequately represent an employer’s or

job seeker’s satisfaction with the

Electronic Labor Exchange?

Q2. How would you rate the

importance of the three types of selfservice outputs (holdings, usage, and

transactions)? Which would be most or

least important for strategic planning,

program management, etc.?

Q3. What other ways would you

suggest for measurement of self-help

services?

Q4. How do you currently or how

would you suggest the national system

collect customer satisfaction data and

information?

Q5. Will the proposed Self-Service

Electronic Labor Exchange measures

enhance State quality improvement

initiatives?

Q6. What specifically defined period

of time would be most appropriate for

self-service performance measurement:

monthly, quarterly, or annually?

B. For Facilitated Self-Help Strategies

Two Facilitated Self-help measures

are being proposed: the number of users

and customer satisfaction. These

measures are intended to determine the

extent to which customers value

facilitated self-help services. The

measures address this by answering two

questions: do customers use the service

and are customers satisfied with the

services?

1. Number of Users of Self-Help Service

The Number of Users of Self-Help

Service is an output measure of the

number of customers who access the

self-help resources which includes the

assistance of a knowledgeable staff

member assigned to facilitate customer

access to and use of the self-help tools.

It is the intent that facilitated self-help

provides service to a large number of job

seekers with a minimum investment of

staff time.

The accumulation of a count of users

can be as rudimentary as an office stroke

tally or a sign-in log. It would not be

necessary to track which self-help tools

are used, although valuable information

needed for the continuous improvement

of facilitated self-help programs might

be gained from a periodic assessment of

which tools are most popular or useful

to job seekers.

2. Customer Satisfaction

Customer Satisfaction can be

measured by conducting periodic formal

telephone or mail surveys or by

collecting customer in-person feedback

using a structured approach at specified

intervals. The measure could capture

the frequency of use of self-help

resources. Customers normally are not

required to register to use the self-help

tools in a labor exchange office or onestop career center. These measures

recognize the customer’s use of and

satisfaction with self-help resources.

Facilitated Self-Help is a widely used

and valid form of service. Self-Help

service is consistent with the reality of

shrinking resources, and relates directly

to customers’ demand. This measure

can contribute to continuous

improvement with input from the

customer.

C. For Staff-Assisted Service Strategies

Measures of Job Seeker Customer

Services

Staff-Assisted Labor Exchange

Services are the core activities of the

modern Labor Exchange Office. The

purpose of these performance measures

is to determine the outcomes,

effectiveness, and system impact of

Staff-Assisted job seeker service.

Performance will be evaluated through

a combination of interrelated

performance measures. These measures

will be useful to legislators, employers,

policy makers, agency administrators

and program managers since they

provide a tool for managing programs,

baseline data for continuous quality

improvement, and feedback that will

allow States to respond to customer

needs. These quantitative measures

should be supplemented by qualitative

measures of customer satisfaction and

are not intended to be the sole measures

of satisfaction.

1. Entered Employment Rate (EER):

The EER is the percentage of job seekers

securing employment after receiving

staff-assisted labor exchange services

divided by the total number of job

seekers who received staff-assisted labor

exchange services. This measure uses as

its denominator only those customers

who receive staff-assisted service(s) and

not the total number of applicants

registered in the labor exchange office

database. This performance measure

looks at the effectiveness of the Job

Search Assistance services that have

employment as the expected outcome.

This measure encourages an increased

level of job order and job seeker followup, tracking and employer feedback.

Service and entered employment data

would be collected and reported by staff

to the labor exchange reporting system.

Unemployment Insurance wage records

can be an additional data source used to

conduct job seeker follow-up on entry to

employment.

[Optional Measure] Job Development

Entered Employment Rate: Is an

additional and optional measure of Job

Search Assistance service. Job

Development (JD) has been an effective

tool to help individuals who have

barriers to employment or difficulty in

finding employment to find jobs. With

the expected expansion in the role of

the Labor Exchange in Welfare-to-Work

programs, it has been suggested that a JD

Entered Employment Rate would be

valuable for program management and

continuous improvement.

Job Development Entered

Employment Rate is the percentage of

job seekers who entered employment

after receiving Job Development referral
32572 Federal Register / Vol. 63, No. 113 / Friday, June 12, 1998 / Notices

compared to the total number of job

seekers who received JD referral.

An outcome measurement of the proactive solicitation of a job opportunity

for an individual or group, this measure

assesses the effectiveness of job

development efforts in contributing to

job seeker entry to employment. Job

development is defined as development

of a job opening for a job seeker or group

of job seekers through direct contact

with potential employers when no

suitable job openings are currently

listed. Job development contacts are

currently reported within the category

of ‘‘Received Reportable Service’’ in the

ETA 9002 report.

This performance measure is intended

to create an incentive for staff to use

their considerable knowledge of the

labor market and employers’ needs to

increase the number and effectiveness of

job development contacts and referrals

on behalf of job seekers.

Measures of Employer Customer

Services

The variety of staff-assisted services

that are made available to employers

can be grouped into three categories: Job

Listing Services, Job Matching, and

Business Assistance service.

The purpose of the proposed

performance measures is to determine

the extent to which employers’ needs

are being fulfilled in terms of the

‘‘Timeliness,’’ ‘‘Quality,’’ and ‘‘Impact’’

of our work. Staff-assisted employer

service performance should be

evaluated by this combination of

interrelated performance measures. The

individual measures should not be

viewed in isolation and these measures

should be supplemented by qualitative

measures of employer satisfaction.

The proposed measures provide an

indirect indication of employer

customer satisfaction. They are not

intended to be the sole measures of

satisfaction. For example, the Return

(Repeat) Business measure may be a

good measure of employer satisfaction

at the macro (state or national) level.

However, federal contractors and others

who are required to list their jobs with

the Agency may or may not be satisfied

employer customers.

It is recognized that services provided

to employers are not currently reported

to the USDOL and may not now be

collected by the labor exchange agency.

These employer service measures may

require States to develop new

administrative reporting or program

record keeping systems.

2. Job Listing Return Business Rate:

The number of employers who list more

than one job order divided by the total

number of employers who use the StaffAssisted Job Listing Services during a

reporting year. This outcome measure of

repeat business serves as a means to

determine employer satisfaction with

the job listing service.

The intent of this measure is to

provide an incentive for service delivery

staff to improve customer service and

relationships including quick response

time, quality referrals, and employer

follow-up contacts. Most of the data

needed for the Return Business measure

is readily available with the possible

exception that some States do not

capture employer identification

numbers on job orders. This measure

would require that employer

identification information be collected

and reported for all job orders in the

agency reporting and Job Bank system.

Some States have expressed a

preference for measuring job openings

rather than job orders. The focus of this

measure is employer satisfaction, not

the volume of job openings or the

number of job orders the labor exchange

receives or an employer market

penetration rate. Repeat business is a

better indicator of employer satisfaction,

and this measure is used in the private

sector as an important indicator of

success in service sector business

enterprises.

Some States and labor exchange staff

were also concerned with the effect of

employer direct job order entry into AJB

and employer access to resumes on ATB

and its impact on the number of job

orders they would receive. In the past

the number of job orders secured by a

local office and listed in the State job

bank has been a key local office

performance measure in some States.

Again, this measure represents the

percent of Return Business which

results from the employer’s satisfaction

with the Staff-Assisted service. Job

orders entered directly by employers are

captured in the electronic labor

exchange measures of AJB and ATB

holdings and are not lost to the state. An

unintended consequence of this

measure may be a counter-productive

competition between a State’s staffassisted service component and the

State’s electronic labor exchange

component. From a local office

viewpoint, this measure may motivate

staff members to discourage employers

use of the job order self entry service

options in AJB to assure that there is a

high rate of repeat job listing business

credited to the staff assisted service

component. Enhancements to the AJB/

ATB systems and report generators are

currently underway as mentioned

above. This labor exchange system

performance issue will be addressed in

future AJB/ATB versions. The

Department would appreciate comments

and suggestions for dealing with this

potential consequence of measuring

Repeat Employer Business in the staffassisted service component of the labor

exchange.

Other States commented on the

impact of having a large number of

small employers or adverse economic

conditions which would reduce the

opportunity for return business during a

reporting period as short as a year. DOL

acknowledges the concern and would

encourage comments which are based

on a review of State Employment

Service, Unemployment Insurance or

Tax administrative data which

demonstrate this phenomenon. DOL

would also be interested to learn if this

is a result of an agency policy to target

small employers who have limited

workforce replacement or expansion

needs.

Finally, even if the suggested defect in

this performance measure is accurate,

the Return Business measure would be

useful for strategic planning,

performance management and

continuous improvement efforts. This

measure is intended to be used by States

for national performance reporting and

state program management as opposed

to being used for State-to-State

comparisons.

3. Business Assistance Service Return

Business: In addition to Job Listing and

Job Matching services, there are a

number of other types of Staff-Assisted

labor market information and services

which employers require. These were

defined in section IV. G. above. The

provision of Business Assistance

Services (BAS) is intended as a means

of stimulating increased employer job

listings and employer support for the

Labor Exchange and Workforce

Development System. The BAS Return

Business Rate is the number of

employers who utilize more than one

BAS service divided by the total number

of employers who use at least one Job

Listing, Job Matching or Business

Assistance staff-assisted service. The

intent is to measure the level of

employer customer satisfaction with

BAS provided and to be an incentive to

increase service to an important

customer.

4. Referral Response Time: Is an

outcome measure of Job Matching

Service (JMS) effectiveness, which

combines the interests of job-seekers

and employers. The Job Order Response

Time measure is the percentage of job

orders for which referral-related action

took place within three business days

compared to the total number of job

orders listed. This performance measure

provides an incentive for quick response
Federal Register / Vol. 63, No. 113 / Friday, June 12, 1998 / Notices 32573

to job orders by job-seeker referral or by

direct referral-related contact with

employers. Three business days have

been identified as the generally

accepted standard for referral and/or

follow-up contact with employers who

use the Job Listing Service (JLS).

Some States indicated concerns

regarding their ability to track employer

follow-up contacts. It is recognized that

current systems may have to be adjusted

to capture this information. Although it

is understood that not all job orders can

have a quality referral made within

three days, this measure encourages

rapid response to the employer’s need

by the provision of other employer

services such as, contact with the

employer to review potential referrals,

to develop a candidate recruitment

strategy and/or to adjust features of the

job order which prevent identification

of candidates for referral, i.e., salary,

experience requirements, etc.

5. Average Time Lapse to Successful

Referral: This outcome measure will

provide a picture of how quickly labor

exchange agencies respond to job orders

with referral of a qualified candidate for

employment. The performance measure

is the average time lapse (in days) from

the date an employer’s job order is listed

to the date of referral of the first agencyreferred individual hired by that

employer. The underlying presumption

is that when the employer decides to

hire one of the labor exchange referrals,

the agency has met the employer’s

candidate qualification need.

Not included in this calculation are

job orders where the employer hires

from another source. This removes from

the equation situations where, although

qualified candidates are referred, the

employer decides to hire an equally

qualified candidate from another source.

This measure focuses on the time

lapse to the date of job-seeker referral.

An employer’s hiring practices which

may delay the offer of employment for

many days or weeks does not affect the

outcome of this measure. This measure

provides a snapshot of the agency’s

referral practices in terms of

responsiveness with a quality feature. It

is intended to balance the rapid referral

measure and mitigates the possibility

that fast, but poor quality, job-seeker

referrals will be the reaction to a rapid

response performance measure.

Job Order Fill Rate: Based on State

comment, an additional measure of the

Employer Satisfaction is proposed. The

Job Order Fill Rate is the number of job

orders for which a placement is made

divided by the total number of

suppressed job orders received within

the reporting year. A job order for which

the employer name, address, and other

identification or contact information is

hidden from the job seekers’ view is

considered to be a suppressed job order.

Employers have the option of listing

a job order in a suppressed or

unsuppressed mode depending on the

level of initial screening, and/or referral

control service the employer wish the

labor exchange to exert. Frequently

employers are guided in the decision to

suppress contact information based on

the local availablity of candidates for

employment who meet the skill,

background and experience required to

be successful candidates for

employment by labor exchange staff.

This outcome measure will provide a

picture of how effectively agencies or

States respond to these job listings with

qualified candidates for employment. In

this measure, the fact that the employer

hires one of the job seekers referred by

the agency is verification that the

candidate was qualified for the job and

that the employer was satisfied with the

service provided.

The intent of this measure is to

provide an incentive to meet and/or

exceed employer customer expectations

by referral of qualified candidates for

employment. Not including in the

denominator of this performance

measure job orders which are listed in

an unsuppressed or broadcast fashion

avoids the possible unintended

consequence that the labor exchange fill

rate performance is measured against

job orders where staff-assisted service is

neither needed nor wanted by the

businesses that want to broadcast

available employment opportunities.

The required data are currently being

collected in many States and are

available.

In some States, a Job Opening Fill

Rate performance measure has been

used for many years as the approach to

gauge the efficiency of local Labor

Exchange offices and operations. In

other States, the Job Opening Fill Rate

has been discontinued because it has

proven to be a disincentive to securing

employer job orders in occupational

areas where local management and staff

would find it difficult to identify local

and immediately available candidates.

By including in the performance

measure denominator only those job

orders and openings which have the

employer contact information

suppressed, there will be less of an

incentive to withhold job orders from

the system or to artificially reduce the

number of job openings. Based on labor

market information, and in consultation

with the employer, job orders that are

not likely to be filled by a local

candidate can be listed in an

unsuppressed fashion and would

therefore not count in the Fill Rate

measure denominator.

Questions

The Department specifically invites

interested persons to provide comments,

data, information and views concerning

the following:

Q7. Are there other services which

your State or agency provides which

have entry to employment as a goal

which should be included in a list of Job

Search Assistance services?

Q8. Veteran service and other target

group service programs continue to

provide a higher level of one-on-one jobseeker service and job development

assistance. Is the level of the job

development assistance activity in your

State or agency significant enough to

require measurement of the Job

Development Entered Employment Rate

separately and apart from the StaffAssisted Entered Employment Rate

(EER)?

Q9. What would be the impact of a

requirement to collect or have the

facilities necessary to associate an

employer ID number with a job order?

Q10. Is the proposed three business

day standard for measuring referralrelated follow-up contact with

employers consistent with your agency

or State approach?

Q11. The Job Order Fill Rate measure

is intended to provide an indication of

employer satisfaction with staff-assisted

labor exchange services. Are the

unintended consequences of a fill rate

performance measure serious enough to

eliminate it from consideration as a

program performance measure?

Q12. Are there other services and/or

service delivery approaches which

should be included in the mix of

services suggested under the Business

Assistance Service definition?

Q13. Will these proposed StaffAssisted Labor Exchange service

measures enhance State continuous

quality improvement initiatives?

D. System Measures

1. Cost Per Entered Employment (CPEE)

This is an outcome measure of the

efficiency of the labor exchange

function. The CPEE is calculated by

dividing the total number of entered

employment counts by the total funds

allocated to the labor exchange function.

Funding sources would include

Wagner-Peyser Act, Veteran

Employment and Training Service

(VETS) allocations to States, and

supplemental State funding provided

for the labor exchange function. This

measure will provide the context for the

public labor exchange systems to
32574 Federal Register / Vol. 63, No. 113 / Friday, June 12, 1998 / Notices

compare its key outcome—entered

employment—against other workforce

development components. For example,

a 40% EER that costs only $300 per

entered employment may seem more

favorable when compared to an 80%

EER when each EE costs $5000.

Another intent of the measure is to

promote efficiency by providing

management with cost-of-operation

information. This information could be

used for strategic planning, allocation

and distribution of scarce resources, as

well as continuous quality improvement

efforts.

Administrative data sources are

available to produce this system

measure, and includes SESA

administrative records and labor

exchange job-seeker service records. In

many States, Unemployment Insurance

wage record data can be used to

improve entered employment

information.

Funding data can be captured from

the annual grant allocation documents

for Wagner-Peyser and Veteran’s

Services. The information on State

funding for the labor exchange function

would have to be collected by

canvassing the States to determine the

level of additional State support. The

cost should be calculated based on the

Wagner-Peyser Program Year basis,

which is consistent with other

employment and training program

funding cycles. The Veterans’ program

grants would have to be prorated since

they are allocated based on a Federal

Fiscal Year. A similar proration would

be required for State funding which is

not allocated on a program year basis.

The CPEE measure is easy to

understand and is generally accepted.

These data are also useful for program

management and the resource allocation

process.

2. Duration of Benefits Compensated

Employers are the primary customers

of Workforce Development programs

nationwide. To meet employers’

workforce needs, public labor exchange

agencies focus on job-seekers who

possess a labor market attachment and

marketable skills. In addition, WagnerPeyser Act funding is provided to the

public labor exchange, in part, to

administer the work test for the State

Unemployment Insurance compensation

system and to provide job search and

placement services for claimants. The

effect and/or impact of services directed

toward UI benefit customers has many

dimensions which can be significantly

affected by movement in the economy

and local business conditions. However,

there is a general belief that labor

exchange service can have a direct and

measurable impact on the duration of

benefits.

Two options are proposed to measure

the impact of staff-assisted service on

unemployment insurance benefit

customers:

The first option is a measure of

duration of benefits compensated for

claimants who receive services

compared to prior year(s)’’ duration.

This measure can be adjusted based on

economic conditions. This measure

seeks to determine the impact of staffassisted labor exchange services on the

duration of benefits for claimants who

received services. Data can be collected

from Unemployment Insurance average

duration data records (UI claims first

payments/weeks compensated), labor

exchange agency records, and local

office data reporting. USDOL

Unemployment Actuarial Unit data on

business cycles and economic

conditions could also be useful in

explaining the impact of job-seeker

services on claimants. Benchmark data

would be available from previous State

reporting.

An easy-to-understand measure of the

success of labor exchange services, this

measure can be adjusted to consider

State and local economic conditions.

Most of the required data is easily

accessible. This duration-of-benefits

model could also be adapted to other

groups such as Welfare-to-Work

customers and Veterans, etc., and could

be expressed as a reduction in the

number of income transfer payments, as

a trust fund savings, or as a part of a

return on investment statement.

The second option is a measure of the

impact of Labor Exchange Services on

the Duration of Benefits Compensated

for UI Claimants Required to Search for

Work (Work-Test Claimants) who

receive Staff-Assisted Labor Exchange

services and enter employment

compared to the average duration-ofbenefits compensated to all claimants

who are required to search for work.

This outcome measure also seeks to

determine the impact of staff-assisted

labor exchange service on the duration

of benefits claimants. In this option, the

group to be studied will consist of UI

claimants who: (1) were not exempt

from an active work search; (2) received

staff-assisted labor exchange job-seeker

services; and (3) entered employment.

Staff-Assisted Labor Exchange

services for claimant job-seekers

includes job search assistance (JSA) and

a variety of facilitated self-help services

enumerated in the definitions above in

section IV. A, B, and C. Since the benefit

customer job-seeker’s identity is known,

they are usually registered in the labor

exchange system.

A claimant who is required to search

for work is a job-seeker who, as a

condition of receiving Unemployment

Insurance benefits, is required to

perform an active work search. This

criterion excludes UI benefit customers

who are on a temporary layoff, required

to secure employment through a union

agent, partially unemployed and receive

benefits under an approved partial

benefit program, or exempt from work

search requirements due to enrollment

in an approved training program.

This performance measure provides

an incentive to deliver high quality JSA

and other service to UI job seekers, to

begin the provision of service early in

the claim, to aggressively follow-up on

job referrals, to track UI job-seekers’

follow-through on work search plans,

and to determine when the benefit

claimant became re-employed.

Administrative data are available for

development of this measure of impact

from Unemployment Insurance Wage

Records (claims filed and average

duration data), ES records (referral and

service data elements in the current

ETA 9002 compliant reporting systems

and the Employment Security Systems

Institute systems), and local office data

reporting. In addition, federal and State

benchmark databases which can be used

to measure the State against its previous

accomplishments are available.

The measure can be used to show the

effectiveness of service when a

reduction in duration can be shown,

and would be useful to legislators,

employers, and policy makers. The

impact or reduction in the duration of

benefits performance measure could be:

(a) translated into a reduction in the

number of weeks compensated and a

calculation of Unemployment Insurance

Trust Fund savings using the State’s

average benefit rate, and

(b) adapted for other target groups

including Welfare-to-Work customers,

Veterans, etc. In this case the result

could be expressed as a reduction in the

number of monthly income transfer

payments, as a dollar savings in benefits

using average benefit rate data, or as

part of a return on investment

statement.

Questions

The Department specifically invites

interested persons to provide comments,

data, information and views concerning

the following:

Q14. Can a measure of UI Benefit

duration provide a meaningful measure

of the impact of labor exchange services

on those claimants who receive staffassisted services?

Q15. Would the measurement, and

comparison of the rate of UI claim
Federal Register / Vol. 63, No. 113 / Friday, June 12, 1998 / Notices 32575

exhaustion be a better system measure

of the labor exchange staff-assisted

service to UI benefit customers?

Q16. Considering the measures

proposed in this FRN, is there an

optimal mix of performance measures

which your State or agency would

suggest?

Finally, it should be noted that the

Workforce Development legislation

currently pending before the Congress

(H.R. 1385 and S. 1186) provides core

performance measures that include, in

addition to entered employment,

retention in employment six months

after entry and increases in earnings.

Under the proposed legislation, these

measures would apply to participants in

the Workforce Development activities

provided through the One-stop System,

including labor exchange activities.

We would appreciate receiving views

on the appropriateness of these

additional measures for measuring

public labor exchange programs.

Paperwork Reduction Act

The notice issued here is not subject

to the requirements of the Paperwork

Reduction Act of 1995 (44 U.S.C. 3501

et seq.) Because it contains no

‘‘collection of information’’ as defined

in 44 U.S.C. 3502(3).

Signed at Washington, D.C., this 8th day of

June 1998.

Raymond J. Uhalde,

Acting Assistant Secretary for Employment

and Training.

BILLING CODE 4510–30–P
32576 Federal Register / Vol. 63, No. 113 / Friday, June 12, 1998 / Notices
Federal Register / Vol. 63, No. 113 / Friday, June 12, 1998 / Notices 32577
32578 Federal Register / Vol. 63, No. 113 / Friday, June 12, 1998 / Notices

[FR Doc. 98–15754 Filed 6–11–98; 8:45 am]

BILLING CODE 4510–30–C