3702 Federal Register / Vol. 60, No. 11 / Wednesday, January 18, 1995 / Notices

DEPARTMENT OF LABOR

Employment and Training

Administration

Unemployment Insurance Program:

Unemployment Insurance Program

Letter No. 13–95 Providing an

Evaluation of a Field Test of

Alternative Unemployment Insurance

(UI) Performance Measures as Part of

the Performance Measurement Review

(PMR) Project

This Unemployment Insurance

Program Letter (UIPL) transmits to the

States, for comment, an ‘‘Interim

Evaluation Report’’ that provides the

results of a field test of alternative UI

performance measures in six State

Employment Security Agencies

(SESAs).

The Unemployment Insurance Service

(UIS), with the involvement of SESA

and Federal UI representatives, defined

alternative timeliness and quality

measures. The intent of the measures is

to promote improved service delivery

thereby strengthening the Federal-State

UI program. The alternative measures

were successfully field tested from July

1993 through September 1994.

This UIPL also provides the status of

other UI Federal oversight components

and their relationship to developing a

coordinated and integrated oversight

system, and efforts to concurrently

develop an integrated data validation

system to ensure the reliability of the

data.

The PMR project ‘‘Interim Evaluation

Report’’ was produced by the field test

evaluation contractor, Mathematica

Policy Research, Inc., of Princeton, New

Jersey. The report is an attachment to

the UIPL.

Because of the length of the report

and substantial cost to reproduce it in

this Federal Register notice, the report

is available only upon request.

Reviewers are invited to comment on

the report; comments will be considered

in developing final implementation

plans.

Copies of the ‘‘Interim Evaluation

Report’’ may be obtained by sending

written requests to the address below or

by telephoning William Coyne or James

Laham at (202) 219–5623 (this is not a

toll free number).

Reviewers are invited to provide their

comments within 60 days of the date of

this notice to: Mary Ann Wyrsch,

Director, Unemployment Insurance

Service, United States Department of

Labor, Employment and Training

Administration, Room S–4231 FPB,

Attn: TEUMC, 200 Constitution Avenue,

NW., Washington, DC 20210.

For Further Information Contact:

William Coyne, PMR Project Officer, or

James Laham, Unemployment Insurance

Program Specialist, at (202) 219–5623.

Signed at Washington, DC, this 4th day of

January 1995.

Doug Ross,

Assistant Secretary of Labor.

Date: January 5, 1995.

Directive: Unemployment Insurance Program

Letter No. 13–95

To: All State Employment Security Agencies

From: Mary Ann Wyrsch, Director,

Unemployment Insurance Service

Subject: Field Test Results, Plans for

Implementation of Unemployment

Insurance (UI) Performance Measurement

Review (PMR) Project, and Status of

Oversight Efforts.

1. Purpose. To convey the results of a sixState field test of performance measures

under the PMR Project and to provide States

with the opportunity to comment on the

interim final report.

2. References. Federal Register Notice

(FRN) No. 54 FR 2238; FRN No. 57 FR 126;

Unemployment Insurance Program Letter

(UIPL) No. 10–89; UIPL No. 13–91; and UIPL

No. 30–92.

3. Background. The Department of Labor

(DOL) initiated the PMR Project in 1988 to

examine, evaluate and improve performance

measurement in the Unemployment

Insurance Service oversight of State

Employment Security Agencies (SESAs). The

UI Quality Appraisal (QA) program had not

been subject to in-depth review since its

inception in the mid 1970s, so it did not take

into account the impact of changing

technologies in the delivery of UI services to

the public. At the same time the Department

recognized a need to integrate the

components of its Federal oversight system.

This UIPL provides current information

regarding PMR design, field test results, and

implementation plans.

4. PMR Design and Field Test. PMR was

designed in three phases. Phase I consisted

of the analysis of existing oversight measures

to determine their legislative basis and to

identify gaps or overlaps in measurement

areas, the development of new or revised

measures and the design for a field test of

selected measures.

With the involvement of State and Federal

UI representatives, the project defined eleven

timeliness measures and five quality

measures covering benefit payments,

adjudications, lower authority appeals, and

the Combined Wage Claim (CWC) program.

Some of the measures added program

categories and reporting intervals to existing

measures derived from the Unemployment

Insurance Required Reports system; some of

the measures looked at UI service areas

which had not formerly been measured.

In Phase II, six States successfully

conducted a 15-month field test of PMR

measures. The test ran from July 1993

through September 1994. UIS provided the

Field Test States—California, Illinois,

Kansas, Missouri, New Hampshire, and

Wisconsin—with the PMR data entry

applications on their existing UI system

platform. UIS National Office designed a data

base system to automate, as much as

possible, the entry, storage, and analysis of

the PMR data. The States developed

programs to extract PMR report data from

their mainframe-based unemployment

compensation records. States also developed

extract programs to select random samples

from a universe of eligible cases.

The first 3 months of the test were a pretest

of system capabilities and State

programming. Corrections were made during

that time with the help of Mathematica

Policy Research, Inc., (MPR) of Princeton,

New Jersey. MPR contracted with the State of

New Hampshire to assess State ability to

implement the PMR design, to validate the

data collected and to provide analysis and

recommendations for establishing levels of

performance.

The ‘‘Interim Evaluation Report’’ dated

December 5, 1994, contains an analysis of the

field test data by MPR and is included as an

attachment to this UIPL. The report shows

that the new performance measures and the

software developed could yield meaningful

and statistically valid information in a costeffective manner, except for the CWC quality

measures. States are invited to review and

comment on the attached ‘‘Interim Final

Report,’’ as comments will benefit the

development of final implementation plans.

In Phase III, PMR will be implemented

nationwide.

5. PMR Implementation. After resolving

comments from stakeholders responding to

this UIPL, UIS will provide States by May

1995, detailed implementation directions

including data systems specifications and an

implementation schedule.

6. Status Report on UI Oversight. DOL’s

oversight programs have evolved over time.

Each component represents a building block

in the Federal UI oversight structure,

beginning with Quality Appraisal (QA) and

Workload Validation (WLV), and growing to

include Cash Management, Benefits Quality

Control (BQC) and Revenue Quality Control

(RQC) programs. The Department recognized

a need to integrate parts of the oversight

system so that they would form a more

coherent whole, and has convened a

workgroup to examine the issue.

a. Performance Enhancement Work Group

(PEWG). The PEWG is a joint Federal/State

work group composed of representatives

from Federal National and Regional Offices,

and State Agencies formed in conjunction

with the Interstate Conference of

Employment Security Agencies. The PEWG

has been working for 13 months to revise

systems for measuring and fostering good

performance in the UI system. The group has

developed a set of partnership principles

which recognize mutual responsibility for the

UI system. Shared ownership is made real by

collaboratively involving State Agencies and

other stakeholders in developing policy and

procedures.

A basic tenet of the PEWG is that all

measures and standards developed for the UI

system should be implemented in such a way

as to foster continuous improvement, and

that the comprehensive oversight system

should recognize the respective interests of

each partner. The PEWG has reviewed the
Federal Register / Vol. 60, No. 11 / Wednesday, January 18, 1995 / Notices 3703

measures the PMR project field tested and

the measures that RQC is currently

implementing. In coming months the PEWG

will address ways to assign numeric

standards for performance to replace the

existing system of Desired Levels of

Achievement and Secretary’s Standards.

The PEWG has specifically addressed three

areas of PMR:

Detection Date. The PEWG has suggested

using the date the issue is detected as the

starting point for measuring and promptness

of adjudications. The PMR Field Test

measured adjudication promptness from the

week ending date of the first week affected

by the adjudication to the date that the

determination was issued.

Redetermination Promptness. The PEWG

has suggested that UIS not implement a

measure of redetermination promptness. The

attached ‘‘Interim Final Report’’ contains a

discussion of the field test results of this

measure.

Combined Wage Claims Quality Measures.

The PEWG has suggested that UIS not

implement measures of Combined Wage

Claims Quality. The attached ‘‘Interim Final

Report’’ contains a discussion of the field test

results of these measures.

A decision on these issues as with all the

other issues surrounding implementation of

the PMR measures, will be influenced by

comments from interested stakeholders in the

system who respond to this UIPL.

b. Major Changes from Quality Appraisal

(QA). AQ currently provides an annual look

at State performance. PMR results are

available with more frequency, that is,

monthly and quarterly rather than annually.

PMR differs from QA in deriving data for

performance measures from universe data

wherever this data is available on States’

automated systems. PMR measures also

contain more reporting intervals and program

categories. PMR universes are generally more

comprehensive than those used for QA,

including all first payments whether total or

partial, for example.

c. Data Validation. Data collected by DOL

are used to support a number of important

indicators that assess the nation’s economic

strength, the performance of States in the

administration of the UI program, and to

determine funding levels for State UI

program administration. To ensure that the

data reported by States adhere to DOL

definitions and are counted properly, DOL is

working toward a unified approach to data

validation in benefits and tax program areas.

Data validity is critical to sound UI program

administration at the State and Federal

levels.

Workload Validation dates back to 1976.

Although it has been modified at times to

adjust to changing economic and legislative

requirements, Workload Validation has not

been reviewed in its entirety since its

inception. Therefore, in 1992, UIS convened

an internal work group that recommended

two coordinated validation efforts: one effort

to validate benefits data used for PMR,

administrative financing, and certain

economic data, and the other to review tax

data used by the RQC program. Through the

joint efforts of PMR and RQC initiatives, DOL

is pursuing an integrated approach to data

validation.

d. Benefits Quality Control (BQC). BQC is

currently being evaluated by a work group

consisting of Federal National Office,

Regional Office, and State UI representatives.

The purpose of this evaluation work group is

to determine the effectiveness and/or

shortcomings of the entire BQC program to

inform PEWG deliberations on the redesign

of BQC. To that end, the work group has

canvassed UI Federal and State personnel

through the use of survey forms distributed

at various staffing levels in these

organizations. These results, along with

available historical data, and other existing

data, will be relied on heavily in producing

BQC program findings and conclusions, and

subsequently providing a report to the

PEWG. This final report will reflect on all

aspects of the program, both positives and

negatives, and will include

recommendations. The report is scheduled

for delivery by the end of January 1995.

f. Revenue Quality Control. The RQC

program assesses State tax functions using

systems reviews, acceptance samples and

method surveys, and a series of computed

measures derived from a required report,

ETA 581. RQC is now concluding a period

of voluntary implementation. New required

reporting on the ETA 581 will become

mandatory in the first quarter of Calendar

Year 1995; the balance of the RQC will

become mandatory effective January 1, 1996.

7. Action Required. SESA Administrators

are requested to:

a. Provide copies of this UIPL and

Attachment to appropriate staff for review

and comment.

b. Forward comments to the National

Office, with copies to the appropriate

Regional Office, within 30 days from the date

of the UIPL. Address comments and inquiries

to: Mr. William N. Coyne, PMR Project

Officer, U.S. Department of Labor, Room S–

4516 F.P.B., Attn: TEUMC, 200 Constitution

Avenue, NW., Washington, DC 20210, FAX

Number: 202–219–8506.

Attachment. ‘‘Performance Measurement

Review Interim Evaluation Report’’

(Mathematica Policy Research, Inc.,

December 5, 1994).

[FR Doc. 95–1124 Filed 1–17–95; 8:45 am]

BILLING CODE 4510–30–M