OVERVIEW OF THE CES REDESIGN

George S. Werking, Bureau of Labor Statistics

Bureau of Labor Statistics, 2 Massachusetts Avenue, NE, Suite 4840, Washington, D.C. 20212

Key Words: Establishment Survey, Sample

Redesign

I. Introduction

A. Employment Statistics in the United States

On the first Friday of each month, the U.S. Bureau

of Labor Statistics releases data on the United States'

employment situation for the previous month. On

release day, the Commissioner of Labor Statistics

appears before the Joint Economic Committee of

Congress and provides a detailed analysis of the

current month's data and trends; at the same time, the

data are made available to the news media and the

financial and business communities. This closely

watched set of statistics is the earliest indicator

available on the previous month's economic activity

and is used as a major gauge of the health of the U.S.

economy. The data in the release cover employment,

hours, and earnings by detailed industry which are

derived from the Bureau's 400,000 unit monthly

establishment survey - the Current Employment

Statistics (CES) survey - along with labor force and

unemployment data which are derived from the

Bureau's 60,000 unit household survey—the Current

Population Survey (CPS).

The establishment survey data have many

important economic uses. Due to the CES survey's size

and timeliness in conjunction with the importance of

the basic payroll statistics which it collects, the CES

monthly estimates are not only used as principal

economic indicators but they also are included in the

development of many of the Nation's other major

economic indicators including: Personal Income

estimates of the National Income and Product

Accounts, the Index of Leading Economic Indicators,

the Index of Coincident Indicators, the Industrial

Production Index, Real Earnings measures, and

Productivity measures. The CES data are not only

widely used on a monthly bases due to its timeliness

and substantial industry detail, but the CES also

provides many continuous monthly industry timeseries

which span over 50 years.

B. Current Employment Statistics Survey

The CES survey, with 400,000 units, is the largest

monthly sample survey in the United States. It is

conducted by the Bureau as a Federal-State cooperative

program under which the Bureau specifies the survey's

sample design and operational procedures while the

States conduct data collection and edit reconciliation

activities. The Bureau produces and publishes

extensive monthly industry detail at the 2, 3, and 4-

digit industry levels for the Nation as a whole while

each State produces monthly State and area (270

Metropolitan Statistical Areas) estimates.

Once a year, complete universe employment

counts for the previous year become available from the

Unemployment Insurance tax records; these counts are

used to annually benchmark (realign) the CES sample

estimates to these universe counts. The annual

benchmark process yields more accurate current

monthly estimates along with providing an annual

estimate of overall survey error. The average

difference in the CES final sample estimate versus the

complete universe count over the past 5 years is under

0.3%. While the CES monthly estimates have been

relatively accurate in tracking the universe counts,

there have always been concerns over the statistical

foundation of the CES program.

II. Background

The CES program was established in the 1920s

and thus predated the development of probability

sampling theory. During the 1950s, 1960s, and 1970s

as most sample surveys were converted to probabilitybased designs, the CES experienced little change to its

fundamental operations and procedures.

Due to the need to support monthly industry detail

at both the National and State level, the CES sample

coverage has been approximately 40% of all

employment in the U.S. since the 1950s. Essentially,

the CES has been operated as an exceptionally large

quota sample where an optimum allocation was used to

establish a required sample size for each sampling cell

(State by detailed industry by size class). Ongoing

solicitation for replacement units was carried out each

month to offset normal sample attrition. As in most

establishment surveys, the coverage of birth units (i.e.,

new business establishment) was limited and, for the

CES survey, an elementary modeling procedure was

put in place to attempt to account for the presumed

missing employment from these units.

Over the years, there have been both formal

outside reviews (Gordon Commission 1960 and

Levitan Commission 1980) and also internal reviews of

the CES program and while in most reviews it was felt

that probability sampling would be useful, they stopped
short of formally recommending the implementation of

a probability design for CES. There were a number of

cost-benefit factors which supported such a position.

• First, it was felt that the large sample

coverage, at 40%, probably produced a fairly accurate

picture of the basic population and the changes

occurring within it.

• Secondly, the CES used a procedure to

annually benchmark to the full population counts of

employment from the Unemployment Insurance Tax

system and this would realign any error in the sample

estimates which occurred over the year.

• Thirdly, while the modeling procedure for

births (known as bias adjustment) was a very simplistic

approach to a difficult measurement issue, it appeared

to be working reasonably well because the annual

revision to the population counts only produced small

adjustments—usually well within a + .5% range.

• Finally, the observed annual benchmark

revisions were believed to be primarily associated with

the problem of sampling and measuring new business

births and these were inherent frame limitations which

would not be resolved by a probability design.

In addition, to the cost-benefit measurement

factors there were also a number of

practical/operational issues.

• The CES survey was a Federal/State

cooperative program where all sampling and collection

activities were done under contract by the States and

changes in the process, particularly major changes,

were quite costly. A change in the sampling and

estimation procedures would require a full rewrite of

50 State processing systems.

• Correspondingly, monthly collection was done

by mail with mail solicitation yielding only about a

30% cooperation rate for the voluntary CES survey. A

change to a probability design would require not only

major systems changes but also a much more staffintensive effort to achieve an acceptable response

rate—this would also require major increases in

program funding.

• Finally, until 1983 the Bureau did not have

responsibility for program funding or State contracts,

the Bureau's role was limited to providing

recommended methods and procedures for the ongoing

program.

Thus, prior to the 1980s there was little empirical

evidence to support major program changes or funding

increases. In 1984, BLS was given responsibility for

the program funds for the Federal/State cooperative

programs and began funding research and

improvement projects. Several of the research projects

subsequently played a pivotal role in preparing for a

major redesign of the CES survey.

III. CES Research and the ASA Panel 

Recommendations

During the 1980s, several major survey

infrastructure improvements were made to the CES

program which provided the foundations for

addressing the issue of implementing a probability

design in the CES program. After BLS received

responsibility for program funding in 1984, a major

effort was launched to develop and export a single CES

processing system to all States. The conversion of all

States to this new system was completed in 1993. The

use of a single exportable State processing system

provided a cost-effective way to change major

processing system modules (e.g., the sampling module)

in a timely manner.

The Bureau also made major changes to the CES

program's data collection procedures. The CES

program had been conducted as a mail survey for over

half a century, however, it was felt that the

decentralized State mail solicitation would never be

able to achieve acceptable response rates for a monthly

probability design. Beginning in 1983 and spanning a

7-year period, BLS conducted extensive research into

alternative data collection approaches including:

CATI, TDE, Voice Recognition, FAX, and EDI

collection. In 1991, BLS received funding to

implement these new collection methods and by 1996,

over 75% of the survey's 400,000 monthly respondents

were reporting under these new automated methods.

In addition to automating much of the ongoing

monthly collection, BLS also created two regional CES

CATI collection centers. These centers are now being

set up to conduct CATI solicitation for the new

probability design.

Another major automation area for the Bureau in

the early 1990s was the creation of a linked

longitudinal universe file of establishment microdata.

Systems were developed for matching the quarterly UI

universe files and creating a longitudinal establishment

database. A 12-month universe file for the 1993

benchmark year (i.e., April 1992 through March 1993)

was extracted from the database for subsequent CES

research on the primary causes of the 1993 benchmark

revision.

In the initial phase of research, the population was

divided into birth units, death units, and continuing

units with a tabulation of over-the-year change from

each component. The CES survey had experienced an

upward revision of 263,000, however, the universe data
showed an offsetting effect for the net birth/death

employment—suggesting that most of the CES survey's

263,000 underestimate was in the continuing

population. This result contradicted earlier beliefs that

most of the underestimation error in the CES was due

to the birth unit measurement problems. To study the

measurement accuracy of the existing CES sample

versus a probability sample, the existing CES sample

was matched and flagged on the universe file and then

an additional 30 independent, identically distributed

(to the CES) random samples were drawn from the

population of continuing units—the results showed

that the current CES sample exhibited a strong

underestimation bias. Further research suggested that

the cause was an age-of-firm bias. Under the CES

survey's quota sample approach, firm's agreeing to

participate could stay in the sample indefinitely and

while there was some replenishment for sample

attrition, the net effect was that the average age of units

in the sample was 9 years older than that of the

population. Additionally, research using the

population file on growth rates by age-of-firm showed

that younger firms exhibited accelerated growth rates

compared to older firms. In summary, this new

research suggested that the CES measurement issues

were not strictly birth measurement issues but were

also significantly affected by the existing nonprobability design. In particular, the research results

provided empirical evidence that even with an

extensive population coverage of 40% of all

employment, a non-random sample can be subject to a

quite significant measurement bias.

At the same time that this research was being

concluded, another major Bureau-sponsored technical

review of the CES program was completed. In 1993,

the Bureau had commissioned the ASA to form an

expert panel to review the CES program and provide a

"Research Agenda" to guide future CES program

improvements. Of the 26 recommendations in the

1994 ASA report, the "first priority" recommendation

was the implementation of a probability design for the

CES survey.

In June 1995, following the conclusion of the

Bureau's internal research effort and the final ASA

report, BLS announced plans for a full redesign of the

CES survey.

IV. CES Redesign Research Phase

The Bureau's CES Redesign proposal called for a

2-year research phase to be followed by a 1-year

production test of systems and procedures and then a

phased-in implementation of the new design.

A redesign research committee was formed

consisting of BLS national and regional office staff,

State staff, and outside consultants. Eleven States

participated on the committee along with consultants

from Westat, NORC, and Michigan Survey Research

Center. The committee met quarterly over the 2-year

period and evaluated the results from a series of

research studies. A brief summary of each research

area is described below, the full technical papers and

results are discussed in this and one other ASA

session:

• User Needs: The initial committee meetings

focused on user needs and survey products at the

national, State, and area level. The results from these

meetings provided guidelines for developing the

basic design. A separate effort studying the feasibility

of collecting "all employee" earnings is still underway.

• Birth Research: This research focused on the

feasibility of constructing a timely birth frame by

obtaining immediate access, at the State level, to

employer applications for new UI account numbers.

Frames were constructed, sampled and interviews were

conducted. The results showed mixed success in terms

of the timeliness, consistency, and accuracy of the

individual State frames. In addition to studying the

feasibility of direct sampling and measurement of the

birth population, research was also conducted on

profiling the birth and death changes over time in the

population and the feasibility of modeling the net

effect. This work has produced favorable results.

While capturing the initial employment of new

units when they first file for UI coverage is important,

the earlier research suggested that it was equally

important to move these units into the design as soon

as possible so their employment movements could be

reflected in the current monthly estimates—this

research had show that younger units grow far faster

than older units. Plans are being made to implement

quarterly sample maintenance procedures to address

this issue.

• Sample Design Research: Research focused

on developing and refining State designs for the eleven

States on the research committee. Initial test

simulations used 12-month linked microdata universe

files and as refinements were made, tests were

expanded to use 60-month files to study design effects

over time. Design options were studied in terms of

their effect on State level estimates, detailed industry

estimates, and MSA estimates; additional

consideration was given to the ability to make small

area estimates.

• Estimator Research: This research focused

on editing/outlier detection, reweighting for atypical

units, imputation options, alternative forms of the
estimator, and the appropriate cell level for applying

population controls. In addition to developing an

employment estimator, separate estimator research was

conducted for the hours and earnings data elements

where unlike the employment data, universe counts do

not exist.

• Solicitation Research: In the redesign, CES

solicitation will be conducted using CATI procedures.

Research was conducted at the two BLS regional

collection centers and at the Michigan Research Center

facility. The research looked at cost-effective methods

of sample refinement, the initial contact materials, the

correct point of entry into the firm, alternative version

of the questionnaire, profiling employer reluctance

issues, and refusal conversion techniques. A final

solicitation protocol is being developed and tested in

the BLS collection centers.

In June of 1997, the Bureau announced the

completion of the research phase and the beginning of

the Production Test/Implementation phase. The results

of the research phase are being documented in a set of

papers being presented at the 1997 Annual Meeting of

the ASA.

V. CES Redesign Production Test and

Implementation Phase

The research phase of the Redesign applied

sampling theory principles to fixed universe files and

through a series of simulations, identified design and

estimator properties which satisfied the survey's

publications goals; the production test phase will now

test the proposed design in a live survey environment.

A sample for the Wholesale Trade industry has

been selected under the new design and is now being

solicited for ongoing monthly collection. The CES

Production Test is being conducted as an independent

parallel sample to the current CES sample. During the

2-year production test, monthly estimates from the new

parallel Wholesale Trade sample will be constructed

and evaluated for conformance to both the design

expectations and the quarterly universe counts.

The initial goals of the Production Test are to field

test and refine the new survey operations and

procedures; to validate systems and data flows; to

provide unit cost and workload measures; and to

identify streamlining options to reduce cost, workload,

and potential error sources. Solicitation and ongoing

response rates will be closely monitored along with the

accuracy of the microdata being collected. Measures of

sampling error will be calculated for levels and overthe-month change and compared to design

expectations.

As quarterly universe counts become available (on

a 9-month lag basis), the monthly sample estimates

will be compared to the corresponding universe counts.

Differences will be studied in terms of disaggregated

error components for births, deaths, and continuing

units. The imputation procedures will also be

evaluated against the monthly population employment

data reported under the mandatory UI tax system. The

evaluation phase will also include an assessment of the

non-economic effects on monthly change estimates due

to survey activities such as quarterly sample

maintenance and imputation updates. As a sufficient

number of monthly estimates become available for the

major publication cells, the seasonal pattern from the

new design will be evaluated against the corresponding

universe series and the current CES sample.

If the results from the evaluation phase are

acceptable, the new Wholesale Trade sample will be

put into ongoing production in June 1999 and the old

sample for the industry will be discontinued.

Subsequently, each major industry division will be

phased-in after a similar period of parallel estimation

and evaluation.

REFERENCES

Butani, S., G. Stamas, and J.M. Brick (1997), "Sample

Redesign for the Current Employment Statistics

Survey", Proceedings of the Section on Survey

Research Methods, American Statistical Association,

in print.

Clayton, Richard L. (1997), "Implementing Fully

Electronic Data Collection in the Redesign of the

Current Employment Statistics Survey", Proceedings

of the Section on Survey Research Methods, American

Statistical Association, in print.

Getz, P.M., J. Kropf, and S. Strifas (1997), "Measuring

the Contribution of Business Births and Deaths to

Overall Employment Movements", Proceedings of the

Section on Survey Research Methods, American

Statistical Association, in print.

Groves, R.M., D.I. Cantor., K. McGonagle, and J. Van

Hoewyk (1997), "Research Investigations in Gaining

Participation from Sample Firms in the Current

Employment Statistics Program", Proceedings of the

Section on Survey Research Methods, American

Statistical Association, in print.

Harter, R.M., and K.M. Wolter (1997), "Estimators for

the Bureau of Labor Statistics' Current Employment

Statistics Program", Proceedings of the Section on
Survey Research Methods, American Statistical

Association, in print.

Stamas, G., K. Levin, D.I. Cantor, and K.L.

Goldenberg (1997), "Sampling for Employment at New

Establishments in a Monthly Business Survey",

Proceedings of the Section on Survey Research

Methods, American Statistical Association, in print.

Werking, George S. (1994), "Establishment Surveys:

Designing the Survey Operations of the Future",

Proceedings of the Section on Survey Research

Methods, American Statistical Association, Invited

Panel on the Future of Establishment Surveys, pp. 163-

169.

Werking, G.S., and R.L. Clayton, "Enhancing Data

Quality Through the Use of Mixed Mode Collection",

Survey Methodology, June 1991, 17, No. 1, pp. 3-14.

Werking, G.S., A.R. Tupek, and R.L. Clayton (1988),

"CATI and Touchtone Self-Response Applications for

Establishment Surveys", Journal of Official Statistics,

4, No. 4, 1988, pp. 349-362.

West, S.A., D.-T. Kratzke, and P. Grden (1997),

"Estimators for Average Hourly Earnings and Average

Weekly Hours for the Current Employment Statistics

Survey", Proceedings of the Section on Research

Survey Methods, American Statistical Association, in

print.