Examining the Discrepancy in Employment Growth

between the CPS and the CES

Thomas Nardone, Mary Bowler, and Jurgen Kropf

Bureau of Labor Statistics

Katie Kirkland

University of Kentucky Center for Poverty Research,

formerly of the Bureau of Labor Statistics

Signe Wetrogan

Bureau of the Census

This paper has been prepared for presentation to the Federal Economic Statistics

Advisory Committee (FESAC) on October 17, 2003. It represents work in progress and

does not represent any agency’s final positions on issues addressed. The FESAC is a

Federal Advisory Committee sponsored jointly by the Bureau of Labor Statistics of the

U.S. Department of Labor and by the Bureau of Economic Analysis and the Bureau of

the Census of the U.S. Department of Commerce.
2

Examining the discrepancy in employment growth between the CPS and the CES

BLS has two monthly surveys that provide information on current labor market

conditions—the Current Employment Statistics (CES) or establishment survey and the

Current Population Survey (CPS) or household survey. Both surveys provide an estimate

of employment in the United States each month. The CPS also provides information on

the number of unemployed and persons not in the labor force.

The estimates of over-the-month employment change from these two surveys usually do

not match in size or even direction. This is not surprising given the many differences in

the surveys, including sample size, estimation procedures, coverage, and definitions.

Over longer periods of time, however, one would expect the separate measures of

employment to track somewhat closely, particularly after accounting for coverage

differences. Historically that has been the case, at least after the revisions that are part of

each survey’s normal estimation procedures.1 As can be seen in chart 1, the surveys do

show the same overall trend in employment for the last half century. Starting in the late

1980s, however, and continuing throughout the 1990s, the establishment survey showed a

greater rate of employment growth than the household survey. This is true even when the

comparison is restricted to the more comparable nonagricultural wage and salary

employment, the universe for the establishment survey. The recent introduction of new

population controls for the household survey reduced the gap, but did not eliminate it.

This paper reviews findings from BLS research into the growth of the discrepancy in the

two surveys’ employment estimates during the 1990s. In examining this discrepancy, we

consider the impact or possible impact of several factors including: universe and concept

differences in the surveys, population controls used in the CPS, undercoverage of certain

population groups, differences in reference periods, and reporting issues. The

contribution of some of these factors to the gap can be demonstrated. For others, their

possible impact can only be roughly gauged from existing data.
3

Differences in Surveys

When studying the CES-CPS discrepancy it is important to bear in mind that their

employment estimates differ because these are two distinct surveys that serve two

different purposes. The CES survey or establishment survey produces an estimate of

nonfarm payroll employment. It is a survey of nonagricultural employers (including

government) that is designed to provide data on the employment, hours, and earnings of

workers on nonfarm payrolls.2 In contrast, the CPS or household survey provides

statistics on the employment status of the civilian noninstitutional population and related

demographic, social, and economic characteristics.3

The surveys differ in their universes, concepts of employment, reference periods, and

other methodologies. Any of these differences are potential sources of the discrepancy in

employment growth. Thus, understanding these differences is key to attempts to explain

the discrepancy. The following section examines the mechanisms through which each

potential factor might have contributed to the growth in the employment discrepancy

during the 1990s. This process rules out some potential factors as the explanation of the

employment growth gap.

Universe and Conceptual Differences

CES nonfarm payroll employment is an estimate of the total number of persons employed

full or part time in nonfarm establishments (including government4

) during the pay period

that includes the 12th of the month. Basically, persons are considered employed if they

receive pay for any part of the reference pay period. CES employment therefore includes

workers on an establishment payroll who are on paid sick leave, holiday or vacation, or

who work during only part of the specified pay period, even if they are unemployed or on

strike during the rest of the pay period.
4

In the CES, persons on the payroll of more than one establishment during the pay period

are counted in each establishment that reports them—whether the duplication is due to

job turnover or multiple jobholding.

Persons are not considered employed if they receive no pay at all for the pay period.

Since the unincorporated self-employed and unpaid family workers do not have the status

of paid employees, they are excluded. Also excluded from the employment estimate are

private household workers, persons who are on layoff, leave without pay, or strike for the

entire pay period and persons who were hired but have not yet started work during the

pay period.

The CPS uses a much broader definition of employment. It includes persons working in

agriculture and nonwage and salary workers, that is, the unincorporated self employed

and at least some unpaid workers in family businesses.

The CPS classifies people as employed if, during the calendar week that includes the 12th

of the month, they did any work at all (at least 1 hour) as paid employees, worked in their

own business, profession, or on their own farm for profit, or worked without pay at least

15 hours in a family business or farm. People also are counted as employed if they were

temporarily absent from their jobs because of vacation, illness, bad weather, childcare

problems, maternity or paternity leave, labor-management dispute, job training, civic or

military duty, or other family or personal reasons--whether or not they were paid or were

seeking other jobs during their time off.

The CPS counts each employed person only once—even if someone holds more than one

job. For purposes of assigning class of worker, occupation, and industry classification,

multiple jobholders are classified and counted in the job at which they usually worked the

greatest number of hours. Included in the total number of the employed are employed

citizens of foreign countries who are temporarily in the U.S. but not living on the

premises of an embassy. Excluded are persons whose only activity consisted of work
5

around their own house (such as painting, repairing, or housework) or volunteer work for

religious, charitable, and other organizations.

It is possible to use information from the CPS to adjust its estimate of employment to

something more akin to that from the CES. Many analysts routinely perform such

reconciliation calculations when comparing the surveys. These differences in the

employment definitions are a logical place to start in any examination of the differences

in employment growth trends.

Table 1 shows the CES and various CPS time series used in reconciliation calculations.

The selection of the time period 1994 to 2000 to illustrate the reconciliation calculations

and growth in the discrepancy was somewhat arbitrary. We choose 1994 as the starting

point because that was when the multiple jobholding data series used became regularly

available. The year 2000 was selected as the end point because it was the last full year of

the economic expansion, the period in which CES employment growth significantly

outpaced the CPS. In addition, the 2000 CPS data are shown before the recent Census

2000-based population control revision so that we can illustrate the growth gap between

the surveys before and after the revision. Annual averages were used instead of monthly

data because some of the components used in the reconciliation are not seasonally

adjusted or not available on a monthly basis.

Much of the conceptual difference between the two surveys is removed by comparing

CPS nonagricultural wage and salary employment to the CES estimate; that is, by

removing agricultural employment, the unincorporated self employed, and unpaid family

workers (see steps 1-3 of table 1). Using other data from the CPS, it is possible to further

reconcile the CPS employment concept with that of the CES. As shown in steps 4 and 5

of table 1, we also remove from the CPS employment estimates of private household

workers and persons who did not receive pay during the reference period. Then we add

in multiple jobholders with secondary nonagricultural wage and salary jobs (step 6).5

(Table 2 shows annual average employment for CES, CPS, and adjusted CPS for each

year from 1994 to 2000.)
6

Even after the adjustment for the surveys’ quantifiable differences, the cumulative

discrepancy remains substantial. Even if an additional adjustment is made for multiple

jobholders who hold more than one extra job, the growth gap persists.6 This suggests that

the discrepancy in employment growth may have been caused by non-quantifiable

conceptual or universe factors (which we will discuss in this section) or by other factors

(which will be discussed in following sections).

We examined several potential sources of discrepancy that stem from universe and

conceptual differences and are not routinely or readily quantified. These include

secondary civilian jobs of active duty military personnel, the CPS age minimum,

employment among the institutional population, and foreign commuters employed by

U.S. companies. Although the data to examine these possible affects are limited, the

evidence that is available suggests these factors are not the chief sources of the gap in

employment growth.

Secondary civilian jobs of active duty military personnel

The CPS sample reflects the civilian noninstitutional population, so active duty military

personnel are ineligible to participate in the CPS. However, active duty military

personnel holding secondary civilian nonfarm payroll jobs are counted in the CES for

their civilian jobs.

If the number of civilian jobs held by active duty military personnel had increased during

the 1990s, this would have increased CES but not CPS employment and therefore would

have contributed to the discrepancy in employment growth. Since military personnel

cannot be identified separately in the CES estimates, the effect of their civilian

jobholding on the discrepancy cannot be quantified. However, the level of the

Department of Defense active duty military personnel fell during the 1990s.7 Only the

most radical assumptions about changes in secondary employment of the active duty

military (that none were employed before and nearly all are now) would explain a

significant proportion of the employment growth gap.
7

CPS age minimum

To be eligible to participate in the CPS, individuals must be 15 years of age or over.

However, CPS data are published only for persons 16 years of age and over, since

younger persons are limited in their work activities by child labor laws and compulsory

school attendance. The CES is not limited by age, so the CPS age minimum is a potential

factor in the employment discrepancy’s growth if the number of workers under age 16

increased.

Unpublished CPS data for 15-year-old nonagricultural wage and salary workers,8 (the

closest estimate of 15-year-olds that are counted in the CES) however, indicate that these

workers were an insignificant factor in the discrepancy’s growth. From 1994 to 2000, the

annual average level of 15-year-old nonagricultural wage and salary workers increased

only slightly. (See table 1.) The change amounted to a very small portion of the growth

in the discrepancy in CES and adjusted CPS employment in the mid-to-late 1990s.

No data are available to assess the direct effect of workers under 15, but they should not

be a large hidden factor in the discrepancy in any case. Even if their employment had

increased during the recent expansion, it seems unlikely to have had any greater impact

on the discrepancy’s growth than the number of employed 15-year-olds.

Commuters to the U.S.

Another difference in the surveys’ employment universes is the CPS exclusion of

workers commuting to work into the U.S. from Mexico or Canada. The CPS estimates

include only persons who live in the U.S. However, daily commuters from foreign

countries are counted in the CES estimates if they appear on U.S. business

establishments’ payroll records. Although, conversely, the CES will not capture U.S.

residents who work in Canada and Mexico, these workers are presumed to be less

numerous than those coming into the U.S. Therefore, the net commuter effect is assumed

to boost the CES employment estimates.
8

Since foreign commuters cannot be identified in the CES, their effect on the discrepancy

cannot be quantified. Nonetheless, BLS has attempted to investigate this issue, looking at

the difference between CES employment and CPS nonagricultural wage and salary

employment estimates from 1994 to 2000 at the state level.9 If daily commuters

contributed significantly to the employment gap, the differences in the surveys’ estimates

should have been pronounced in states that border Mexico and/or Canada.

This limited analysis, however, did not provide any clear-cut answers. The primary

Mexican border states—Texas, Arizona, and California—had significant discrepancies in

growth between the CES and CPS employment measures. Texas, for instance, had the

largest discrepancy of any state, and Arizona exhibited a fairly substantial gap relative to

its size. California’s difference, on the other hand, was not exceptionally large relative to

its size. Among the Canadian border states, only New York and Minnesota had notable

discrepancies although in the case of New York, the difference relative to the size of the

state was not exceptional. In addition, a number of non-border states exhibited

substantial gaps: Florida, Massachusetts, Ohio, North Carolina, Pennsylvania, Virginia,

Maryland, and Indiana. No consistent pattern of discrepancy existed among either

Mexican or Canadian border states, nor were there any obvious common factors among

the states that registered the largest differences. That being said, many of the states with

large discrepancies appear to be ones that may have seen a significant influx of

undocumented aliens during the past decade. (Undocumented workers are discussed in a

later section.) (See table 3.)

In addition to comparing state-level estimates, we attempted to gauge the potential effect

foreign commuters had by looking at overall growth in seven metropolitan areas near

border crossings: San Diego, El Paso, Laredo, McAllen, Brownsville, Detroit, and

Buffalo. Using CES annual average employment estimates for 1994-2000, total

employment growth for these areas was 611,000 over the time period. Even assuming a

large proportion of this growth was due to foreign commuters, it would resolve little of

the cumulative discrepancy in two surveys from the mid-to-late 1990s. Consequently,

while foreign commuters may contribute to the employment survey differences in some
9

states, they are not likely to play a major role in the overall discrepancy between the

national CES and CPS estimates. (See table 4.)

Since the conceptual differences between the surveys do not appear to account for much

of the gap in employment, other factors need to be considered. One characteristic of the

gap is useful in determining what other factors are likely to be reasonable explanations.

As can be seen from chart 1, the gap was not the result of a one-time break. Rather, the

size of the discrepancy widened over time. This suggests that an explanation, or

explanations, are phenomena that became more prevalent during the decade. One factor

that fits this criterion and which can be demonstrated to have contributed to the gap was

the understatement of population controls used in CPS estimation.

Population Controls

The distribution of the population selected for the monthly CPS sample may differ

somewhat, by chance, from the population as a whole in such characteristics as age, sex,

race, ethnicity, and state of residence. Because these characteristics are closely correlated

with labor force participation and other principal measurements made from the sample,

the survey estimates can be substantially improved when weighted appropriately by the

known distribution of these population characteristics

Monthly independent population controls developed by the Population Division of the

U.S. Census Bureau provide the base to which the monthly CPS results are applied to

estimate current employment levels. If the controls understate the level of population

growth, the growth in CPS employment also will be understated. The intercensal

population projections are adjusted every 10 years to reflect new information from the

decennial census. Following the 1980 and 1990 censuses, changes to population

estimates led to sizable increases in CPS employment and the narrowing of past

discrepancies in CPS and CES employment growth.10 Indeed, when population controls

based on information of Census 2000 were introduced retroactively to the CPS, they

raised the level of CPS employment in 2000 by 1.7 million. These revised estimates
10

significantly narrowed the cumulative gap in employment growth between the CES and

CPS from the 1990s. (See table 1.) Moreover, an estimated higher level of population

growth from 2000 to 2002 increased CPS employment by an additional 533,000 over

previous estimates. Finally, an adjustment to intercensal population estimates made in

January 2003 boosted CPS employment by 576,000. Since population controls can have

such a large impact on CPS employment estimates, the following section describes their

derivation during the 1990s and changes made to their estimation following Census 2000.

The monthly independent population controls for the nation by age, sex, race, and

Hispanic origin and for states by selected age groups and broad race categories are

developed using the basic demographic accounting formula whereby the population from

the latest decennial data are updated using data on the components of population change

– births, deaths, and net international migration; with net internal migration as an

additional component in the state population estimates. The estimates begin with the

latest decennial census as the base and incorporate the latest available information on

births and deaths along with the latest estimates of net international migration. Data on

births and deaths are supplied by the National Center for Health Statistics. The net

international migration component in the population estimates includes: (1) legal

migration to the U.S., (2) emigration of foreign born and native people from the U.S., (3)

net movement between the U.S. and Puerto Rico, (4) estimates of temporary migration,

and (5) estimates of net residual foreign-born population, which include unauthorized

migration.

Following the 1990 decennial census, BLS chose to use CPS population controls that

used as its base the 1990 Census with adjustments obtained from the Post-Enumeration

Survey (PES), a post-census quality control check that assessed the estimated

undercount.11 The Census Bureau's Population Estimates Program further modified the

1990 Census race data for use in the production of post-1990 population estimates and

projections. Because of needs to have census data comparable with the reporting

categories used by state and local agencies and for compiling other administrative data

used in producing population estimates and projections, the Census Bureau developed a
11

procedure to assign an Office of Management and Budget (OMB) race category--White;

Black or African American; American Indian and Alaska Native; Asian; and Native

Hawaiian and Other Pacific Islander--to those who reported "Some other race" in the

census.

The adjusted and modified base data were updated throughout the decade using the latest

information on births and deaths available from the National Center for Health Statistics.

During the 1990s, the core data source for the estimation of international migration was

the Immigration and Naturalization Service (INS) public use immigrant file. This annual

file contains records for citizens of foreign countries immigrating or establishing legal

permanent residence in the U.S. and accounts for the majority of international migration

to the U.S. However, the file contains no information on emigration, undocumented

immigration, or nonimmigrant moves into the country. Research conducted at the Census

Bureau (Robinson, 1994) produced an allowance of 225,000 net unauthorized migrants

per year. The intercensal estimates developed throughout the 1990s incorporated this

annual allowance as a constant.

The remaining category of foreign-born migration to the United States, not included in

the administrative immigrant data from INS is the flow of legal temporary residents;

persons who reside in the country long enough to consider themselves ‘‘usual residents’’

while they are here, but who do not have immigrant visas. These include foreign students,

scholars, some business persons (those who establish residence), and some professionals

who are provided a special allowance to work. INS data on nonimmigrants would provide

records of admission for temporary visa holders; however, there would be no reliable data

source for their departures. The stock of this category of foreign-born persons

enumerated in the 1990 census was estimated at 488,000. The intercensal estimates

developed during the 1990s assumed this number to remain constant throughout the

decade. As noted earlier, the administrative data from the INS contained no data on the

emigration of legal permanent residents of the United States to abroad. The estimates

developed throughout the 1990s assume a constant emigration of 222,000 per year, of
12

whom 195,000 are foreign-born (Ahmed and Robinson, 1994) and 27,000 are native born

emigrants.

A final class of international migration that can only be estimated roughly as a net flow is

the movement of persons from Puerto Rico and the outlying areas to the United States.

No definitive administrative data exist for measuring volume, direction, or balance of

these migration flows. The migratory balance from all areas except Puerto Rico is

assumed to be zero. For Puerto Rico, we assumed an allowance of roughly 7,000 net

migration (arrivals minus departures) per year.

After each decennial point, the intercensal estimates are rebenchmarked to use the new

decennial census population as its base and start the estimation forward with estimates of

the components of population change. As part of the evaluation process, the Census

Bureau Population Division compared the set of intercensal estimates developed for the

weighting of the April 2000 CPS (developed in April 2000 using the 1990 census

adjusted as the base population) to Census 2000 consistent data. The Census 2000 count

of the total civilian noninstitutional population was 2.7 million higher than the 1990

based estimate. In particular, the Census 2000 count of Hispanic males was much larger

than the estimated number. In reviewing the estimates developed during the 1990s, the

Population Division has speculated that the difference between the 1990 based estimate

and the Census 2000 count (error of closure) is due to a combination of factors including

the differences in coverage (a difference that occurred even with the adjustment in 1990

for undercount) and a failure in the estimates to account for a larger number of residual

foreign born population.12

Additional studies carried out by the Census Bureau Population Division as part of the

estimates evaluation indicated that the estimates of unauthorized migrants that were used

in the 1990 based intercensal estimates were too low. The evaluations indicated that the

residual foreign born population increased by about 5 million during the 1990 to 2000

decade rather than the 2.25 million (10 x 225,000) assumed in the 1990 based estimates.

The results of the evaluations also confirmed the Population Division’s reservations that
13

an assumption concerning the annual flow of the residual foreign born population,

developed at the onset of the decade, could be used unchanged throughout the decade.

In response to these concerns, the Population Division has undertaken an aggressive

project to study and develop estimates of the size, characteristics and impact of the

foreign born population.

Beginning in January 2003, the independent population controls used to weight the CPS

shifted to a Census 2000 base. Consistent with a December 2001 decision and reaffirmed

again in March 2003, the Census Bureau decided that the intercensal population estimates

would be based on the official Census 2000 results rather than on a population base

adjusted using revised estimates of census coverage.

Thus, for the post 2000 period, the independent estimates used to weight the Current

Population Survey begin with the Census 2000 results as enumerated. Consistent with

the modifications introduced in 1990, the Census 2000 data by race have been modified

to assign each of the persons in the "Some other race" category in the census to an OMB

race category. The modified race data include 31 race groups which are the single and

multiple race combinations of the OMB race categories: White; Black or African

American; American Indian and Alaska Native; Asian; and Native Hawaiian and Other

Pacific Islander. Identical to the 1990 decade, the monthly independent population

controls for the nation by age, sex, race, and Hispanic origin and for states by selected

age groups and broad race categories are developed using the basic demographic

accounting formula whereby the population from the latest decennial data are updated

using data on the components of population change – births, deaths, and net international

migration; with net internal migration as an additional component in the state population

estimates. Data on births and deaths continue to be supplied by the National Center for

Health Statistics.

In an effort to use continuously updated data rather than once-a-decade measures used in

the 1990s, for the first time, the national estimates of the international migration

component incorporated data available from the Census 2000 Supplementary Survey
14

(C2SS) and the 2001 Supplementary Survey (SSO1) along with estimates developed

from the Demographic Analysis and Population Estimates (DAPE) project.13

An estimate of the net change in the combined components of legal immigration to the

United States, emigration of the foreign born population from the United States,

temporary migrants, and the net residual foreign-born population for the July 1, 2000 to

July 1, 2001 period were developed from the survey data. The C2SS data on the foreign

born population were survived forward and compared to the estimates of the foreign born

population derived from the SS01 data. The estimates from the DAPE project provided

the estimates of the emigration of the native population and the net movement between

the United States and Puerto Rico.

Independent estimates of each of the components of international migration are not

available. However, the analysis of the data supplied by the Immigration and

Naturalization Service on the flow of legal permanent residents for the 2000 to 2001

period support the estimates derived from the survey data.

It is important to remember that the increased study of the size and characteristics of the

foreign born population along with new ways of estimating the annual flows of

international migration components are in the infancy stages. As such, new

methodologies that result from future studies are likely to lead to alternative measures of

the annual flows of international migration.

Although the post Census changes to CPS population controls eliminated a substantial

proportion of the gap in employment growth that had accumulated over the decade, a

substantial discrepancy remained after these adjustments. Thus, additional factors related

to both surveys have been considered. Some that have been examined include

undercoverage of certain population groups in the household survey, the reporting of

undocumented workers, the sample used for the household survey during the 1990s, the

effect of reference period differences in the surveys, and response error in both surveys.
15

Undercoverage

Even when assuming the underlying population controls are precise, CPS employment

estimates are subject to potential measurement inaccuracies due to undercoverage.

Undercoverage refers to bias in the household data that can occur when housing units are

missed in the sample or when persons in sample households are missed. Research has

shown that undercoverage in the CPS differs by age, sex, race, and Hispanic origin. In

general, it is more significant among the male population than among females and also

more significant among blacks, persons of Hispanic origin, and other races than among

the non-Hispanic white population. The likelihood of undercoverage is estimated by

examining coverage ratios—the estimated number of persons in a given population group

as measured by the survey divided by an independent population level for that group.14

Based on data from 1996, for example, the CPS covered about 92 percent of the

population 16 years of age and over as measured by the 1990 decennial census (adjusted

for the census undercount). For blacks, though, the coverage ratio was only 83 percent,

with black males age 20 to 29 having the lowest coverage rate of any age/race group, just

66 percent. It should be made clear, however, that persons missed in the survey are

accounted for in the final CPS estimates, but they are assigned labor force characteristics

like their cohorts who were surveyed. Bias may then affect the estimates because those

missed may differ in their labor force characteristics from those surveyed.15

A wide range of procedures is in place to prevent undercoverage from entering into the

survey process at all, such as quality control reviews of housing unit lists used for the

sample frame. To reduce undercoverage bias that does occur, the CPS methodology

includes a ratio adjustment procedure that utilizes independent age-sex-race-Hispanicorigin population controls. Three sets of independent population controls are updated

and used each month.16 This second-stage ratio adjustment, however, cannot fully

compensate for potential biases if, to some extent, missed persons in missed households

and missed persons in interviewed households have different characteristics than

interviewed persons in the same age-sex-race-origin group.
16

Reporting of Undocumented Workers

Another possible source of discrepancy between the CES and CPS employment may

occur in the reporting of undocumented workers. While it is an undisputed fact that these

workers are part of the U.S. economy, how many there are and their relative prevalence

in the labor force remains a subject of debate. Regardless of the extent to which

undocumented workers are employed by U.S. businesses, though, they probably are more

likely to be reflected in the CES employment figures than in the CPS. Employers

provide the data to the CES program and to administrative tax authorities such as the

state unemployment insurance programs and they generally would be inclined to include

undocumented employees in their reports. This is based on the presumption that

undocumented workers are using false identification information, such as a Social

Security number, and are doing so with or without the knowledge and assistance of the

employer. If the employer does not realize that the worker is undocumented, they would

have no reason to omit them from employment surveys and administrative tax reports. If

the employer is in fact knowingly hiring an undocumented worker, it may be in their best

interest to report the employee in surveys and tax reports and claim ignorance if

discovered by authorities. (While employer participation in the CES survey is voluntary,

reporting and remitting tax payments to state unemployment insurance programs--the

source of the CES benchmark employment levels--is mandatory and carries monetary

penalties for noncompliance.) This hypothesis is supported by reports from the Social

Security Administration (SSA) about significant growth in the number of W-2 statements

(reports of wages paid) submitted by employers where the employee names and Social

Security numbers fail to match official SSA records. SSA Audit Reports note a strong

correlation between many of these unmatched W-2s (known as “suspended wage items”)

and industries and employers that tend to hire immigrant workers.17

With regard to the household survey, although the CPS survey materials and interviewers

routinely provide reassurances about confidentiality to potential and actual respondents,

undocumented residents probably are more likely than legal residents to refuse to
17

participate in the CPS. If they do participate, they may not be forthcoming about their

labor market activities or a proxy respondent for the household may omit them from the

household roster altogether. These scenarios would all result in undercoverage bias in the

household employment estimates, as previously discussed.

Given the likelihood that the proportion of undocumented workers in the economy grew

in the 1990s, this may represent another source of discrepancy between the CES and

CPS, one that would reflect the widening gap exhibited during that time. Because

estimates of the overall number of undocumented workers are imprecise, and because

such workers cannot be identified in either the CES or CPS, BLS cannot quantify their

possible effect on the discrepancy.

Current Population Survey Sample

Although the CPS sample is scientifically designed to represent the civilian

noninstitutional population of the nation, not all sub-state areas in the country are

included. In designing the sample frame for the CPS, the Census Bureau has divided the

entire United States into geographically defined primary sampling units, or PSUs--

typically a single populous county or a group of smaller contiguous counties within a

state. In general, metropolitan areas fall into the category known as self-representing

PSUs and are always selected in the sample. PSUs in nonmetropolitan areas, however,

are stratified into groups of like size with similar unemployment patterns and

demographic characteristics. Only one of the nonmetropolitan PSUs (referred to as a

nonself-representing PSU) is selected from each of the resulting strata for the survey and

it is used to represent the other PSUs in its stratum. If the selected nonself-representing

PSUs experienced less growth than the PSUs they represented, it could have resulted in

understated employment levels in the CPS during the 1990s and hence contributed to the

discrepancy between the household and payroll surveys.

BLS researched the possibility of missed employment growth in the nonself-representing

strata of the CPS sample by examining the employment trends in those counties from
18

1994 to 2000. Employment data from the BLS Covered Employment and Wages, better

known as ES-202, program were used because they are available at the county level and

represent a universe employment count not subject to sampling error.18

Collectively, growth in the nonself-representing counties tracked well with growth in the

counties not in the sample. Between 1994 and 2000, employment in the nonselfrepresenting counties grew by 12.4 percent, compared with 10.0 percent for the counties

not in the sample. (See table 5.)

On a state-by-state basis, only 5 states19 registered discernibly greater growth in the notin-sample counties than in the nonself-representing counties. Even in these 5 states,

employment growth in the non-sample counties did not significantly outpace growth in

the nonself-representing counties. The average rate of growth in the not-in-sample

counties was 10.9 percent from 1994 to 2000 compared with 6.3 percent in the nonselfrepresenting counties. Individually, none of the 5 states showed a particularly large

disparity in growth between the non-sample and nonself-representing counties. At most,

there was only about a 7 percentage point difference in growth rates between the nonsample and nonself-representing counties of these states.

In summary, the results of the research indicate that understated employment growth in

the nonself-representing sample strata did not occur and so was not a source of

discrepancy between the CPS and CES surveys during the mid-to-late 1990s. Overall,

employment in the nonself-representing counties increased somewhat more than in the

counties not in the sample. Even among the small number of states where the not-insample counties experienced greater growth than nonself-representing counties, the

relative differences were minor.20

Different Reference Periods

Under certain assumptions, differences in employment growth as measured by the two

surveys could arise due to the difference in the length of the reference periods. The CPS
19

counts the number of persons who are employed during its reference period, which is the

calendar week that includes the twelfth of the month. The CES employment count is

comprised of the number of workers that employers report from their payroll records for

the CES reference period, which is the pay period including the twelfth of the month. As

employers’ pay periods differ, the CES reference period is not limited to only one

calendar week, as is true for the CPS, but may be biweekly, monthly, or some other

frequency longer than one week. With a longer reference period, the CES has a greater

chance than CPS to pick up new hires each month. In combination with other factors,

this might account for a portion of the growing discrepancy in employment witnessed in

the 1990s. In addition, job turnover can affect the establishment employment given that

the same person potentially can be counted in two jobs (old and new) during the same

reference period by the establishment survey. The effect could be compounded further if

a greater share of establishments had longer pay periods in the 1990s than in earlier years.

The following examples help illustrate how CES might count more employment than

CPS. 1) During a multi-week reference period, the CES might count as employed a

newly hired person who during the first week (the CPS reference week) was unemployed

or not in the labor force in the CPS. Of course, there will always be some persons

moving from employment to unemployment or outside the labor force to offset this

increase. During an expansion, however, the net change in employment from the overall

flow of persons in and out of employment and the labor force would tend to be positive

and might show an increasing trend during periods of particularly rapid economic

growth.

2) If a person leaves one job and starts another during a relatively short time span, they

could appear on both employers’ payrolls for the CES reference period. They would be

counted twice—once for each job--in the CES survey but only be counted once as an

employed person in the CPS. This would have the same effect as multiple jobholding

and inflate CES employment growth. (But since they would not be classified as multiple

jobholders in the CPS, they would not be accounted for in the reconciliation process.)
20

The issue here is not that the CPS reference week is any less representative than other

weeks in the month with regard to employment status changes. Rather, it is simply that

the CPS reference week presents a narrower time frame in which to capture these

changes. In terms of comparing the one-week CPS reference period to the CES pay

period concept, the typically longer CES reference period potentially allows it to capture

more payroll employment movements in a given month. This would tend to have a

positive effect on CES employment, especially in expansions.

It is generally more common for persons to enter the labor force or change jobs rapidly

during an expansion and less likely during a recession. It also is the case that during

expansions, CES employment tends to increase faster than CPS employment; in

contractions, CES employment tends to decline more rapidly than CPS employment.

This pattern might partly explain why the employment discrepancy increases during

expansions and diminishes during recessions—the CES may be picking up more jobs in

expansions due to greater net movement into the workforce and increased job-to-job

movement.

An increase in job-to-job movement during individual pay periods coupled with an

increase in the proportion of multi-week pay periods reported in the CES could have

resulted in higher employment growth in CES than CPS in the 1990s. Over time there

has been a trend toward longer pay periods in the CES. From its 1962 investigation of

the two surveys, the Gordon Committee reported that at least one-fifth of the total CES

establishments had pay periods longer than a week.21 Since then, the length of pay

periods has grown. By 1998, nearly half (47 percent) of the CES private establishments

reported pay periods that were biweekly, semimonthly, or monthly.22

According to CES data, reporters in the services industry division tend to have pay

periods longer than one week, whereas manufacturing firms tend to have week-long pay

periods. (Sixty-seven percent of services reporters have pay periods longer than a week,

compared to only 26 percent of those in manufacturing.)23 The services division’s 39-

percent employment growth (from 1992 to 2000, based on annual averages) might have
21

been subject to potential inflation by the effect of its reporters’ longer pay periods picking

up more new hires and double-counting due to turnover. And, since employment in the

services division amounts to a third of total CES employment (according to 2000 annual

averages), and it has the most reporters of all the industry divisions, this might have

affected the CES employment estimate and, consequently, the surveys’ growing

employment discrepancy.

To further determine the likelihood that lengthened pay periods and rapid job changing

have played a role in the growing employment discrepancy, BLS looked at available data

on job changing and turnover. Since the CPS redesign of 1994, respondents classified as

employed in consecutive survey periods have been asked whether they still work for the

employer that they named in the prior survey period. The resulting information (not

normally tabulated or published) can be used as a proxy measure to evaluate job changing

over relatively short time periods.24 A review of these data over the period 1994 to 2000

showed that although a significant number of persons indicated their employer changed,

there was no increase in their relative proportion over the time period.

While the BLS does not have labor turnover statistics for the 1990s,25 outside data are

available from the Bureau of National Affairs (BNA, Inc.). The BNA is a nongovernmental source of turnover rates, based on job separations reported by firms in their

survey.26 The BNA reported that in 1999 employee turnover reached its highest level in

nearly two decades.27 By industry, non-manufacturing firms experienced a particularly

sharp rise in turnover in 1999. The BNA data, however, are perhaps a less precise proxy

for job-to-job changes because they merely reflect separations; the separated employees

may or may not move into other employment right way.

The authors of a BLS working paper from 1976 examined the possibility of workers’

being double-counted in the payroll survey due to labor turnover. They used a quit

variable to measure the proportion of workers who would account for some of the job

turnover effect on the employment discrepancy. However, the authors found that the

results were inconclusive.28
22

In summary, increased job-to-job movement and lengthened pay periods could

potentially contribute to the growing discrepancy between the CES and CPS employment

estimates and might help explain at least some of the cyclical pattern that the gap has

historically exhibited. The proxy data available on job-to-job movement and turnover,

however, do not provide clear-cut evidence as to whether rapid job changing was a

definite component of the increasing employment gap. Even if the relative frequency of

job changing was stable in the 1990s, though, the increased proportion of employers with

longer pay periods may have heightened the probability of jobchangers being

doublecounted in the CES. Unfortunately, the effects on the discrepancy are not known

because double-counting in the CES due to job-to-job turnover cannot be identified nor

can the general effects of the differing survey reference periods be measured.

Estimation Methodology in the Current Employment Statistics Survey

Noncovered Employment in the CES Benchmark

Employment data from the CES survey are benchmarked annually. 29 The principal

source of benchmark data for private industries comes from employment data provided to

state employment security agencies by employers covered by state unemployment

insurance (UI) laws and shared with the BLS as the basis of the aforementioned ES-202

program. The ES-202 data represent almost 98 percent of nonfarm employment. BLS

supplements this primary benchmark with universe counts for Federal employees derived

from summaries prepared by the U.S. Office of Personnel Management for the executive,

legislative, and judicial branches. In addition, BLS uses several other sources to establish

benchmarks for industries partially covered or exempt from mandatory UI coverage,

accounting for the nearly 2.5 percent residual nonfarm employment.30 This residual

employment is referred to as the “presumed noncovered.” The contribution of presumed

noncovered employment to an affected industry ranges from less than 0.05 percent in

newspaper publishing to 100 percent in Class I railroads.
23

If the presumed noncovered benchmark estimates were to overstate growth, they could be

a source of discrepancy with the CPS. The statistical reliability of the benchmark sources

for presumed noncovered employment varies to some degree. Some sources provide a

complete count of employees such as those covered by the Railroad Retirement Act;

others, like the Annual Survey of Government Employment, have a calculated sampling

error. Presumed noncovered employment derived from County Business Patterns is a

universe count but is subject to non-sampling error and to variability in reporting

practices. These latter sources are available only on a one- or two-year lagged basis.

BLS extrapolates the current level of non-covered employment based on the trends in the

UI-covered portion of these industries, introducing another potential source of error.

As part of the BLS research, the presumed noncovered benchmark sources underwent

review for upward bias. No discernible pattern of overstatement was identified in any of

the presumed noncovered benchmarks. Over the ten-year period from 1990 to 1999,

presumed noncovered employment contributed 790,000 (excluding Federal government

employment) to the increase in employment in the CES program. The relative amount of

employment growth over this time period contributed by the presumed noncovered sector

also makes it unlikely to have played any significant role in the gap between the CES and

CPS employment estimates.

Response Error Issues

Response error is inevitable when collecting data. BLS has procedures and mechanisms

in place to minimize its occurrence and to identify and correct such errors wherever

possible. In spite of best practices, however, some level of error will remain. For that

reason, we address some specific types of response error that could be factors in the gap

between CES and CPS employment in the mid-to-late 1990s.
24

Response Error in the Current Employment Statistics Survey: Paycheck Counts

Response errors in the CES survey and its principal benchmark, the ES-202 program, can

lead to inconsistencies between these two establishment data series, as well as contribute

to the discrepancy between the employment measured by the CES and CPS surveys. In

particular, reporting errors caused by employers submitting a paycheck count rather than

an actual employment level will overstate employment data from establishment sources

like CES and ES-202. This is because in addition to issuing paychecks for regular

earnings, some employers issue separate checks to workers for bonuses, vacation pay,

and other special pay. Consequently, reporting a count of checks issued in lieu of the

actual number of workers on the payroll for the pay period often will cause some

employees to be counted more than once. Detecting this type of response error through

routine data editing can be quite difficult since it typically causes only a modest

overstatement in the individual establishment’s employment level, not enough to be

flagged for follow up and correction by standardized data editing programs. The

cumulative effect of this type of reporting error, however, can be significant.

In 1992, BLS discovered this type of paycheck count reporting error among numerous

establishments nationwide that used third party payroll processing services. It led to an

unusually large—and widely publicized—downward benchmark revision of 640,000 to

CES employment for 1991.31 If this type of reporting error continued to exist on a

widespread basis, it could be a contributing factor in the gap between CES and CPS

employment in the 1990s. Furthermore, if the use of third party payroll processing and/or

payroll software has been on the rise in the past decade, and some large and growing

amount of employment was overstated by these sources, it also could account for some of

the growth in the discrepancy.

Since the 1991 benchmark, BLS has undertaken several studies aimed at determining the

causes of paycheck counts and related reporting errors and also made significant efforts

to prevent them. In a 1996 Response Analysis Survey, BLS conducted 8,000 telephone

interviews of CES and ES-202 reporters to review reporting practices for both programs
25

and the concepts and definitions of employment. Results indicated that 94 percent of the

respondents in both programs reported unduplicated employee counts. BLS also

analyzed the quality of reports from payroll processing firms, interviewing a large sample

of 250 firms.32 From the information they provided, BLS estimated that the approximate

coverage of employees reported by all such payroll firms was 24 million.33 Ninety-five

percent of the employment reported by payroll firms was correct; that is, they provided

an unduplicated count of employees.34

Concurrent with undertaking these studies, BLS implemented procedures to educate ES202 and CES respondents using means such as special brochures and targeted telephone

interviews with larger employers and employers with large differences in employment

reported to the two programs. BLS also initiated increased communication and

cooperation with third party payroll processing firms and payroll software developers and

has developed a close relationship with trade associations representing the interests of

payroll processors and professionals. These outreach efforts have continued though at a

reduced level. Although BLS cannot quantify the response error that may occur from

paycheck counting, the outreach practices reduces the possibility that widespread

reporting errors are present in the establishment data and play more than a minor role in

the discrepancy between CES and CPS employment.

Employment among the prison population

The CPS sample represents the civilian noninstitutional population. By definition, it

excludes persons in institutions such as prisons. The CES employment definition also

excludes prisoners employed in government-sponsored prison work programs or as

contract workers for private firms. However, it is unclear as to whether such workers

might have been included in the CES benchmark data.35 Given this uncertainty, along

with the fact that the prison population has swollen in recent years, and frequent media

reports on increased use of prison labor, we looked at the possible effect of prison

workers on the employment discrepancy’s growth in the 1990s. According to the Bureau

of Justice Statistics, as of June 30, 2000, 1.3 million persons were in Federal and State

prisons.36 The Correctional Industries Association reported that the total number of U.S.
26

inmates employed in industries was nearly 84,000 in 1999, an increase of about 27

percent since 1990.37 Based on these data, it would seem that even if all incarcerated

workers were included inadvertently in the CES, they would account for only a small

proportion of the employment discrepancy.

Response error in the Current Population Survey

Response error may occur in the CPS for different reasons—for example, the respondent

might misinterpret the question.38 Response error can originate from many different

sources.39 This section examines the respondent, time in sample, and interview mode.

Each of these three potential sources of response error is examined in terms of whether it

could have produced a clear understatement of CPS employment or has increased over

time. We also discuss the following indicators of potential nonsampling error—proxy

reporting rates, month-in-sample indexes, and mode-of-interview biases.40 Overall, it has

been determined that CPS response error has not contributed to the growth in the CESCPS employment discrepancy in the 1990s because it has neither generated a clear

understatement of CPS employment nor has it increased over time.

Respondent

In order to collect all the information for the household timely and efficiently, the

interviewer generally interviews any knowledgeable household member 15 years of age

or older. Respondents who provide labor market information for other household

members are called proxy reporters.41 Nonsampling error may occur because some

household members may not know or be able to report correct information for other

household members.42 There has been concern that since it is a potential source of

response error, the use of proxy reporting in the CPS might have heightened the

CES/CPS employment discrepancy by affecting the CPS employment estimate.

According to research, self-reporting is more reliable than proxy reporting, especially

when there are motivational reasons for self and proxy respondents to report differently.
27

However, in some circumstances, such as in responding to sensitive questions, proxy

reporting is more accurate.43

Generally, the level of proxy reporting in the CPS has been around 50 percent.44 Since

the incidence of proxy reporting has not changed during the 1990s, it would seem an

unlikely source of increasing response error in the CPS and therefore could not have

contributed to the growth of the CES-CPS employment discrepancy.

Time in sample

A sample household in the CPS has eight interviews during a 16-month period. First, a

household is interviewed 4 consecutive months, then is not in the sample for the next 8

months, is interviewed the next 4 months, and then retires from the sample.45 Overall,

though, since the sample rotation pattern has been the same since 1953, it could not have

caused an increasing amount of response error and therefore it is unlikely that it affected

the growth of the CES-CPS employment discrepancy.

Interview mode

A household’s first and fifth months’ interviews are done in person whenever possible.

The remaining months’ interviews may be done over the telephone either by the field

interviewer or an interviewer from a centralized telephone facility.46 With the

implementation of the revised CPS in January 1994, there was an increased reliance on

the use of centralized telephone centers for conducting interviews. Telephone

interviewing is the preferred data collection method because it is more time and cost

efficient. Approximately 60 percent of the households in any given calendar month are

interviewed by telephone.47

The major difference in a computer-assisted personal interview (CAPI) and a computerassisted telephone interview (CATI) is that a CAPI case is likely to be done by the same

interviewer all months in sample (MIS), whereas a CATI case has a definite change in
28

interviewer between MIS 1 and 2 and 5 and 6. Cases in centralized telephone centers

often have different interviewers each month. However, it is not clear whether the results

from the centralized CATI or the CAPI interviews are more accurate.48

BLS tested the effect of centralized telephone interviewing on selected labor force

characteristics using data from January to December 1996. All cases within eligible

primary sampling units were randomly assigned to the CATI panel or the control (CAPI)

panels. No significant differences were found between the CATI test and control panels

for the labor force participation rate or the employment-population ratio.49 Therefore, it

seems that interview mode would not have affected the CPS employment estimate or the

CES-CPS discrepancy.

Note on changes at business turning points

Generally, analysts of the employment data consider the establishment survey a better

gauge of monthly job growth than the household survey. Due to its larger sample size,

establishment survey estimates tend to be less volatile than those from the household

survey. In addition, the payroll survey estimates are anchored annually to the near

universe count of jobs from the ES-202 program. The household survey has no such

benchmark to an administrative count of jobs.

Some analysts contend, however, the household survey provides a more accurate picture

of change at business cycle turning points. The foundation for this contention is the

difficulty measuring the birth and death of new firms. To the extent that the

establishment survey might over or understate the birth and death of new firms, it could

over or understate employment growth. Generally, the contention has been that the

establishment survey overstates job growth going into a downturn because it is not fully

accounting for business closures and understates growth coming out of the downturn

because it is not fully capturing new business formation. The CES redesign undertaken

in 2000 addressed this criticism by updating the sampling frame and CES sample more

frequently—twice a year—and redrawing the full sample every year.
29

Clearly, gauging business activity at turning points is difficult, and it may remain a

potential issue for the establishment survey. However, there also is an issue with the

household survey that may have affected the estimates for the period from 2000 through

2002. As discussed above, one determinants of CPS employment growth is the estimated

level of population growth. In February 2003, the new population controls for 2000,

2001, and 2002 were introduced. These new population estimates included not only a

one-time bump for the difference between prior estimates and the results of Census 2000

but also a slightly higher population growth rate from 2000 through 2002. This higher

population growth in turn raised the level of CPS employment growth for the period. As

shown in chart 2, CPS-CES employment over those 3 years tracked more closely prior to

the introduction of the new population controls.

While the population growth estimates are reviewed each year, they are not adjusted for

changing economic conditions. It could be argued that one of the key factors in

population growth, undocumented immigration, is responsive to those economic

conditions. That certainly seemed to be the case in the 1990s, when CES showed more

employment growth each year than the CPS, in part because the estimate of population

growth was too low. Population growth may have eased as the labor market softened in

the early part of this decade. The boost in estimated population growth introduced from

January 2000 forwarded could be an overstatement and could be resulting in an

overstatement of employment growth from the household survey.

Finally, those wishing to analyze the employment data around the recent business cycle

turning point should bear in mind that both surveys underwent significant changes to

their methodology since the last economic downturn during the early 1990s. Moreover,

the labor market itself underwent major transformations during that decade. Thus, the

observed behavior of the surveys during past business cycles may not be a good predictor

of their behavior in the current situation.
30

NOTE TO REVIEWERS: In accordance with our usual practice, BLS will make a

preliminary announcement on the size of the March 2003 benchmark revision on October

3, 2003, in the Employment Situation news release. This preliminary estimate of the

benchmark will be presented at the FESAC conference.

Summary

During the 1990s, employment growth as measured by the CPS lagged growth as

measured by the establishment survey. The quantifiable universe and definitional factors

between the surveys explained only part of this discrepancy. The update of CPS

population controls following Census 2000 narrowed but did not close the gap. To date,

then, BLS has not been able to pinpoint a source or sources of these differing trends in

employment growth. It may be that multiple factors, including CPS undercoverage,

survey reference period differences and rapid job changing, and miscellaneous reporting

and response errors are all likely to have contributed to varying degrees.

Questions for consideration by FESAC

Of the areas discussed for which information is incomplete or unavailable, which would

seem most promising for further study? Can you suggest approaches for further research

in these areas?

Are there other explanations for the discrepancy that BLS should investigate?

Are there other data sets that might be useful for research into this issue?

1 The surveys have shown differing trends for substantial periods in the past. These differences were

examined in several articles, including Green, Gloria, “Comparing employment estimates from household

and payroll surveys,” Monthly Labor Review, December 1969, pp 9-20; National Commission on

Employment and Unemployment Statistics, Counting The Labor Force, Labor Day 1979, pp 193-199;

Christopher G. Gellner, “A 25-year look at employment as measured by two surveys, Monthly Labor

Review, July 1973, pp 14-23; and Paul O. Flaim, “How many new jobs since 1982? Two surveys differ,”

Monthly Labor Review, August 1989, pp 10-15.

2 For more information on the establishment survey, refer to Chapter 2 of the Handbook of Methods, on the

Internet at http://www.bls.gov/opub/hom/homch2_itc.htm (last visited Nov. 14, 2001).
31

3 For more information on the household survey, refer to the “Technical notes to household survey data

published in Employment and Earnings,” on the Internet at http://www.bls.gov/cps/cpstn952.htm (last

visited Nov. 14, 2001).

4 The CES employment statistics for government refer to civilian employees only.

5 The availability of monthly multiple jobholding data from the CPS since 1994 has helped in reconciling

this part of the CES-CPS employment growth gap. In the monthly internal reconciliation process, the

number of multiple jobholders who are classified as nonagricultural wage and salary workers on their

secondary jobs is added to the unadjusted CPS employment estimate.5 This is done so the CPS

employment estimate will more closely match the payroll survey’s “job count.” In years prior to 1994,

monthly multiple jobholding estimates were imputed from information collected less frequently.

During the period of the employment discrepancy’s growth in the 1990s, the number of multiple jobholders

who were nonagricultural wage and salary workers on their secondary jobs increased but the growth in

multiple jobholders during this period accounted for only a small share of the growth in the discrepancy

between the CES and otherwise-adjusted CPS estimates. While the CES-CPS employment discrepancy

grew substantially and increasingly, multiple jobholding exhibited unsteady and then dampened growth.

So although multiple jobholding does contribute to the CES-CPS employment discrepancy, it did not

contribute much to the discrepancy’s growth.

6 In addition to identifying persons who have more than one job during the survey reference week, the CPS

has information on the number of jobs held up to 4 or more. About 93 percent of multiple jobholders have

2 jobs. The number of persons holding 3 or more jobs remained around 550,000 from 1994 to 2000.

Further, the class of worker and industry status is only collected for the primary and second job, so it is not

known to what extent 3rd and 4th jobs would be nonagricultural wage and salary employment.

7 For more information see Department of Defense Active Duty Military Personnel Strength Level, Fiscal

Years 1950 – 2000, Department of Defense, Directorate for Information Operations and Reports, Statistical

Information Analysis Division--Personnel, on the Internet at

http://web1.whs.osd.mil/mmid/mmidhome.htm (visited April 23, 2001).

8 Nonagricultural wage and salary workers are a class of worker from the Current Population Survey.

CPS wage and salary workers receive wages, salary, commissions, tips, or pay in kind from a private

employer or from a government unit. This is the group of employed persons from the CPS that is most

closely related to the CES employment estimate of nonfarm payroll employees. For more information, see

“Collection and Coverage,” from “Technical notes to household survey data published in Employment and

Earnings,” Labor Force Statistics from the Current Population Survey, Bureau of Labor Statistics, on the

Internet at http://www.bls.gov/cpstn1.htm (visited Feb. 28, 2001).

9 The 2000 CPS data were not adjusted for Census 2000 population controls.

10 Following the 1990 decennial census, BLS chose to use CPS population controls that incorporated the

results and adjustments obtained from the Post-Enumeration Survey (PES), a post-census quality control

check that assessed the estimated undercount. Thus, BLS began the decade estimating CPS employment

using the best available data for the population controls. Even using the PES-adjusted population controls,

however, the total revision in 1990 was an increase of just 880,000, or 0.75 percent of the prior year’s

employment. This was far less significant than the approximate 2 percent upward revision introduced from

the 1980 census. For more information on the Post-Enumeration Survey, see Current Population Survey:

Design and Methodology, Technical Paper 63, pp. D-20-D-22. On the Internet at

http://www.census.gov/prod/2000pubs/tp63.pdf (visited Mar. 16, 2001).

11 Using this population base and the subsequent procedures described implies the assumption of a constant

size undercount.
32

12 In 1999, Juhn and Potter of the Federal Reserve Bank of New York published an article on the

discrepancy in which they proposed that CPS employment levels likely were understated due to perceived

deficiencies in the census population data (“Explaining the Recent Divergence in Payroll and Household

Employment Growth,” Federal Reserve Bank of New York, Current Issues in Economics and Finance,

December 1999, Vol. 5, No. 16, pp. 1-6, on the Internet at

http://www.ny.frb.org/rmaghome/curr_iss/ci5-16.html). They hypothesized that an undercount of the

working-age population data underlying the CPS employment figures could explain a sizable portion of the

discrepancy between the two series. They further observed that the demographic groups most likely to be

undercounted were those “whose employment status is very sensitive to business cycle fluctuations.” In

Juhn and Potter’s assessment, these factors taken together could account not only for a significant portion

of the gap itself, but also explain its cyclical behavior and marked increase through the 1990s. They

illustrated their case with the example of black men, a demographic group widely acknowledged as prone

to be undercounted, particularly the younger cohorts. Increases in employment among black men during

the 1990s expansion were likely to be captured fully in the payroll survey, they asserted, but understated in

the household survey. For the overall population, they projected that an undercount of 2 percent could

translate into a gap of 2 million between the household and payroll surveys, if half of that 2 percent found

employment during the latter half of the decade.

13 See Deardorff and Blumerman, 2001, "Evaluating Components of International Migration: Estimates of

the Foreign-Born Population by Migrant Status in 2000", Population Division Working Paper Series No.

58.

14 For additional information on the calculation of the CPS coverage ratios, see Current Population Survey:

Design and Methodology, Technical Paper 63, U.S. Department of Labor and U.S. Department of

Commerce, March 2000, page 16-1, on the Internet at http://www.census.gov/prod/2000pubs/tp63.pdf

(visited Mar. 16, 2001).

15 For a detailed discussion of coverage error in the CPS, see Current Population Survey: Design and

Methodology, Technical Paper 63, U.S. Department of Labor and U.S. Department of Commerce, March

2000, pp. 15-2-15-4 and pp. 16-1-16-2, on the Internet at http://www.census.gov/prod/2000pubs/tp63.pdf

(visited Mar. 16, 2001).

16 For more information on the second-stage ratio adjustment process, see Current Population Survey:

Design and Methodology, Technical Paper 63, pp. 10-5-10-7. On the Internet at

http://www.census.gov/prod/2000pubs/tp63.pdf (visited Mar. 16, 2001).

17 See “Patterns of Reporting Errors and Irregularities by 100 Employers with the Most Suspended Wage

Items,” Audit Report A-03-98-31009, September 1999, and “The Social Security Administration’s

Earnings Suspense File Tactical Plan and Efforts to Reduce the File’s Growth and Size,” Evaluation Report

A-03-97-31003, February 2000, Office of the Inspector General, Social Security Administration. On the

Internet at http://www.ssa.gov/oig/index.htm (visited April 16, 2001).

18 The BLS Covered Employment and Wages, or ES-202, program produces a complete count of

employment and wages for workers covered by state unemployment insurance (UI) laws and federal

government workers covered by Unemployment Compensation for Federal Employees. The program

serves as a virtual census of nonfarm employment and wages with significant geographic and industry

detail and is the primary benchmark for the CES employment estimates. In this analysis, the ES-202

employment data were adjusted to a concept similar to CES by omitting private households and agricultural

industries that are not included in the CES. For more information on the ES-202 program, see

<http://www.bls.gov/cew/home.htm>.

19 Alaska, Georgia, Idaho, Oklahoma, and Wyoming.

20 Nationwide, the ES-202 employment (as adjusted to a CES concept) rose by 17.1 million, or 15.3

percent, over the 1994-2000 period. When evaluated by county-defined categories corresponding to the
33

CPS sample, employment in the self-representing counties grew by 15.5 percent, compared with the 12.4

percent in the nonself-representing counties, and 10.0 percent in the non-sample counties.

Seven states and the District of Columbia are comprised entirely of self-representing PSUs (Connecticut,

Delaware, District of Columbia, Massachusetts, New Hampshire, New Jersey, Rhode Island, and Vermont)

and therefore were not directly relevant for the research. A ninth state—Hawaii—has PSUs defined at the

sub-county level that prevented its inclusion. Data for the remaining 42 states, then, were the basis for the

comparison of the nonself-representing and the non-sample counties.

It should be noted that, at any point in time, the ES-202 program has a small proportion of

employment that cannot be classified in specific counties. These include employer reports that do not

provide county detail, employer reports for which the county information is temporarily unavailable, and a

very small number of cases where the UI-covered employment is based in foreign locations. In 1994, 1.3

percent of the ES-202 employment in this research was not classifiable by county; in 2000, it was 2.0

percent. The employment associated with the unknown county locations was categorized separately in this

research. There is no reason to believe that this employment is more likely to fall into one CPS sample

category over another, that is, self-representing versus nonself-representing or not in sample. Hence, their

exclusion should not have had any notable effect on the results of the analysis.

21 Measuring Employment and Unemployment, President’s Committee to Appraise Employment and

Unemployment Statistics, 1962, p. 360.

22 See Getz, Patricia, Jurgen Kropf, and Michael Roosma, “Causes and Treatments of Non-Economic

Fluctuations in Average Weekly Hours and Average Hourly Earnings Series,” Bureau of Labor Statistics,

1999, on the Internet at http://www.bls.gov/ore/pdf/st990180.pdf (visited Feb. 27, 2001), p. 3.

23 See Getz, Patricia, Jurgen Kropf, and Michael Roosma. Page 3.

24 It should be noted that the CPS question that asks whether the respondent’s employer has changed was

not designed to gather or track information about job changing. Rather, the question is used as an indicator

of whether the industry in which the respondent is employed has changed and hence to determine whether

the individual’s industry and occupation information should be carried forward.

25 In the late 1990s, BLS initiated a new survey program, the Job Openings and Labor Turnover Survey

(JOLTS), which gathers data on labor turnover by industry from a sample of 16,000 business

establishments. The earliest data available are for December 2000. For more information, see the JOLTS

homepage on the BLS Internet site at http://www.bls.gov/jlt/ (visited Jan. 11, 2002).

26 The Bureau of National Affairs is a private publisher. BNA’s rates of permanent separation exclude

layoffs, reductions-in-force, long-term leaves of absence, and departures of temporary staff. BNA’s

quarterly turnover data are derived from a survey of human resource and employee relations executives

representing a cross-section of U.S. employers, both public and private.

27 See “Economy Spurs Highest Turnover Rates in Nearly 20 Years, BNA Survey Finds,” BNA press

release, March 13, 2000, on the Internet at http://www.bna.com/press.trn98r1.htm (last visited Jun. 4,

2001).

28 For more information, see Joseph R. Antos, Anthony J. Barkume, J. Welson Mixon, and Jack E. Triplett,

“Why Employment Estimates Differ: A Study of Discrepancies between BLS Household and Payroll

Estimates,” Working Paper 65, Bureau of Labor Statistics, October 1976, pp. 12-14. The authors

constructed the quit variable by multiplying the manufacturing quit rate (produced from the former BLS

Labor Turnover Survey until the end of 1981) by the estimate of total nonfarm payroll employment. This

provided an estimate of the total number of quits in the nonagricultural sector. However, the authors

identified two measurement problems that produced errors in the estimate. The quit rates were only

available for manufacturing and mining and therefore were not perfectly correlated with economy-wide

averages. In addition, the reference periods for turnover and employment data differed.
34

29 Annual benchmark revisions are a standard part of the CES estimation process. Benchmarking is a reanchoring of the sample-based employment estimates to total nonfarm employment counts.

30 The U.S. Census Bureau’s County Business Patterns, based on data for employees covered under Social

Security laws, augment the ES-202 for non-office insurance sales workers, child daycare workers, religious

organizations, and private schools and hospitals. The Annual Census of Governments conducted by the

Census Bureau provides benchmarks for State and local government hospitals and educational institutions.

Universe data for interstate railroads are obtained from the Railroad Retirement Board.

31 For further information about the 1991 CES benchmark revision, see Patricia M. Getz, “BLS

Establishment Estimates Revised to March 1991 Benchmarks,” Employment and Earnings, Vol. 39, No. 6,

June 1992, pp. 6-11, and “Revision of Payroll Survey Employment Estimates to March 1991 Benchmarks,”

Bureau of Labor Statistics, Office of Employment and Unemployment Statistics, June 3, 1992.

32 See Goldenberg, Karen L, Susan E. Moore, and Richard J. Rosen, “Commercial Payroll Software and the

Quality of Employment Data,” at http://stats.bls.gov/orest/st940240.htm (visited March 12, 2001).

33 It should be noted that payroll processing firms are more likely to provide the unemployment insurance

tax reports that are the source of ES-202 employment than they are to file the CES survey on behalf of the

employer. Hence, most response errors stemming from third party payroll processing firms probably will

originate in the ES-202 program. Since the ES-202 is the primary benchmark for the CES employment,

though, the CES employment also will reflect any overstatements from the ES-202.

34 BLS also contacted and interviewed payroll software developers in this latter study. Using information

about the number of leased or licensed software products and the number of copies of payroll software

sold, it was estimated that approximately 25 to 40 million employees are covered by such payroll software

products. This estimate could substantially increase if user-developed software is also considered. The

accuracy obtained from the software products was not readily determined, since many components of

correct reporting were the responsibility of the end users. Software product accuracy, then, is probably

similar to the percentage of incorrect reports received from respondents using self-developed software or

no software at all.

35 Generally speaking, prison workers should not be included in the payroll employment data. The CES

program considers these workers out of the survey’s scope. However, the primary employment benchmark

data for the CES come from State unemployment insurance (UI) tax reports provided to BLS for the ES202 program (see endnote 18). Prison workers were not formally excluded under federal UI coverage law

until 1997. Even though excluded under federal law, States can choose to cover such workers under State

law; whether any States do so cannot be readily determined. More importantly, though, government and

private employers in practice could be including prison workers inadvertently in their CES reports and/or

their unemployment insurance tax reports. Although BLS has not identified any instances where prison

workers have been reported in error, the inclusion of other out-of-scope and non-covered workers in

establishment data has occurred on occasion. Such reporting errors are extremely difficult to detect.

36 Prison and Jail Inmates at Midyear 2000, Bulletin, (Bureau of Justice Statistics, March 2001), on the

Internet at http://www.ojp.usdoj.gov/bjs/pub/pdf/pjim00.pdf (visited Apr. 24, 2001).

37 See 2000 CIA Directory, Correctional Industries Association. For more information, the CIA website

may be accessed on the Internet at http://www.correctionalindustries.org (visited June 13, 2001).

38 For more information, see Technical Paper 63 Current Population Survey: Design and Methodology,

March 2000, 15-6, on the Internet at http://www.census.gov/prod/2000pubs/tp63.pdf (visited Jan. 10,

2002).
35

39 For more information on sources of response error, see Technical Paper 63 Current Population Survey:

Design and Methodology, March 2000, pages 15-6 – 15-8, on the Internet at

http://www.census.gov/prod/2000pubs/tp63.pdf (visited Jan. 10, 2002).

40 For more information about indicators of potential nonsampling error, see Technical Paper 63 Current

Population Survey: Design and Methodology, March 2000, Chapter 16, on the Internet at

http://www.census.gov/prod/2000pubs/tp63.pdf (visited Jan. 10, 2002).

41 For more information about the respondent, see Technical Paper 63 Current Population Survey: Design

and Methodology, March 2000, pages 7-3, 15-7, and 16-10, on the Internet at

http://www.census.gov/prod/2000pubs/tp63.pdf (visited Jan. 10, 2002).

42 See Technical Paper 63 Current Population Survey: Design and Methodology, March 2000, page 16-10,

on the Internet at http://www.census.gov/prod/2000pubs/tp63.pdf (visited Jan. 10, 2002).

43 See Technical Paper 63 Current Population Survey: Design and Methodology, March 2000, page 15-7,

on the Internet at http://www.census.gov/prod/2000pubs/tp63.pdf (visited Jan. 10, 2002).

44 See Technical Paper 63 Current Population Survey: Design and Methodology, March 2000, page 16-10,

on the Internet at http://www.census.gov/prod/2000pubs/tp63.pdf (visited Jan. 10, 2002).

45 For more information about the rotation of the CPS sample, see Technical Paper 63 Current Population

Survey: Design and Methodology, March 2000, page 3-15, on the Internet at

http://www.census.gov/prod/2000pubs/tp63.pdf (visited Jan. 10, 2002).

46 A household’s initial interview is to be done in person by a field representative. For households in the

second, third, and fourth months in sample, the field representative may conduct the interview over the

telephone (this must be approved by the respondent). The fifth-month interview must be done in person.

The purpose of the in-person interview is to reestablish the relationship with the household after its being

out of the survey for 4 months. Households in their sixth, seventh, or eighth month-in-sample are eligible

for telephone interviewing. See Technical Paper 63 Current Population Survey: Design and Methodology,

March 2000, pp. 7-3 - 7-5, on the Internet at http://www.census.gov/prod/2000pubs/tp63.pdf (visited Jan.

10, 2002).

47 See BLS Handbook of Methods, Chapter 1. “Labor Force Data Derived from the Current Population

Survey,” U.S. Department of Labor, Bureau of Labor Statistics, pages 9-10, reprinted from April 1997, on

the Internet at http://www.bls.gov/opub/hom/home.htm (visited Jan. 10, 2002). For a discussion of the

incidence of telephone interviewing in the CPS and related indicators, see Technical Paper 63 Current

Population Survey: Design and Methodology, March 2000, pages 16-6 – 16-7, on the Internet at

http://www.census.gov/prod/2000pubs/tp63.pdf (visited Jan. 10, 2002).

48 See Technical Paper 63 Current Population Survey: Design and Methodology, March 2000, page 16-7,

on the Internet at http://www.census.gov/prod/2000pubs/tp63.pdf (visited Jan. 10, 2002).

49 See Technical Paper 63 Current Population Survey: Design and Methodology, March 2000, pages 16-7 -

16-8, on the Internet at http://www.census.gov/prod/2000pubs/tp63.pdf (visited Jan. 10, 2002).
Chart 1. Employment from CES, CPS total, and CPS Nonag Wage and Salary, 1948-2003

0

20,00040,00060,00080,000

100,000120,000140,000160,000

Jan-48

Jan-50

Jan-52

Jan-54

Jan-56

Jan-58

Jan-60

Jan-62

Jan-64

Jan-66

Jan-68

Jan-70

Jan-72

Jan-74

Jan-76

Jan-78

Jan-80

Jan-82

Jan-84

Jan-86

Jan-88

Jan-90

Jan-92

Jan-94

Jan-96

Jan-98

Jan-00

Jan-02

(In thousands)

CES CPS CPS Nonag Wage and Salary

Note: CPS data for January 2000 forward reflect Census 2000-based population controls.
Annual averages, in thousands

CES employment 114,291 131,785 17,494

CPS employment 123,060 1135,208 12,148

Discrepancy in CES and CPS employment estimates -8,769 -3,423 5,346

Reconciliation for quantified differences in CES and CPS:

CES employment 114,291 131,785 17,494

CPS employment 123,060 1135,208 12,148

Adjust CPS to better match CES estimate

Less:

1. Agricultural employment 3,409 13,305 -104

2. Nonagricultural unincorporated self-employed persons 9,003 18,674 -329

3. Nonagricultural unpaid family workers 131 1101 -30

4. Nonagricultural private household wage and salary workers 966 1890 -76

5. Unpaid absences, nonagricultural wage and salary workers 1,991 11,998 7

Plus:

6. Multiple jobholders with secondary wage and salary jobs 4,916 15,284 368

7. Agricultural services wage and salary workers2 716 1964 248

Adjusted CPS employment 113,192 126,488 13,296

Discrepancy in CES and adjusted CPS employment 1,099 5,297 4,198

Additional adjustments to the CPS

Plus: 15-year-old nonagricultural wage and salary workers3 354 433 79

Plus: multiple jobholders with more than 2 jobs3 524 545 21

Plus: Census 2000 population control revision to CPS 1,683 1,683

Adjusted(2) CPS employment 114,070 129,149 15,079

Discrepancy in CES and adjusted(2) CPS employment 221 2,636 2,415

Note: Shaded area illustrates the routine monthly reconciliation exercise carried out by BLS to make the CPS better match

the CES concept. Since January 2003, the monthly reconciliation has been modified somewhat from this example in terms of

the data series used. Conceptually, however, the adjustments are the same.

1 These data reflect CPS employment estimates prior to revisions incorporating the Census 2000-based population controls.

2 This subset of agricultural employment, based on the 1987 Standard Industrial Classification (SIC) and 1990 Census

Industrial Classification, was included in the CES survey's scope and hence added back in to the CPS after removing total

agricultural employment in step 1. Due to the redefinition of the agricultural sector under the North American Industry

Classification System (NAICS) and the derivative 2002 Census Industrial Classification, this adjustment is no longer

necessary in current reconciliation exercises.

3 Special CPS microdata tabulations. This information is not routinely tabulated from the CPS .

Table 1. Reconciliation of employment estimates from the CES and the CPS

Adjusting the discrepancy between 1994 and 2000 for routinely quantified conceptual and universe differences

Item 1994 2000 Growth
Table 2. Employment as measured by the CES and CPS, annual averages, 1994-2000

(Numbers in thousands)

Year CES CPS

wage and

CPS nonag

salary

wage and

CPS nonag

salary,

adjusted for

private

workers,

household

unpaid

services, and

absences, ag

multiple

jobholding CES CPS

wage and

CPS nonag

salary

wage and

CPS nonag

salary,

adjusted for

private

workers,

household

unpaid

services, and

absences, ag

multiple

jobholding

CES minus

CPS

CES minus

CPS nonag

wage and

salary

CES minus

CPS nonag

wage and

salary,

adjusted

household

for private

workers,

unpaid

absences,

ag

services,

and

multiple

jobholding

1994 114,291 123,060 110,517 113,1921995 117,298 124,900 112,448 115,559 3,007 1,840 1,931 2,367 1,167 1,076 6401996 119,708 126,708 114,171 117,380 5,417 3,648 3,654 4,188 1,769 1,763 1,2291997 122,776 129,558 116,983 120,443 8,485 6,498 6,466 7,251 1,987 2,019 1,2341998 125,930 131,463 119,019 122,389 11,639 8,403 8,502 9,197 3,236 3,137 2,4421999 128,993 133,488 121,323 124,551 14,702 10,428 10,806 11,359 4,274 3,896 3,3432000 131,785 135,208 123,128 126,488 17,494 12,148 12,611 13,296 5,346 4,883 4,198

Note: The CPS data series reflect estimates prior to revisions incorporating the Census 2000-based population controls.

Cumulative change since 1994 Discrepancy in growth since 1994 Levels
Table 3. Difference between CES and CPS nonagricultural wage and salary employment

growth for selected states, 1994 and 2000 annual averages

(In thousands)

CES employment

State

1994 2000 1994 20001

Mexican border

Arizona 1,692 2,243 1,686 2,061 176

California 12,160 14,488 12,267 14,292 304

New Mexico 657 745 624 705 7

Texas 7,751 9,433 7,828 8,858 652

Canadian border

Alaska 259 284 251 267 9

Idaho 461 560 466 543 22

Maine 532 603 488 574 -14

Michigan 4,147 4,674 4,173 4,637 64

Minnesota 2,310 2,676 2,178 2,389 154

Montana 340 388 330 373 4

New Hampshire 523 622 534 594 39

New York 7,831 8,635 7,430 7,930 304

North Dakota 295 328 258 277 13

Vermont 264 299 254 282 7

Washington 2,304 2,711 2,223 2,613 18

Total, border states 1,758

Selected non-border states

Florida 5,799 7,081 5,776 6,640 417

Massachusetts 2,904 3,323 2,743 2,903 259

Ohio 5,076 5,625 4,838 5,139 247

North Carolina 3,359 3,934 3,133 3,466 242

Pennsylvania 5,192 5,691 4,999 5,271 227

Virginia 3,004 3,517 2,979 3,279 213

Maryland 2,146 2,450 2,408 2,499 213

Indiana 2,713 3,000 2,670 2,749 208

Cumulative

difference

in growth,

1994-2000

1 Not adjusted for Census 2000 population controls

CPS nonagricultural

wage and salary

employment
Table 4. Growth in CES employment in selected metropolitan areas

near border crossings, 1994 and 2000 annual averages

(In thousands)

Metropolitan Area 1994 2000

Mexican border

Brownsville-Harlingen-San Benito, 91 109 19

El Paso, Texas 231 256 26

Laredo, Texas 57 69 12

McAllen-Edinburg-Mission, Texas 119 157 38

San Diego, California 955 1,194 239

Canadian border

Buffalo-Niagara Falls, New York 535 558 24

Detroit, Michigan 1,950 2,204 254

Total 611

Change,

1994-

2000
Table 5. Employment growth in nonself-representing and not-in-sample

counties, 1994 and 2000 annual averages1

(In thousands)

Employment 1994 2000

Total, United States 111,796 128,929 17,133 15.3

Self-representing counties 87,088 100,555 13,467 15.5

Nonself-representing counties 7,830 8,802 971 12.4

Counties, not in CPS Sample 15,440 16,981 1,541 10.0

Non-classifiable counties 1,438 2,591

as a percent of total employment 1.3 2.0

Change,

1994-

2000

Percent

change,

1994-

2000

1 ES-202 employment, as adjusted to a CES concept by removing private households and

agricultural industries not included in the CES
Chart 2. Employment from the household and establishment surveys,

January 2000 - August 2003

0.990.991.001.001.011.011.021.02

Jan-00

Mar-00

May-00

Jul-00

Sep-00

Nov-00

Jan-01

Mar-01

May-01

Jul-01

Sep-01

Nov-01

Jan-02

Mar-02

May-02

Jul-02

Sep-02

Nov-02

Jan-03

Mar-03

May-03

Jul-03

Employment indexed to January 2000

CPS Nonag W&S empl (new pop controls adjusted for 01/03 bump) CPS Nonag W&S empl (old pop controls) CPS Nonag W&S empl (new pop controls) CES Total nonfarm employment