1

Report of the FESAC Subcommittee on the Discrepancy in CPS-CES 

Sub-committee members include: Patricia Anderson, Ernst Berndt, Barry Bosworth, 

Robert Groves, John Haltiwanger, Randall Kroszner, Judith Lessler, and Lisa Lynch 

(Chair). 

This report was recommended to BLS by the FESAC on December 9, 2005. 

I. Introduction 

In May 2005 the Bureau of Labor Statistics requested that the Federal Economics 

Statistics Advisory Committee review the BLS’ current practices for collecting data on 

employment from two surveys – the Current Population Survey (CPS) on the household 

side, and on the employer side, the payroll survey known as the Current Employment 

Statistics (CES). Specifically, the BLS raised a number of questions related to the 

discrepancy in employment trends that have arisen since the late 1990s. The nature of 

this discrepancy is two-fold. First, from 1998 until the end of 2002, an adjusted 

household survey employment series (i.e one that was smoothed for population control 

revisions and adjusted to an employment concept more similar to the payroll survey) 

appeared to suggest lower CPS employment relative to the CES series. The two series 

then apparently began to converge. However, since March 2005 net employment in the 

adjusted CPS series appears to be increasing a bit faster than in the CES series. 

Second, when one compares the CPS and CES without making any adjustment for 

differences in who is covered in the definition of employment, the level of employment 

from the CPS series is always higher than that in the CES. This reflects in large part 

differences in employment concepts. However, while both series showed employment 

decreases during the recession of 2001, CPS employment recovered more quickly to its 

pre-recession level than did CES employment. So prior to the 2001 recession, adjusting 

for employment concept, CES employment was growing faster than CPS, but since the 

2001 recession the unadjusted CPS employment series appears to have recovered faster 

than the CES series. This raises the issue of whether or not our employment concepts 

are accurately capturing areas of the labor market that may be increasingly important 

for the economy. 

BLS provided a detailed report on this discrepancy in a presentation to FESAC in 

October of 2003 and now maintains a monthly update on the CES-CPS discrepancy 

www.bls.gov/web/ces_cps_trends.pdf. Nevertheless there remain questions about the 

nature of this gap that this sub-committee has attempted to address along with 

suggestions for future BLS work on this topic. 

II. Questions related to the adjusted CPS-CES difference 

There are likely to be multiple potential sources for the CPS-CES discrepancy. The 

following questions that were posed to FESAC by the BLS relate primarily to the first 

dimension of the discrepancy between the CES and CPS – higher CES employment in 
2

the late 1990s and early 2000s relative to an adjusted measure of employment from the 

CPS. 

Question 1. One issue that might explain the CPS-CES differences would be 

changes in amount of off-the-books employment. Such employment would 

not be captured by the CES. The CPS might capture some off-the-books 

employment but probably would do so imperfectly. Changes in off-the-books 

employment also might help explain the cyclical nature of CPS-CES 

differences. Can the subcommittee recommend specific methods for research 

in this area? Can the subcommittee suggest other sources of data that might 

be used to track changes in off-the-books employment? 

RESPONSE: 

By its very nature, off-the-books employment is difficult to measure. However, 

some sectors of the economy are more likely to have such employment than others—

specifically residential construction/renovation and household services. Other sectors 

of the economy that may have a higher incidence of off-the-books employment include 

retail, agriculture, taxi services, hotels and eating establishments. Certain groups of 

workers are also more likely to be working off-the-books – in particular, illegal 

immigrants. So how might this phenomenon explain either of the two dimensions of 

the CES-CPS discrepancy? With the 2000 Census we saw that the amount of 

immigration into the United States during the 1990s was significantly under-estimated. 

As BLS has already concluded, the population controls that were used in the 1990s 

(that did not include all of this growth in immigration) contributed significantly to the 

discrepancy between the household and payroll employment series. 

Off-the-books employment may explain in part a more rapid recovery in CPS 

employment post 2001. Given the importance of residential construction in this most 

recent economic recovery the difficulty in capturing off-the-books employment may 

have become more important. Specifically, if employers fail to report using illegal 

labor but households do report such employment then this may explain some of the 

discrepancy in the two series since the 2001 recession. However, quantifying this is not 

easy because respondents are likely to want to hide such work. One strategy that has 

been used in the UK is to conduct household surveys of demand and supply of off the 

books activity. Alternatively one could possibly utilize time use surveys to capture 

under-reporting of off the books work. See for example Colin Williams, Small 

Businesses in the Informal Economy: making the transition to the formal economy 

Evidence and key stakeholder opinion http://www.sbs.gov.uk/SBS_Gov_files/sbc/SBCInformal-EconomyEvBase.pdf#search='measuring%20the%20informal%20economy%20in%20the%20%

20UK’. In both of these cases one could then look to see if certain sectors that have a 

higher incidence of off-the-books employment were growing faster in the CPS than 

CES series and then consider how this might contribute to the size of the discrepancy 

over time. 
3

Question 2. Changes in employment relationships may have led to the growth 

of independent contracting and self employment. While self employment is 

measured within the CPS, it is possible that growth of such employment is 

understated (and growth in wage and salary employment overstated) in the 

CPS due to response error. Such error would complicate the analysis of CPSCES differences. Can the subcommittee suggest data sources or research that 

could be used to check the trend in self employment as measured by the CPS? 

 

RESPONSE: 

Support for ongoing research that compares self-employment information from the 

CPS, detailed earnings records from the IRS, and information on sole proprietorships 

from the Census Business Register would be helpful to understand the impact of the 

business cycle on self-employment. With such data it would also be possible to try to 

model aspects of self-employment and independent contracting. Additionally, surveys 

such as the contingent work survey should be maintained on a regular cycle similar to 

the displaced workers survey so that we can generate a time series on how contingent 

work and self-employment fluctuate over the business cycle. This might be an area 

where the type of supplemental surveys discussed below could have a tangible payoff. 

Question 3. Past research on CPS-CES differences shows that the population 

controls used for the CPS can result in understatements of employment 

growth. BLS and others have suggested that in the early 2000s, these controls 

may have overstated the growth in employment. Can the subcommittee 

suggest approaches for examining whether population growth is overstated? 

RESPONSE: 

The committee concluded that ensuring better measurement of our population that 

picks up changes in immigration in a timely manner is one of the highest priorities for 

research and funding for BLS and Census. While measurement problems with 

population controls seem a probable explanation for some part of the CPS-CES 

discrepancy, it is important to think about how this could result in the CPS 

underestimating employment growth in the 1990s and overestimating it in the early 

2000s. The evidence for underestimation in the 1990s is strong, given the availability 

of the 2000 Census to verify that the within decade population adjustments were 

insufficient. Much of that understatement seems to come from grossly understating the 

net growth in immigrants. Thus, the immigrant population seems like the logical place 

to start in considering whether we are now overstating population growth. To the 

extent that illegal immigration (which we have the most trouble measuring) is 

procyclical (i.e. it increases in economic expansions and contracts during recessions) it 

remains a promising explanation for both the under- and over-estimation. 

Efforts to improve our understanding of how the flow of immigrants (legal and 

illegal) changes over time will be critical to improved measurement of our population. 

For example, the American Community Survey seems particularly well suited (subject 

to sufficient funding) to provide feedback to intercensal estimates of the population. 

Another way to examine the impact of the business cycle on the employment of illegal 
4

immigrants is to look at the unmatched SSNs from the SSA Audit Reports. Could a 

time-series be created from these reports to determine if they are indeed procyclical? 

The October 2003 FESAC paper notes that the Census Population Division has 

“undertaken an aggressive project to study and develop estimates of the size, 

characteristics and impact of the foreign born population.” This seems like it should 

remain a high priority area, with a special emphasis on being able to better predict the 

cyclical component of immigration, rather than just assuming constant rates. 

Additionally, work on how the discrepancy varies at the state and regional level 

might provide some insight into the relative importance of illegal immigration for the 

gap between the CES and CPS. This type of work is likely to lead to suggestions for 

more effort to develop population estimates at the state and regional level using 

additional data sources (driver licenses, etc.) to provide regional checks on the national 

estimates. 

Question 4. The CES captures the number of payroll jobs that exist during a 

pay period. By definition, this includes cases where a worker moves from one 

employer to another during a pay period. As a result, increases in turnover 

would inflate the count of jobs from the CES and decreases in turnover would 

deflate the CES count. BLS has examined this issue and believes the impact 

is fairly small. Can the subcommittee review the work BLS has done and 

suggest areas for further research or alternative approaches? 

RESPONSE: 

There are mixed views on the committee about the relative importance of this issue 

compared to some of the other potential contributors to the discrepancy. While results 

from past work on this issue have not been promising, it remains for some committee 

members an attractive partial explanation, given the cyclicality of the CPS-CES gap. 

While the October 2003 FESAC paper focused on the employment growth in the CES 

being larger than the CPS over the 1990s, in the current decade employment growth in 

the CPS has outpaced that from the CES (see www.bls.gov/cps/ces_cps_trends.pdf). 

The difference appears to be the stronger economic growth in the earlier period. Thus, 

it appears that overstatement in the CES (or understatement in the CPS) is procyclical, 

while overstatement in the CPS (or understatement in the CES) is countercyclical. 

It is clear that the potentially longer payroll period in the CES would imply that the 

CES should have higher employment levels than the CPS. The current attempt to 

adjust for this (see www.bls.gov/ces/cesjobch.pdf) does indeed estimate lower levels. 

However, it does not appear to change any of the basic patterns from the CES. Its 

growth is still stronger than that from the CPS in the late 90s and weaker coming out of 

the 2001 recession. While the procedure used for this adjustment appeared generally 

sensible, there is some question of whether the CPS sample is large enough to pick up 

the cyclicality of turnover. Perhaps an experiment could be done using UI wage 

records. The problem here, of course, is that wage records are quarterly. Presumably, 

though, one could try assuming that separations and accessions are distributed 
5

uniformly throughout the quarter and divide the quarterly rates by 3 and then use the 

same methods for weighting the monthly rates from the CPS. 

One problem with this current method is that it can’t really pick up cyclicality in 

how likely a worker is to move within a pay period. Presumably in good times, job-tojob moves are more likely. One way the UI wage records might be able to address this 

is to follow individuals across firms, and calculate “earnings weeks lost” as in 

Anderson and Meyer (Brookings Micro, 1994) to estimate how fast the transition was. 

In their data (from over twenty years ago), over fifty percent of quarterly separations 

with reemployment appeared to move to the new job within 2 weeks. For firms with 

monthly pay periods, this could easily result in double-counting in the CES. 

Again, while the current efforts in this regard have not been promising, it is also 

true that outside data on turnover from BNA points to 1999 as having the highest 

turnover in two decades, a cyclical change that is not clear in the job changing rates 

calculated from the CPS, but which does line up fairly well with a time of increased 

CES employment. Thus, it may be worth at least looking at whether UI wage records 

seem to show more cyclicality than the CPS turnover estimates do. 

More generally, the BLS may wish to study the role of very short duration jobs in 

the labor market. The potential role of short duration jobs can be illustrated by 

comparing UI wage records and 202/QCEW employment numbers for a given payroll 

period. The UI wage records capture all employer-employee covered jobs over a 

quarter regardless of duration. When one compares the total number of employeremployee matches to the total number of covered jobs in the 202/QCEW for the quarter 

(using for example the average of the monthly numbers from 202/QCEW monthly 

reports), the number of matches far exceeds the point in time number of jobs. The 

reason for this is that many jobs have a very short duration. While the UI wage records 

do not provide duration directly, the UI wage records do provide earnings. Many of 

these jobs have very low earnings suggesting duration often measured in days. It may 

be that the relative extent of very short duration jobs varies systematically by industry, 

with the cycle, etc... and this in turn impacts the size of the CPS-CES discrepancy over 

time. 

III. Questions related to improving the monthly estimates 

The following questions address the broader issues of data quality and definition of 

employment that are likely to be related to the differences in the pattern of employment 

growth in the CPS and CES post 2001 recession. While the CPS measure of monthly 

employment change is more comprehensive than the CES measure, the former is 

subject to more variability and therefore tends to be ignored in analysis of the monthly 

Employment Situation. A near-exclusive focus on changes in CES employment levels 

creates the potential for overlooking labor market developments in non-wage and salary 

employment. BLS has considered four alternatives for developing more reliable, 

comprehensive measures of monthly employment change. 
6

Question 5. Should the CPS sample be increased to provide less volatile and, 

therefore, more useful estimates of monthly employment change? In a letter 

to Congressman Dreier, BLS noted that to make the CPS standard error on 

over-the-month change in employment equal to that of the CES, the CPS 

sample would have to be about ten times larger. Costs for the program would 

increase proportionately. BLS can provide other information about the effect 

and cost of increasing the CPS sample at the subcommittee’s request. 

RESPONSE: 

It does not appear that imprecision per se in the CPS causes systematic deviation in 

the two employment series but rather other factors. Therefore, increasing the CPS 

sample tenfold to make the over-the-month standard error the same seems like an 

enormous waste of money that could instead be better spent on supplemental surveys of 

the CPS that probed more on contingent work and self-employment. The committee 

believes that the areas of work outlined below would be a more productive use of 

scarce BLS staff time and resources. 

Question 6. Should BLS investigate the use of modeling to develop a CPSbased national employment series that would provide estimates of the 

underlying trend? BLS has done research into the use of an experimental 

model-based method to remove the effects of sampling error as well as 

seasonality from the time series for selected demographic groups. BLS could 

try to apply these techniques to total employment. 

RESPONSE: 

The committee believes that this is a very constructive way to proceed and should 

be a priority for BLS. Any modeling that is meant for a subset of the data should 

perform equally well on the full sample. It also seems like a simple extension of 

current work that would be a relatively low cost approach to improving more reliable 

measures of monthly employment change. 

Question 7. Another alternative would be to model the portion of CPS 

employment that is outside the coverage of the CES. The CPS collects data 

on the self employed, agriculture, and other types of employment not included 

in the CES. Like total CPS employment, these data tend to be noisy from 

month to month. Again, modeling might be used to estimate the underlying 

trend, and the trend estimate for non-CES employment could be published 

along with the monthly estimates from the CPS and CES. Does the 

subcommittee believe such estimates would contribute to the analysis of 

current employment developments? 

RESPONSE: 
7

This is a productive undertaking by BLS. Recent research on self-employment 

done by researchers such as Robert Fairlie may be useful for identifying supplemental 

questions for the CPS that would be more informative about these areas of 

employment. It would also be helpful to re-examine the issue of respondent bias in the 

CPS. Reliance on a single respondent to answer questions for all members of the 

household, especially for more non-traditional work may result in mis-coding of work. 

Exploring the relative magnitudes of variation for respondents, spouses, and other 

household members would be useful. 

Question 8. Yet another alternative would attempt to obtain IRS 

administrative records and develop a model-based monthly estimate of selfemployment “jobs” that could be combined with the CES count of wage and 

salary jobs for a comprehensive measure of monthly employment change. 

RESPONSE: 

Again, this seems a productive path to pursue. In general, more data sharing 

among the statistical agencies would improve the quality and interpretation of data for 

public policy purposes. To date, while progress has been made working with the IRS to 

gain data sharing access, these efforts have been limited. 

IV. New research 

While it is likely that further analysis of the observational data from CES and CPS 

will yield insights into the causes of the CES-CPS differences, it is unlikely that those 

insights will be sufficient to answer all questions. For that reason, it may be useful to 

propose a set of small, highly controlled auxiliary studies, with new data collections. 

These might be labeled as small scale tests of explicit hypotheses about causes of the 

differences between the two series. The following example illustrates how this might 

be done. 

Decision-making Regimen for Auxiliary Methodological Studies’ 

Proposals 

The regimen for suggesting these hypotheses might involve the following steps: 

1. for each question raised about possible sources of the CPS-CES discrepancy, 

assess the weakness of current data to answer the question 

2. identify specific hypotheses that need to be tested 

3. identify additional secondary data analyses that address these hypotheses 

4. discuss research design alternatives that could test these hypotheses 

5. assess what magnitude of a design is feasible, given practical concerns (e.g., time 

constraints, budget constraints) 

6. propose a methodological study 
8

Two Likely Designs for Auxiliary Methodological Studies 

A. Qualitative Investigations into Causes for Reporting Error

Several of the questions posed to FESAC about the CPS-CES discrepancy suggest 

that there are subgroups of the target populations (i.e., employers and persons) that may 

disproportionately contribute to the differences between the two series. Examples 

include new employers on the CES side, changing categories of employment and 

misreporting of self-employment, and differential inclusion of the “residual foreign 

born” across the two surveys. Insight into the CPS-CES discrepancy could be obtained 

by small scale qualitative studies seeking detailed insight into employer record keeping 

practices and person-level concepts of working. The studies would not be open-ended 

but guided by hypotheses generated from the statistical analysis of the CES-CPS 

comparisons. These qualitative investigations would yield additional information on 

potential causes that could then be examined via additional secondary data analyses. 

It is likely that there will be many questions remaining that could only be answered 

through some primary data collection. 

B. “Cross-Over” Structured Measurement for Quantitative Estimation

There is one “cross-over” design that seems likely to emerge from the five steps – a 

data collection that mimics the CES survey at the employer level and the CPS design 

for the employee/person level. The cross-over design might have two sampling frames, 

one of employers and the other of households. A sample of employers would collect 

both CES data (using the CES methods) and sample and measure employees (using 

CPS methods). A sample from the household frame (in geographical areas adjacent to 

the sample employers’ establishments) would be measured using CPS methods, and 

among those reporting that they were employed, a follow-up would take place with 

their employer using CES methods, including a specific check about whether the person 

would have been included in that employer’s CES report. Given the geographic and 

industrial variability in the CES and CPS discrepancy, one could choose pairs of areas 

that have different levels of discrepancy and select these areas so that they span the 

variability in likely explanations for differences. Specific hypotheses and appropriately 

powered studies would be used. 

What These Auxiliary Studies Will and Won’t Do 

These auxiliary studies will most likely not be used to provide quantitative 

estimates of components of the differences between CES-CPS due to specific causes; 

however, they will generate preliminary evidence to support or refute alternative 

hypotheses of the causes. Furthermore, these auxiliary studies will not identify how the 

CES-CPS differences might be diminished by changes in survey design or estimation ; 

they may, however, suggest what steps might be considered for future design changes.