1 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

A Household Survey Perspective on 

Re-Engineering an Establishment 

Survey

Adam Safir

Bureau of Labor Statistics

Re-Engineering and Modernizing the 2017 U.S. Economic Census 

Federal Committee on Statistical Methodology 

March 9, 2018
2 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Re-engineering and Modernizing 

the 2017 Economic Census

Discussant Remarks

 Consumer Expenditure Surveys

 CE Redesign Process

 High-Level Challenges of Redesign/Re-engineering

 Comments on Presentations
3 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Consumer Expenditure Surveys

 Ongoing survey which collects detailed expenditure information from a national 

sample of households

 Conducted by the Census Bureau on behalf of the Bureau of Labor Statistics

 Interview Survey

 1-hour interview, 4 waves, 3 months apart

 Collect ‘big ticket’ and recurring expenses

 Approximately 26,000 completed interviews per year, 63 percent response rate (2016) 

 Diary Survey

 2 one-week self-administered paper diaries

 Collect daily, small and easily forgotten expenses

 Approximately 13,000 completed diaries per year, 61 percent response rate (2016)

3
4 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Data from the CE Program

 Provide the only source of information on the complete range of consumer 

expenditures & incomes in the United States, as well as consumer characteristics

 Provide budget shares (i.e., weights) for detailed expenditure categories in the 

Consumer Price Index (CPI) 

 Provide data essential for administering certain federal and state government 

programs 

 Provide critical information for policy and economic analyses

4
5 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

CE Redesign Process

 Motivation

 Evidence of measurement error

 Changes in technology and spending behaviors

 Need for greater methodological and operational flexibility

 Constraints

 Meet user requirements (w/in reason, i.e. not being all things to all users)

 Operationally be cost neutral (aside from start-up costs)

 Stages

 Gather (2009-13): Identify user needs, collecting external input, define data quality for us

 Decide (2014): Data requirements, design decisions, research roadmap

 Implement (2014-): Test, implement, evaluate 

5
6 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Key Inputs

 Expert panels (e.g., CNSTAT)

 External events

 Outreach with stakeholders

 Collecting information and advice

 Ongoing research and testing on key topics
7 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Similarities/Differences in Redesign/Re-Engineering

7
8 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Comparison of Redesign Goals

8

Economic Census CE Surveys

Transition to 100 percent electronic 

collection

{Transition to online diary}

Reduce overall respondent burden Reduce measurement error 

Automate operations to increase

productivity

Improve methodological and 

operational flexibility 

Improve data products to reflect 

changing economy

Respond to changes in technology and 

spending behaviors
9 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Comparison of Redesign Goals

9

Goals Economic Census CE Surveys

Increase online collection Transition to 100 percent 

electronic collection

{Transition to online diary}

Increase data quality Reduce overall respondent 

burden

Reduce measurement error 

Improve operations Automate operations to increase

productivity

Improve methodological and 

operational flexibility 

Respond to environmental 

changes

Improve data products to reflect 

changing economy

Respond to changes in 

technology & spending behaviors
10 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Redesign Challenges (1)

1. Requirements. Defining survey requirements, e.g., in terms of varied user needs 

AND balancing those requirements against respondent task burden

2. Resource Uncertainties. Managing uncertainties regarding funding and staffing, 

which limit research, testing (sample sizes), evaluation, and implementation

3. Stakeholder Communication. Reinforcing purpose (e.g., meas. error vs. burden/RRs)

4. Decision making. Accommodating top-down decision making 

5. Synthesizing results. Managing ongoing research efforts while making redesign 

decisions , i.e., synthesizing test results into ongoing redesign testing & 

implementation

10
11 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Redesign Challenges (2)

1. “Federal” technology. Negotiating the promises and pitfalls of emergent commercial 

technologies, many of which have limitations that are not yet fully understood or 

well suited to the federal government environment – e.g., data security login 

requirements for online forms

2. Pace of technology. Keeping pace with technology; not attempting to predict the 

future but not also committing to the present (moving target)

3. Trend lines. Managing breaks in trend lines , i.e., accuracy vs. consistent 

measurement

4. Comparison point. Managing a sometimes higher bar for redesign vs. design

5. Expectations. Managing stakeholder expectations (i.e., bang for buck, results)

11
12 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

An Overview of the Improved 2017 Economic 

Census (Moore & Samples)

1. Data serve as a benchmark for PFEIs (slide 4)... how important is accuracy vs. 

maintaining consistency in trend lines or avoiding breaks in data series?

2. Move to 100% electronic data collection (slide 6)... was this based on testing, or 

more driven by a need to meet respondent or stakeholder expectations?

3. Reporting via secure online portal (slide 9)... were there any significant issues with 

login burden/complexity?

12
13 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Executing a Multi-Year Multi-Method Electronic 

Data Collection Re-engineering (Riemer )

1. Requirements gathering from internal stakeholders (slide 10); noted some issues in 

the logistics of multiple stakeholders... were there also challenges in reaching 

compromise for conflicting stakeholder needs, or sometimes just saying no?

2. Trade-offs in paper mock-ups vs. semi/fully functioning sites (slide 12)... were 

expectations met for the purpose and implementation of each through the testing 

process?

3. Evolving research plans and coordinating research plans is difficult when faced with 

tight timelines, as well as managing and coordinating separate research threads 

(slide 19)... additional thoughts on improving that process?

13
14 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

2017 Economic Census Contact Strategy: Using 

Data to Make Decisions (Johnson)

1. Noting the estimated savings of 24.3 percent over prior year (slide 3) in the annual 

surveys, what are cost savings expectations for the Economic Survey?

2. Any unexpected issues in transferring test results from the annual surveys to the 

Economic Census?

14
15 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

From Research to Implementation of Product 

Est. in the 2017 Economic Census (Davie et al.)

1. Any unexpected issues in the communication and/or cooperation of the research 

and implementation teams (slide 9), and how were those addressed? 

2. In terms of reaching consensus for group decisions (slide 26), was any negotiating 

required for top-down requests/demands from stakeholders?

3. How were unexpected results in implementation testing (slide 35) addressed?

4. To what degree did not having time for detailed investigations (slide 35) affect 

confidence in results?

15
16 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Presenter Comments & Audience Questions