Consumer Expenditure Survey

Research & Plans for the Future

Adam Safir

Branch Chief

Research and Program Development

Consumer Expenditure Survey

October 16, 2012
 

 

 

 

 

 

 

 

 

 

 

 

Presentation Overview

1. Background 

2. Gemini Project Redesign Objectives

3. Focus of Recent CE Research

4. Gemini Project Design Team Plans

5. Use of CNSTAT’s Report

2 
 

 

 

 

 

 

 

 

 

 

 

 

Continuous Survey 

Improvement Process

 Biennial – CEQ instrument revisions

 2013 – Inc Tax Estimation w/NBER Calculator

 2009 – Telephone thresholds (CEQ)

 2005 – Diary keying and auto-coding system

 2005 – User friendly diary form

 2005 – Contact History Instrument (CHI)

 2004 – Income imputation

 2004 – CAPI (CED) demographics and income

 2003 – CAPI (CEQ)
 

 

 

 

 

 

 

 

 

 

 

 

Redesign Objectives

 Reduce measurement error, and in 

particular, underreporting

 Reduce burden

 Hold neutral, or reduce, costs

 Monitor redesign results

 Research agenda
 

 

 

 

 

 

 

 

 

 

 

 

Focus of Recent CE Research: 

Reduce Measurement Error

5 

 Reduce number of interviews

 Reduce interview length

 Reduce reference period length

 Reduce proxy reporting

 Maximize record use & minimize recall

 Incorporate new technology

 Incorporate multi-mode interviewing
 

 

 

 

 

 

 

 

 

 

 

 

Focus of Recent CE Research: 

Reduce Burden

6 

 Reduce number of interviews

 Reduce interview length

Streamlining the questionnaire

Split questionnaire research

Global questions

– Diary to interview imputation

– Backcasting

– Within quarter imputation

– Auxiliary source data
 

 

 

 

 

 

 

 

 

 

 

 

Focus of Recent CE Research: 

Reduce Costs

 Reduce interview length

 Reduce number of interviews

For example, eliminate the bounding 

interview

7 
 

 

 

 

 

 

 

 

 

 

 

 

Focus of Recent CE Research: 

Monitor Redesign Results

Data Quality Profile

 A consistent, well-defined set of metrics can be used 

to establish baselines for monitoring trends in the 

quality of survey activities and evaluate the impact of 

survey design options

Measurement Error Analysis

 Determine sources of measurement error on 

expenditure reporting and develop a methodology for 

tracking and evaluating changes in measurement 

error due to design changes

Burden Index

 Measure the effect of alternative design options 8 
 

 

 

 

 

 

 

 

 

 

 

 

In-Progress & 

Planned Research

In-Progress & FY2012

 Records Information & Feasibility

 Web Diary Test, Individual Diary Test

FY2013

 Redesign Monitoring

 Redesign Feasibility Testing

FY2014

 Additional Redesign Feasibility Testing

 Additional Analyses of Completed Studies

9 
 

 

 

 

 

 

 

 

 

 

 

 

Gemini Project Design Team

 Initiated in July, 2012

 Joint team with the Bureau of the Census

Objective 

 Create a single proposal for a redesigned CE that 

expects to result in a verifiable reduction in 

measurement error

Scope

 Synthesize inputs from 2009-12 Gemini Project 

activities, as well as from independent design 

proposals (Abt-SRBI, CNSTAT, University of Nebraska, 

University of Wisconsin, Westat) 10
 

 

 

 

 

 

 

 

 

 

 

 

Gemini Project Design Team: 

Timeline 

 Jul 2012 – Kick-off

 Oct 2012 – Data collection recommendations, by quex group

 Nov 2012 – Redesign proposal outline (i.e., table shell) 

 Nov 2012 – Decisions regarding independent design proposals

 Dec 2012 – Summary of FR, Census, BLS staff redesign ideas

 Dec 2012 – High level decisions re: survey redesign elements 

 Feb 2013 – Present update at CE Annual Meeting

 Feb 2013 – Detailed overview of proposed redesign elements

 Mar 2013 – Draft redesign proposal report 

 Jun 2013 – Final redesign proposal report

11
 

 

 

 

 

 

 

 

 

 

 

 

Gemini Project Design Team: 

Challenges 

 Synthesizing research results into a 

comprehensive redesign plan

 Budget for research, testing, evaluation, and 

implementation

 Sample size requirements

 Timing of research findings

 Pace of social & technological change

12
 

 

 

 

 

 

 

 

 

 

 

 

Pace of change.... 

January 12, 2012: “In Seoul, a virtual grocery 

store in the subway” (CNN Online)

13
 

 

 

 

 

 

 

 

 

 

 

 

October 8, 2012: In Forest Glen, a virtual grocery 

store in the subway (Washington, DC suburbs)

14
 

 

 

 

 

 

 

 

 

 

 

 

“Half of U.S. adults now own a tablet 

or smartphone” (PEJ, 2012*) 

15

* “The Explosion in Mobile Audiences and a Close Look at What it Means for News”

A. Mitchell, T. Rosenstiel , L. Houston Santhanam of PEJ, and L. Christian of the Pew Research Center

October 1, 2012 
 

 

 

 

 

 

 

 

 

 

 

 

CNSTAT Report: Value

16

High-level thinking, justification, and guidance from 

nationally-recognized experts in economics, statistics, 

and survey methodology on factors affecting the quality 

and cost of CE data, and improved options for collecting 

those data, culminating in:

 Concurrence on Issue Areas

 Broad Recommendations

 Specific Design Proposals

 Cost Estimates for Maintaining Budget Neutrality

 Advocacy for Additional Resources
 

 

 

 

 

 

 

 

 

 

 

 

CNSTAT Report: 

Promising Design Features

17

 One sample design

 Flexible recall periods & interview structure

 Increased use of technology, e.g., tablets

 Use of tech to encourage ‘in the moment’ reporting

 Increased reliance on self-administration

 Increased use of records

 Reduce proxy reporting

 Mixed mode data collection

 Large incentives

 Modular design, with a core survey
 

 

 

 

 

 

 

 

 

 

 

 

CNSTAT Report: 

Panel’s Recommendations

18

6-01 Prioritize CE Data Uses for Redesign Trade-offs

6-02 Implement a Major Redesign ($)

6-03 Fund Several Major Feasibility Studies ($) 

6-04 Sync Reference Periods for Exp & Non-Exp Items

6-05 Use Tablet for Self-Administration (w/Paper) ($)

6-06 Develop a Redesign Roadmap within 6 Months

6-07 Use Incentives ($)

6-08 Pursue a Long-Term Research Agenda

6-09 Increase Size & Capability of Research Staff ($)

6-10 Engage Outside Experts in App Dev ($)

6-11 Target Research on CNSTAT Recommended Topics

6-12 Fund a Methods Panel (or Research Sample) ($)
 

 

 

 

 

 

 

 

 

 

 

 

Thank You, 

from CE to CNSTAT!

19
 

 

 

 

 

 

 

 

 

 

 

 

Contact Information

Adam Safir

safir.adam@bls.gov

20