Consumer Expenditure Survey

CNSTAT Panel Briefing

Measurement Error & 

Gemini Project Overview

Adam Safir

Division of Consumer Expenditure Survey, BLS

February 8, 2011
 Measurement Error

 Evidence

 Possible Sources

 Internal Research

 Gemini Project Overview

 Motivation for Redesign

 Major Tasks

 Activities Completed & Upcoming

 Research

Outline
Measurement Error: Evidence

From 1984 until the present, the ratios of 

aggregate expenditure estimates from the 

CE compared with Personal Consumption 

Expenditures (PCE) data from the National 

Accounts show a declining trend for a

number of spending categories

3
Measurement Error: Evidence, cntd.

 The 2008 CE Program Review Report 

cites item nonresponse and measurement 

error as areas of concern

 Subgroup comparisons show differences 

in expenditure reporting by mode of 

interview and use of recall aids

4
Measurement Error: Possible Sources

 Complexity: detailed information

 Conditioning: 22 sections, 5 waves

 Proxy reporting: household respondent

 Recall effects: 3-month reference period

 Survey length: 60 minute average

 Telephone administration: 33%

5
Measurement Error: Internal Research

Complexity

 What Does ‘Usual’ Usually Mean? (Edgar, 2009)

Conditioning

 Learning to Say No: Conditioned Underreporting in 

an Expenditure Survey (Shields & To, 2005)

Proxy reporting

 The Quality of Proxy Reports on the Consumer 

Expenditure Survey (Kojetin & Jerstad, 1997)

6
Internal Research, cntd.

Recall effects

 Analysis of Recall Effect on Reporting of 

Expenditures for CE (Steinberg et al., 2006)

Survey length

 Forthcoming results from Measurement Issues and 

Order Effects studies

Telephone administration

 Mode Effects in a Survey of Consumer Expenditures 

(Safir & Goldenberg, 2008) 7
Gemini Project

Mission 

 Improve data quality through a verifiable 

reduction in measurement error, with a particular 

focus on under-reporting. 

Objective

 Develop a detailed research plan for implementing 

a redesign of the CE surveys, including piloting, 

evaluation, and transition.

8
Gemini Project: Redesign Motivation

Technological advances

 Transformation of respondent behavior

 Emergence of new data collection options

Need for more flexibility

 Allow for faster implementation of questionnaire 

changes 

 Allow for mixed-mode interviewing

9
Redesign Motivation, cntd.

Evidence of measurement error

 Possibly due to complexity, conditioning, proxy 

reporting, recall error, survey length, mode effects

Same basic design for past 30 years

 Different environment today

10
Gemini Project: Major Tasks by Year

Year Tasks

2009-10 (1) Research database

(2) Data requirements and priorities

(3) Data quality definition

2010-11 (4) Hold research events, produce reports

2010-13 (5) Conduct ongoing supporting research

2012 (6) Assess user impact of design alternatives

(7) Recommend survey redesign

(8) Propose transition roadmap

2013+ Piloting, Evaluation, Transition 11
Gemini Project: Activities To Date

Date Report

Oct 2009 Data Quality Definition Report (public)

Mar 2010 Research Tracking System Report (internal)

Jun 2010 CPI Requirements Paper (public)

Aug 2010 Conference Team Report (internal)

Aug 2010 Data User Needs Report (internal)

12
Activities to Date, contd.

Date Event

Jan 2010 Survey Redesign Panel

Mar 2010 Data Capture Technology Forum

May 2010 Panel on Respondent Record Use

Jun 2010 Data User Needs Forum

Dec 2010 Methods Workshop

13
Gemini Project: Upcoming Activities

Date Event

Mar 2011 CE Data Priorities Summary

May 2011 Household Survey Data Producers Workshop

May 2011 AAPOR Presentations: Burden, Global 

Questions, Data Capture & Financial Software

Sep 2011 CE Redesign Options Workshop

Mar 2012 CNSTAT Panel Recommendations (pre-pub rpt)

14
Gemini Project: Ongoing Research

• Alternative designs

• Technology

Data capture and 

financial software

• Cost

• Reporting quality

Diary double 

placement

• Recall period, interview frequency

• Split questionnaire

Measurement 

issues study

• Alternative designs

• Technology

Online diary 

prototype

• Interview length

• Reporting quality Order effects test

15
Gemini Project: Research, cntd.

• Interview length

• Statistical issues

Split 

questionnaire

• Availability

• Utility Records study

• Feasibility

• Effectiveness

Balancing edit 

(proposed)

• Cost

• Reporting quality

Bounding 

(proposed)

• Proxy reporting

• Response rates

Individual diaries 

(proposed)

16
Gemini Project Website

http://www.bls.gov/cex/geminimaterials.htm:

 Gemini overview

 Gemini events

 Gemini presentations

 CE survey methods research

 CE methods and survey forms

 Other redesign related events and studies

17