The Gemini Project:

The Consumer Expenditure 

Survey Redesign Initiative 

Adam Safir

Office of Prices and Living Conditions

Division of Consumer Expenditure Survey

BLS – Statistics Canada Meeting

November 24, 2015
Presentation Outline

1. CE Program Overview

2. CE Methods Improvements

3. Redesign Overview

4. Proof of Concept Test Overview

5. Challenges Ahead
Survey Goals

 To meet the need for timely and detailed 

information about the spending patterns of 

different types of households

 To provide the basis for revising the 

expenditure weights for the Consumer Price 

Index (CPI) every two years

CE Program Overview
Survey History

 First BLS survey was in 1888-1891, conducted 

sporadically until 1940s, and then every 10 years

 The continuing CE began in 1980, with a recognized 

need for a constant source of data on the economic 

condition of America’s consumers

 The continuing CE allowed CPI to revise weights more 

often (initially a 3 year cycle, now a 2 year cycle with 

sample expansion in 2000)

 Current design based on 1972-73 survey format

CE Program Overview
Scope & Sample

 Nationwide survey designed to represent the U.S. 

civilian non-institutionalized population

 Data are collected from consumer units (CUs) -

people living at one address who share living 

expenses (in most cases, same as HHs)

 The survey is designed to represent all urban and 

rural areas in the U.S. (of note, CPI only uses the 

urban sample)

 Sample selection begins with definition and selection 

of geographic areas (currently 91 PSUs in sample)

CE Program Overview
Data Collection

 The U.S. Census Bureau collects CE data for the BLS 

using two different surveys, with separate samples:

Interview, quarterly, 4 interviews, n=28K per year

Diary, 2 weekly diaries, n=14K per year

 Respondents in both surveys are asked to describe 

“all” of their spending. Business expenses and 

reimbursements are excluded.

CE Program Overview
Interview & Diary Improvements (1)

 CAPI Interview (2003)

 CAPI Diary demographics and income (2004)

 Contact History Instrument (2005, 2006)

 User Friendly diary (2005)

 Diary auto-coding system (2005)

 Biennial Interview questionnaire revisions (2005-15)

 Elimination of bounding interview, reducing survey 

from 5 to 4 waves (2015)

CE Methods Improvements
Interview & Diary Improvements (2)

 Timely and relevant questionnaire revisions

Medicare prescription drug program (2006)

Tax stimulus: amount and how used (2008, 2009)

 Statistical methods improvements 

Income imputation (2004)

Improved Diary/Interview source selection for best 

publication estimates (biennially since 2007)

Income tax estimation implementation (2013)

CE Methods Improvements
Gemini Project: Motivation

 Evidence of measurement 

error

 High level of burden 

(presumed to negatively 

affect data quality)

 Need for greater operational 

flexibility

 Changes in spending 

behaviors and technology

Redesign Overview
Gemini Project: Motivation

Redesign Overview
Gemini Project: Objectives

 Verifiable reduction in measurement error, with a 

particular focus on underreporting

 No harmful effect on response rates

 Neutral impact on budget

 Secondary objective: Reduction in burden 

Redesign Overview
Data Requirements

 Annual expenditure estimates of total household 

spending for the U.S. population

 Month of expenditure(s) for each expenditure 

category

 Data collected at a minimum of 2 points in time, 1 

year apart 

 Minimum set of expenditure/non-expenditure data 

elements from each household

Redesign Overview
Key Proposal Inputs

 National Academies’ Committee on National Statistics 

(CNSTAT)

 Expert panels 

 External discussion events

 Ongoing research on key topics

 Census staff & Field Representative (FR) input

Redesign Overview
Post Wave 1 

Non -monetary Incentive:

CE Experience Package

Respondent 

engagement 

mailings

Diary Week

& Records Collection

Advance Mailing

Wave 1

Visit 1: Recall Interview

Visit 2: Records Interview

Wave 2

12 months 

between waves

Advance Mailing

Visit 1: Recall Interview

Diary Week

& Records Collection

Visit 2: Records Interview

Design Proposal

Redesign Overview
Comparison of Design Features

Redesign Overview
Current vs. Redesign: 

Sample Design

Advantage:

 Reduced costs of maintaining two samples

 Diary level detail for all CUs

Disadvantage:

 Potential burden for household completing both the 

Interviews and Diary

Current Redesign

Two independent Samples (Diary and 

Interview)

One Integrated Sample (Interview, 

with Diary keeping component)

Redesign Overview
Current vs. Redesign: 

Interview

Advantage:

 Ease respondent burden with half of typical content of the 

current interview

 Better quality data for records focus

Disadvantage:

 Rely on completion of 2nd interview for complete spending 

from one household

Current Redesign

Single Interview for large, recurring 

expenditures

2 interviews: Recall based and 

Records based

Redesign Overview
Current vs. Redesign: 

Diary (Mode)

Advantage:

 Electronic diary – more flexible and easier for respondent

 Electronic diary – in the moment reporting

 Respondent uses mode most comfortable with (Electronic 

or Paper)

Disadvantage:

 Design of electronic diary does not meet respondent 

expectations

Current Redesign

Two one-week open ended paper 

diaries

Electronic one week diary with paper 

back-up

Redesign Overview
Current vs. Redesign: 

Diary (Mode)

Challenges Ahead
Current vs. Redesign: 

Diary (Level)

Advantage:

 Reduce error due to proxy reporting

 Spreads burden across household members, instead of on 

an individual respondent

 Increased salience among participants

Disadvantage:

 Requiring all household members to complete diary could 

reduce response.

Current Redesign

Single paper diary per household Individual diaries for all households 

15 and over. 

Redesign Overview
Current vs. Redesign: 

Incentives

Advantage:

 Increase response rates

 Increase respondent motivation/Improved data quality

 Fewer contact attempts needed to complete interview 

(saves money)

Disadvantage:

 Costly 

Current Redesign

No Incentives Performance-based incentives at the 

individual and household level

Redesign Overview
Current vs. Redesign: 

Expenditure Categories

Advantage:

 Fewer questions – reduced burden and interview length

Disadvantage:

 Less detailed expenditure data

Current Redesign

Detailed UCC level More highly aggregated

Redesign Overview
Advantage:

 Improvement of annual variance estimates 

 Reduced measurement error resulting from conditioned 

underreporting

 Analysis of 12-month change in expenditures/income

Disadvantage:

 FR may lose rapport with respondent between waves

 Some users need one year of expenditures/income for analysis

Current Redesign

4 waves - 12 consecutive months of 

expenditures per CU, each wave 

treated independently

2 waves - 3 months of expenditures, 

set 12 months apart, each wave 

treated independently

Current vs. Redesign: 

Waves

Redesign Overview
Linking Redesign Features with 

Data Quality Improvements

Redesign Feature Link to Data Quality Improvement

Incentives Increase cooperation & engagement

Technology Promote real-time data capture

Individual diaries Reduce proxy reporting

Reduced interview length

Reduced survey content

Reduce burden

Increased record use Increase accuracy

Redesign Overview
Redesign Features: 

Benefits of Record Use

Redesign Overview
Redesign Features: 

Benefits of Incentives (1)

Redesign Overview

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20+

Cumulative percent

No. of attempts at contact

Number of contact attempts to attain final disposition

2009 study(n=21,733) 2015 study(n=18,301)
Redesign Features: 

Benefits of Incentives (2)

Redesign Overview

23.9

27.1

43.9

15.6

10.6

14.7

35.7

20.9

 Records used >90% of the

time (% n)

 Information book used

>90% of the time (%n)

 Number of reported

expenditures

 Prevalence of edited

expenditure (% MTAB

records)

CE Interview Survey Doorstep concerns and reporting behavior in Wave 5

(Kopp, McBride, and Tan (2013; Table 11)

Doorstep concerns: None(n=1,360) Low (n=657) High (n=484)
Proof of Concept Test (POC): 

Overview

 Ensure that the basic underlying structure 

and components of the new design are 

feasible

 Designed to mirror the proposed design to 

the fullest extent possible

 Conducted with Census production FRs

 Sample performance –

53 percent response rate (preliminary)

n=515 (expected)

POC Overview
Proof of Concept Test (POC): 

Research Objectives

 Methodological issues - Ability & willingness of 

respondents to complete all components, i.e. provide 

recall information, complete diary, provide records 

 Operational factors - Contact attempts needed, length 

of visits, technical issues with individual diaries, 

effectiveness of or logistical issues with incentives

 Experiential factors –

Respondent experience (task difficulty, level of effort, 

reactions to materials/incentives)

FR experience (difficulties completing interview, time 

spent on each case)

 Effectiveness of system and material improvements

POC Overview
POC Web Diary

POC Overview
POC Overview
POC Preliminary Findings

 Response rates exceeded expectations based on 

earlier tests (although still lower than production)

 Feedback about the new design from the FRs via 

debriefings was very positive

Liked the ability to hand the incentives to the 

respondent after completed interview

Felt incentives helped gain cooperation and 

improve quality throughout the process

 An early look at Visit 2 shows record use to be 

strong, with at least 1 record for 88% of interviews 

(and almost half of the 10 sections)

POC Overview
Timeline for Redesign 

Development & Implementation

Design Proposal and Planning

Approved redesign proposal released (July 2013)

User impact of redesign (July-December 2013)

Roadmap developed (September 2013)

Field Tests (Develop, Field, Analyze)

Individual and Web Diaries (2012-15)

Proof-of-Concept Test (2015)

Incentives Test (2016)

Large-Scale Feasibility Test (2019)

Dress Rehearsal (2021)

Implementation date depending on funding availability

Challenges Ahead
Challenges Ahead: Organizational

 Meeting (or amending) redesign requirements

Reducing respondent burden due to data security 

requirements for online instruments

Enabling FR handling of incentives

Returning “data of value” to respondents through a 

spending summary report

 Administrative data linkage projects

Incorporating innovations and lessons learned from 

using auxiliary housing and income data files from 3rd

party aggregators (in conjunction with CARRA)

Challenges Ahead
Challenges Ahead: Timing

 Challenge: Synthesizing test results into in-progress 

redesign testing and implementation

 Question: With the redesign project schedule 

constrained by time and funding, how to develop a 

structured approach to designing and implementing 

successor tests without complete information from 

predecessor tests, and options for updating test plans 

in progress?

Challenges Ahead
Challenges Ahead: 

Technological (1)

 Challenge: Keeping up with the pace of technological 

change & meeting respondent expectations

 Question: With the increased pace of technological 

innovation and social penetration/adoption, thoughts 

regarding extent of the program office’s redesign 

resources to be allocated to ongoing environmental 

scanning and evaluating new approaches, in the 

context of an existing full scale redesign effort; also 

thoughts on opportunity and cost structure tradeoffs?

Challenges Ahead
Challenges Ahead: 

Technological (2)

 Challenge: Negotiating the promises and pitfalls of 

emergent private sector technologies, many with 

limitations that are not yet fully understood or well suited 

to the Federal government data collection environment?

 Example: Frequently hear some variation of “why don’t 

you just use Mint.com”, but no discussion of:

respondent concern about government data linking

unbanked respondents

incomplete account linking

lack of entries for cash transactions

insufficient details in big box store transactions

vast differences in technological sophistication by both 

FRs and respondents 

Challenges Ahead
Contact Information

Adam Safir

Branch of Research and Development

Consumer Expenditure Survey

www.bls.gov/cex

202-691-5175

safir.adam@bls.gov