CE’s Gemini Redesign: Updates on 

Development, Implementation, and 

Evaluation

Laura Erhard 

Office of Prices and Living Conditions

Division of Consumer Expenditure Survey

CE Survey Methods Symposium

July 15, 2014
Outline

1. Gemini Project to Redesign CE

a. Overview of the Redesign 

b. User Impact Findings

c. Proof of Concept Test Development

d. A Look Ahead

2
Redesign Objectives

 Verifiable reduction in measurement error, with a 

particular focus on underreporting

 No harmful effect on response rates

 Neutral impact on budget

 Secondary objective: Reduction in burden 

3
Post Wave 

Non-monetary Incentive:

CE Experience Package

Respondent 

engagement 

mailings

Diary Week

& Records Collection

Advance mailing

Wave 1

Visit 1: Recall Interview

Visit 2: Records Interview

Wave 2

12 months 

between waves

Advance mailing

Visit 1: Recall Interview

Diary Week

& Records Collection

Visit 2: Records Interview
Data User’s Impact
Data User’s Impact

 Main Objectives:

Obtain feedback from users on any serious 

issues

Educate users on the structure and content 

of the new CE redesign proposal

Educate users on the implementation 

timeline
Data User’s Impact

 Outreach activities:

Redesign of the Gemini Website

>5400 site visits since update

Presentations at conferences, meetings, and 

webinars

~200 attendees in total

Federal Data users Meetings

5 agencies

Notice in the Federal Register

No responses

User’s Impact Survey

22 respondents
Data User’s Impact

 What we wanted to know:

What are the positive impacts on users’ research?

What are the negative impacts on users’ research?

Concerns about these specific design changes, 

such as: 

– 12-month change instead of 12 months 

consecutive data (assuming no attrition)

– Fewer, more aggregated expenditure categories

– Diary reference period not aligned with Visit 1 

and Visit 2 reference periods

What information/training is needed prior to 

implementation and how much lead time?
Data User’s Impact

 General findings:

Generally positive feedback

Pleased with focus on improved data quality

 Concerns of select users:

Limiting the detail collected in the survey 

Inability to capture of expenditures with 

longer transaction time and one-time vs 

regular expenses (e.g., health care and 

reimbursements, car purchases)

Seasonality issues at micro level since 

respondents reporting for same time of year at 

each wave

Inability to use 4 consecutive quarters
Data User’s Impact

 Resources needed for users: 

Details on the break in the 

series/Crosswalk between current design 

and new design

Information on new file structure and 

variables

Guidance on creating a synthetic panel

At least a month lead time for changes
Proof Of Concept Test (POC)
POC Overview

 Ensure that the basic 

underlying structure 

and components of the 

new design are 

feasible.

 Designed to mirror the 

proposed design to the 

fullest extent possible
POC Goals

 Methodological issues

R willingness to complete all components

R ability & willingness to provide recall 

information

Ability & willingness to for eligible HH 

members to complete diary
POC Goals

 Operational factors

# contact attempts needed to complete 

case

Length of visits

Technical issues with individual diaries

Logistical issues with incentives
POC Goals

 Experiential factors

R interview experience: 

– Perceptions of burden

–Task difficulty

– Level of effort 

–Reactions to materials/incentives

FR experience

–Causes for difficulties completing 

interview

–Non-interview time spent on each case
POC – Details
POC - Details

 Incentive use for diary keeping, how do you 

determine a “complete” diary?

 Increased incentive for records interview –

how do you define “records used”?

 What will constitute a “complete” interview 

(for response rates)?

 Do we need visits spaced throughout the 

month (as in current diaries)?
POC –Details

 How to train respondents to keep diaries?

 What’s the best diary follow up procedures 

for FRs mid-week  logistics of getting FRs 

needed information

 How to train respondent to collect records?

 How to conduct the records visit? 
POC Design
POC Design – Advanced Mailing

 Advanced letter sent via priority mail

Contains $2 bill (token cash incentive)

Contains CE advance mailing language 

modified to describe the POC Visit 1/Ind. 

Diary/Visit 2 design

Includes description of the incentive 

structure for each piece of the completed 

interview
VISIT 1
POC Design – Visit 1 Recall

 Visit 1 (V1) interview very similar to current CE 

Quarterly Interview

 V1 conducted with a modified production CAPI 

instrument

 Allow V1 to be completed during Visit 2 (V2) to 

allow time for Diary placement/Records training 

(if necessary)

Test Order Production

Section #

Section Description

1 8 Home Furnishings and Related Household Items

2 6 Appliances, Household Equipment, and Other Selected Items

3 17 Subscriptions, Memberships, Books and Entertainment Expenses

4 18 Trips and Vacations

5 19 Miscellaneous Expenses
POC Design – V1 Diary Placement

 Gather all available HH members

 Discuss Diary keeping tasks

 Identify eligibility and mode for each participant, 

provide user IDs/passwords or paper diary

Main respondent decides on mode for absent 

members

Collect email and phone number of each 

individuals, asking permission to contact midweek

 Leave materials for respondents

 Leave back-up paper diary
POC Design –

V1 Records Instructions

 The FR will describe the records interview 

and request the respondent to collect 

relevant records from the reference period (3 

months prior to the V1 interview) 

 The respondent will be 

provided with a detailed 

checklist identifying which 

records to collect
DIARY WEEK
POC Design –

Diary Week: Respondents

 Respondents keep open-ended diary 

 Diary kept online (accessible either via web or mobile device) OR fill out paper 

diary.

 Main respondent will record expenditures for ineligible respondents, HH 

expenses, and for any HH member not participating. 

Categories: 

- Food and Drink Away From Home

- Food and Drink For Home Consumption 

- Clothing, Shoes Jewelry, & Accessories 

- All Other Products & Services

 Web diary not yet developed but will be designed to be similar to the mobile 

diary with minor modifications for ease of use on PC

 Help available via a call-center help desk and email
POC Design –

Diary Week: FRs

 FRs will have access to a summary screen of Diaries for their cases

 FRs will call the main respondent within 2 days of placement if a R 

hasn’t logged in OR if R hasn’t made entries

 Phone calls are made to the reference person (main respondent)

 FR can use their discretion to follow up with individuals that 

provided contact information

 FRs will call the household mid-week regardless of entries, to see if 

there are any questions and remind them to collect records for V2

Contact 

Flag

Diary Mode # Successful 

Log Ins

# Unsuccessful 

Log-ins

# Expenditure 

reports

Date of last 

successful Login

Member 1 Web/Mobile 5 1 9 July 9

Member 2 Web/Mobile 0 15 0 July 10

Member 3 Paper -- -- -- July 13

Member 4 Web/Mobile 2 0 5 July 13
VISIT 2
POC Design – V2 Diary Pick Up

 Conduct diary recall process with main R and 

any available HH members

 For each present respondent:

Diary Entries?

Yes

Review of Diary 

Entries

Any additional 

expenditures missed/ 

Receipts collected?

Enter missed 

expenditures in CAPI 

No

Within 2 Days – Total 

Recall

Between 2-5 daysTotal Recall with 

receipts

After 5 days – no 

recall
POC Design –

V2 Records Interview

 Exact protocol TBD

Who handles the records, the FR or 

Respondent?

What order should the interview occur in –

based on record order or predetermined 

order (such as CAPI order)

 CAPI will default to earliest uncompleted 

question (incl. V1 questions) and follow main 

path for records sections

 FR will be able to move between sections by 

using tabs, as currently done in production
Visit 2 – Records Interview

 Conduct records interview with respondent

Test 

Order

Production

Section #

Section Description

6 1 General Housing Characteristics

7 2 Rented Living Quarters

8 3 Owned Living Quarters and Other Owned Real Estate

9 4 Utilities and Fuels

10 10 Rented and Leased Vehicles

11 11 Owned Vehicles

12 13 Insurance Other than Health

13 14 Health Insurance

14 15 Medical and Health Expenses

15 21 Occupations, Work Experience and Income

16 22 Assets and Liabilities
Visit 2 – Respondent 

Debriefing

 A series of questions related to V1 interview, 

diary keeping, recording collection, and V2 

interview

EX:

Reactions to the incentives

General feedback

Any technical difficulties with diaries

Where diary was completed

Amount of time spent collecting 

records

Willingness to share records
FR Debriefings

 FR debriefing questions will be programmed 

in the instrument after each visit with 

questions about each part of the test 

After V1: recall, diary placement, and 

records training

After V2: diary week contacts, diary recall, 

and records.
Looking Ahead
Looking ahead

 Individual and Web Diaries (2012-14)

 Proof-of-Concept Test (2015)

 Question development (2015-16)

 Protocol development (2015-16)

 Incentives test (2016) 

 Large-Scale Feasibility Test (2018)

 Dress Rehearsal (2021)

 Implementation date depending on availability of 

funding and resources throughout process

35
Contact Information

Laura Erhard

Senior Economist

Branch of Research and Development

Consumer Expenditure Survey

www.bls.gov/cex

202-691-5119

erhard.laura@bls.gov