1 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Revising the CE Surveys 

to Collect Outlets 

Erica C. Yu

Office of Survey Methods Research

CE Survey Methods Symposium

July 2018
2 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bl s.gov bl s.gov

Background: Inputs to the CPI

 CE Surveys: identify what is in the market 

basket and how expenses should be weighted

 Telephone Point-of-Purchase Survey: build a 

frame of businesses where consumers shop
3 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bl s.gov bl s.gov

Considerations of Adding 

Point-of-Purchase Questions to CE

 Fact: Increase in number of questions asked

 How would that affect the survey experience?

Will answering these questions be effortful?

Will these questions change the ‘survey context’?

 How will the quality and characteristics of the 

point-of-purchase data be affected? 

Will the CE survey yield enough data?

Will the CE survey yield high quality data?
4 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bl s.gov bl s.gov

Challenges of Integrating 

Two Surveys

 Different definitions of item categories

 Different reference periods

 Different sample sizes

 Different modes

 Different materials and aids
5 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bl s.gov bl s.gov

Research Overview

 Step 1: Exploratory lab studies to evaluate 

feasibility (any show-stoppers?) and feedback 

on perceived burden for Diary and Interview

 Step 2: Limited implementation in production 

to evaluate data quality and objective burden

 Step 3 (ongoing): Exploratory online studies to 

try new questions to collect more data 
6 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bl s.gov bl s.gov

Step 1a: Feasibility in CE Diary

 n = 60 participants in the lab

 Simulated data entry from receipts

 Computer task

 Three group comparison:

Group 1: “item-based” data entry

Group 2: “transaction-based” data entry

Group 3: control, no outlet entry 
7 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bl s.gov bl s.gov

Item-Based Data Entry
8 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bl s.gov bl s.gov

Transaction-Based Data Entry
9 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bl s.gov bl s.gov

Step 1a: Feasibility in CE Diary

 Analysis focused on comprehension of data 

requirements, time taken to enter data, 

participant feedback about ease of use and 

preference

 No significant differences between groups 

though trend towards preference for 

transaction-based diary
10 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bl s.gov bl s.gov

Step 1b: Feasibility in CE Interview

 n = 642 participants online

 Participants “purchased” items as part of a 

fictional narrative and then answered CE 

interview questions to recall those items

 Three group comparison:

Group 1: “item-based” outlet as follow-up prompt

Group 2: “transaction-based” prompts

Group 3: control, no prompts for outlets
11 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bl s.gov bl s.gov

Item-Based Recall
12 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bl s.gov bl s.gov

Transaction-Based Recall
13 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bl s.gov bl s.gov

Step 1b: Feasibility in CE Interview

 Analysis focused on number of items 

reported, price reporting error, duplicate 

reporting, and participant feedback about 

ease of use and preference

 No significant differences between groups on 

number of items or price accuracy 

 Participant ratings of the transaction-based 

group showed higher burden and behavior 

showed more duplicate reporting
14 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bl s.gov bl s.gov

Conclusions on Feasibility

 Addition of point-of-purchase questions likely 

to have no major effect on data quality or 

perceived respondent burden

 Optimal format varies by survey

Diary: Transaction-driven format lends itself to a 

transaction-based diary

Interview: Long reference period with recall from 

long-term memory lends itself to an item-based 

interview with follow-up prompts
15 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bl s.gov bl s.gov

Step 2: Adding Questions to 

CE Interview

 A handful of point-of-purchase questions were 

added to interviews

2016 Quarter 3, 4th wave interviews

New questions spanned across 5 interview 

sections
16 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bl s.gov bl s.gov

Step 2: Adding Questions to 

CE Interview

 Audit trails were used to gauge how much time it 

takes to collect point-of-purchase data

Store name: 19 seconds

Purchase mode: 7 seconds

Location: 15 seconds

Respondent confusion leads to more interviewer 

probing (e.g., point-of-purchase for gasoline)

 Expenditure data were used to assess impact on 

CE data quality

Don’t knows and Refusals not changed

Possible data quality check for re-mapping expenses
17 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bl s.gov bl s.gov

Conclusions from Implementation

 Addition of point-of-purchase questions leads 

to a non-negligible amount of time added to 

the interview (approximately 40 seconds per 

item category)

 Interviewers reported that collecting point-ofpurchase information is a natural extension

 Evidence of possible improvement of CE data 

quality through re-mapping

 The interview may not yield enough outlet 

data for CPI
18 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bl s.gov bl s.gov

New Design - Interview
19 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bl s.gov bl s.gov

New Design - Diary
20 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bl s.gov bl s.gov

Future Challenges

 Educating interviewers about the reasons for 

the questionnaire changes

 Monitoring the impact of collecting point-ofpurchase in the Diary

Burden unknown, difficult to measure

New tech developments: Receipts?

 Incorporating the new point-of-purchase data 

into review processes to improve data quality
Contact Information

21 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Erica C. Yu

Office of Survey Methods Research

Bureau of Labor Statistics

www.bls.gov/osmr

202-691-7924

Yu.Erica@bls.gov