Redesign Options for the 

Consumer Expenditure Survey 

Presented at the 

Consumer Expenditure Survey 

Redesign Options Workshop 

Washington, D.C. 

October 26, 2011 
Pat Cunningham, Bob Fay, David Judkins, 

Bob Patchen, Abie Reifer, Lou Rizzo and Pat Ward 

Westat 

David Cantor, Sid Schneider, Brad Edwards 

Westat

Anonymous reviewers from the expert panel

Carol House, 

Committee on National Statistics (CNSTAT)

Authors

Acknowledgements 
Outline of presentation 

Review of major issues 

Proposed redesign 

Diary 

Quarterly Interview 

Summary and discussion of proposed design 

Evaluation 

3 
Review of major issues 

4 
CE Basic Design 

CPI = 

5 

Diary Quarterly 

Interview

+ 
Parameters from the CNSTAT 

for recommendations 

• Reduce measurement error 

• Adapt to changing retail 

and technological landscape 

• Provide respondents with more flexibility 

• Think‘out of the box,’but ground 

recommendations in as much

survey experience as possible

• Do not let budget constrain ideas too much 

6 
Causes of measurement error 

• CE is very burdensome 

> Quarterly Interview averages 1 hour 

> Recall is difficult 

> Proxy reporting for other consumer unit (CU) members. 

> High burden leads to errors (e.g., “satisficing,”

nonresponse; interviewer shortcuts)

• Retrospective recall is subject to error 

> Omissions, telescoping, estimation 

> Failure to encode (expense amounts) 

7 
Changing retail environment 

• Transactions occur through multiple modes 

> Brick and mortar stores 

> Online expenditures 

 Bill payment 

 Online shopping 

• Electronic records for most purchases 

8 
Which transactions leave an electronic record 

that could be used for CE data collection? 

9 

Cash

29%

Money 

Order 

1%

Bank Account Payment 

5%

Check

13%

Online Bill Pay 

5%

Prepaid Card

1%

Credit Card 

17%

Debit

Card

29% Total = 64.5 transactions 

per consumer per month, 

2009
Transactions with 

e-records/month/consumer, 2009 

% with e-records 

Bill payments 87.0%

Online payments 98.0%

Retail, service, 

person-to-person transactions 59.3% 

All transactions 71.0% 

10
With consolidation in retail industry, 

relatively few retail chains 

• 20 largest grocery chains 

• Walgreens, CVS, Rite Aid 

• Sam’s Club, Costco 

Cooperation from several chains = 

Data for a high proportion of CU expenditures 

Total Sales 

61.6% 

Total 

Pharmacy

Sales 

73.2% 

Warehouse 

Club Sales 

89.5% 
12

Proposed redesign 
Summary of key features of proposed design 

13

Both Diary & Interview Surveys

Scan receipts or mail paper records; 

Download financial files 

Increase monitoring 

Enhance recall interview 

Monetary & nonmonetary incentives 

Vendor Survey
Summary of key features of proposed design 

14

Both Diary & Interview Surveys

Scan receipts or mail paper records; 

Download financial files 

Increase monitoring 

Enhance recall interview 

Monetary & nonmonetary incentives 

Vendor Survey

Diary Survey

Two 7-day collection periods 

Estimates for food, alcohol & other frequent purchases
Summary of key features of proposed design 

15

Both Diary & Interview Surveys

Scan receipts or mail paper records; 

Download financial files 

Increase monitoring 

Enhance recall interview 

Monetary & nonmonetary incentives 

Vendor Survey

Diary Survey

Two 7-day collection periods 

Estimates for food, alcohol & other frequent purchases 

Quarterly Interview Survey

Two 3-month collection periods, 12 months apart 

Estimates for all other purchases
Proposed design: 

use of personal records 

• Ask respondents to download electronic 

information from 

> Credit card 

> Bank 

> Mortgage 

> Any other online source 

• Emphasis on collecting receipts 

• Create records using other methods 

16
17
Proposed design: 

use of personal records 

• Ask respondents to download electronic 

information from 

> Credit card 

> Bank 

> Mortgage 

> Any other online source 

• Emphasis on collecting receipts 

• Create records using other methods 

18
19
20
Proposed design: 

use of personal records 

• Ask respondents to download electronic 

information from 

> Credit card 

> Bank 

> Mortgage 

> Any other online source 

• Emphasis on collecting receipts 

• Create records using other methods 

21
Potential problems 

with increased record use 

• Reduced response rate 

> Countermeasure: accept self-reports too. 

Avoid extremes. 

> Empirical question on effect on total survey error. 

Measurement error vs. non-response bias 

• How many respondents will comply? 

22
Proposed design: 

survey of vendors 

• Ask vendors to provide purchases 

made by respondents 

• Use information to substitute directly for data 

• Use information to improve data quality 

> Fill in data missing from respondents’ reports

> Obtain a measure of accuracy, completeness 

of respondents’ reports

23
Potential problems 

with vendor surveys 

• Timing of collection may not coincide 

with publication schedule 

• Increased cost (assuming sample size is fixed) 

> Possible offset by efficiency gains 

> Empirical question on interaction between 

costs and TSE 

24
Proposed design: develop 

enhanced recall interview

• Recall interview for all respondents 

> Respondent burden mitigated with record use 

• Cues to store data about expenditures 

(e.g., type; amount of expenditure; retail outlet). 

> Consider use of Event History Calendar 

> Refer to any records or receipts. 

• CARI for monitoring question performance 

and interviewer behavior 

25
Enhanced recall interview: 

Pros and Cons 

• PRO 

> Reduce error due to omission and telescoping 

> Data for those with no records or receipts 

• CON 

> Potentially increased interview length 

26
Proposed design: monetary 

& nonmonetary incentives 

• $50 – $70 for main respondent 

$10 – $20 secondary 

• Reports of expenditures 

• Charts showing how household expenditures 

compare with others 

27
Bundle.com compares spending of users and 

others with similar demographic characteristics 

28
29

Proposed redesign 

Diary 
Proposed design: 

multiple diary keepers 

• All individuals 14+ fill out a diary 

• Main diary keeper instructs others in CU 

• Edgar et al (2006) show increases in 

mean expenditures with multiple diaries/CU 

30
Multiple diary keepers: pros and cons 

• PRO 

> Improved measurement 

> Reduced burden on main CU respondent 

• CON 

> Increase in cost 

> Reduction in response rate 

• How do cost and error balance? 

31
Redesigned Diary survey schedule 

• Start: Initial in-person interview 

• 2-3 days: Telephone call 

• 7 days: Telephone or in-person interview 

• 10 days: Telephone call 

• 14 days: Telephone or in-person interview 

32
Respondent data reporting alternatives 

• Electronic 

> Download financial data files 

> Scan receipts, paper forms 

> Use respondents’ smartphones

• Paper 

> Mail-in receipts, monthly statements 

> Keep diary, as now 

33
Respondents download financial data 

from Web sites of financial institutions 

• Credit card accounts 

• Checking accounts 

• Bank accounts for debit card 

or electronic payments 

• Software for downloading 

> Financial software packages like Mint 

> New software using Yodlee Software 

Development Kit 

34
Creating a Record 

• Types of records 

> Receipts from vendor 

> On-line receipts 

> Information entered into smartphone 

> Short paper form describing purchase 

• Transmitting receipts 

> Scan 

> Mail 

35
36
37
Repository system 

• Software automatically transfers 

to central repository 

> Scanned receipts 

> Downloaded financial data files 

• Converts to text using optical character 

recognition (OCR) software 

• Extracts individual purchases from text 

> Expert system reads text 

> Finds description, cost of purchase (shirt, $20) 

> Ignores irrelevant text 

38
39
40
Repository matches purchases, 

generates Web survey 

• Compares 

> Purchases documented in receipts 

> Purchases documented in financial data files 

• Matches by total cost, date 

• Generates Web survey 

> CE-relevant questions for each purchase 

41
42
Example #1 

• Repository finds 

> Expenditure for $20.25 in credit card data file 

> Receipt for $20.25 which lists 

 Hammer $15.00 

 Nails $3.00 

 Tax $2.25 

> Match data file with receipt 

43
Example #1 

• Repository finds 

> Expenditure for $20.25 in credit card data file 

> Receipt for $20.25 which lists 

 Hammer $15.00 

 Nails $3.00 

 Tax $2.25 

> Match data file with receipt 

44
Example #2 

• Repository finds 

> Expenditure for $34 in checking account data file 

 Check cashed July 30, 2011 

> No corresponding receipt for $34.00 

near July 30, 2011 

> No receipt was scanned 

45
Example #2 

• Repository finds 

> Expenditure for $34 in checking account data file 

 Check cashed July 30, 2011 

> No corresponding receipt for $34.00 

near July 30, 2011 

> No receipt was scanned 

46
Example #3 

• Repository finds 

> Receipt for $5 from dry cleaner 

dated July 15, 2011 

> No corresponding check or credit card charge 

for $5 near July 15, 2011 

> Paid in cash 

47
Example #3 

• Repository finds 

> Receipt for $5 from dry cleaner 

dated July 15, 2011 

> No corresponding check or credit card charge 

for $5 near July 15, 2011 

> Paid in cash 

48
Will many respondents cooperate 

and actually save receipts? Maybe yes. 

• SIPP encouraged use of administrative 

records of income 

> Interviewers encouraged, instructed respondents 

> Telephoned respondents to remind them 

> Asked respondents to get missing records, 

including return visits 

> Achieved “astonishingly high” levels of compliance

 Record use increased from 25 to 87 percent 

49
Will many respondents cooperate 

and actually save receipts? Maybe no. 

• MEPS respondents asked to keep 

receipts, calendar 

> 13.7 % used receipts to report data 

> 22.5 % used calendar 

50
Administrative record survey for Diary 

• Approach major retail chains 

• Ask for purchasing histories 

• Identify respondent households 

by loyalty card number 

51
Uses of personal records 

and recall interviews for Diary 

• Use receipts, financial data files 

as memory aids in recall interview 

• Enter transaction dates 

on Event History Calendar 

52
Diary survey 

data collection timeline 

• Start: in-person interview 

> Sign consent for electronic information 

> Select paper or electronic data collection method 

> Provide envelopes/box for receipts 

> Provide scanner if electronic method used 

> Provide paper diary if that method used 

• Interviewer monitors respondents' recordkeeping 

> Scans of receipts, mailed-in paper receipts 

53
Initial followup 

• 2-3 days: telephone call 

> Identify barriers, problems 

> Motivate 

• 7 days: telephone/in-person 

> If respondent has been sending adequate records, 

by telephone 

> Otherwise, in-person recall interview 

• 10 days: telephone call 

> Identify barriers, problems 

> Motivate 

54
14 days: telephone/in-person 

• If respondent meets criteria for 

recordkeeping, conduct interview 

by telephone; then close out 

• Otherwise, in-person recall interview 

55
Estimates of cost 

• Costs estimated using: 

> CE Diary costs provided by CNSTAT 

 For proposed design, anchored on total cost 

from current design 

 Computed a “per complete” cost by dividing total 

cost by number of completes

> Costs from Medical Provider Component 

of the MEPS, Energy Supplier Survey for RECS 

• Many assumptions required 

56
Table of Diary costs 

57

Current design Proposed design 

CE Units 7,449 7,449 

Interview cost $5.1M $7.5 M 

Administrative 

records cost NA $0.8 M 

Total Diary Cost $5.1M $8.3 M 
Impact of the redesigned Diary survey 

on precision of estimates 

• Sample design unchanged, changes in precision 

are a function of cost 

• If only one respondent is interviewed, Diary cost 

would not increase 

• For fixed budget, new design with >1 respondent 

could collect approximately 60% as much data 

> This would increase standard errors 

by about 29% 

> If no administrative data are collected, 

the increase would be about 21% 

58
59

Proposed redesign 

Quarterly Interview 
New interviewing schedule 

for CE quarterly interview survey 

• Two recall interviews, 

spaced 12 months apart 

• Collect details on purchases other than food and 

alcoholic beverages 

• Vary the reference periods 

across expenditures 

60
61

The interviewing structure 

Set up interview 

3 month collection period 

Year 1 Quarters Year 2 Quarters Year 3 Quarters

Panel 1 2 3 4 1 2 3 4 1 2 3 4

1

2

3

4

5

6

7

8
62

Quarterly interview 

reference periods for recall 

Months 

12 11 10 9 8 7 6 5 4 3 2 1

Frequent purchases 

Majority of purchases 

Purchases that are 

easy to recall or 

estimate accurately 

over 12 month period 
Proposed structure: Pros and Cons 

• PRO

> Reduces number of CE interviews 

> Reduces design effects for annual estimates 

> Can ask reference periods >3months 

• CON 

> Increased cost due to more interviews 

> No longitudinal data on quarterly change 

> No annual data for individual CUs 

63
Redesigned 

Quarterly Interview schedule 

• Wave 1 

> Start: initial in-person interview 

> 2 weeks, 1 month, and 2 months: telephone call 

> 3 months: In-person interview 

• Wave 2, 12 months later 

> Start: telephone or in-person interview 

> 2 weeks, 1 month, and 2 months: telephone call 

> 3 months: in-person interview 

64
Respondent data reporting alternatives 

• Electronic 

> Download financial data files 

> Scan receipts, paper forms 

> Enter information using smartphone 

• Paper 

> Mail-in receipts, monthly statements 

> Keep receipts and other records 

65
Other survey design parameters 

• One main person interviewed in the CU 

> Consider experimentation 

with using more than 1 person 

> Encourage main respondent 

to consult others in CU 

• Incentive provided for each recall interview 

66
Wave 1, initial in-person interview 

• Sign consent for electronic information 

• Select paper or electronic data 

collection method 

• Provide envelopes/box for receipts 

• Provide scanner if electronic method used 

• Conduct bounding interview 

67
Followup 

• Telephone contacts at 2 weeks, 1 month, 

and 2 months 

> Interviewer monitors respondents’ recordkeeping

 Scans of receipts 

 Mailed-in paper receipts 

> Identify barriers, problems 

> Motivate 

• In-person interview at 3 months 

> Review records provided 

> Recall interview 

68
Wave 2 

• 12 months after initial contact 

> Mail instruction packet to CU 

> Followup telephone call to orient respondent 

and conduct bounding interview 

> If household has changed, use initial 

contact protocol 

• Followup 

> Repeat wave 1 procedures 

69
Administrative record survey 

• Approach major retail chains, utility, mortgage 

companies 

• Ask for purchasing histories and collect 

expenditures as they occur 

70
Quarterly Interview survey cost caveats 

• Same as for the Diary 

• Plus: difficult to partition current costs by mode 

71
Table of quarterly interview costs 

72

Current design Proposed design 

CE Units 28,000 28,000 

Interviews/ 

CE Unit/quarter 1 2 

Quarters 4 4 

Interview cost $17.0 M $24.7 M 

Administrative 

Records cost NA $10.2 M 

Total Interview Cost $17.0 M $34.9 M 
Estimates of precision 

for proposed design 

• Estimates for expenditures 

that are currently based 

on the Quarterly Interview Survey 

• Estimates of precision simulated 

using the 2009 public use file 

73
Methodology for estimating precision 

• Simulated effects on within-PSU sampling 

variance for 1-, 3-, and 12-month 

reference periods 

• Assumed each 3-month interview was 

uncorrelated and computed variances 

• For 12-month period used constant reduction 

based on length of reference period 

• For 1-month period used most recent month 

of 3-month reference period 

74
Percent reduction in sampling variance 

for a 3 month reference period 

75
Percent reduction in sampling variance 

for a 3 month reference period 

76

MEDIAN = 

19%
Percent reduction in sampling variance 

for a 12 month reference period 

77
Percent reduction in sampling variance 

for a 12 month reference period 

78

MEDIAN = 

39%
Percent reduction in sampling variance 

for a 1 month reference period 

79
Percent reduction in sampling variance 

for a 1 month reference period 

80

MEDIAN = 

-36%
Combining costs and precision 

• The additional costs of the new design 

are partially offset by reductions in variance 

• Cutting 4 waves to 2 waves increases 

costs by 40% 

> A 20% reduction in sample achieves equal or better 

precision on the annual estimates for most expenditures 

> Shifting to a 12-month reference period for some items 

would reduce variances further. 

• The administrative record component 

adds to the cost 

81
Percent reduction in sampling variance with a 

20% sample cut and a 3 month reference period 

82
83

Proposed redesign 

Summary and discussion 
Methods to improve 

measurement & reduce burden 

• Increase use of records 

> Personal 

> Administrative 

• Reduce number of quarterly interviews 

• Use of multiple diary keepers 

• Enhanced recall interview 

• Use of incentives 

84
Design is flexible 

• Provides respondents with several different 

ways to report 

> Need to make this a positive feature of the survey 

> Avoid making it more complicated to respond 

• Is adaptable as new technologies and 

consumption methods evolve 

• If records become principal mode, 

permits two surveys to merge 

85
Oh No! The cost is twice as much!! 

86
Additional costs of proposed 

design relative to current design 

87

Method to Reduce Burden 

and Measurement Error Diary Interview Total

Use of personal records N N N 

Administrative records .8 M 10.2 M 11.0 M 

Reduction in CE interviews NA 7.6 M 7.6 M 

Multiple diary keepers 2.4 M NA 2.4 M 

Enhanced recall interview N N N 

Total 3.2 M 17.8 M 21.0 M 

N= not a significant cost factor 

NA = not applicable
But... 

• New design can cut sample by at least 20% 

and maintain or exceed current precision 

> Efficiencies with selective use of a 12 month 

reference period 

• Modifications to administrative data collection 

are possible, e.g. 

> Restrict to utility companies. This would reduce 

the cost from $10 million to $3 million 

> Subsample retailers to fit resources 

88
Design priorities 

in managing costs 

• First priority 

> Personal records and technology 

> Multiple diary keepers 

> Incentive 

• Second priority 

> Administrative data collection 

• Third priority 

> New panel design 

89
90

Evaluation 
Administrative data 

• Will retail, utility, mortgage companies 

provide data? 

• What are the barriers? 

• What procedures elicit cooperation? 

91
Outcome measures for evaluation 

• Level of expenditures 

• Match reported expenditures = administrative data 

• Proportion of expenditures reported with records 

• Cost of data collection 

• Nonresponse 

• Perceived level of burden, satisfaction 

• Proportion of respondents requiring 

recall interview 

92
Diary survey field test 

• Single respondent vs. multiple respondents 

• Low incentive vs. high incentive 

• Respondent chooses data collection method 

vs. interviewer chooses 

93
Quarterly Interview field test 

• Single respondent vs. multiple respondents 

• Low incentive vs. high incentive 

• All 3-month reference period vs. 

1, 3, or 12-month reference periods 

94
David Cantor 

DavidCantor@westat.com 

Sid Schneider 

SidSchneider@westat.com 

Brad Edwards 

BradEdwards@westat.com

Thank you!