For 

Measuring What We Spend: Toward a New 

Consumer Expenditure Survey

Workshop Discussion of NRC Report

October 16, 2012

Don A. Dillman, Chair

National Academies Panel on Redesign Options for 

the Consumer Expenditure (CE) Surveys

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

1 

Issues with The 

Current Consumer 

Expenditure Surveys
My Purpose 

 Discuss measurement and non-response issues 

associated with the current CE Surveys.

 Provide background for understanding recommendations 

of the National Academies Panel on Redesigning the 

BLS Consumer Expenditure Surveys.

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

2 
But, first some greatly 

deserved thank-you’s

 To the Bureau of Labor Statistics for:

 The Gemini Project, which had already done part of 

what a NA Panel might otherwise be asked to do. 

 Assistance provided regularly throughout the Panel’s 

work.

 Presenters and discussants at Panel sponsored 

meetings and workshops.

 Contractors for two proposed redesigns of the CE 

surveys. 

 The CE Panel members who brought survey design 

expertise from a variety of scientific perspectives.

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

3 
Three considerations that made coming to 

grips with issues facing the CE surveys 

challenging

 Diverse data requirements drive the identification of CE 

design issues.

 Consumer Price Index.

 Administration of federal/state government programs.

 Policy analyses and economic research.

 Different CE design issues are linked to different data 

requirements

 Needed survey design expertise is held by different 

organizations and people.

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

4 
Diverse Expertise 

 Data collectors—organizations and field representatives 

 Respondents who must be willing and able to answer the 

questions.

 Agency staff who summarize, analyze and report CE 

data.

 Professionals outside government who analyze data, 

often from various disciplinal perspectives.

 The scientific disciplines involved in constructing valid 

and reliable measures of consumer expenditures and the 

survey methods used to collect them.

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

5 
Why the CE Surveys need to 

be redesigned (See Conclusions 5-1 and 5-2)

 Underreporting of expenditures for both the 

Diary and Interview are a concern.

 Underreporting varies across categories and 

appears to be less for interview.

 The questions are different in each mode as is 

the context for asking them and that contributes 

to differences in measurement between modes. .

 Both the diary and interview can and need to be 

improved. 

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

6 
Sources of Underreporting 

(Conclusions 5-3 to 5-14)

 I will begin with the interview and then move 

on to the diary survey.

 Our goal here was to identify specific issues 

that might be focused on with hope of getting 

improvement through redesign efforts.

 Panel conclusions stemmed from a variety of 

sources: published literature, disciplinal 

perspectives, and the experiences of panel 

members who completed the CE Interview 

and/or diary. Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

7 
Synopsis of the interview task

 5 in-person interviews, ~65 minutes each, are 

conducted with each consumer unit (household) three 

months apart.

 Interview #1 asks demographic information, major 

durable goods, and a one month recall of expenditures. 

 Interviews. #2-5 are quite repetitive asking respondents 

to report details of expenditures for the last three 

months.

 A respondent “booklet” includes 36 pages, each with 7-

70 items on it to help identify specific expenditures. 

 Detailed assets and changes from earlier time periods 

are generally asked in interview. #2 and #5.

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

8 
Some additional aspects of 

response situation (from field 

representative debriefing surveys)

 Interview is usually completed with one 

person in the consumer unit who needs to 

report for all persons. 

 Records less likely to be extensively used 

in interview (31%) than NOT used (39%).

 Respondent has to learn rules for 

assigning dates of purchase, delivery and 

payment, which may differ.

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

9 
Detail requested for the last 3 

months (interviews 2-5) can be 

daunting, e.g.

 Did you purchase any “pants, jeans, or 

shorts” (in the last three months)?

 Please describe the item

 Was this purchased for someone inside or outside 

of your household?

 For whom was this purchased? (Enter name, age, 

sex).

 When did you purchase it?

 How much did it cost?

 Did this include sales tax?

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

10
A difficult asset question from 

the fifth interview

 On the last day of last month, what was the 

total balance or market value (including 

interest earned) of checking accounts, 

brokerage accounts and other similar 

accounts? 

 How does the amount your household had 

on the last day of last month compare with 

the amount your household had on the last 

day of last month one year ago?

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

11
Two major barriers to accurate 

interview reporting

 Knowledge and recall of expenditure 

amounts and details.

 R may not know this information.

 R may have known, but cannot remember. 

 Motivation to report amounts and details

 R may be unwilling or reluctant to share what is 

known.

 R may be unwilling to make effort to determine 

amounts and details of expenditures. 

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

12
Why is knowledge and recall a 

barrier to accurate reporting?

 Only one person likely to do the interview.

 Records are infrequently used.

 Even if records are used, they may not 

correspond to rules for reporting date of 

expenditure and details.

 Purchasing and paying for products and 

services has become much more 

complicated over the last 30 years.

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

13
Societal Changes in how consumer 

expenditures are made makes recall 

more difficult

 Greater range of consumer expenditures.

 Retail outlets more likely to sell unrelated items in single 

purchase (e.g. food + luggage + motor oil +socks)

 On-line purchases are growing.

 Multiple payment methods may be used by one 

person—cash, debit-card, credit card, and checks.

 Automatic deductions {paycheck(s), credit card(s), bank 

account(s)} may not be registered in memory .

 Transactions often made without cognitive attention to 

amount; the card is simply “swiped” and for smaller 

purchases no signature required.

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

14
Societal Changes in how consumer 

expenditures are made makes recall 

more difficult(2)

 List prices set to allow discounts on discounts.

 Store credits and discounts sometimes applied to total 

amount of all purchases, making knowledge of purchase 

price less obvious.

 Enormous variation in receipt structures, including 

abbreviations, makes knowledge of cost more difficult.

 Day item obtained may not be date payment made.

 Observation: Purchasing behavior is much more 

complex than when CE interviewing (and Diary) 

methods were developed, and these changes work 

against the recall of specific purchases and cost. 

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

15
Societal changes in household 

structure add to the recall problem

 More households have multiple wage earners.

 Households are more likely to include unmarried 

partners.

 Household members more likely to have expenditure 

obligations to other households (e.g. children of divorced 

parents).

 Partners/spouses less likely to inform each other of all 

expenses.

 Partner agreements, “I pay for food, you pay for rent,” 

may compound the knowledge problem.

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

16
For example: How much did this 

necktie cost?

 Sign in store: “Neck wear: assorted vendors. Buy 1 

(59.95). Get 1 free” (Sign was on multiple tables)

 Bought two ties (and a third item)

 Clerk: Use of store credit card “today” gives additional 

20% off of purchase.

 Time of year Charity donation today would give 10% off 

current purchase and 10% of all expenditures would 

come as gift certificate after December 31st. I declined.

 Clerk offers to send receipt by email instead of my taking 

receipt with me. 

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

17
Outcome: Necktie purchase

 Price was 59.95 on tie, but receipt said 41.65 

was “today’s price”.

 Clerk gave me 20% off of 41.65 for total of 

33.32/tie.

 Two for one did not apply because I provided 

store credit card and clerk took the 20%. 

 Checked online billing the next day to see what 

was on credit card—30 lines of single spaced 

code for bill.

 Very difficult to know how much this tie cost.

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

18
Adding up the challenge

Purchases from more sources . . .

+ less association of product with store

+ fewer purchases at known list price

+ more payment streams (methods)

+ automatic deductions for some expenditures

+ less absorption of final price at purchase time

+ less sharing of detail within households

= less certainty on specific expenditures

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

19
The knowledge/recall problem 

in methodological perspective

 One cannot report what one does not know, e.g. 

expenditures never “encoded” or partner expenditures 

not shared.

 The interview cannot rely on establishing recall of details 

by connecting expenditures to a particular payment 

method (check book), physical location of purchase 

(shopping trip), or routine behavior of a member of the 

consumer unit. 

 Cognitive methods that could be used to enhance 

memories (e.g. calendar of events over last 3 months, or 

personalizing recall methods to individuals) are likely to 

make an already long interview more burdensome. Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

20
Motivation has multiple 

aspects

 Belief by some that answering detailed 

expenditure and asset questions are an 

invasion of privacy or beyond the need of 

government to know.

 The interview situation discourages obtaining 

information from other members of the 

household unless they are present.

 Partner/teenager expenses

 Searching for receipts and perusing records

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

21
Motivation in Methodological 

Perspective 

 Some requests for information (alcohol purchases, 

gambling costs) reduce reporting because of social 

undesirability. 

 Seeking great detail that requires significant effort to 

calculate or find , encourages satisficing: “This “number” 

is good enough”.

 Seeking information respondent cannot easily provide 

encourages estimation.

 Estimation often results in telescoping, ie. reporting 

significant expenditures for more than three months.

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

22
Motivation in Methodological 

Perspective (2) 

 Answering “yes” to questions, e.g. Did you take an 

overnight trip or did you purchase any other clothing, 

typically leads to additional questions.

 All five interviews can include dozens of potential 

screener questions, and respondents soon learn that a 

“no” or “not that I remember” results in fewer questions 

and a shorter interview.

 The inability of respondents to give accurate answers 

and a belief that the answer they can give will not be 

useful, may help justify saying “no”.

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

23
Mutual interests of respondent 

and the interviewer

 The respondent typically wants interview to 

be shorter and less work.

 The Interviewer also wants interview to be 

shorter so the respondent will agree to 

another interview.

 Thus, interviewers may not encourage 

respondents to look for receipts, go through 

records, divide receipts, or use other recall 

methods. 

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

24
Effects of switching data 

collection to the telephone

 38% of CE interviews are completed by 

telephone, especially the later ones.

 Recall visual aid (the notebook)not used.

 Telephone interviews obtain fewer “yes” 

answers to screener questions.

 Receipts and other records are less likely to 

be used.

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

25
In summary, interview data 

quality may be negatively 

affected by...

 Interviewing only one person in multi-person households, 

i.e. proxy reporting.

 The long (three month) recall period.

 Lack of knowledge respondent has for providing extreme 

detail requested.

 Lack of motivation for making effort needed to provide 

accurate answers.

 The repetitive nature and length of the interviews.

 The easy avoidance of additional questions by saying 

“No, I didn’t purchase any of that.”

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

26
The Diary completion task is 

also demanding

 Households are asked to report everything they 

purchase for two weeks.

 Week 1 diary; in-person delivery with selected 

questions and instructions.

 Week 2 diary; in-person delivery when Week 1 

diary is picked up.

 Interviewer returns a third time to pick up Week 

2 diary.

 Respondent is encouraged to make diary 

entries each day purchases are made. Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

27
Learning to complete the diary 

can be confusing

 44 numbered pages + covers + flaps = 52 surfaces of 

information 

 Respondent asked to write in names of others in 

household for whom they are reporting.

 15 of pages provided instructions, 28 pages are laid out 

by “day” and have labeled tables for:

 1 Food and Drinks Away From Home

 2 Food and Drinks for Home Consumption

 3 Clothing, Shoes, Jewelry and Accessories

 4 All Other Products, Services and Expenses

 Respondent asked to provide up to six pieces of 

information for each entry.

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

28
Understanding requires roundabout processing, e.g.

 P. 9 asks for Day 1 Food and Drinks for Home.

 Right flap of back cover provides answers to frequently 

asked questions.

 P.3 has instructions for “How to fill out your diary” 

including request to record expenses each day.

 P.2 has general instructions and section on what not to 

record.

 Examples for Food and Drinks are on P. 5, e.g. specify 

white, wheat or rye bread, and instant vs. ground coffee.

 Subsequent Daily pages for Food and Drinks are on 

Pp.13, 17, 21, 25, 29 and 33; “additional pages”. for 

overflow are Pp. 38-41

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

29
Graphical navigation guidance 

is quite limited

 All pages mix blue, gray and black positive 

and negative print; full color limited to outside 

covers. 

 Numbers are assigned to Day (1-7) as well 

as expenditure categories (1-4) and pages 

(1-44), so may not be an effective guide.

 If diary is not filled out each day expenses 

are made, flipping of non continuous pages is 

required. 

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

30
Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

31
Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

32
Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

33
Knowledge as a barrier to 

Completing the Diaries

 In principle less of a problem unless

 Respondent doesn’t ask other unit members for 

expenses.

 Receipts not kept for some purchases.

 Receipts make it difficult to identify items 

(abbreviations)

 Knowing and applying rules may produce inaccurate 

recordings that are difficult to catch.

 Information may not be available in a timely way, e.g. 

auto deductions from bank account.

 Society changes (see interview discussion) also apply 

here. Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

34
Motivation as a barrier

 Recording requested detail on daily basis 

may be seen as a considerable burden.

 Long itemized receipts may be difficult to 

enter, e.g. if discount applied to an entire bill 

and abbreviations used.

 Abbreviations may be impossible to interpret.

 Respondents may delay shopping trip to 

avoid the “need” to record.

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

35
In Sum, Diary responses are 

negatively affected by... 

 ...both knowledge (e.g. proxy reporting) and 

motivation. 

 In addition, design and layout procedures 

present problems—some respondents decide 

only to collect receipts because of how 

difficult the task seems to be.

 An outcome. Some diaries (especially second 

week) do not get completed.

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

36
Is one of these modes (interview vs. 

diary) fundamentally better than the 

other? 

 A difficult question to answer.

 The original goal was to collect different kinds of information—big 

and recurring expenses from interview and small detailed ones from 

diary.

 Each now asks some of the same topics, but in different ways, e.g. 

food for home consumption.

 Interview- “What has been your or your household usual 

WEEKLY expense for grocery shopping?”

 Diary—A listing of each item that asks, e.g. white vs. wheat 

bread, types of meat, and if purchase was for someone else.

 These are quite different questions, subject to different 

kinds of errors associated with “estimation” vs. “reporting 

of details”. 

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

37
A cognitive perspective on the 

search for improved accuracy

 Typical approaches to getting better 

measurement include:

 Ask more detailed questions.

 Create a memory context, e.g. daily activities and 

event history of last three months.

 Personalize questions to how people are most 

likely to recall expenditures.

 Observation: These kinds of efforts make 

a demanding survey interview/diary even 

longer and more demanding? Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

38
Possible Synopsis of Major 

problems of Interview vs. Diary 

as now implemented 

While the interview asks people to report detail 

they never knew or can’t recall (primarily a 

knowledge problem)...

The diary asks people to do something they 

won’t do (primarily a motivation problem). 

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

39
A related question: Is the 

extreme detail necessary?

 The CE surveys serve multiple purposes:

 CPI Index

 Administration of government programs

 Policy analyses

 Are less detailed estimation questions adequate for 

some of these purposes, but not others?

 Is the extreme detail necessary for any of these 

purposes (e.g. white vs. wheat bread)?

 Would recall and reduction of proxy reporting in selfadministered reporting produce better data than on-thespot-estimates of expenditures.

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

40
Other demand issues (Conclusions 5-

15 to 5-16).

 Response rates for diary and interview have 

both been declining and are in the low 70’s.

 Interview quality seems to go down with the push 

Field Representatives face from respondents to 

use the telephone.

 Diary quality seems to go down with the second 

week of collection.

 Administrative data may provide a potential to 

“fill-in” data. 

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

41
Conclusion

 The issues described today are only some of 

the issues that involved extensive discussion 

among panel members, and with most panels 

there were many viewpoints expressed.

 These discussions provided background for 

developing recommendations that will be 

discussed by Carol House, National 

Academies Study Director for the CE Panel 

and Andy Peytchev, a member of the Panel.

Don A. Dillman, NRC Workshop on CE Survey 

Redesign , October 16, 2012

42