Consumer Expenditure Survey 

Research Highlights 

Adam Safir 

Branch Chief 

Research and Program Development 

Consumer Expenditure Survey 

July 17, 2012 
Symposium Objectives 

1. Share CE research findings and 

redesign progress with stakeholders 

and other interested researchers 

2. Spur a productive discussion about 

how CE and other survey data 

producers develop, implement, and 

evaluate their redesigns 

Questions and feedback are encouraged 

2 
Symposium Organization 

 Morning 

 CE program & research highlights 

 Gemini project to redesign the survey 

 CNSTAT’s efforts on CE’s behalf

 Afternoon 

 Other large-scale redesign initiatives 

 Additional methods research findings 

 Different perspectives on similar topics 

 Additional time reserved at end of day 3 
Presentation Structure 

I. CE Overview 

II. Redesign Motivation 

III. Research Findings 

A. Reduce Measurement Error 

B. Reduce Burden 

C. Reduce Costs 

D. Monitor Redesign Results 

IV. Redesign Challenges 

4 
I. CE Overview 

 Collected for the Bureau of Labor 

Statistics by the Census Bureau 

 Consists of 2 national HH surveys 

 Provides: 

 Information on the buying habits of 

America’s consumers, including 

expenditure, income, and HH data 

 The basis for revising the cost weights 

and associated pricing samples of the 

Consumer Price Index (CPI) 5 
I. CE Overview 

 Only Federal survey to provide 

information on a complete range of 

consumer expenditures, income, and 

HH characteristics 

 Data users include economic 

policymakers, businesses, academic 

researchers, other Federal agencies, 

and CPI 

6 
I. CE Overview: 

CE Quarterly Interview 

 5 quarterly interviews, 4 used in final 

data 

 CAPI, in-person (some telephone) 

 3-month recall 

 Length: ~60 minutes 

 Annual Sample: ~28,000 interviews 

 Avg. Response Rate: 74% (CY 2010) 

7 
I. CE Overview: 

CE Diary 

 2 independent “weekly” diaries, 2 total 

weeks of diary-keeping 

 Paper diary form, only 1 instrument, 

i.e., no individual diaries 

 3 interviewer visits, sometimes only 2 

 Total recall / receipt entry conducted 

 Annual Sample: ~14,000 one-week 

diaries 

 Avg. Response Rate: 77% (CY 2010) 8 
I. CE Overview: 

Survey Improvements 

 2003 – CAPI (CEQ) 

 2004 – CAPI (CED) demographics and income 

 2004 – Income imputation 

 2005 – Contact History Instrument (CHI) 

 2005 – User friendly diary form 

 2005 – Diary keying and auto-coding system 

 2009 – Telephone thresholds (CEQ) 

 Ongoing – Biennial CEQ instrument revisions 
II. Redesign Motivation 

 Despite all of these improvements: 

Evidence of underreporting 

– from benchmarking and subgroup analyses 

CE is burdensome 

CE is expensive 

Trend of declining response rates 

 Further, the basic design has been the 

same since 1980 
II. Redesign Motivation: 

Objectives 

 Reduce measurement error 

in particular, underreporting 

 Reduce burden 

 Reduce costs 

 Monitor redesign results 

 Research agenda 
III.A Reduce Measurement Error 

 Reduce number of interviews

CE studies have found that there is little 

evidence of CEQ respondents are 

satisficing after more than one wave of the 

CEQ Survey (Edgar, 2005; Yan & Copeland, 

2010) 

 Reduce reference period length

Reducing to one month from one quarter 

had a positive impact on expenditure 

reporting rates, but higher attrition and 

burden (Creech et al. 2011) 12
III.A Reduce Measurement Error 

 Reduce interview length

Research has been conducted to identify 

the impact of interview length on data 

quality, but the results have been 

inconclusive (Brattland et al. 2011) 

Identifying the impact of interview length 

is ideally done in an experimental study, 

which is prohibitively expensive for the CE 

(2012 JPSM Practicum, forthcoming) 

13
III.A Reduce Measurement Error 

 Reduce proxy reporting 

Proxy reporting is a source of 

underreporting in the CE surveys (Kojetin & 

Jerstad, 1997) 

One approach is to give all household 

members a diary to record their expenses 

3 studies have found this to be an effective 

way to increase expenditure reports, but 

with risk to response rates (Edgar et al. 

2006; NORC, 2001; Westat 2005) 

Internet individual diary study planned 14
III.A Reduce Measurement Error 

 Maximize record use & minimize recall

Findings show that recall aid use is 

associated with higher expenditure 

reporting (Safir & Goldenberg, 2008) 

Confirmed anecdotally by interviewers 

(Shields, 2004) 

But, encouraging use can be problematic 

(Edgar & Fricker, 2010; Geisen et al. 2011) 

And recall aid use can be prohibitively time 

consuming (NORC, forthcoming) 

15
III.A Reduce Measurement Error 

 Incorporate new technology 

CEQ/D: Financial software (PC or SP) 

CED: Web, SmartPhone diaries 

 Incorporate multi-mode interviewing

CEQ: In-person, plus telephone 

interviewing 

CED: Paper, plus web and/or SmartPhone 

diaries 

16
III.B Reduce Burden 

17

 Reduce number of interviews

As noted earlier, CE studies have found 

little evidence that CEQ respondents are 

satisficing after more than one wave of the 

CEQ Survey (Edgar, 2005; Yan & Copeland, 

2010)

Suggests that although asking respondents 

to participate in 5 interviews is likely 

burdensome, there is no reason to reduce 

the number of interviews in an attempt to 

improve data quality 
III.B Reduce Burden 

18

 Reduce interview length

Split questionnaire research 

– Preliminary simulation results indicate that split 

questionnaire designs for the CE can reduce 

survey length by at least 50%, with the impact 

on variances “varying” depending on the type 

of expenditure category (Gonzalez, 2012) 

– Depending on the type of split questionnaire 

design employed, there is the possibility of 

improving some other aspect of the survey 

process 

– Responsive split questionnaire designs show 

promise for improving data quality
III.B Reduce Burden 

19

 Reduce interview length

Global questions 

– A mixture of global questions (asked first) and 

detailed questions lead to higher data quality 

and more expenditure reports (Creech et al. 

2011) 

– However, respondents use seemingly unreliable 

response strategies to arrive at answers to 

global questions (Edgar, 2012) 
III.B Reduce Burden 

20

 Reduce interview length

Other methods associated with dropping 

questions: 

– Diary to interview imputation 

– Within-quarter interview imputation 

– Backcasting 
III.C Reduce Costs 

 Reduce interview length

Very little money is saved by shortening an 

interview (Elkin, 2011) 

Most of the expense of an interview 

(especially the first one) is from contacting 

the respondent 

21
III.C Reduce Costs 

 Reduce number of interviews

Bounding interview elimination 

– First interview data are used for bounding, and 

collecting rostering and inventory information 

– CE research has shown that the bounding 

interview may only be minimally effective in 

addressing telescoping errors (Elkin, 2012) 

– Significant cost savings could be realized by 

dropping the bounding interview 

– One implementation challenge is incorporating 

the rostering and inventory questions into the 

2

nd interview 22
III.D Monitor Redesign Results 

 Data Quality Profile

A consistent, well-defined set of metrics 

can be used to establish baselines for 

monitoring trends in the quality of routine 

survey production activities over time 

(Fricker & Tan, 2012) 

These metrics also can be drawn upon to 

evaluate the impact of survey design 

options under consideration, as well as 

external interventions that affect the 

survey 23
24

III.D Monitor Redesign Results 
III.D Monitor Redesign Results 

 Measurement Error Tracking

Determine sources of measurement error 

on expenditure reporting 

– Is the CE Program’s focus on under-reporting 

appropriate? (e.g., records study results) 

– Distinguish between under-reporting 

(unreported incurred expense) and 

underestimation (incurred expenditure reported 

at a lower value) 

Develop a methodology for tracking and 

evaluating changes in measurement error 

due to design changes 25
III.D Monitor Redesign Results 

 Burden Index

Burden (actual or perceived) has been 

posited as one of the contributing causes 

to measurement error (Fricker et al. 2012) 

The ability to measure the effect of 

alternative design options on respondent 

burden would be a useful evaluation tool 

Further, a measure of burden could 

facilitate a more systematic examination of 

the association between burden and other 

survey measures of interest 26
IV. Challenges 

 Synthesizing research results into a 

comprehensive redesign plan 

 Budget for research, testing, evaluation, 

and implementation 

 Sample size requirements 

 Timing of research findings 

 Pace of technological change 

Rs  PC  Laptop  SmartPhone  ??

27
References 

28

Reduce Measurement Error

Number of Interviews

Measurement Issues Study Final Report 

Creech, B., J. Davis, S. Fricker, J. Gonzalez, M. Smith, L. Tan, and N. To (2011)

CE Data User’s Survey 

Edgar, J. (2005) 

Panel Conditioning in Consumer Expenditure Interview Survey

Yan, T., and K. Copeland (2010) 

Interview Length

Order Effects Test Final Report

Brattland, J., J. Edgar, S. Maloney, P. Murphy, B. Steinberg, and N. Tseng (2011) 

2012 JPSM Practicum Report

JPSM Practicum students (forthcoming)
References 

29

Reduce Measurement Error

Proxy Reporting

Individual Diary Feasibility Test

Edgar, J., Spell, S., Verlander, R., & Wetzel, G. (2006) 

The Quality of Proxy Reports on the Consumer Expenditure Survey

Kojetin, B. and S. Jerstad (1997)

Individual Diaries: Literature Review

NORC (2001) 

A Field Test of a Multiple Diary Procedure for the Consumer Expenditure Survey

Westat (2005)
References 

30

Reduce Measurement Error

Record Use

CE Validation Study (presentation) 

Edgar, J. & Fricker, S. (2010) 

U.S. Consumer Expenditure Records Study

Geisen, E., A. Richards, C. Strohm (2011) 

Records Information and Feasibility of Use Report

NORC (forthcoming)

Mode Effects in a Survey of Consumer Expenditures 

Safir, A., and K. Goldenberg (2008)

CEQ Field Conference Focus Groups Results from Supervisors Group

Shields, J. (2005)
References 

31

Burden

Measurement Issues Study Final Report 

Creech, B., J. Davis, S. Fricker, J. Gonzalez, M. Smith, L. Tan, and N. To (2011) 

CE Data User’s Survey

Edgar, J. (2005)

Global Clothing Questions Cognitive Testing Results

Edgar, J. (2011) 

The Use of Responsive Split Questionnaires in a Panel Survey (presentation)

Gonzalez (2012)

Panel Conditioning in Consumer Expenditure Interview Survey

Yan, T., and K. Copeland (2010)
References 

32

Costs

Cost Savings from Shortening Interview 1 Project Report

Elkin, I. (2011)

Recommendations Regarding the Use of a CE Bounding Interview

Elkin, I. (2012)

Monitoring Results

A proposal for a preliminary framework for monitoring and reporting on data quality for the 

Consumer Expenditure Survey 

Fricker, S., and L. Tan (2012)

Exploratory Research on the Construction of a Summary Index for Respondent Burden

Fricker, S., C. Kreisler, and L. Tan (2012) 
Contact Information 

Adam Safir 

safir.adam@bls.gov

33