RTI International

RTI International is a trade name of Research Triangle Institute. www.rti.org

Evaluating Measurement Error in the 

Consumer Expenditure Survey

*Emily Geisen, Ashley Richards, Charles Strohm

**M. Christopher Stringer

***Janel Brattland, Jeanette Davis, Brandon Kopp, Neil Tseng

*RTI International

**U.S. Census Bureau

***Bureau of Labor Statistics

The findings and conclusions in this presentation are those of the authors and do not

necessarily represent the views of the U.S. Census Bureau or the Bureau of Labor Statistics
RTI International

Introduction

 U.S. Consumer Expenditure Quarterly Interview 

Survey (CEQ)

– Monthly survey designed to capture continuous 

information about buying habits of Americans.

– U.S. Census Bureau conducts survey for Bureau of 

Labor Statistics

 Data from the CEQ are used to calculate the 

weight of index items in the Consumer Price 

Index (CPI)

– CPI is one of the nation’s leading economic indicators 
RTI International

Background

 The CEQ presents a number of challenges

– Long (5 one-hour interviews over 12 months)

– Detailed questions

– Burdensome to respondents

 Belief that some of the CEQ data are underreported or 

misreported, due to

– Recall error

– Panel conditioning

– Respondent fatigue

 One possible solution: incorporate the use of respondent 

records instead of relying on recall only
RTI International

CE Records Study Objectives

 Census Bureau contracted with RTI 

International to conduct the CE Records Study

 Study objectives:

1. Evaluate the availability and feasibility of respondent 

records 

2. Measure the direction and magnitude of 

measurement error 

3. Understand respondents’ interview experience 
RTI International

Methodology

 Participants completed 2 interviews within a 7-day period

– First interview mimicked typical CEQ interview

– Second interview matched records to expenditures reported in 

first interview

– In-person interviews conducted by Field Interviewers (FIs)

 Convenience sample of 115 participants in RaleighDurham, NC and Washington, DC area

 Recruited a diverse sample with respect to: age, 

education, employment status, income, household size 

 Participants were compensated $100 total

 Informed consent obtained for each interview
RTI International

Methodology – First Interview

 Interview 1: 

– Subset of regular CEQ instrument

– Standard CEQ procedures

– Short debriefing questionnaire

 Between interviews: 

– Participants asked to collect records (e.g., receipts, 

bank statements, credit card bills) for all items asked 

about in Interview 1
RTI International

Methodology – Second Interview

 FIs asked for a record for each item 

reported in Interview 1

 When applicable, FIs asked participants to 

explain why:

– Reported amount did not match

– No record available

– Item not reported in Interview 1
RTI International

Results – Availability of Records

 Records available for 36% of items reported

– Only 8% of those records were inaccurate (not gold standard)

 More likely to have records based on 4 factors:

– Demographic characteristics. Non-Hispanic white, DC 

participants, married and home-owners

– Date or frequency of purchase. Items purchased regularly or 

items purchased recently

– Significance of the item. More expensive items; participants 

often said they did not keep records for insignificant items

– Online records. Common reason provided for not having 

records was that they were online; participants were unwilling or 

unable to print them
RTI International

Results – Extent and Direction of Error

 Distribution of reports values:

– 37% of reports were underestimated

– 33% overestimated

– 30% equaled the record value

 Reported value considered a “Match” if:

– Reported amount within 5% of the record for items 

$200 or more

– Reported amount within 10% for items less than $200

 Using this criteria, 53% of reports matched
RTI International

Results – Magnitude of Measurement Error 

 Determining magnitude of difference:

– Difference between reported and record amount

– Divided by smaller value

– Retains relative value between under/overestimates

 Across all items, magnitude of dif. was 36%

 For misreported items, magnitude of dif. was 52%

– Magnitude of underestimates was 48%

– Magnitude of overestimates was 57%
RTI International

Results – Respondent Experience

 95% of participants said that the length of interview 1 (60 

minutes) was just about right

 Almost half said they either sometimes, often or always 

guessed the cost of expenses, yet almost all thought 

their responses were accurate

 When not accurate, usually said “I guessed or estimated”

 Perceived accuracy associated with actual accuracy

– Those who were more confident about responses, had slightly 

more accurate reports

– But participants who said they were “extremely certain” only had 

matching reports for 63% of reported items
RTI International

Discussion

 Relatively low level of record availability (36%)

 Participants tended to both underestimate and 

overestimate, by a substantial amount

 Factors associated with availability of record are not 

associated with accuracy of records

 Participants are overconfident about the accuracy of 

their responses

 CE Records Study was retrospective; next steps are to 

examine whether participants are able and willing to 

retain records for purchases they make in the future
RTI International

Limitations

 Exploratory research study with non-probability sample

 Only 115 participants, demographic analyses limited

 Asked about a subset of CEQ, not full survey

 Participants only provided records for 36% of items – no 

way to know if the accuracy of reports when records 

were provided are similar to the accuracy of reports 

when records are not provided
RTI International

More Information

Emily Geisen

Corresponding Author

919.541.6566

egeisen@rti.org