Investigating Signs of Interview Fatigue:

Decreased Reporting of Category 

Expenditures

Brett McBride,

U.S. Bureau of Labor Statistics

68th Annual AAPOR Conference

May 19, 2013
 

Outline

 Background: Consumer Expenditure Survey

 Research Questions

 Response Pattern to Filter Questions (FQ) in survey

 Data and Methods

 Findings

Response Patterns

Associated Correlates of Reporting Changes

 Summary

2
 

Background

Consumer Expenditure (CE) Survey

3

 National household panel survey that collects 

information about buying habits of consumers

 Used to calculate the weights of items tracked by 

the Consumer Price Index (CPI), also used by 

economists, academics, and market researchers

 Study focus: Quarterly Interview Survey (CEQ) –

survey of larger, easier-to-recall purchases

 CEQ: 1-hour interviews conducted 5 times over 

15 months
 

Example:

CEQ “Interleafed format”

4

“Since April 1, have you purchased any of the 

following items ... 

[FQ#1]: -coats, jackets or furs?

[if ‘yes’]: 

What did you buy?

Was this purchased for someone inside 

or outside of your household? ...

How much did it cost?

[FQ#2]: -sport coats or tailored jackets? ...
 

Research Questions

5

1. Is there decreased endorsement of FQs (‘FQ 

trend’) over the course of the interview?

2. Is decreasing FQ trend due to low motivation?

3. Is decreasing FQ trend due to fatigue from 

cognitive burden? 

 Decreased FQ trend may suggest need to revise 

CEQ’s interleafed format to grouped format
 

Data and Methods

6

 Data from May 2011 interviews (n=539) 

 303 FQs with collected data, asked of all 

respondents

 Calculate proportion of respondents endorsing 

each FQ

 Divide interview into 3 segments to calculate 

average endorsement rate in first third and last 

third (FQ trend)

 OLS Regression to examine association of 

motivation, burden, controls on FQ trend 
 

Findings: FQ Trend (1)

7

0%

5%

10%

15%

20%

25%

1 2 3 4 5 6 7 8 9 10 11 12

Average FQ endorsement levels

Sequential FQ Segments

Linear 11% decline - first third to last third
 

Findings: FQ Trend (1)

8

0%

5%

10%

15%

20%

25%

1 2 3 4 5 6 7 8 9 10 11 12

Average FQ endorsement levels

Sequential FQ Segments

Linear 11% decline - first third to last third

housing clothing, insurance
 

Findings: FQ Trend (2)

9

0%

5%

10%

15%

20%

25%

1 2 3 4 5 6 7 8 9 10 11 12

Average FQ endorsement levels

Sequential FQ Segments

Polynomial „U-shaped‟ pattern
 

Findings: Regression Model

10

Variable Coefficient P-value

Intercept -5.089 <0.001

Years of education 0.165 <0.001

Years of age 0.029 <0.001

Number in household -0.157 0.070

Own home -1.719 <.001

Proportion spent necessities -0.012 0.025

In-person interview 0.003 0.994

Expressed ‘doorstep’ concern -0.245 0.316

Number of expenditures 0.177 <.001

Number of „don‟t know‟s 0.146 0.050

Survey duration (minutes) -0.009 0.348

Dependent Variable: FQ trend (last third – first third)
 

Findings: Regression Model

11

Variable Coefficient P-value

Intercept -5.089 <0.001

Years of education 0.165 <0.001

Years of age 0.029 <0.001

Number in household -0.157 0.070

Own home -1.719 <.001

Proportion spent necessities -0.012 0.025

In-person interview 0.003 0.994

Expressed ‘doorstep’ concern -0.245 0.316

Number of expenditures 0.177 <.001

Number of „don‟t know‟s 0.146 0.050

Survey duration (min) -0.009 0.348

Dependent Variable: FQ trend (last third – first third)
 

Findings: Regression Model

12

Variable Coefficient P-value

Intercept -5.089 <0.001

Years of education 0.165 <0.001

Years of age 0.029 <0.001

Number in household -0.157 0.070

Own home -1.719 <.001

Proportion spent necessities -0.012 0.025

In-person interview 0.003 0.994

Expressed ‘doorstep’ concern -0.245 0.316

Number of expenditures 0.177 <.001

Number of „don‟t know‟s 0.146 0.050

Survey duration (min) -0.009 0.348

Dependent Variable: FQ trend (last third – first third)
 

Findings: Regression Model

13

Variable Coefficient P-value

Intercept -5.089 <0.001

Years of education 0.165 <0.001

Years of age 0.029 <0.001

Number in household -0.157 0.070

Own home -1.719 <.001

Proportion spent necessities -0.012 0.025

In-person interview 0.003 0.994

Expressed ‘doorstep’ concern -0.245 0.316

Number of expenditures 0.177 <.001

Number of „don‟t know‟s 0.146 0.050

Survey duration (min) -0.009 0.348

Dependent Variable: FQ trend (last third – first third)
 

Summary: 

Findings

14

 Limited decline in FQ trend in survey

 Motivation (‘doorstep’ concern) association 

with declining FQ trend not significant

 Burden (survey duration) association not 

significant but in expected direction

 Burden (number of expenditures, ‘don’t know’ 

responses) association significant but not in 

expected direction

 Burden (number in household) borderline 

significant association
 

Summary: 

Caveats and Applications

15

 Difficult to disentangle FQ trend (i.e., impact of 

respondent conditioning/underreporting) from 

content of specific FQ questions

 Insufficient evidence of declining responses to 

FQs to warrant revising interleafed format 

 Need for interviewers to provide 

encouragement to low-motivation households 

in later stages of interview
Contact Information

mcbride.brett@bls.gov
 

FQ Trend Comparisons

17

Do not 

own home 

(n=200)

Own 

home 

(n=345)

Change in FQ trend -0.55 -0.79

CAPI

(n=460)

CATI 

(n=85)

Change in FQ trend -0.67 -0.86

No doorstep 

concern

(n=293)

Doorstep 

concern 

(n=252)

Change in FQ trend -0.55 -0.87

Change in FQ trend -0.70
 

Subgroup Analysis

18

 Examined FQ trend among older respondents 

(≥65 years old, n=109)

 Group expected to have health expenditures 

 Those not reporting health expenditures 

hypothesized to be satisficing/fatigued

 Of interest: Was group’s FQ trend showing 

reduced reporting at other FQs as well? 

(indicative of satisficing/fatigue)

Reported health 

expend. (n=87)

Did not report

expend. (n=22)

Δ

Diff.

FQ trend 0.4 -3.4 3.8***

FQ trend (excluding 

health FQs) -1.7 -3.4 1.7**

**p<0.01, *** p<0.001