Response Burden: What 

Predicts It and Who is 

Burdened Out?

Ting Yan

Institute for Social Research

University of Michigan

Scott Fricker Shirley Tsai

U.S. Bureau of Labor Statistics
Acknowledgment

 Study funded by ASA/NSF/BLS 

Fellowship 

2
Outline

 How is response burden defined?

 How is response burden measured?

 What are the effects of response 

burden?

 What causes response burden?

 What can be done to reduce or 

counteract negative effect of response 

burden?

3
Gaps in the field 

“The topic of respondent burden is not a neat, 

clearly defined topic about which there is an 

abundance of literature” (Bradburn, 1978: p49)

“Response burden is not a straight forward area 

to discuss, measure and manage” (Jones, 2012: p1)

4
Gaps in the field 

 Undeveloped conceptualization 

 Lack of good measurement 

 Lack of empirical research on

What predicts response burden

The impact of burden on data quality and 

statistical estimates

5
Perceived/Subjective 

Burden

 “...perceived response burden ... negative feelings such as 

annoyance, frustration or inconvenience which may be 

experienced by survey participants” (Frankel, 1980: p1)

 “...respondent burden ... the presumed hardships entailed 

in being a survey participant” (Sharp and Frankel 1983: p36)

 “...respondent’s experience...” (Haraldsen 2004: p398)

 “... perception of time and burden associated with the 

response task” (Giesen 2012: p1-2)

 “[T]he degree to which a survey respondent perceives 

participation in a survey research project as difficult, time 

consuming, or emotionally stressful...” (Graf 2008: p740)

6
defined?

Actual/Objective Burden

 “... characteristic of research activity intervening 

between the survey instrument and response activity 

which, if increased, will decrease the probability of 

the respondent providing the full information 

required...” (Corbin 1977: p9)

 “... respondent can feel burdened whenever the 

question appears either threatening or difficult...” 

(Warriner 1991: p256)

 “the length of the interview” (Groves et al. 1991: p251)

 “the number and size of the respondent’s tasks” 

(Hoogendoorn and Sikkel 1998: p189)

7
How is response burden 

measured?

8

 Characteristics of survey/tasks causing burden

 Length of interview (Groves et al. 1999; Singer et al., 1999; Hoogendoorn, 

2004)

 Frequency of interview (Hoogendoorn et al., 1998)

 Difficulty of response tasks (Filton, 1981)

 Rs’ attitude towards and beliefs about surveys

 Self-reports

– Interest in survey (Sharp et al., 1983; Hoogendoorn, 2004; Fricker et al. 2011; 

2012)

– Importance of interview (Sharp et al., 1983)

 Interviewer notes

– Rs’ complaint about survey burden (Martin et al., 2011)

 Effects of response burden

 Willingness to be re-interviewed (Sharp et al., 1983; Fricker et all., 2011; 

2012)

 Feeling of exhaustion (Stocke and Langfeldt; 2004)
What are effects of response 

burden?

 Leading to unit nonresponse (e.g., Groves et al. 1999; Rolstad, 

Adler, and Rydén 2011)

 Leading to panel attrition (e.g., Martin et al. 2001; Fricker et al. 

2011)

 Leading to item nonresponse (e.g., Warriner 1991)

 Leading to break-offs (e.g., Galesic 2006)

 Leading to delayed responses (e.g., Giesen 2012)

 Leading to inaccurate response (e.g., Warriner 1991; Kennedy 

and Phipps 1995; Haraldsen and Jones , 2007 ; Giesen and Haraldsen, 2012 )

 Leading to negative evaluations of surveys (Stocke and 

Langfeldt; 2004)

9
What causes burden?

10

Survey and 

Task 

Characteristics

Respondent 

Characteristics Respondent 

Perception of 

Survey

Response 

Burden
This talk examines...

 Respondent level characters

Cognitive capacity

Motivation

(General) Attitude

Task difficulty

 Survey design characters

 Mediation by perception of survey

 Direct and Indirect paths 

11
Data

 Consumer Expenditure Interview Survey (CE)

Longitudinal survey conducted by BLS

– Providing information on buying habits of American 

consumers

• Expenditures, income, consumer characteristics

– Rotation panel design

• Panel members are interviewed every quarter up to 

five times

• In each interview quarter, 5 panels in different stage 

of panel life

 Pooled cases who completed their 5th interviews 

between October 2012 and December 2012

A total of 5,143 cases used 12
Data and method

 Structural equation modeling (SEM) 

To incorporate structural models and 

measurement models 

To estimate direct and indirect paths

 PROC CALIS is used to conduct SEM

 ML estimation is used to estimate 

parameters

13
Structural Model

14

Motivation

Cognitive 

Capacity

Attitude

Recruitmen

t Effort

Task 

Difficulty

Perception 

of Content

Perception 

of Task

Response 

Burden
Results: Model fit 

statistics

 x

2

(93)=1697, p<.0001

 Standardized RMSR (SRMSR): 0.0454

 RMSEA Estimate: 0.0579

 Adjusted GFI (AGFI): 0.9379

 Bentler Comparative Fit Index: 0.9004

15
Measurement Model: 

Motivation

 Interviewer assessment of R’s level of effort

 A lot of or moderate effort (=1) vs. bare minimum (=0)

 Interviewer assessment of R’s level of cooperation

 Very cooperative (=1) vs. other(=0)

 Level of reluctance through door step concerns

 Easy (=1) vs. busy or reluctant (=0)

 Whether or not R refused at least once to survey

 Never refused (=1) vs. refused at least once (=0) 16

Motivation

Effort Cooperation Reluctance Refusal

0.65*

0.51*

0.37* 0.39*
Measurement Model: 

Cognitive capacity

 Age: >60 (=0) vs. <=60 (=1)

 Education: high school or less (=0) vs. some college 

or more (=1)

17

Cognitive 

Capacity

Education Age

0.01* 22.28*
Measurement Model: 

Attitude

 How sensitive did you feel the questions I asked you 

today were?

 Not sensitive at all (=1) vs. other (=0)

 I trust that the U.S. Census Bureau to safeguard the 

information that I provided them

 Strongly agree or agree (=1) vs. other (=0)

18

Attitude

Sensitivity Confidentiality

0.49* 0.95*
Survey Design 

Characteristics

 Contact attempts:

Equal to or less than median (=0) vs. more than 

median (=1)

 Personal visits

Equal to or less than median (=0) vs. more than 

median (=1)

19

Recruitment 

Effort

# of contact 

attempts # of personal visits

0.76* 0.64*
Measurement Model: 

Task Difficulty

 Size of household

 Multiple-person households (=1) vs. single-person 

households (=0)

 Total time spent

 Longer than median (=1) vs. equal to or less than median 

(=0)

20

Task Difficulty

# of Household 

members

Total time spent on 

interview

21.78* 0.01*
Measurement Model:

Perception of Survey

 Perception of survey

 Very or somewhat interesting (=1) vs. other (=0)

 Perception of questions

 Very easy (=1) vs. other (=0)

21

Perception of 

Survey content

Survey 

interesting

Survey easy to 

answer

0.77* 0.58*
Measurement Model:

Perception of Survey

 Perception of length

 Survey too short or about right(=1) vs survey too long (=0)

 Perception of survey requests

 Reasonable number of interviews (=1) vs. too many 

interviews (=0)

22

Survey not too 

long

Reasonable number 

of interviews

Perception of 

Survey task

0.72* 0.85*
Results: Structural Model (1)

23

Motivation

Cognitive 

Capacity

Attitude

Recruitment

Effort

Task 

Difficulty

Perception 

-0.001 of Content

0.13*

0.86*

-0.0004

-0.005

-0.01

Response 

Burden
Results: Structural Model (2)

24

Motivation

Cognitive 

Capacity

Attitude

Recruitment 

Effort

Task 

Difficulty

Perception 

of Task

0.001

0.32*

-0.59*

-0.004*

0.05*

-0.56* Response 

Burden
Results: Structural Model (3)

25

Motivation

Cognitive 

Capacity

Attitude

Recruitment 

Effort

Task 

Difficulty

Perceived 

Burden

-0.07*

-0.001

-0.20*

0.009

-0.0002
Conclusions

Total Effects Direct Effects Indirect Effects

Motivation -0.26*** -0.07*** -0.18***

Cognitive Capacity -0.001* -0.001# 0

Attitude 0.13*** -0.20*** 0.32***

Recruitment Effort -0.02 0.01 -0.03*

Task Difficulty 0.002** 0 0.002***

Perception of 

Survey Content

-0.01 -0.01 0

Perception of 

Survey task

-0.58* -0.58* 0

26

#p<.10; *p<0.05; **p<0.01; ***p<0.001; 
What causes response burden & who is 

burdened out?

 Survey and task characteristics have weak 

impact on response burden

 Respondent motivation and attitudes have 

strong impact on response burden

Unmotivated respondents

Respondents with positive attitudes

 Perception of survey task has strong impact 

on response burden

Those who perceived the survey as too long and 

too many

27
Limitations and Next Steps

 Model fit is not great

Modifying both measurement and 

structural models 

 Interaction between respondent, 

survey, and task characteristics not 

considered

Including interaction terms in the structural 

models

28
Contact Information

Scott Fricker

Senior Research Psychologist

Office of Survey Methods Research

www.bls.gov/osmr

202-691-7390

fricker.scott@bls.gov