Monetary Incentives and Response Rates 

in Household Surveys: 

How much gets you how much?

Andrew Caporaso, Westat

Andrew Mercer, Pew Research Center

David Cantor, Westat

Reanne Townsend, Westat
Response Rates, 2000–2014: All Studies

2

55%

60%

65%

70%

75%

80%

85%

90%

95%

100%

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014

NHIS NSDUH GSS NCVS MEPS MCBS NHANES NSFG

(Williams, Brick & Hubbard (2016))
Incentives and Response Rates 

 Incentives improve response rates

– Monetary > Non-monetary

– Prepaid > Promised

• Is this true across for all modes of data collection?

– More money > Less money

• $0 vs $1 vs $2 vs $5 vs $10?

 “...no good evidence for how large an incentive 

should be.” (Singer & Ye, 2013)

• Should depends on desired response rate and costs
Our Research Questions

 What is the expected improvement in response rate 

per dollar of incentive?

– The dose-response relationship

 How is this relationship impacted by...

– Incentive timing (prepaid or promised)?

– Data collection mode?

– Burden & sponsorship?

 Has the relationship changed over time?

– We look at research from the past 2 decades

4
Methodology – Meta-analysis

• Meta-analysis of experimental* literature 

on incentive use...

– Published 1992 or later

– Pertaining to a mail, telephone or in-person survey

– Targeted at general population samples

– Testing monetary incentives offered at the onset of the 

survey request (prepaid or promised)

– In cross-sectional survey, or 1st round of panel

*comparison of two or more incentives in the same survey

5
Methodology – Data Captured

 DV: Response rate;

 IV: Incentive amount, converted to $2012 USD, natural log transformation

 Incentive timing (Prepaid, Promised);

 Mode (Mail, Telephone, or In-person);

 Survey sponsor (Government/University, or Private);

 Survey considered burdensome (Yes, No);

– Definition from Singer, et al. (1999). 

• Longitudinal; > 1 hour; sensitive questions, addt’l tasks besides survey 

 Year of experiment or year/publication;

6
Meta-analysis – Literature Search

• After conducting a thorough literature 

search...

– Over 200 reports found on incentive effects

• 40 met criteria for meta-analysis

– 55 experiments summarized in 40 reports

• 178 conditions tested across 55 experiments

7
Data Summary I

8

Telephone

(n = 59)

Mail

(n = 94)

In-Person

(n = 25)

Timing

Prepaid 17 73 8

Promised 21 2 11

No Incentive 21 19 6

Prepaid Values ($2012)

Min Value $1.38 $1.06 $1.54

25th Percentile $2.50 $2.82 $10.29

Median Value $2.82 $6.38 $18.44

75th Percentile $6.89 $11.27 $32.53

Max Value $7.04 $56.94 $56.38

Promised Values ($2012)

Min Value $5.88 $5.54 $1.54

25th Percentile $12.15 $7.14 $28.96

Median Value $21.46 $8.75 $36.85

75th Percentile $29.27 $10.35 $52.98

Max Value $50.07 $11.96 $74.65

Distribution of experimental conditions by mode of experiment, 

incentive timing, and incentive value details.
Data Summary II

9

Telephone

(n = 59)

Mail

(n = 94)

In-Person

(n = 25)

Year

1987–1991 2 0 4

1992–1996 11 4 3

1997–2001 21 54 3

2002–2006 21 23 6

2007–2011 4 13 9

Burden

Low Burden 59 38 3

High Burden 0 56 22

Sponsor

Government 25 36 21

University 24 16 4

Private 10 42 0

Distribution of experimental conditions by mode of experiment, year, 

burden, and survey sponsorship.
Methodology – Statistical Analysis

 Hierarchical regression model

– Level 1: Conditions

– Level 2: Experiments

 Weighted by precision

– Based on condition sample size 

10
Parameter B S.E.

Intercept 0.26 (0.062)*

ln($) [prepaid] 0.06 (0.009)*

Mail 0.01 (0.056)

In-Person 0.37 (0.092)*

High Burden -0.27 (0.063)*

Gov./Univ. Sponsor — —

Year minus 2013 -0.02 (0.004)*

ln($) x Promised -0.04 (0.011)*

ln($) x Mail 0.03 (0.011)*

ln($) x In-Person -0.02 (0.016)

ln($) x Burden — —

ln($) x Gov./Univ. Sponsor — —

ln($) x Year — —

ln($) x Mail x Promised -0.05 (0.019)*

ln($) x In-Person x Promised 0.03 (0.020)

* p < .05

Regression Model Predicting Response Rate

11
Change In RR by Mode/Timing
2012$ Mail prepaid Phone 

prepaid

Phone 

promised In person

$1 +.06 +.04 +.01 +.02

$2 .10 .07 .02 .03

$3 .12 .08 .03 .04

$4 .14 .10 .03 .05

$5 .16 .11 .04 .05

$10 .22 .14 .05 .07

$15 .25 .17 .06 .08

$20 .27 .18 .06 .09

$30 .31 .21 .07 .10

$40 .33 .22 .07 .11

$50 .35 .24 .08 .12

13

Estimated improvement in response rate relative to no incentive 

by incentive value, timing and mode.

Estimates based on little or no experimental evidence
Incentive Conclusions

• Effects on response rates:

• Dependent on mode and incentive timing

• Still much variability across studies

• Effect of survey characteristics:

• Incentive timing matters

• Most important in telephone surveys

• Least important for in-person surveys

• Inconclusive for mail 

• Findings inconclusive for sponsorship, burden

• Effects over time: 

• No changes observed over time*

• *mitigated by declining response rate overall

14
Response Rates, 2000–2014: All Studies

15

55%

60%

65%

70%

75%

80%

85%

90%

95%

100%

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014

NHIS NSDUH GSS NCVS MEPS MCBS NHANES NSFG

(Williams, Brick & Hubbard (2016))
GSS Incentives, 2004–2014

16

0%

20%

40%

60%

80%

2004 2006 2008 2010 2012 2014

Proportion Receiving

$31.59 

$37.91 

$43.33 

$41.25 

$44.76 

$49.37 

 $-

 $15

 $30

 $45

 $60

2004 2006 2008 2010 2012 2014

Average Amount Paid

(Williams, Brick & Hubbard (2016))
Incentives and Longitudinal Surveys

 Incentives work similarly in panel surveys as they do in 

cross-sectional surveys

– “the evidence suggests consistently that attrition rates 

would be higher in the absence of incentives, but we have 

limited knowledge of what the optimum strategies are for 

any given design...” (Laurie & Lynn, 2009)

 Some evidence that cost of increased incentives 

partially offset by reductions in other data collection 

costs over time
Incentives, Data Quality, and Costs

 Literature mixed on impact of incentives on other data 

quality indicators (Singer & Ye, 2013)

– Sample composition

– Item missing, completion rates

– Nonresponse bias

 Some evidence that incentives can facilitate reductions 

in other data collection costs

– Cost-savings greatest when these costs are high
Future Research

 Incentives for web surveys

– Lotteries, panel points

 Quasi-monetary incentives

– Difference between cash vs debit cards? 

 Promised mail incentives

– Mail to web? 

 Incentives for other populations?

– Establishments, physicians, low income?

19
Thank You

• Bureau of Labor Statistics

• Andrew Mercer, Reanne Townsend, David Cantor

• Aaron Maitland, Roger Tourangeau, Mike Brick, 

Doug Williams, Darby Steiger, Jon Ratner

• Everyone in AAPOR & beyond who contributed research

andrewcaporaso@westat.com

20