1 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

You’ve got mail: 

Estimating the Effects of Postcards 

and Monetary Incentives on 

Respondent Recruitment

Gray Jones, Economist

Division of Consumer Expenditure Surveys

AAPOR Conference

May 13, 2021
2 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Outline

 Design of embedded postcard and monetary 

incentive experiment

 Findings of experiment

 Conclusions and recommendations for future 

research
3 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Literature on Pre-Notification and 

Prepaid Incentives

 Pre-notification has been shown to have some 

positive impact on participation 

In epidemiological studies (Koitsalu, M., et al. 

2018)

And web surveys (Dykema, J., et al. 2011)
4 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Literature on Pre-Notification and 

Prepaid Incentives

 Prepayment of monetary incentives generally 

increase participation rates (Singer, E. & C. Ye, 

2013)

 Incentive via debt card was ineffective for 

increasing CE Diary Survey participation in 

past studies (McGrath, D., et al. 2007) 
5 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Postcard and Incentive Experiment 

Overview

 All groups received an advance letter (AL) 

through USPS priority mail

 Advance material groups

Control (AL only)

Postcard Only

Both
6 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

LSF Postcard and Incentive 

Experiment Results

1. Postcard Results 

2. Incentives Results

3. Contact Attempts and Time Spent in 

Instrument
7 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

LSF Postcard Experiment

Postcard

Assignment N Percent

Postcard

Group 1,332 75.2%

Control

Group 439 24.8%

Total 

Sample 1,771 100%
8 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

LSF Postcard Experiment

Postcard Receipt Status N Percent

Question Not Asked 466 35%

Question Asked 866 65%

Reported Received 506 58%

Not Reported Received 360 42%

Total Postcard Group 1,332 100%
9 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Participation by Postcard Group 

Assignment (n=1771)

52% 53%

48% 47%

0%

10%

20%

30%

40%

50%

60%

No Postcard Postcard

Outcome Percentage

Type A Nonresponse Good
10 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Participation by Postcard 

Reported Received (n=866)

32%

25%

68%

75%

0%

10%

20%

30%

40%

50%

60%

70%

80%

No Postcard Postcard Reported Received

Outcome Percentage

Type A Nonresponse Good
11 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Advance Letter and Incentives 

Overview

 Subsequent to the 

postcard

 A random half 

contained a $5 

incentive 
12 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Response Rate by Incentive Group

Incentive 

Status Good Type A Nonresponse All Response 

Rates

Incentive group 412 467 879 46.9%

Control group 423 466 889 47.6%

Total 835 933 1,768 47.2%
13 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Advance Letter and Incentives

Advanced Letter Receipt 

Status N Percent

Question Not Asked 561 31.7%

Question Asked 1,210 68.3%

Reported Received 1,017 84%

Not Reported Received 193 16%

Total Letter Sample 1,771 100%
14 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Advance Letter and Incentives

Incentive 

Assignment N Percent

Incentive 

Group 879 49.7%

Control

Group 889 50.3%

Total 

Sample 1,768 100%
15 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Advance Letter and Incentives

Advanced Letter Receipt 

Status for Incentive 

Group

N Percent

Question Not Asked 285 32.4%

Question Asked 594 67.6%

Reported Received 501 84.3%

Not Reported Received 93 15.7%

Total Incentive Group 879 100%
16 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Participation by Advance Letter 

Received Status (n=1,210)

51%

27%

49%

73%

0%

10%

20%

30%

40%

50%

60%

70%

80%

No Letter Letter Reported Received

Outcome Percentage

Type A Nonresponse Good
17 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Incentives and Advance Letter 

Received Status (n =1,208) 

49%

27%

53%

27%

51%

73%

47%

73%

0%

10%

20%

30%

40%

50%

60%

70%

80%

No Letter Incentive Letter Reported

Received Incentive

No Letter NonIncentiveLetter Reported

Received NonIncentive

Outcome Percentage

Type A Nonresponse Good
18 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Incentives and Mode Choice

Incentive 

Group N Online Diary

Paper 

Diary

Incentives 412 43.7% 56.3%

Control group 423 44.2% 55.8%

Total 835 43.9% 56.1%
19 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Respondent Debriefing on 

Incentives (n = 332)

Question Response 

Options

% of CUs

In the first letter we 

sent you, explaining 

the survey and asking 

for you to participate, 

we included a fivedollar bill. Did your CU 

receive this money?

Yes 65.1%

No 11.1%

Don’t know 21.4%

Refused 2.4%
20 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Respondent Debriefing on 

Incentives (n = 216)

Question Response Options % of 

CUs

How did the money 

your CU received affect 

your decision to 

complete the diary?

It made me more willing to 

complete the diary

31.9%

It made me less willing to 

complete the diary

1.4%

It had no effect on my 

decision to complete the 

diary

65.3%

Don’t know 1.4%
21 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Comparing Contact Attempts

Between Incentive Groups

Average number of contact attempts Complete Type A Total

LSF cases receiving incentive 5.8

(n=412)

4.7

(n=467)

5.2

(n=879)

LSF cases not receiving incentive 5.9

(n=423)

4.8

(n=466)

5.3

(n=889)

Average Number of contact attempts 

to placement outcome of the diary 

Complete Type A Total

LSF cases receiving incentive 3.0

(n=412)

4.2

(n=467)

3.6

(n=879)

LSF cases not receiving incentive 2.9

(n=423)

4.2

(n=466)

3.6

(n=889)
22 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Comparison of Total Time Spent in 

the Diary by Incentive Groups

Total Time Spent in Diary Complete

LSF cases receiving incentive 48.6 minutes

(n=412)

LSF cases not receiving incentive

47.7 minutes

(n=423)
23 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Conclusions from Experiment

 Pre-notification postcards had no 

measureable effect on participation 

 Prepaid cash incentives made no discernable 

impact on the behavior of the LSF sample 

units.
24 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Future Research

 Incentive effect by income

 Advance material receipt results 
25 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Gray Jones

Economist, BRPD 

Division of Consumer Expenditure Surveys 

www.bls.gov/cex

202-691-6869

jones.graham@bls.gov

Contact Information