1 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Incentives in the CE Interview Survey: 

Present Findings and Future Research

Ian Elkin

Bureau of Labor Statistics

2017 CE Survey Methods Symposium

O7/18/2017

Any opinions expressed in this presentation are those of the authors and do 

not constitute policy of the Bureau of Labor Statistics.
2 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Incentives or How I get my Child to do 

Things He Does Not want to do

Ian Elkin

Bureau of Labor Statistics

2017 CE Survey Methods Symposium

O7/18/2017

Any opinions expressed in this presentation are those of the authors and do 

not constitute policy of the Bureau of Labor Statistics.
3 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Prior to Incentive Administration
4 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Post-incentive Administration
5 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Outline

I. Background

II. Study Design

III. Analysis

IV. Conclusions

V. Next Steps
6 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

I. Background
7 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Incentives Test - Background

 Study objectives included developing a plan for -

 Operationalizing & implementing incentives

 Researching & recommending incentive amounts

 Proposing incentive distribution procedures, including procedures to capture respondents that 

generally do not respond to classic incentives

 Analyze test data to make a recommendation regarding incentive implementation 

 Past CE incentives research -

 Gemini Incentive Structure Review: Summary of Incentive Experiences 

 CE incentives operational summaries

 CE Interview Incentives Test Report

 CE Diary Incentives Test Report 
8 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

II. Study Design
9 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Incentives Test - Overview

 Test Information and Conditions:

 July 2016 – March 2017 

Experiment

 First Interviews

 Test Sample Size: 1,350 per 

treatment group

 Control Sample Size: 1,950

 Conditional and unconditional 

incentives
10 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Incentives Test – Treatment & Control Groups

Treatment and Control Groups

$5 Token 

Incentive 

(unconditional)

$40 Survey 

Incentive 

(conditional)

$20 Records Use 

Incentive 

(conditional)

All $5 $40 $20

No Token None $40 $20

No Record $5 $40 None

Control (1st Interviews) None None None
11 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

III. Analysis
12 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

CE Data Quality Indicators

 Response rate

 Number of expenditures

 Number of contact attempts

 Mode – collection level

 Records usage

 Length of interview

 Doorstep concerns

 Converted refusal
13 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Incentives Test Analysis – Response Rate

Response Rates, Second Interview

Difference (Treatment Minus 

Control)

All 5.0 % points

No Token 3.2 % points

No Record 5.1 % points

Response Rates, First Interview

Difference (Treatment Minus 

Control)

All 4.6 % points

No Token 1.4 % points

No Record 5.0 % points
14 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Incentives Test Analysis – Number of First 

Interview Expenditures

Number of Expenditures, First Interview

Difference in Medians

(Control group baseline)

Difference in Means

(Control group baseline)

All +1.0 +1.1

No

Token +1.0 +1.2

No

Record +1.0 +1.0
15 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Incentives Test Analysis – Number of Second 

Interview Expenditures

Number of Expenditures, Second Interview

Difference in Medians

(Control group baseline)

Difference in Means

(Control group baseline)

All +1.0 +1.6

No

Token +1.5 +2.2

No

Record +1.5 +2.4
16 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Incentives Test Analysis – First Interview Contact 

Attempts

Number of Contact Attempts, First Interview

Difference in Medians

(Control group baseline)

Difference in Means

(Control group baseline)

All 0.0 -0.2

No Token 0.0 -0.4

No Record 0.0 -0.3
17 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Incentives Test Analysis – Second Interview Contact 

Attempts

Number of Contact Attempts, Second Interview

Difference in Medians

(Control group baseline)

Difference in Means

(Control group baseline)

All 0.0 +0.2

No Token 0.0 -0.3

No Record 0.0 -0.3
18 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Incentives Test Analysis – Mode of Collection

Difference in Mode of Interview (Treatment Minus Control)

Telephone In-Person Mix

All -5.4 % points 4.5 % points 0.9 % points

No Token -4.4 % points 4.2 % points 0.2 % points

No Record -1.8 % points 1.7 % points 1.1 % points
19 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Incentives Test Analysis – Records Usage

Records Usage

Difference (Treatment Minus 

Control)

All 29.6 % points

No Token 29.3 % points

No Record 14.3 % points
20 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Incentives Test Analysis – Length of Interview

Total Survey Time 

Difference in 

Medians

(Control group baseline)

Difference in 

Means

(Control group baseline)

All 2.6 minutes 2.9 minutes

No

Token 5.5 minutes 3.0 minutes

No

Record 0.9 minutes 3.2 minutes
21 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Incentives Test Analysis – Doorstep Concerns

Doorstep Concerns (Treatment Minus Control)

Not 

Interested/Hostile Time Privacy Other

All -2.1 % points -5.9 % points -5.5 % points 0.6 % points

No Token 1.8 % points -5.2 % points -0.1 % points 1.0 % points

No Record -4.2 % points -3.0 % points -2.9 % points -1.2 % points
22 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Incentives Test Analysis – Converted Refusal

Converted Refusals

Difference (Treatment Minus Control)

All -3.3 % points

No Token -0.9 % points

No Record -3.4 % points
23 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

IV. Conclusions
24 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Incentives Test - Conclusions

 Increase in response rates

 Increase in expenditures

 Decrease in contact 

attempts

 Increase in records use

 Decrease in converted 

refusals

 Increase in interview time

 What does this all mean?
25 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

V. Next Steps
26 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Incentives Test – Next Steps

 Continue analysis of 

expenditure data focusing 

on imputed expenditures

 Analyze introduced bias and 

add demographic controls

 Determine impact on 

respondent burden

 Analyze cost effectiveness
27 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Incentives Test – Next Steps

 Are incentives right for CE?

 What amount is the right 

amount?

 Are token incentives cost 

effective?

 What is the most impactful 

way to distribute incentives?
Contact Information

28 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Ian J. Elkin

Senior Economist

Division of the Consumer Expenditure Surveys

www.bls.gov/cex

202-691-6865

Elkin.Ian@bls.gov