1 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Enhancing Data Quality Using 

Expenditure Records 

Safia Abdirizak, Laura Erhard, Yezzi Angi Lee, Brett McBride

Bureau of Labor Statistics

American Association of Public Opinion Research

May 19, 2017
2 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Outline

I. Background- CE Redesign (Gemini Project) 

Proof of Concept Test 

II. Record Use 

How frequently did respondents consult records?

Section-Level Expenditure Comparisons 

III. Response Quality

Data Quality Measurement 

Missing Data Rates

Extent of Rounded Responses 

IV. Summary

V. Next Steps
3 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

I. Background
4 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

CE Redesign (Gemini Project) 

In 2009, the Bureau of Labor Statistics (BLS) Consumer 

Expenditure Survey (CE) initiated the multi-year Gemini Project for 

the purpose of researching, developing, and implementing an 

improved survey design. 
5 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Proof of Concept Test

 Objective: To access the feasibility of the redesign by testing a singlesample expenditure survey design consisting of

 two personal interviews, 

 personal diaries,

 incentives, 

 record usage,

 and technology usage (online diaries)

 Fielded from July 2015-October 2015 divided across four Census Regions: Atlanta, 

Chicago, New York, and Denver.

 Final number of 520 completed cases for a response rate of 50 percent. 
6 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Test Design

 Visit 1: 

 Recall Interview

 One week of diary keeping and records collection

 Visit 2: 

 Records Interview

 Incentives including, 

 $2 token incentive with the advanced mailing,

 $20 after Visit 1, if respondent completed all sections of Recall Interview,

 $20 for each eligible household member that entered at least 1 expenditure in diary,

 and $20 debit card mailed to the respondent after completing the records interview with an 

additional $20 debit card included if 1+ records were used. 

Note: Our analysis compared the Proof of Concept Test sample (“test” group) with Matched 

Production sample responses to corresponding sections in the current survey (“control” group). 
7 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

II. Record Use 
8 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

How frequently did respondents consult records? 

78%

73%

67% 65% 64%

54%

50%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Utilities (n=512) Non-Health

Insurance (n=344)

Health Insurance

(n=302)

Owned Housing

(n=312)

Owned Vehicles

(n=174)

Medical (n=280) Rented and Leased

Vehicles (n=52)

Percent Frequency of Record Use (among CUs with 1+ expenditure)
Section-Level Expenditure Comparisons 

Did record use cue additional expenditure 

reports, within the test sample? 
10 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Average Count of Entries by Record Use

(among CUs with 1+ section expenditure)

2.97

4.26

1.19 1.25

1.83 1.7

3.61

2.86

3.41

1.04

1.26 1.36 1.3

2.19

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Owned Housing ***Utilities Rented and Leased

Vehicles

Owned Vehicles ***Non-Health

Insurance

***Health

Insurance

***Medical

Test

With Record Use Without Record Use

*p<.05; **p<.01; ***p<0.001
11 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Section-Level Expenditure Comparisons 

Did record use cue additional expenditure 

reports, overall by sample? 
12 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Average Count of Entries by Sample

1.77

4.01

0.11

0.43

1.13

0.91

1.58

1.86

3.92

0.13

0.43

1.26 1.22

1.48

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

Owned Housing Utilities Rented and Leased

Vehicles

Owned Vehicles *Non-Health

Insurance

***Health

Insurance

Medical

Test(N=520) Control (N=1483) *p<.05; **p<.01; ***p<0.001
13 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Section Level Expenditure Comparisons

Did respondents report increased expenditure 

values when referencing records, within the test 

sample?
14 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Average Expenditure Totals by Record Use

(among CUs with 1+ section expenditure)

$2,835 

$1,292 

$959 

$4,027 

$736 

$1,047 

$736 

$2,660 

$1,226 $1,363 

$3,505 

$552 

$936 

$494 

$0

$500

$1,000

$1,500

$2,000

$2,500

$3,000

$3,500

$4,000

$4,500

Owned Housing Utilities Rented and Leased

Vehicles

Owned Vehicles *Non-Health

Insurance

Health Insurance ***Medical

Test

With Record Use Without Record Use

*p<.05; **p<.01; ***p<0.001
15 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Section Level Expenditure Comparisons

Did respondents report increased expenditure 

values when referencing records, overall by 

sample?
16 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Average Expenditure Totals by Sample

$1,664 

$1,258 

$116 

$1,284 

$454 

$587 

$336 

$1,783 

$1,290 

$156 

$1,404 

$518 

$772 

$334 

$0

$200

$400

$600

$800

$1,000

$1,200

$1,400

$1,600

$1,800

$2,000

Owned Housing Utilities Rented and Leased

Vehicles

Owned Vehicles Non-Health

Insurance

Health

Insurance***

Medical

Test (N=520) Control (N=1,480)
17 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

III. Response Quality
18 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Data Quality Measurement

We explored the extent of rounding in the records interview and 

evaluated item non-response through analyses of respondentprovided invalid blanks in both the test and control samples.
19 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Missing Data

Was there less item non-response, in the form of don’t 

knows? 
20 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Distribution of Number of Don’t Knows by Sample 

69%

14%

13%

4%

79%

11%

9%

2%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

0

1

2-5

6+

Test (N=520) Control (N=1,483)
21 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Extent of Rounded Responses

With the use of records, was there less evidence 

of rounding to multiples of $25? 
23 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Average Percent of Rounded Expenditure Responses

22%

34%

0%

5%

10%

15%

20%

25%

30%

35%

40%

***Test (n=510) Control (n=1,432)

*p<.05; **p<.01; ***p<0.001
24 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Average Percent of Rounded Expenditure Responses by Question 

19%

13%

46%

11%

23% 22%

29%

3%

16%

25%

44%

23%

58%

17%

23%

26%

34%

7%

26%

35%

0%

10%

20%

30%

40%

50%

60%

70%

Owned

Housing

Utilities Rented and

Leased

Vehicles

Life Insurance Long Term

Care Insurance

Automobile

Insurance

Homeowners'

Insurance

Renter's

Insurance

Health

Insurance

Medical

Test (n=510) Control (n=1,432)
25 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Record Use and Rounding - Utilities Example

78%

36%

Test (n=510) Control (n=1,432)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage Frequency of Record Use 

(among CUs with 1+ Section 

Expenditures)

13%

23%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Test (n=118) Control (n=631)

Comparison of CU average percent of 

rounded expenditures
26 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

IV. Summary
27 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Summary

1. Did record use cue additional expenditure reports?

 Yes, all sections (except Owned Vehicles) had higher entry averages when CUs 

used records. 

2. Did respondents report increased expenditure values when referencing records?

 Though not consistently significant, record use was associated with increases 

in expenditure values in both the control and test sample.

3. Was there less item non-response?

 Yes, respondents in the test sample provided fewer missing values for 

expenditure variables than respondents in the control sample. 

4. Was there less evidence of rounding to multiples of $25? 

 Yes, for questions in the records interview the test sample had higher record 

use with lower levels of rounding compared to the control sample. 
28 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

V. Next Steps
29 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Next Steps

 Explore incentives data once 

available 

 Further investigate section level 

expenditure comparisons

 Continued development of records 

interview protocol

 Large Scale Feasibility Test 

(tentatively 2020)

Larger sample sizes (2,000 completes)
Contact Information

30 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Safia Abdirizak

Economist

Division of Consumer Expenditure Surveys

www.bls.gov/cex

202-691-5137

abdirizak.safia@bls.gov