1 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

An Examination of Nonresponse Bias 

in the Consumer Expenditure Surveys 

During the COVID-19 Period

Stephen Ash

Price Statistical Methods Division

Office of Prices and Living Conditions

Federal Committee on Statistical Methodology 

2021 Research and Policy Conference

November 3, 2021
2 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Research Question

 Did nonresponse bias increase during the 

COVID-19 Pandemic?

 Did the COVID-19 Pandemic impact the 

effectiveness of the nonresponse adjustments 

within sample weighting?

 Were the variables used in the nonresponse 

adjustment as effective as in the past?
3 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Current Noninterview Adjustment

 Weighting cells approach.

 Partition the sample (both interviews and 

noninterviews) into cells.

 Calculate the response rates (probability of 

response) within cells.

 The Noninterview adjustment is the inverse of 

the response rate within cells.
4 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Current Noninterview Adjustment

 Variables used to define 192 cells

Number of Contacts (1, 2, 3-4, 5+)

Number of persons in CU (1, 2, 3-4, 5+)

Zip-code level IRS income (Top 10%, Middle, bottom 10%)

Census Region (Northeast, South, Midwest, West)

 Collapsing for large values of the noninterview 

adjustment and small sample sizes within cells
5 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

CE Interview Survey Response Rates 

by Number of Contact Attempts
6 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Measuring Nonresponse Bias

 We want to measure the bias of estimates due 

to nonresponse.

 We don’t know the truth. 

We will never know the expenditures for 

noninterviews.

 What’s the next best thing?
7 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Our Research Approach – Build a Better 

Mousetrap for Nonresponse Bias

 Developed a noninterview adjustment tailored 

specifically for the COVID-19 period (2020Q2 

and 2020Q3).

 Compared current estimates with estimates 

derived from alternative noninterview 

adjustments.

 Limitation: just because we didn’t catch mice, 

doesn’t mean there aren’t mice. Could mean 

we need an even better mousetrap.
8 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Better Mousetrap for Nonresponse Bias

 Used a logistic regression model to model response.

 Considered current variables and additional 

variables for defining the noninterview adjustment. 

All variables need to be known for interviews and 

noninterviews.

Included additional variables from the Census 

Planning Database.

Included variables that were significant in the 

logistic regression model
9 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

CE Interview Survey

Alternative Noninterview Adjustments

Current

A2

(2020Q2)

A3

(2020Q3)

Number of Contacts X X X

Number of People in CU X X X

Census Region X

IRS Zip Code-Level Income X

Tenure X X

Census Division X X

Interview Number X X

Was the PSU SR or NSR? X X

Quartiles of proportion aged 65+ X

Quartiles of proportion HS Grad X
10 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Distribution of 2020Q2 Interview Survey 

Noninterview Factors
11 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

2020Q2 Interview Survey

Annual Estimates of Expenditures

Current 

Method

Alternative Method 

(A2)

Alternative Method 

(A3)

Mean of ... Estimate Estimate Percent 

Difference Estimate Percent Difference

Total Expenditures 56,314 

(848)

56,519

(781) 0.4 56,668(832) 0.4

Housing Expenditures 19,563

(268)

19,672

(273) 0.6 19,705(281) 0.6

Transportation 

Expenditures 

9,399 

(490)

9,420

(498) 0.2 9,454

(517) 0.2

Food Expenditures 8,536 

(115)

8,549

(106) 0.2 8,557(107) 0.2

Personal Insurance 

Expenditures 

6,583 

(139)

6,586

(121) 0.0 6,615(128) 0.0

Health Expenditures 4,867 

(106)

4,850

(110) -0.3 4,854(109) -0.3
12 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

2020Q3 Interview Survey

Annual Estimates of Expenditures

Current 

Method

Alternative Method 

(A2)

Alternative Method 

(A3)

Mean of ... Estimate Estimate Percent 

Difference Estimate Percent Difference

Total Expenditures 57,455 

(636)

58,208

(712) -1.3 58,428(701) 1.7

Housing 

Expenditures 

20,294

(250)

20,608

(297) -1.5 20,700

(282) 2.0

Transportation 

Expenditures 

9,503 

(332)

9,618

(357) -1.2 9,614

(357) 1.2

Food Expenditures 8,528 

(98)

8,604

(104) -0.9 8,624

(101) 1.1

Personal Insurance 

Expenditures 

6,339 

(132)

6,405

(138) -1.0 6,444

(140) 1.7

Health Expenditures 4,949 

(116)

4,952

(113) -0.0 4,964(111) 0.3
13 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

CE Diary Survey

Alternative Noninterview Adjustments

Current

A2

(2020Q2)

A3

(2020Q3)

Number of Contacts X X X

Number of People in CU X

Census Region X

IRS Zip Code-Level Income X X

Tenure (owner/renter) X X

Census Division X X

Quartiles of Hispanic Origin X X

Month X
14 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Distribution of 2020Q3 Diary Survey Noninterview 

Factors
15 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

2020Q2 Diary Survey

Monthly Estimates of Expenditures

Current 

Method

Alternative Method 

(A2)

Alternative Method 

(A3)

Mean of ... Estimate Estimate Percent 

Difference Estimate Percent Difference

Total Expenditures 2,825 

(175)

2,754

(206) -2.5 2,840(236) 0.5

Meal Purchases Away 

from Home

124

(8)

123

(9) -0.6

125

(9) 1.5

Food Purchased for 

Home Consumption

407 

(15)

393

(13) -3.4 398(14) -2.2

Clothing 82

(12)

81

(13) -0.5 83(13) 1.8

Other 2,214 

(168)

2,158

(198) -2.5 2,234(228) 0.9
16 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

2020Q3 Diary Survey

Monthly Estimates of Expenditures

Current 

Method

Alternative Method 

(A2)

Alternative Method 

(A3)

Mean of ... Estimate Estimate Percent 

Difference Estimate Percent Difference

Total Expenditures 3,422 

(184)

3,317

(172) -3.1 3,388(186) -1.0

Meal Purchases Away 

from Home

165

(9)

164

(9) -0.7 163(9) -1.0

Food Purchased for 

Home Consumption

415

(12)

417

(14) 0.6 421(14) 1.5

Clothing 94

(7)

97

(8) 2.7 95(8) 0.8

Other 2,789

(182)

2,639

(172) -4.0 2,709(185) -1.4
17 — U.S. BUREAU OF LABOR STATISTICS TATISTICS • bls.gov bls.gov

Conclusion

 Not enough evidence of nonresponse bias.

 A future research project with Census will 

consider using Administrative data from the 

IRS in the noninterview adjustment. 
Contact Information

18 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Stephen Ash

Price Statistical Methods Division

Office of Prices and Living Conditions

ash.stephen@bls.gov