1 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Yezzi Angi Lee 

David Biagas, Jared Ogden, Sharon Pham

Barry Steinberg, Meaghan Smith, and Melissa Edwards

Division of Consumer Expenditure Surveys

76th Annual AAPOR Conference

May 13, 2021

Impact of the COVID-19 Pandemic on 

Consumer Expenditure Surveys 
2 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Overview

• Timeline of events—protocol changes 

• Guidance to Field Staff 

• Non-respondent reclassification 

• COVID-19 impact on CE interview data quality 
3 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Consumer Expenditure Surveys

 Interview 

Large purchases and recurring 

payments

3 months recall

Rotating panel

4 waves

In-person and telephone 

surveys

 Diary 

Small or frequently purchased 

items

Two consecutive 1-week periods

In-person placement and pick 

up

Paper diary 
4 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Protocol Changes
5 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Timeline

March 2020

* Online version of Infobook

* Telephone interviewing only

* Census Nation Processing 

Center (NPC) closed

April & May 2020

* Advance Letters

June 2020

* Online Diary 

* Stimulus payment questions 

added

July 2020

* Some personal visits are allowed for certain 

geographic areas (optional)

* Disposable information booklets were provided 
6 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Guidance to Field Staff - Interview

 Telephone interviewing

For Wave 2 to Wave 3 interviews, Field interviewers (FRs) likely have a 

working phone number

FRs use Census resources to obtain phone numbers for those cases not 

yet contacted/without an available phone number

 COVID flag was added for non-respondents 

 Use of online version of the Information Booklet 

 July 1st – Personal visits are optional
7 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Guidance to Field Staff - Diary

 Conduct telephone interview to collect demographic and 

income section

 Send a web link to Diary forms or ship Diary form 

 COVID flag was added for non-respondent 

 June 1st – Provide option of online diary

 July 1st – Personal visits are optional 
8 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Non-respondent Reclassification
9 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Non-respondent Reclassification

 Beginning mid-March, the COVID-19 pandemic forced the U.S. Census 

Bureau to mandate that interviewers make contact attempts and conduct 

interviews only by phone

 Many addresses were “unreachable” by phone, making their eligibility status 

unknown

 AAPOR’s Standard Definitions Report recommends being conservative with 

response rates

 Increase in eligible nonresponse results in lower response rates but more 

accurate 

 FRs were initially instructed to code the cases of unknown eligibility as 

ineligible
10 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Non-respondent Reclassification

 CE changed some ineligible non-response to eligible nonresponse 

based on following rules

 Interview

If the interviewer attached a COVID-19 note to the case, and the CU was 

eligible in a previous interview

When the interviewer attached a COVID-19 note to the case, but we did not 

have information on the CU’s prior eligibility, randomly reclassified them based 

on prior eligibility rates for the region

 Diary

For Diary, we used Interview Survey eligibility rates and either both weeks were 

reclassified or none were
11 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Interview Reclassification Summary

Response Rate 

Before Reclassification

Response Rate 

After Reclassification 

April 63 % 46%

May 63 % 47%

June 66 % 45%
12 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Diary Reclassification Summary

Response Rate 

Before Reclassification

Response Rate 

After Reclassification 

March 49 % 39 %

April 52 % 29 %

May 52 % 29 %
15 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

COVID Impact on Interview Data Quality
16 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Interview Data Quality 

 CE interview Survey 

From July 2019 to June 2020 

Sample size of 13,246

Restricted to those who completed the interview, have change of 

mode information, and removed first interviewers

Pre COVID-19: Before the start of the COVID-19 pandemic (July 2019 –

March 2020)

During COVID-19: After the start of the COVID-19 pandemic (April 

2020-June 2020)
17 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Background

 Since beginning of the March 2020

Covid-19

Switched to all telephone interview

 Mode effect (Biagas, 2020; Young, 2020)

Telephone respondent tend to have higher salaries, levels of education, 

and more likely to be homeowner and white

Personal visit respondents are more likely to use records during 

interview and reported lower expenditures
18 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Quality Metrics

 Total Interview Time

 Number of Entries

 Proportion of Rounded Entries

 Total Expenditures
19 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Mode Changes in the data

Time Prior Mode Current Mode N

Pre COVID-19

In-person In-person 4,982

Telephone Telephone 3,050

In-person Telephone 1,610

Telephone In-person 461

During COVID-19 Telephone Telephone 1,412

In-person Telephone 1,677
20 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Hypothesis #1 

 Naïve approach

telephone to telephone would show the effect of COVID-19 pandemic 

on changes in data quality
21 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Hypothesis #2 

*
22 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Demographic Characteristics 

 In-person to Telephone in Post COVID 

Home owner

White 

Older 
29 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Data Quality – Interview Time

0

10

20

30

40

50

60

70

80

90

Jul-19 Aug-19 Sep-19 Oct-19 Nov-19 Dec-19 Jan-20 Feb-20 Mar-20 Apr-20 May-20 Jun-20

Telephone to In-person

In-person to Telephone

In-person to In-person

Telephone to In-person
30 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Data Quality – Interview Time

0

10

20

30

40

50

60

70

80

90

Jul-19 Aug-19 Sep-19 Oct-19 Nov-19 Dec-19 Jan-20 Feb-20 Mar-20 Apr-20 May-20 Jun-20

Telephone to In-person

In-person to Telephone

In-person to In-person
31 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Data Quality – # Entries

0

5

10

15

20

25

30

35

40

45

50

Jul-19 Aug-19 Sep-19 Oct-19 Nov-19 Dec-19 Jan-20 Feb-20 Mar-20 Apr-20 May-20 Jun-20

Telephone to In-person

In-person to Telephone

In-person to In-person

Telephone to In-person
32 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Data Quality – % Rounded Entries

20

25

30

35

40

45

50

55

60

65

70

Jul-19 Aug-19 Sep-19 Oct-19 Nov-19 Dec-19 Jan-20 Feb-20 Mar-20 Apr-20 May-20 Jun-20

Telephone to In-person

In-person to Telephone

In-person to In-person
33 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

Jul-19 Aug-19 Sep-19 Oct-19 Nov-19 Dec-19 Jan-20 Feb-20 Mar-20 Apr-20 May-20 Jun-20

Telephone to In-person

In-person to Telephone

In-person to In-person

Data Quality – Total Expenditures
34 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Result Summary

 Trend 

Interview Time: No change

Number of Entries: No change

Proportion of Rounded Entries: No change

Total Expenditures: Decrease
35 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Conclusion

 Impact of COVID-19 pandemic

Shift to telephone interview

Online Diary implemented

Reclassification had major impact on response rate

No significant impact on data quality 

 Lesson learned 

Flexibility & Communication was very important in responding to the 

COVID-19 pandemic
Contact Information

36 — U.S. BUREAU OF LABOR STATISTICS • bl s.gov

Yezzi Angi Lee

Division of Consumer Expenditure Survey

www.bls.gov/cex

Lee.Yezzi@bls.gov