Discussion: Collecting Spending Data during a Pandemic

Stephanie Eckman

Fellow, Survey Research Division

www.rti.org

RTI International is a trade name of Research Triangle Institute. RTI and the RTI logo are U.S. registered trademarks of Research Triangle Institute.

Tracking Trends

2

Average Z

Rather steady, slow trend upward over time

But estimates are different due to CHANGE IN TRUE VALUE as well as CHANGE IN ERROR

2

Change in Trends due to Pandemic

Important questions:

Are households eating out less?

Which households increased spending on home improvement during pandemic?

How much did spending on home office equipment increase?

Trends by income groups, age, race/ethnicity, gender, region, etc.

3

Applies to other questions as well

3

4

Measuring Change

         Change in Z from time 1 to time 2

true value

error

change in true value

change in error

true value

error

4

Factors Affecting Measures of Change

Focus before pandemic was on stable 

ME, NR somewhat understood

Pandemic changed drastically

ME, NR changed

5

1 -- Few changes in data collection protocol over years

Little understanding of how changes WOULD affect data

2 – when pandemic hit and DC methods changed, little 

Due to COVID-19 we cannot separate true changes from changes in errors

Errors before pandemic (e1) somewhat understood

But e2 is not – we struggled simply to collect the data

Irony is that ...

5

Factors Affecting Measures of Change

6

Changes in true value of Z

Reduced income

Stockpiling

Delivery 

Changes in error term

Nonresponse 

Mode effects

Nonresponse 

Measurement error

How to distinguish?

External source 

Testing before pandemic

NONRESPONSE

At home patterns

Ash paper explores this

MODE

Affects who responds

Spam calls

Internet access, ability

Literacy

Interviewers encourage response

Affects responses

Aural vs visual

Self-administered vs interviewer

EXTERNAL – Rigg & Creech

6

Thought Experiment

What would we have done, if we’d had time to prepare?

What experiments could have helped us understand ?

7

1 -- Few changes in data collection protocol over years

Little understanding of how changes WOULD affect data

2 – when pandemic hit and DC methods changed, little 

Due to COVID-19 we cannot separate true changes from changes in errors

Errors before pandemic (e1) somewhat understood

But e2 is not – we struggled simply to collect the data

Irony is that ...

7

Prepare for Next Disruption

8

Run parallel web survey

Impute face-to-face estimates from web survey data

When jump to web data collection necessary, we have adjustments ready
| Source              | Background Vars | Z F-to-F | Z Web |
|---------------------|-----------------|----------|-------|
| Face-to-Face Survey | X               | X        |       |
| Web Survey          | X               |          | X     |

֍

Similar to design used in: Eckman (2021) 10.1093/jssam/smab024

Relies on assumptions -- but so does doing nothing

MI not embraced within some agencies

8

9

Thank You

seckman@rti.org

http://stepheckman.com

@stephnie

9

# image1.png

# image2.png

# image5.png

# image8.png

# image9.png

# image10.png

# image11.png

# image12.png

# image13.png

# image14.jpeg

# image7.png