Survey Isolation during COVID-19: The Effects of Suddenly Relying on Address-Matched Phone Numbers for Interviewing HouseholdsJonathan EgglestonU.S. Census BureauFCSM 2021

1

I would like to thank Yarissa Gonzalez, Tim Trudell, John Voorheis, as this presentation is an offshoot of our joint research on weighting with the Current Population Survey. Any opinions and conclusions expressed herein are those of the author and do not necessarily reflect the views of the U.S. Census Bureau. The Census Bureau's Disclosure Review Board and Disclosure Avoidance Officers have reviewed this data product for unauthorized disclosure of confidential information and have approved the disclosure avoidance practices applied to this release. DRB Delegated Authority Number FY2021-CES010-017.

1

Introduction

U.S. Census Bureau conducts in-person interviews for numerous surveys, including the Current Population Survey (CPS) and American Community Survey (ACS)

March 2020 

In-person interviewing suspended

For the multimodal ACS, paper response options suspended.  Materials could not be handled because of the closure of the Census Bureau’s National Processing Center 

2

2

How Were Interviews Conducted After March 2020?

Decentralized phone interviews from an interviewer’s home (i.e. non-CATI phone calls)  

Respondents contacted in two ways

Letter mailed with interviewer’s phone number --  some respondents took initiative to call their interviewer

Interviewers used phone numbers from either Census Bureau’s Contact Frame or a commercial lookup service

3

3

Census Bureau Contact Frame

Dataset with phone numbers for every household address in the United States (if the address has a phone number).

Phone numbers from a variety of sources

Prior respondents to the American Community Survey

Alaska Permanent Fund Dividend data 

SNAP/WIC/TANF administrative data for a handful of states

But for most of the U.S. Population, phone numbers come from commercial providers

4

4

Census Bureau Contact Frame

Source of commercial data: Exact details proprietary, but some components include

Magazine/periodical change of address information

Utility records

Concern: Lower income households may be less likely to be in these data

Fewer utilities: No internet at home, utilities part of apartment rent

Less likely to subscribe to magazines

Hypothesis: Low-income households are less likely to be contacted with these commercial phone numbers

Rest of Talk: Provide some evidence in support of this hypothesis

5

5

Nonresponse and Income-CPS

Match IRS and other administrative data to CPS sampled addresses, including nonrespondent households

Compare means between respondents and nonrespondents for each CPS month

Adjusted gross income (AGI) from tax returns

Characteristics of address-matched phone numbers: model-based quality score  

Higher quality score → More confidence phone number is associated with address in question 

Income results similar to Rothbaum and Bee (2020), who look at same measures for the CPS ASEC (“March” CPS)

6

6

Income of Respondents Higher After March 2020

7

Source: 2017-2020 Basic Monthly CPS + Administrative Data

Last Month: September 2020

Income of Respondents Higher After March 2020

8

Source: 2017-2020 Basic Monthly CPS + Administrative Data

Change in Phone Number Quality Too

9

Source: 2017-2020 Basic Monthly CPS + Administrative Data

Nonresponse and Income

Change in income of respondents associated with change in phone number quality of respondents

However, unclear if phone number quality is the driving factor or something else

Additional evidence comes from phone numbers reported by respondents to the American Community Survey (ACS) 

10

10

Regression: ACS Phone Number Matching Other Census Sources

Sample: Respondents sampled in 2018 for the ACS who had a landline phone number linked to their address

Further Restriction: Household matched to at least one IRS or other administrative record

For various technical reasons, only have data for the landline ACS reported number

Dependent variable: Binary variable =1 if the phone number the ACS respondent reported matches any of their landline phone numbers from the contact frame

Linear Probability Model

Explanatory variables: Economic and demographic data from administrative records

Takeaway: Higher income households more likely to have a matched landline number

11

11

Demographics(Inferred from IRS data, 2010 Census, and demographic data from the Social Security Administration)

12
| Any Household Member who is...   | Coefficient (Percentage Points) |
|--------------------------------|---------------------------------|
| Over 60                        | 16.47%***                       |
| Under 10                       | -8.82%***                       |
| Black                          | 1.58%***                        |
| Hispanic                       | -4.22%***                       |
| Foreign Born                   | -3.35%***                       |
| Married                        | 5.93%***                        |

Households with a member over 60 are 16.47 percentage points more likely to have their ACS reported phone number match one of their landline contact frame phone numbers

Source: 2018 ACS + Administrative Data

***: p<.01

Adjusted Gross Income
| AGI Amount                   | Coefficient (Percentage Points) |
|------------------------------|---------------------------------|
| Less than $25,000            | (Excluded Outcome)              |
| Between $25,000 and $50,000  | 1.06%***                        |
| Between $50,000 and $100,000 | 0.74%***                        |
| Over $100,000                | 1.44%***                        |

13

Source: 2018 ACS + Administrative Data

***: p<.01

Asset Income From IRS Records
| Received Any...   | Coefficient (Percentage Points) |
|-----------------|---------------------------------|
| Interest Income | 6.02%***                        |
| Dividend Income | 5.20%***                        |

14

Source: 2018 ACS + Administrative Data

***: p<.01

Structure Type
| Housing Unit Type     | Coefficient (Percentage Points) |
|-----------------------|---------------------------------|
| Trailer/Other/Unknown | (Excluded Outcome)              |
| Single-family         | 7.96%***                        |
| Multi-Unit            | -1.22%***                       |

15

Source: 2018 ACS + Administrative Data

***: p<.01

Number of Jobs
| Most W2s a Household Member Has Is.....   | Coefficient (Percentage Points) |
|-----------------------------------------|---------------------------------|
| 1                                       | (Excluded Outcome)              |
| 2                                       | -2.43%***                       |
| 3                                       | -3.42%***                       |
| 4 or more                               | -3.59%***                       |

16

Source: 2018 ACS + Administrative Data

***: p<.01

Conclusion

Takeaway: Lower income households less likely to have a phone number matched to their address

Implications for other surveys: Potential bias from linkage of addresses to phone numbers

Some surveys use addresses linked to phone numbers as alternative to landline RDD (e.g. Link et al. 2009 for Nielsen TV Ratings Diary)

However, nonlinkage bias may exclude the same types of people excluded from landline RDDs

17

Some Sources:

https://www.pewresearch.org/methods/2017/05/15/what-low-response-rates-mean-for-telephone-surveys/

Stephen J. Blumberg, Julian V. Luke, Coverage Bias in Traditional Telephone Surveys of Low-Income and Young Adults, Public Opinion Quarterly, Volume 71, Issue 5, 2007, Pages 734–749, https://doi.org/10.1093/poq/nfm047

Link, M., Gail Daily, C. Shuttles, Tracie L. Yancey, A. T. Burks, and C. Bourquin. "Building a new foundation: Transitioning to address based sampling after nearly 30 years of RDD." In 64th Annual conference of the American Association for Public Opinion Research, Hollywood, FL. 2009.

17

Contact Information

Jonathan Eggleston

Senior Economist

Survey Improvement Technical Lead

Survey and Economic Research Group 

Center for Economic Studies

U.S. Census Bureau

Office: 301.763.2357 

jonathan.s.eggleston@census.gov

18

18

Income of Respondents Higher After March 2020

19

Source: 2017-2020 Basic Monthly CPS + Administrative Data

# image2.png

# image3.png

# image4.png

# image5.png

# image1.png