agricultural research, and beyond (Gourlay et al. 2021; Zezza et al. 2022; Glazerman et al. 2023). In this regard, phone surveys, such as those we study, address two common criticisms of the usefulness of (in-person) survey data for health policy: their low temporal frequency and sparse coverage in conflict-affected or hard-to-access areas. Our study explores the reliability of phone survey data for vaccine research on COVID-19 and beyond.

The remainder of this paper proceeds as follows. Section 2 describes our data and Section 3 our empirical strategy. Section 4 presents our results. Section 5 discusses the implications of our results for health policy and research. Section 6 concludes.

# 2 Data

We use data from three sources: phone surveys, in-person surveys, and administrative records.

Phone surveys have become key tools to fill information gaps when in-person data collection came to a near complete halt during the COVID-19 pandemic (Wollburg, Contreras, et al. 2022). As a result, they have become widespread and enabled repeated experimentation at high frequency for the purpose of this analysis (Gourlay et al. 2021; Glazerman et al. 2020).

Specifically, we use data from longitudinal and cross-country comparable national phone surveys implemented between March 2021 and January 2023. These multi-topic phone surveys were conceived in order to track the effects of the COVID-19 pandemic in the absence of in-person data collection (Himelein et al. 2020). Our experimentation with survey design choices draws on five of these surveys in Sub-Saharan Africa that were supported by the Living Standards Measurement Study (LSMS) team at the World Bank and implemented by the respective National Statistical Offices. These surveys are re-contact surveys, drawing their samples from the latest nationally representative, in-person LSMS-ISA household survey conducted in each country before the pandemic. As part of the LSMS-ISA surveys, phone contact numbers of all household members (where available) as well as from a reference contact such as a neighbor were collected (Gourlay et al. 2021). The list of households with a phone contact, or a random subset of it, constituted the sample to be contacted for the phone surveys and covered between 73% (Malawi) and 99% (Nigeria) of households included in the in-person LSMS-ISA survey. This approach also led to response rates that compare favorably to other phone surveys, especially those employing random digit dialing (Dillon et al. 2021; Gourlay et al. 2021; Henderson and Rosenbaum 2020). It also allowed to draw on a rich set of household characteristics to attenuate coverage biases through reweighting techniques (Ambel et al. 2021; Himelein et al. 2020; Brubaker et al. 2021). The respondent for each phone survey interview was purposively selected as an adult (15+) household member that is knowledgeable of the affairs of the household, typically the household head.

The data we use in this study comes from a harmonized survey module on COVID-19 vaccination that was first fielded in August 2020 and then periodically repeated. The content of the survey module on vaccines varied over time in response to changing data demands as vaccination campaigns progressed. The focus of this paper is a question on whether the respondent had been vaccinated for COVID-19 which was included in the survey module after COVID-19 vaccines 5