# 1. Introduction

Investments in large-scale vaccination efforts have led to drastic reductions in child mortality over the last decades (WHO 2020a). Much of this investment has targeted low- and middle-income countries, which will receive USD 1.2 billion in support between 2021 and 2025 from Gavi, a multi-donor initiative aiming to cut the share of children without any routine immunization in half by 2030 (Gavi, The Vaccine Alliance 2022b; 2021). Achieving further reductions in the share of unvaccinated children requires reliable data on vaccine coverage, which children remain unvaccinated, where and why (Galles et al. 2021; Danovaro-Holliday et al. 2021; Scobie et al. 2020; WHO 2020a; Cutts et al. 2016). A substantial share of funding is thus directed toward reliable and fit-for-purpose data systems (Gavi, The Vaccine Alliance 2022c; WHO 2020a).

A strong emphasis on data quality is a core principle of the WHO’s Immunization Agenda 2030 and reflects the doubtful reliability of vaccine coverage estimates to date (WHO 2020a; Galles et al. 2021; Cutts et al. 2016). Specifically, it is not uncommon to see estimates of vaccine coverage differ by double digits between administrative and survey-based sources (Galles et al. 2021; Sandefur and Glassman 2015; Miles et al. 2013; Burton et al. 2009; Dykstra et al. 2019). In 2017, less than half of Gavi-supported countries reported survey estimates of vaccine coverage that were within 10 percentage points of administrative figures (Gavi, The Vaccine Alliance 2022c). Such misalignment is taken by donors as the core metric by which progress in data quality is evaluated (Gavi, The Vaccine Alliance 2022c) and is the subject of this study.

Previous research on this topic has focused on routine immunization and in-person data collection, comparing different data sources such as self-reports, home-based records, and health facility or administrative data (Dansereau et al. 2020; Sandefur and Glassman 2015; Miles et al. 2013; Lim et al. 2008). In these studies, the scope of experimentation to explore different design choices and their effect on the reliability of vaccine uptake estimates has been small. Differences in design choices can have a direct impact on the quality of data collected and affect the policy conclusions and investment recommendations drawn (De Weerdt et al. 2020a). This makes their assessment an important empirical question to investigate (Dillon et al. 2020).

In this study, we document that substantial misalignment between survey and administrative data is also pervasive in COVID-19 vaccine coverage figures. In a sample of 36 LMICs, survey-based estimates suggest COVID-19 vaccine coverage that is on average 47% higher than what is documented in administrative sources. This pattern displays distinct regional variation that is particularly striking and consistent in Sub-Saharan Africa.

We then systematically investigate the sources of this misalignment in Sub-Saharan Africa. Our empirical strategy exploits the unprecedented availability of high-frequency administrative data and concurrently collected (phone) survey data on COVID-19 vaccinations, which allows us to directly compare vaccine coverage estimates from these two sources at multiple points in time and in multiple countries. We conduct a series of (randomized) survey experiments using cross-country comparable, longitudinal (phone) surveys across five LMICs in Sub-Saharan Africa. Each experiment exogenously varies one aspect of the survey design that we hypothesize could give rise to the observed discrepancy between survey and administrative data-based estimates of the 2