COVID-19 vaccination rate. We broadly classify these design choices into questions of measurement and representation (Groves 1989; Groves and Lyberg 2010). Together, the design choices we study span some of the most common sources of error covered under the total (survey) error framework that has been the guiding paradigm for research on data quality (Amaya et al. 2020; Groves and Lyberg 2010; Groves 1989).

We find that the substantial misalignment between survey and administrative data-based estimates of vaccine coverage is only partly explained by measurement and representation issues in survey data. After accounting for observed errors of representation and measurement at the household and individual level, there remains a statistically significant gap of 6%-52% between survey and administrative estimates in our study countries. Our findings indicate that design choices determine the reliability and suitability of the survey data for capturing vaccine coverage. At the same time, our explorative analysis suggests administrative data also suffers from flaws and inaccuracies. Without giving due consideration to both survey design and potential inaccuracies in administrative records, both data sources can lead to different findings and different policy conclusions. Our findings have implications for research and policy making on COVID-19 vaccination but also for future data collection on routine immunization and supplementary immunization activities.

Our study relates and contributes to several strands of literature. Conceptually, we frame our analysis as part of the literature on total survey error and total survey quality (Amaya et al. 2020; Groves and Lyberg 2010; Biemer 2010) as well as studies analyzing the role of design choices in producing credible research and value for development (De Weerdt et al. 2020a; Jolliffe et al. 2023). This research emphasizes the sensitivity of both survey and administrative data to nonclassical sources of measurement error. In this framing, conducting robust research is akin to an optimization problem in which the researcher tries to maximize credible knowledge subject to budget and data quality constraints (Dillon et al. 2020). Improvements in data quality provide opportunities to improve the quality of inference conducted. In line with Dillon et al. (2020), we argue that research on vaccination has paid insufficient attention to the sensitivity of data quality to design choices. Our study addresses this gap.

Secondly, for vaccination data specifically, we contribute to a body of research documenting and analyzing misalignment between survey-based and administrative sources (Wolter et al. 2022; Nguyen et al. 2021; Galles et al. 2021; Bradley et al. 2021; Sandefur and Glassman 2015; Miles et al. 2013; Murray et al. 2003; Lim et al. 2008; Burton et al. 2009). This literature stresses that both data sources are subject to potential errors. The strength of administrative data is its spatial granularity and frequency, but it often suffers from numerator (number of people vaccinated) and denominator (size of the target population) issues.<sup>1</sup> Survey data, for example from large household survey programs such as the Demographic and Health Surveys (DHS), Multiple Indicators Cluster Survey (MICS), or Living Standards Measurement Study (LSMS) is independent of public record keeping and can also capture vaccination obtained through private or non-governmental providers

> 1 For example, numerator issues may occur when administrative data misses out on vaccinations conducted through the private health care sector and or when aggregating figures from lower administrative levels. Denominator issues may relate to outdated census data or inaccurate population projections (Burton et al. 2009).

3