Note: Bivariate OLS regression of vaccination status on a dummy for different respondent selection options. Panel 1 tests the effect of randomly selecting the respondent (vs. purposive selection). Panel 2 tests the effect of eliciting the vaccination status of all household members aged 15+ via proxy reporting (vs. purposive selection). Panel 3 tests the effect of randomly selecting the respondent (vs. proxy reporting for all members aged 15+). Vaccination rate estimated using survey weights and as reported in administrative data reported at bottom. Burkina Faso - Survey dates: August 30 - September 25; Administrative data: October 2. Malawi - Survey dates: July 26 - September 7; Administrative data: September 4. Uganda - Survey dates: August 5 - September 1; Administrative data: September 4. Administrative data for adult coverage assumes no children were vaccinated at the time of data reporting. This is assumption likely overstates the true vaccination rate among adults where countries started vaccinating children already. This is the case for children under 16 years of age in Malawi (starting Dec 4, 2021) and Uganda (starting Jul 26, 2021) Standard errors in parentheses. *** p<0.01, ** p<0.05, * p<0.1 Further, there is a statistically significant difference between the estimated vaccination rate based on random respondent selection and based on proxy reports for all household members across all countries (Panel C), which goes against our prediction that both approaches should lead to statistically indistinguishable estimates.

There are several possible explanations for this difference, relating both to errors of representation and errors of measurement. These include errors in proxy reporting (see section 4.2) and enumerator demand (see section 4.3).

After weighting our random respondent estimates using the re-calibrated phone survey weights (see Section 3), the gap between survey data and administrative records remains substantial in all cases (11 percentage points or 46% of administrative coverage in Burkina Faso, 19.5 percentage points or 76% in Malawi, and 8.9 percentage points or 13% in Uganda; Table 4).<sup>17</sup> Estimates are closest to the survey data when using (proxy-reported) data for all household members. However, differences remain statistically significant in all countries and non-negligible in Burkina Faso (5.7 percentage points or 42% of the administrative coverage rate) and Malawi (5.4 percentage points or 36%).

# **4.3 Measurement errors**

# **Survey mode**

We argue that if phone survey mode effects were driving the misalignment between administrative records and survey estimates of vaccine uptake, there should be no misalignment between administrative records and _in-person_ survey estimates. We test this by comparing the administrative data to the sample of Ethiopia phone survey respondents who were also interviewed in person in the Ethiopia Socioeconomic Survey (ESS). We find a significant discrepancy of 12.6 percentage points (or 38%) between administrative records (33.4% vaccine coverage)<sup>18</sup> and inperson survey estimates (46.0% coverage) in this sample (Table 5). This difference remains substantial at 10.4 percentage points (or 52%) and statistically significant when estimating vaccine coverage based on the full ESS sample, which is representative of the general population. Survey

> 17 At the time of data collection, Malawi and Uganda had already been vaccinating children under the age of 16 for over 8 and over 13 months, respectively. This means that our assumption of zero vaccinated children under the age of 15 that we have to make for comparability between survey data estimates for the adult population (15+) and administrative data is likely too conservative and understates the true gap in this case.

> 18 Conservatively assuming that at the time of reporting no vaccinated children were included in the administrative data.

21