Recent years have seen the resurgence of Yellow fever outbreaks in Africa and Latin America. 1 Regarding Africa specifically, five alerts have been issued for the first semester 2020 alone (Uganda, South Sudan, Ethiopia, Togo, Gabon). 2 As a response to the large-scale Angola 2015-2016 outbreak, the World Health Organization (WHO) launched the Eliminate Yellow fever Epidemics (EYE) initiative in 2017. 3 This strategy aims at preventing sporadic cases sparking urban outbreaks and potentially triggering international spread. It relies on various vaccination activities, including Preventive Mass Vaccination Campaigns (PMVCs) that target all or most age groups in a specific area. Evaluating the health impact of such campaigns is key to inform further PMVCs within or beyond the EYE strategy, to ensure population acceptance and adherence to vaccination campaigns, and to sustain domestic and international efforts for vaccination activities.
Previous attempts were made in order to estimate the impact of vaccination activities, including PMVCs. [4] [5] [6] These attempts mostly relied on mathematical models to estimate PMVCs impact in terms of deaths or cases prevented on the long term. However, few studies aimed at quantifying the effect of vaccination campaigns on the risk of outbreak. Regardless of the number of cases they may generate, outbreaks can possibly lead to healthcare, economic and social destabilizations of entire regions. As an example, the west-African 2013-2016 Ebola outbreak strained health systems and generated fear of the disease. This caused excess deaths due to neglected need for malaria control. 7, 8 When assessed at the population level, the association of vaccination activities and risk of outbreak can be approached within a classical epidemiological perspective. As individuals would be in a cohort study, populations (for instance populations living in well-defined geographical areas) may be followed over time while tracking both exposure (vaccination activities) and events (outbreaks). In such observational studies, a risk of confounding arises when both exposure and event share a same cause. This risk is high when measuring the association between PMVCs and yellow fever outbreaks because PMVCs usually target areas that are assessed at particularly high risk by public health officials, due to the disease circulation in the past or based on expert view or risk assessment. 9 Such a risk of confounding is usually overcome in the statistical analysis by conditioning, generally adjusting, on the shared common cause; in this case the baseline risk of yellow fever in the area. However, the environmental or demographic drivers of yellow fever are not fully understood, 10 leading to a situation in which residual confounders may bias the measure of association.
The self-controlled case series (SCCS) method is a case-only epidemiological study design for which individuals are used as their own control. 11 As all known and unknown time-invariant confounding are implicitly controlled for, the method is a relevant alternative to classical cohort or case-control study designs when the risk of residual confounding is high. The SCCS method has successfully been applied at the individual level, comparing exposure vs. non-exposure periods within individual cases. 12 However, to our knowledge, this method has never been used for population-level case series to evaluate the health effects of a public health intervention in specific regions, countries, or other predefined geographical clusters that may be considered as group-level cases.
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)
The copyright holder for this preprint this version posted July 11, 2020. . https://doi.org/10.1101/2020.07.09.20147355 doi: medRxiv preprint
Here, we illustrate the use of the SCCS method at the population level by assessing the association between the implementation of PMVCs and the occurrence of yellow fever outbreaks at the province level in the African endemic region between 2005 and 2018.
Considering the yellow fever vaccine's high level of efficacy, 13 and given the fact that PMVCs target all or most age groups in targeted areas, we expect to detect a substantial preventive effect of PMVC on the risk of outbreak. We also expect to detect this association in a cohort design, providing confounders in the association between exposure to PMVC and outbreak are adequately controlled for (no model misspecification). A SCCS model would avoid the risk of residual confounding, at least for time-independent variables, but would reduce statistical power as compared to a cohort-design analysis. 14 
We compiled location and dates of yellow fever reported in Africa between 2005 and 2018 from international epidemiological reports, namely the WHO Weekly Epidemiological Reports (WER) and the Diseases Outbreak News (DON). 15, 16 Locations were resolved at the first sub-national administrative level, thereafter called province, and data were recorded for each outbreak with the date of occurrence. Outbreak reports that could not be located at the province level were excluded.
We compiled data regarding PMVCs conducted as part of the Yellow Fever initiative since 2006, 17 and additional campaigns further conducted under the EYE strategy. 1 Starting dates and locations of PMVCs were collected, and the resulting list of vaccination campaigns was compared with data from the WHO International Coordinating Group (ICG) on Vaccine Provision, while resolving any discrepancy.
Estimates of population-level vaccine-induced protection against yellow fever were obtained from Hamlet et al. 18 These estimates were obtained by compiling regularly updated vaccination data from different sources (routine infant vaccination, reactive campaigns, PMVCs) and inputting these into a demographic model.
For our main analysis, we used the SCCS method to compare to estimate the incidence rate ratio (IRR) of yellow fever outbreak after vs. before the implementation of a PMVC. We used the province as unit of analysis, so that the main outcome represents the risk for a province to be affected by an outbreak. As the dependency between potential outbreak recurrences in the same province could not be excluded, we limited the analyses to the first outbreak occurrence per province for the main . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)
The copyright holder for this preprint this version posted July 11, 2020. . https://doi.org/10.1101/2020.07.09.20147355 doi: medRxiv preprint analysis. 19 We used a conditional Poisson model with logit link to model the occurrence of outbreaks. 14 Provinces included in the SCCS analysis were those both affected by an outbreak and targeted for a PMVC over a study period from 1 st Jan 2005 to 31 Dec 2018. We defined the unexposed period as the pre-PMVC period. Previous research found that a single dose of yellow fever vaccine provides a longlasting immunity with high efficacy. 13, 20 Therefore, and given the relatively short observation period of the study (14 years), we assumed the exposure period starting at the date of the first PMVC and lasting until the end of the observation period. This assumption was made regardless to estimated achieved coverage or intra-province geographic extent of the campaigns. In other terms, we assumed the campaigns to achieve uniform high coverage in all age groups across provinces.
In order to allow for possible variation in coverages achieved across PMVCs, we considered the estimated population-level vaccine coverage as an alternative time-varying, quantitative exposure (considered as categories with 20% bandwidth).
We also used alternative SCCS models to assess the influence of several assumptions on our results (Table 1) . 11 We conducted a SCCS analysis considering all outbreaks, instead of the first one only, in order to evaluate the influence of the assumption of non-independent recurrence. Additionally, as it is possible that the occurrence of an outbreak could affect subsequent exposure, we conducted a SCCS analysis including a 3-year pre-exposure period.
As the precise date of outbreaks and PMVCs start were not always available, we assumed where missing, that outbreaks started in the middle of the year and that exposure to PMVCs started at the end of the year. The influence of these assumptions was explored in sensitivity analysis.
To assess whether spatial autocorrelation could affect our results, we conducted multiple resampling. In each re-sampling from the SCCS study sample, we only sampled one random province per country and re-estimated the IRR of the association between exposure and the event. This implicitly accounts for spatial autocorrelation within, but not across countries.
We compared the results obtained using the SCCS method with those obtained using a classical cohort design. The study sample was constituted of all provinces belonging to the 34 African countries at high or moderate risk for yellow fever. 3 We used univariate and multivariate Poisson regression models with robust variance, considering exposure alternatively as a binary (pre-versus post-PMVC) or continuous (vaccination coverage) time-dependent variable.
In a cohort design, the choice of covariates to include is critical to prevent bias due to residual confounding. However, there is currently no clear consensus about the demographical and environmental drivers of yellow fever. We thus considered two (partially overlapping) sets of covariates (Supplementary Text S1). Both sets of variables were documented to reproduce well the . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted July 11, 2020. . https://doi.org/10.1101/2020.07.09.20147355 doi: medRxiv preprint presence and absence of yellow fever records at the province level. The first set of covariates was previously used in a statistical model whereas the second was used in a mechanistic model. 4, 10 Statistical models aims to describe the patterns of in the associations between species (including infectious agents species) and environmental variables while mechanistic models aim at explicitly representing biological processes in their occurrence. 21 The association between each covariate and the exposure status was explored using modified Poisson regression.
For each province , we estimated the expected number of outbreaks averted by PMVC, , using the formula:
Where is the total time of observation, is the time at which PMVC was implemented (if no PMVC in province , thus = ), is the rate of outbreak occurrence in a Poisson process in the absence of PMVC, and IRR is the incidence rate ratio after vs. before PMVC implementation. With being the number of outbreaks observed in the province during the pre-PMVC, an estimator of is = ⁄ , which leads to:
We obtained 95% confidence intervals for using bootstrap (10,000 resampling). For each resampling, a value of was randomly sampled based on the parameters estimated in the SCCS analysis.
Finally, based on and , the total number of outbreaks observed, we obtained the outbreaks prevented fraction, , with:
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted July 11, 2020. . https://doi.org/10.1101/2020.07.09.20147355 doi: medRxiv preprint 
Among the 479 provinces within the African endemic or at-risk region for yellow fever, 81 (16.9%) from 18 countries (of 34 countries) experienced at least one yellow fever outbreak between 2005 and 2018, including 12 provinces experiencing more than one outbreak (total number of outbreaks: 96) ; and 124 (25.9%) were targeted for at least one PMVC (Figure 1 ). The SCCS study sample was constituted from 33 (6.9%) provinces having experienced both outbreak and PMVC implementation over the study period. Temporal trends in the estimated population-level vaccination coverage for this sample are displayed in supplementary material (Supplementary Figure S1 ). The median of the difference between the post-and the pre-PMVC estimate of vaccination coverage was 24.2% (interquartile range: 9.4 -42.7%) (Supplementary Figure S2) . 
Among the SCCS study sample, the first outbreak occurred during the unexposed period in 26 (78.8%) provinces versus 7 (21.2%) occurring in the exposure period ( Figure 2 ). Under baseline assumptions, this corresponded to a significantly reduced incidence rate ratio of 0.14 (95% Confidence interval, CI: 0.06 -0.34) for the exposed versus unexposed periods. A similar protective . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted July 11, 2020. . https://doi.org/10.1101/2020.07.09.20147355 doi: medRxiv preprint association was observed when considering all outbreaks instead of the first one only (IRR = 0.19, 95% CI 0.09 -0.39) or when including a 3-year pre-exposure period (IRR = 0.14, 95% CI 0.05 -0.40). Considering estimates of population-level vaccine coverage as a categorical variable based on 20% allowed observing a reduced risk of outbreak for higher levels of coverage (Table 2 ). Considering vaccine coverage as a continuous linear exposure ensured a better fit of the model (likelihood ratio test: p =0.44). Doing so, we estimated that a 10%-increase in vaccine coverage decreased the risk of outbreak by 41% (IRR 0.59; 95% CI 0.46 -0.76).
The negative association between exposure to PMVCs and outbreak remained significant across a range of assumptions regarding the imputation of the date (within the same year) of PMVCs implementation and outbreak starting date (when missing) (Supplementary Table S1 ).
When re-sampling 100 times the SCCS study sample while allowing only one sampled province per country, and after excluding re-sampling yielding to random zero in the corresponding contingency table (N=16 with no outbreak occurring during exposed periods, thus leading to infinite confidence . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted July 11, 2020. . https://doi.org/10.1101/2020.07.09.20147355 doi: medRxiv preprint interval surrounding the association measure), we obtained an averaged IRR of 0.09 (95% CI 0.01-0.62). . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted July 11, 2020. . https://doi.org/10.1101/2020.07.09.20147355 doi: medRxiv preprint
In a cohort design, over the 81 outbreaks (first outbreaks only) that occurred over the study period, 74 occurred during unexposed periods versus 7 occurring in exposed periods. Most of the environmental covariates we explored were associated with exposure to PMVCs (Supplementary  Table S2 ). Exposure to PMVCs was associated to a significant reduced risk of outbreak (IRR = 0.37, 95% CI 0.15-0.92) when adjusting on the covariates obtained from a statistical model. When adjusting on covariates obtained from a mechanistic model, exposure to PMVC was not significantly associated with the risk of outbreak (IRR=0.65, 95% CI 0.26-1.65) (detailed results in Supplementary  Tables S3 and S4 ). For both sets of covariates, we observed an inversed U-shaped association between the estimates of vaccination coverage and the risk of outbreak, with the risk decreasing for lowest and highest values of vaccination coverage ( Table 2) .
Based on the value of IRR estimated in the main analysis (IRR = 0.14, 95% CI 0.06 -0.34), we estimated that PMVCs implemented over the study period averted in median 50 (28 to 80) outbreaks, corresponding to a prevented fraction of 34% (22% to 45%).
In this paper, using the SCCS method, we quantified the preventive effect of PMVCs on the risk of outbreak at the province level, documenting a 86% (66 to 94%) reduction of the risk of outbreak occurrence for provinces that were targeted by a PMVC. This result was robust over a range of assumptions. When using an estimate of population-level coverage as exposure, we also observed a dose-response preventive effect on the risk of outbreak. Considering the scale of PMVCs implementation during the study period, this corresponded to an estimated 22% to 45% of outbreaks averted by PMVCs in Africa between 2005 and 2018. Based on a cohort design analysis, the association between PMVC and outbreak was sensitive to the choice of adjustment variables. Moreover, we observed a challenging U-shape association between vaccination coverage and the risk of outbreak in the cohort analysis. Overall, these results suggest a risk of residual confounding that the SCCS method, but not cohort design, could overcome, at least for time-independent confounder. To our knowledge, this is the first time a SCCS analysis was conducted at the population level.
Considering evidence of yellow fever vaccine efficacy at the individual level, a preventive effect of PMVC on outbreak risk was indeed expected. This is why we think that the cohort analysis results may be biased by residual confounding ; whereas we consider the results obtained from the SCCS method to be more trustworthy. Indeed, the decision of targeting a province for PMVCs partly relies on a risk assessment. 3 For the results of the cohort-design analysis to be valid, one needs to account for all possible confounders in the association between PMVCs and outbreak. This is particularly challenging as the environmental and demographic drivers of yellow fever are not fully understood yet. 9 Another result suggesting residual confounding in the cohort-design analysis is the U-shaped . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted July 11, 2020. . https://doi.org/10.1101/2020.07.09.20147355 doi: medRxiv preprint relationship between vaccination coverage and outbreak risk. The yellow fever vaccine has not been introduced in large regions of Eastern Africa yet, as the risk of yellow fever is usually considered as low, though existing (eg. Kenya). This setting can actually yield to a spurious negative association between low level of vaccination coverage and outbreak risk when confounders are not controlled for. In the SCCS analysis, we did observe a linear relationship in the expected association between vaccination coverage and outbreak risk, which in another evidence for lower residual confounding. Previously, in a study setting at important risk for residual confounding, SCCS have been documented to outperform cohort or case-control designs. 22 Although the SCCS method has been originally developed to be conducted at the individual level, we ensured that our analysis complied with all its requirements. 11, 14 Exposure and outcomes were ascertained independently. The list of PMVCs was compiled based on information provided by international funders. Outbreak occurrence were compiled from WHO sources, which themselves compile outbreak notification from countries as per the 2005 International Health Regulation. The observation period was chosen in order to maximize the chance that cases experienced the exposure period. Indeed, our observation period started few time before the launch of the first Yellow Fever initiative, which boosted the use of PMVCs that have been very rare since the 1960s. 4, 17 The choice of the long and unlimited exposure period was based on evidence regarding the long-lasting protection conferred by the yellow fever vaccine, and the SCCS method has been previously used successfully while considering long and unlimited risk periods. 23 Under the assumption of causality, the incidence risk ratio we estimate represents the average effect for a province of being targeted by a PMVC, which corresponds to the average treatment effect in the counterfactual framework. This average effect is likely to mask large heterogeneity in the local effect of PMVC. Indeed, PMVCs occur in population with various baseline levels of immunity, and they may achieve various levels of post-intervention coverage. The dose-response relationship we observed in the association between vaccination coverage and outbreak risk brought additional evidence for a causal link between PMVC and reduced outbreak risk. When looking at higher values of vaccination coverage, it is notable that several outbreaks (n=4) occurred at estimated levels of vaccination coverage >80%, an empirical threshold that has been often suggested as protecting from outbreaks. 24 While keeping in mind all the limitations such province-based estimates of vaccination coverage may have (outbreaks could occur in small pockets with low vaccination coverage even in provinces with high coverage), this can be viewed an argument to ensure high vaccination coverages homogeneously in at-risk areas, and to sustain them after PMVCs by ensuring routine infant vaccination.
Relying on our estimate of the preventive effect of PMVC, the timing of implementation of these PMVCs and the number outbreaks observed during the study period, we further estimated that PMVCs have averted from 28 to 80 outbreak in Africa between 2005 and 2018, corresponding to a prevented fraction lying between 22% to 45%. Garske et al. previously estimated that vaccination campaigns conducted up to 2013 averted between 22 to 31% of yellow fever cases and deaths in Africa, while Shearer et al estimated that all vaccination activities (including routine infant vaccination) conducted up to 2016 have averted 33 to 39% of cases. 4, 5 Our estimates were in a comparable range, although direct comparison with these model-based estimates is not straightforward. Indeed the latter are expressed as proportions of all yellow fever cases, including sylvatic cases that are not linked to outbreaks. Preventing outbreaks of epidemic prone diseases is . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint this version posted July 11, 2020. . https://doi.org/10.1101/2020.07.09.20147355 doi: medRxiv preprint critical for ensuring global health security yet there are few empirical studies that quantify the impact of public health interventions like immunization have on the risk of outbreaks.
A main limitation of our study is that it does not account for possible time-varying confounders. Environmental changes affecting vector-borne diseases have been documented across tropical Africa over the study period, probably the main being changing land-use such as deforestation. 25, 26 Demographic changes, more frequent intrusions of humans into forest and jungles, increasing human mobility between endemic and non-endemic areas, have also been suggested to have affected the yellow fever risk in the recent period. 27 Similarly, recent international emphasis about yellow fever may have led to better surveillance of the disease in the recent years. However, these various factors are likely to have increased the risk of outbreaks and the probability of outbreak detection in the recent period, which overlaps with the post-PMVCs period in our study sample. This may have led to an underestimate of the association between PMVC and yellow fever outbreaks.
Previous quantification of the outstanding health impact of vaccination activities have mainly focused on cases or deaths prevented while relying on mathematical models, which structures and assumptions may be difficult to understand by a non-expert audience, whether it be decision-makers or targeted populations. 28, 29 Here we further document this impact using an empirical, maybe more intuitive approach thus allowing for a triangulation of methods to further document the beneficial impact of yellow fever vaccine campaigns. This method relies on data that are quite easily accessible. Thus, our method could be applied to other diseases for which PMVCs are implemented, such as polio, meningitis or cholera. Due to the COVID-19 pandemic, WHO recommended to temporarily suspend preventive campaigns while assessments of risk, and effective measures for reducing COVID virus transmission were established. In consequence, regarding yellow fever specifically, four countries postponed vaccination campaigns. 30 Our results provide additional evidence to encourage a rapid rescheduling of these vaccine campaigns in order to prevent further outbreaks of preventable disease.
Evidence before this study In 2013, a systematic review of the literature concluded that a single dose of yellow fever vaccine confers sustained life-long protective immunity (Gotuzzo et al). We searched Pubmed on June 7 th , 2020 using the search terms "yellow fever" [All Fields] AND "vaccin*" [All Fields] AND "campaign*" [All Fields] without language or date restriction. The search returned 127 results. Nine articles evaluated yellow fever mass vaccination campaigns based on secondary criteria, namely postcampaign coverage, adverse events following vaccination, operational campaign costs. Two modelling studies evaluated the short-term impact of reactive campaigns conducted during the 2015-2016 outbreak affecting Angola and the Demographic Republic of Congo. One study (Shearer et al.) estimated the global number of cases averted by the vaccination coverage levels achieved in 2016, thus encompassing all vaccination activities (routine childhood immunization vaccination, outbreak response campaign and preventive mass vaccination campaigns, PMVCs). Two studies relying on mathematical models focused specifically on the impact evaluation of PMVCs. Jean et al. estimated the long-term number of cases and death averted by hypothetic future PMVCs conducted according to various vaccination scenarios. Garske et al. estimated the impact of PMVCs conducted in Africa between 2006 and 2012. They estimated that PMVCs had reduced the number of yellow fever cases and deaths by 27% across the African at-risk zone, achieving up to an 82% reduction in countries targeted by these campaigns. To date, no study has quantified the effect of yellow fever preventive vaccination campaigns on the risk of outbreak occurrence. Moreover, to our knowledge, the evidence about the public health impact of yellow fever vaccination available to date stems from mathematical models, whereas few empirical evidence is available.
Added value of the study Based on dates and locations of both yellow fever reported outbreaks and PMVCs conducted in Africa between 2005 and 2018, we estimated the incidence rate of yellow fever outbreaks at the province level and compare the pre-and post-PMVCs incidence rate. Relying on the Self-Controlled Case Series (SCCS) method, which allows to use each case as its own control and thus eliminates all time invariant confounding, we estimated that PMVC reduces the risk of yellow fever outbreak by 86% (66% to 94%) at the province level. We further estimated that PMVCs achieved a 34% (22% to 45%) reduction in the number of outbreaks in Africa over the study period.
Implication of all the available evidence Beside evidence on efficacy and duration of immunity after yellow fever vaccination and modelbased estimates of yellow fever vaccine impact, our study provides new empirical evidence of the high impact of yellow fever vaccination campaigns in preventing outbreaks. Moreover, this study illustrates the potential of the SCCS method applied at the population level in order to evaluate public health interventions.
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)
The copyright holder for this preprint this version posted July 11, 2020. . https://doi.org/10.1101/2020.07.09.20147355 doi: medRxiv preprint