# Interaction of Vaccination and Reduction of Antibiotic Use Drives Unexpected Increase of Pneumococcal Meningitis

Matthieu Domenech de Cellès, Margarita Pons-Salort, Emmanuelle Varon, Marie-Anne Vibet, Caroline Ligier, Véronique Letort, Lulla Opatowski, Didier Guillemot

► To cite this version:

Matthieu Domenech de Cellès, Margarita Pons-Salort, Emmanuelle Varon, Marie-Anne Vibet, Caroline Ligier, et al. Interaction of Vaccination and Reduction of Antibiotic Use Drives Unexpected Increase of Pneumococcal Meningitis. Scientific Reports, 2015, 5, pp.11293. 10.1038/srep11293. hal-01223296

HAL Id: hal-01223296

https://hal.sorbonne-universite.fr/hal-01223296

Submitted on 2 Nov 2015

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
SCIENTIFIC REPORTS

OPEN

# Interaction of Vaccination and Reduction of Antibiotic Use Drives Unexpected Increase of Pneumococcal Meningitis

Received: 30 November 2014

Accepted: 11 May 2015

Published: 11 June 2015

Matthieu Domenech de Cellès<fnref n="1" />,²,³,⁴,*, Margarita Pons-Salort<fnref n="1" />,²,³,⁴,*, Emmanuelle Varon⁵,⁶, Marie-Anne Vibet<fnref n="1" />,²,³, Caroline Ligier<fnref n="1" />,²,⁴, Véronique Letort⁷, Lulla Opatowski<fnref n="1" />,²,⁴ & Didier Guillemot<fnref n="1" />,²,⁴,⁸

Antibiotic-use policies may affect pneumococcal conjugate-vaccine effectiveness. The reported increase of pneumococcal meningitis from 2001 to 2009 in France, where a national campaign to reduce antibiotic use was implemented in parallel to the introduction of the 7-valent conjugate vaccine, provides unique data to assess these effects. We constructed a mechanistic pneumococcal transmission model and used likelihood to assess the ability of competing hypotheses to explain that increase. We find that a model integrating a fitness cost of penicillin resistance successfully explains the overall and age-stratified pattern of serotype replacement. By simulating counterfactual scenarios of public health interventions in France, we propose that this fitness cost caused a gradual and pernicious interaction between the two interventions by increasing the spread of nonvaccine, penicillin-susceptible strains. More generally, our results indicate that reductions of antibiotic use may counteract the benefits of conjugate vaccines introduced into countries with low vaccine-serotype coverages and high-resistance frequencies. Our findings highlight the key role of antibiotic use in vaccine-induced serotype replacement and suggest the need for more integrated approaches to control pneumococcal infections.

*Streptococcus pneumoniae* (the pneumococcus) is a Gram-positive bacterium that frequently colonizes the human nasopharynx, but can also invade other body sites to cause a broad spectrum of infections<fnref n="1" />. The pneumococcus imposes a high public health burden, accounting for an estimated 11% of all deaths in children <5 years old worldwide². In the early 2000s, pneumococcal conjugate vaccines (PCV) were licensed for young children, and were shown protective against carriage and infection by a limited number of serotypes that caused a high proportion of invasive diseases in the United States³. Because of the vaccine's serotype specificity, however, serotype replacement following widespread vaccination caused early concern⁴. After the introduction of the 7-valent PCV (PCV7), two markedly different outcomes were observed. Several authors reported little serotype replacement and a sharp decline

<footnote n="1">¹Institut Pasteur, Unité de Pharmaco-Épidémiologie et Maladies Infectieuses, F-75015 Paris, France. ²INSERM, U1181, F-75015 Paris, France. ³Univ. Pierre et Marie Curie, Cellule Pasteur UPMC, F-75005 Paris, France. ⁴Univ. Versailles Saint Quentin, UFR des Sciences de la Santé Simone-Veil, EA 4499, F-78180 Montigny-le-Bretonneux, France. ⁵AP-HP, Hôpital Européen Georges-Pompidou, Laboratoire de Bactériologie, F-75015 Paris, France. ⁶Centre National de Référence des Pneumocoques, F-75015 Paris, France. ⁷École Centrale Paris, Laboratoire de Mathématiques Appliquées aux Systèmes, F-92290 Châtenay-Malabry, France. ⁸AP-HP, Hôpital Raymond-Poincaré, Unité Fonctionnelle de Santé Publique, F-92380 Garches, France. *These authors contributed equally to this work. Correspondence and requests for materials should be addressed to M.D.D.C. (email: matthied@umich.edu)</footnote>
of invasive pneumococcal diseases in children <2 years old and, to a lesser extent, in other age groups not directly targeted by the vaccine, as a consequence of herd immunity⁵⁻⁷. In other regions, however, considerable serotype replacement was observed⁸, and data even indicated small or no net benefits of vaccination—e.g., Spain⁹, and France¹⁰. Although several mechanisms have been proposed (e.g., secular trends in serotype distribution, changes in diagnostic practices), the causes of this variability of outcomes remain unclear¹¹. Antibiotic use has been suggested to affect serotype replacement¹²,¹³, but rigorous evaluation of this hypothesis is required.

The reported increase of pneumococcal meningitis (PM) in France, where a national campaign to reduce antibiotic use was implemented in parallel to PCV7 introduction, provides unique data to examine these effects. Here we combined these incidence data and national data on antibiotic use and vaccination to construct a mechanistic model of pneumococcal transmission. Using likelihood, we aimed to assess the ability of competing hypotheses to explain that unexpected increase.

## Results

**Unexpected dynamics of pneumococcal meningitis in France.** As shown in Fig. 1A, PCV7 was introduced in 2003 for children <2 years old, in whom the seven targeted vaccine-serotypes were responsible for 70% of invasive pneumococcal diseases in France at that time. PCV7, initially recommended for children at high risk of infection, was then universally recommended for children <2 years old in 2006; vaccine coverage gradually increased from 2003 and exceeded 80% by 2006. Concurrently, a national campaign to reduce antibiotic use was launched at the end of 2002, and achieved substantial prescription reduction over the ensuing 5 years¹⁴. Throughout this period, the French National Reference Center for Pneumococci (NRCP) collected PM-incidence data classified by serotype and antibiotic susceptibility. For simplicity, we grouped the serotypes according to inclusion in PCV7 (vaccine or nonvaccine serotype) and penicillin susceptibility (penicillin-susceptible with MIC ≤ 0.06 mg/L, or penicillin-resistant with MIC > 0.06 mg/L).

The data presented in Fig. 1B and 1C reveal shifts of the serotypes causing PM and demonstrate substantial serotype replacement following PCV7 introduction. Before 2003, most PM were caused by vaccine-serotype penicillin-resistant (VR) strains and nonvaccine-serotype penicillin-susceptible (NVS) strains. Thereafter, VR-PM gradually decreased following PCV7 introduction, concomitant with a dramatic NVS-PM increase and a less pronounced NVR-PM increase (mostly caused by serotype 19A, Supplementary Fig. S7). Overall, a ~70% decrease of vaccine-serotype PM was observed from 2001–2003 to 2008–2009, outweighed by a ~120% rise of nonvaccine-serotype PM; this pattern resulted in a ~20% PM increase during this period¹⁵. Comparable trends were observed in young children and old adults (Supplementary Fig. S1), suggesting a common serotype-replacement mechanism. During the study period, the serotypes 19F, 14, and 23 were the most common vaccine serotypes; the serotypes 19A, 3, and 7F, which have been identified as major non-PCV7 serotypes in other countries⁷,¹⁶⁻¹⁸, were the most common nonvaccine serotypes. Notably, there was little indication of substantial trends in these serotypes prior to the introduction of PCV7 (data not shown).

**Mechanistic model and inference framework.** To understand these paradoxical trends, we constructed a mechanistic, population-based model of pneumococcal transmission and PM infection (Fig. 2 and Methods). The model was structured according to carriage, vaccination and antibiotic-exposure statuses. Detailed data on vaccination and antibiotic exposure were obtained from the National Health Insurance (NHI) system and included as covariates in the model. We hypothesized that biological differences between pneumococcal serotypes might have caused the observed PM dynamics in France. Specifically, we evaluated support for two classes of hypotheses represented by two different models (Fig. 2). The Vaccine/NonVaccine (VNV) model explored transmissibility or invasiveness differences between vaccine and nonvaccine serotypes; the Susceptible/Resistant (SR) model addressed transmissibility or invasiveness differences between penicillin-susceptible and penicillin-resistant strains. Both models integrated additional hypotheses on between-strain competition (defined as the relative rate of acquisition in carriers vs. noncarriers, Fig. 2) and PCV7 efficacy on carriage (defined as the reduction in the acquisition risk of vaccine-serotypes in vaccinees¹⁹), reflected in two estimated parameters. We parameterized both models by defining transmissibility and invasiveness ratios (Fig. 2), which were estimated from the monthly incidence data (Fig. 1B) using a state-of-the-art likelihood-based, sequential Monte-Carlo method allowing for measurement error, non-stationarity, and inclusion of covariates²⁰. All model details are given in the Methods and in text S1 of the Supplementary Information.

**Fitness cost of penicillin resistance.** The estimation results were unequivocal: the SR model outperformed the VNV model by a large margin (ΔAIC = 22, Supplementary Table S5). The SR-model parameter estimates revealed lower transmissibility (β<sub>R</sub>/β<sub>S</sub> = 0.95, 99% CI: (0.95–0.96)) and lower invasiveness of penicillin-resistant strains (ρ<sub>R</sub>/ρ<sub>S</sub> = 0.78, 99% CI: (0.54–0.93))—that is, a fitness cost of penicillin resistance. The estimates also indicated moderate between-strain competition for carriage (relative risk of acquisition in carriers vs. noncarriers: θ = 0.82, 99% CI: (0.60–1)).

The SR model provided good agreement with data (Fig. 3). The associated R² values were 0.68 and 0.69 respectively, for the VR- and the NVS-PM series, which represented most of the PM occurring during the study period. As a result, the R² associated with the whole PM series was 0.68. The largest
<figure><img src="image_2.png" /><figcaption>**Figure 1.** Public health interventions and dynamics of pneumococcal meningitis (PM) in France. (A) Monthly number of antibiotic prescriptions (beta-lactams and macrolides) in France (black curve), and 7-valent pneumococcal conjugate-vaccine (PCV7) 2-dose coverage in children <2 years old (red curve). Antibiotic data are from the NHI system; vaccine coverage data are yearly estimates from a permanent 1/97 NHI sample (red squares, text S2 and Table S2) (B) Monthly and (C) annual PM cases (per 100,000 population) reported to the National Reference Center for Pneumococci. The biennial pattern apparent in C is caused by different notification rates between odd and even years; we corrected for this effect in our analyses (Supplementary Text S1 and Text S2).</figcaption></figure>

discrepancy between model and data occurred for NVR PM. Notably, trends for the NVR-PM series were largely dominated by penicillin-resistant 19A, which increased steadily throughout the study period. This increase caused the resistance frequency among nonvaccine serotypes to remain relatively constant, while that frequency decreased among vaccine serotypes. To better understand how the dynamics of serotype 19A may have affected our results, we also performed additional estimations with a subset of data, from which we removed serotype-19A PM. These estimations indicated independent dynamics for this serotype, possibly because fitness cost was absent (Supplementary Text S3).

In agreement with carriage studies in French children²¹,²², the SR model predicted extensive replacement in carriage, with a gradual decrease of vaccine serotypes from 2004 and a concomitant increase of nonvaccine serotypes, particularly associated with penicillin-susceptible strains (Supplementary Fig. S6). Notably, although the model did not include a seasonal transmission term, it predicted a small seasonality in carriage. This pattern was caused by seasonal variations in antibiotic use and the fitness cost of resistance, which led to an increased circulation of penicillin-susceptible strains and a lower resistance frequency during summer months²³.
<figure><img src="image_3.png" /></figure>

<table><thead><tr><th>Model</th><th>Transmission hypotheses</th><th>Invasiveness hypotheses</th><th>Estimated parameters</th></tr></thead><tbody><tr><td>VNV</td><td>β<sub>V</sub> = β<sub>V S</sub> = β<sub>V R</sub><br>β<sub>NV</sub> = β<sub>NV S</sub> = β<sub>NV R</sub></td><td>ρ<sub>V</sub> = ρ<sub>V S</sub> = ρ<sub>V R</sub><br>ρ<sub>NV</sub> = ρ<sub>NV S</sub> = ρ<sub>NV R</sub></td><td>β<sub>V</sub>, ρ<sub>V</sub><br>β<sub>NV</sub>/β<sub>V</sub>, ρ<sub>NV</sub>/ρ<sub>V</sub></td></tr><tr><td>SR</td><td>β<sub>S</sub> = β<sub>V S</sub> = β<sub>NV S</sub><br>β<sub>R</sub> = β<sub>V R</sub> = β<sub>NV R</sub></td><td>ρ<sub>S</sub> = ρ<sub>V S</sub> = ρ<sub>NV S</sub><br>ρ<sub>R</sub> = ρ<sub>V R</sub> = ρ<sub>NV R</sub></td><td>β<sub>S</sub>, ρ<sub>S</sub><br>β<sub>R</sub>/β<sub>S</sub>, ρ<sub>R</sub>/ρ<sub>S</sub></td></tr></tbody></table>

**Figure 2.** General model structure and derivation of the vaccine/nonvaccine (VNV) and susceptible/ resistant (SR) models. (Top) Simplified model diagram. The population was divided according to carriage status: S, susceptibles; C<sub>VS</sub>, carriers of a vaccine-serotype, penicillin susceptible (VS) strain; C<sub>VR</sub>, carriers of a vaccine-serotype, penicillin-resistant (VR) strain; C<sub>NVS</sub>, carriers of a nonvaccine-serotype, penicillin- susceptible (NVS) strain; C<sub>NVR</sub>, carriers of a nonvaccine-serotype, penicillin-resistant (NVR) strain. The per- susceptible rate of pneumococcal acquisition, λ<sub>X</sub>, is serotype-dependent. Serotype-specific transmission rates β<sub>X</sub> (where λ<sub>X</sub>=β<sub>X</sub>C<sub>X</sub>/N) are assumed constant over time. Carriers of a given serotype can acquire another serotype (super-acquisition) at a rate reduced by a factor 1 − θ compared with noncarriers. Carriers develop PM at a serotype-dependent rate ρ<sub>X</sub>, which is assumed to be seasonal. In the full model, the population is further stratified according to vaccination and antibiotic exposure status, so that this simplified diagram is repeated 4 times (individuals unexposed to antibiotics and unvaccinated, exposed and unvaccinated, unexposed and vaccinated, exposed and vaccinated). All model details are given in Supplementary Text S1. (Bottom) The table summarizes the simplifying transmission and invasiveness hypotheses that define the VNV and SR models. In the VNV model, strain's transmissibility and invasiveness differ between vaccine and nonvaccine serotypes; in the SR model, strain's transmissibility and invasiveness differ between susceptible and resistant serotypes.

To further assess the predictive power of the SR model, we separated the data into a training period (years 2001–2007) and a testing period (years 2008–2009) to perform out-of-fit (i.e., out-of-sample) predictions (Supplementary Fig. S10). This had little impact on parameter estimates (β<sub>R</sub>/β<sub>S</sub> = 0.95,
<figure><img src="image_4.png" /><figcaption>**Figure 3.** Model fit to data. For each strain, the observed numbers of PM (red curve), and the results of 10 stochastic runs of the SR model (thin grey lines) are shown. For completeness, the total number of PM (panel TOTAL) is also given; this series was not used for statistical inference. Note the different y-axis for each graph. VS: vaccine-serotype penicillin-susceptible; VR: vaccine-serotype, penicillin-resistant; NVS: nonvaccine-serotype penicillin-susceptible; NVR: nonvaccine-serotype penicillin-resistant.</figcaption></figure>

$ρ_R/ρ_S = 0.76$), and the SR model had high predictive power for the most prevalent types (vaccine-type, penicillin-resistant VR and nonvaccine-type, penicillin-susceptible NVS) and overall.

**Sensitivity analyses.** To assess the robustness of the SR model estimates, we conducted extensive sensitivity analyses. First, we varied the values of fixed model parameters and performed the estimations as before. For all the parameters tested, the estimates varied little (overall range of estimates: $β_R/β_S$ in 0.90–0.98 and $ρ_R/ρ_S$ in 0.64–0.90); crucially, the estimates always highlighted the existence of the fitness costs (Supplementary Table S6).

Second, because individual serotypes may differ in fitness, we tested how removing data from a single serotype affected our estimates (Supplementary Table S7). To do this, we identified in the French data
the 3 most prevalent vaccine serotypes (19F, 14, and 23F, by decreasing order) and nonvaccine serotypes (19A, 3, and 7F, by decreasing order). We then created 6 new data sets by removing the data for each of these 6 serotypes, and we performed the estimations as before (that is, as we did for serotype 19A, cf above). While the estimated fitness cost on invasiveness remained <1 after removing any of serotype 14, 19F, 23F, and 19A (overall range of estimates: 0.23–0.95), this was not the case after removing serotype 3 (range: 0.5–1.2) or serotype 7F (range: 0.75–1.5). By contrast, the estimated fitness cost on transmission remained remarkably robust after removing any of these 6 serotypes (overall range: 0.95–0.96). Thus, although the estimated fitness cost on invasiveness appeared less robust, our estimate of the fitness cost on transmission, which drives the dynamics of replacement in carriage, remained unaltered.

**Model extension to include age structure.** Because pneumococcal epidemiology varies by age, we considered an extended version of our model to include a simple age structure: [0, 3) y, [3, 16) y, and ≥16 y²⁴. To do this, we extracted age-specific antibiotic use data and used data from the POLYMOD study to fix age-specific contact rates²⁵; this extension also allowed us to consider age-specific durations of carriage²⁶. Although we did not attempt to perform parameter estimation on this extended model, we found that a fitness cost on transmission remarkably close to our original estimate (0.945 instead of 0.954) explained qualitatively well the age-stratified incidence data in France (Supplementary Fig. S9). In particular, this model was able to qualitatively reproduce the rebound in pneumococcal meningitis in children aged [0, 3) y, the fast decrease of V-type PM concomitant with the strong replacement by NV-type (mostly penicillin-susceptible) PM in all age groups, and the overall increase in all age groups.

**Counterfactual scenarios of public health interventions.** Using the simple, non age-structured model, we tried to assess the relative contributions of PCV7 and antibiotic use to PM incidence during 2001–2009 by simulating the SR model for alternative scenarios of public health interventions in France (Fig. 4A–C). In a single-intervention–simulated scenario with no PCV7 and reduced antibiotic use (Fig. 4B), penicillin-susceptible PM continued to increase by the end of 2002, concurrent with a steady diminution of penicillin-resistant PM. Because of the higher transmissibility and invasiveness of penicillin-susceptible strains, these variations resulted in an overall, though moderate, PM rise during 2001–2009.

The single-intervention–simulated scenario with PCV7 and no antibiotic reduction displayed more complex dynamics, with higher strain replacement (Fig. 4C). Here, PCV7 introduction promptly interrupted the VR-PM increase caused by unrestricted antibiotic use; a sharp decline was predicted after 2004, causing a transient, overall PM reduction. With increasing vaccine coverage, however, PCV7 progressively conferred a selective advantage to nonvaccine strains, particularly more transmissible and more invasive NVS strains. Although this effect was delayed (starting after 2005), it reversed the overall PM decrease and caused an increase after 2007. Thus, this scenario highlighted two opposing PCV7 effects: quick reduction of vaccine serotypes, followed by a delayed, but more pronounced expansion of nonvaccine serotypes, particularly NVS serotypes. Crucially, the latter effect occurred because of the high resistance frequency and elevated prevalence of nonvaccine serotypes at PCV7 introduction.

Comparing these two scenarios with the reconstructed PM dynamics under the reference scenario (Fig. 4A) provides insight into the mechanisms of strain replacement in France. Results from the single-intervention scenario with no PCV7 suggested that the reduced antibiotic use mainly accounted for the 2003–2004 dynamics, with a small NVS-PM increase concomitant with a small VR-PM decrease. Akin to the single-intervention scenario with PCV7, PCV7 introduction then engendered the prompt VR-PM decline and a progressive NVS-PM rise. Because of the higher NVS-PM prevalence at PCV7 introduction, the two effects canceled each other out until 2006, resulting in a relatively constant total PM incidence during the period of low PCV7 coverage. After 2006, the NVS-PM rise drove an overall PM increase, qualitatively similar to that predicted in the single-intervention scenario with PCV7. Quantitatively, however, the NVS increase with the combined interventions (+120% from 2001–2002 to 2008–2009, Fig. 4A) exceeded the sum of the predicted increases of the two single-intervention scenarios (+28%, Fig. 4B and +36%, Fig. 4C). These results suggest an unexpected interaction between PCV7 and antibiotic-use reduction in enhancing the spread of NVS pneumococci, which, we propose, eventually caused the observed PM increase from 2006 to 2010.

**Interaction of vaccination and antibiotic-use reduction.** Our scenario analysis suggests that, in addition to the synergistic interaction of the two interventions, the initially low vaccine-serotype coverage (that is, the proportion of PM caused by vaccine serotypes) and high resistance frequency were critical for high VR-strain replacement by NVS strains. Because those two quantities may vary widely among countries, we attempted to determine their impact on the expected benefits of vaccination with PCVs. To do so, we simulated a range of public health scenarios with PCV introduction and different reductions of antibiotic use (0 to 30%), while varying initial vaccine-serotype coverage and resistance frequency (Fig. 4D). In the scenarios with no antibiotic-use reduction, our model consistently predicted lower PM incidences, ranging from 0% to 30%, with reductions being higher for higher prevaccine vaccine-serotype coverage, regardless of the resistance frequency. In the scenarios with antibiotic-use reductions, however, the model predicted that low vaccine-serotype coverage and high antibiotic-resistance frequency could lead to a higher PM incidence, similar to the French experience. This impact was more pronounced for
A Reference scenario: PCV7 and antibiotic reduction

<figure><img src="image_5.png" /></figure>

B Antibiotic reduction and no PCV7

<figure><img src="image_6.png" /></figure>

C PCV7 and no antibiotic reduction

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

**Figure 4.** Simulated counterfactual scenarios of interventions in France and expected benefits of vaccination in various settings. (A–C) Model predictions according to intervention scenarios in France, 2001–2009: (A) Reference scenario with two interventions, (B) no PCV7 and antibiotic reduction, (C) PCV7 and no antibiotic reduction. The curves represent mean predicted annual cases (per 100,000 population) and the 95% prediction intervals for each type (green: VS, blue: VR, red: NVS, yellow: NVR) and for the total number of PM (black), obtained from 1,000 stochastic simulations of the best (SR) model with parameters fixed at their maximum likelihood values. In A–C, to avoid the biennial notification pattern, the simulated number of PM is represented before applying the observation model. In (A), the triangles represent the observed annual PM-incidence data from Fig. 1C, corrected for this biennial notification pattern (that is, divided by the reporting probability for each year). (D) Effects of vaccination and reduced antibiotic use. We simulated the effects of introducing a conjugate vaccine and reducing antibiotic use, while varying the initial vaccine-serotype coverage (proportion of PM caused by vaccine serotypes, x-axis) and the initial resistance frequency (y-axis). Here, we assume that, after a 2-year period without intervention, the two interventions are implemented simultaneously, and result in 100% vaccine coverage (all scenarios) and 0% (top left), 10% (top right), 20% (bottom left), or 30% (bottom right) reduction of antibiotic use. The colors represent the mean predicted variations of PM incidence (in %) 5 years after the implementation of the two interventions. Negative values correspond to decreases, while positive values correspond to increases.
greater antibiotic reductions, indicating increasing interaction between the two interventions, consistent with our interpretation of the data in France. These outcomes suggest that public health interventions aimed at lowering pneumococcal resistance might counteract the benefits of vaccination with PCVs.

## Discussion

The difficulty of interpreting PM data from France, because of the gradual vaccine introduction and the concomitant campaign to lower antibiotic use, has been reported before¹¹. Indeed, our findings indicate a complex association between the two interventions: the PCV7 effect gradually combined with the impact of antibiotic-use reduction to amplify NVS-pneumococcus replacement to finally cause the PM increase. That interpretation was only possible here because of our method, i.e., a mathematical model that integrated comprehensive data on vaccine and antibiotic use and that was confronted to detailed PM-incidence data using statistical inference. Previous studies highlighted a variety of mechanisms responsible for serotype replacement after vaccination, e.g., competition between vaccine and nonvaccine serotypes²⁷⁻²⁹, capsular switch³⁰, invasiveness of nonvaccine serotypes¹¹, or high pneumococcal antigenic diversity⁴,³¹. However, most of those studies focused on differences between vaccine and nonvaccine serotypes as the main forces driving serotype replacement postvaccination. Our findings also suggest the key roles of antibiotic use and competition between susceptible and resistant strains in modulating the effects of conjugate vaccines. While the heterogeneity of pneumococcal epidemiology worldwide precludes unequivocal recommendations, our findings have two potentially notable public health implications. First, under certain conditions (low vaccine-serotype coverage and high resistance frequency), initiating concomitantly vaccination and antibiotic-use reduction programs can lead to unexpected trends. Second, circulating serotypes and susceptibility to antibiotics, in addition to crude incidence estimates, should be carefully monitored after the rollout of vaccination.

Our results indicate lower transmissibility and lower invasiveness of penicillin-resistant pneumococci—that is, a fitness cost of resistance³². Determining the existence of this fitness cost, and, if so, its extent has been a central question in the epidemiology of pneumococcal resistance³³. Previous evidence for a lower transmissibility of resistant pneumococci came, first, from laboratory studies that demonstrated a lower growth rate of resistant pneumococci in competition experiments *in vitro*³⁴ or in mice³⁵; and, second, from epidemiological studies that documented a rapid decrease of resistance after a reduction of antibiotic use²³,³⁶. Our estimate of the epidemic fitness cost is also consistent with that of previous modeling studies that used statistical inference on infection³⁷ or carriage³⁸,³⁹ data. Regarding the impact of resistance on invasiveness, laboratory studies indicated a relationship between beta-lactam resistance and loss of virulence in mice⁴⁰,⁴¹. Such a relationship was also suggested by epidemiological data, which documented a lower frequency of resistance in invasive isolates vs. noninvasive isolates⁴²,⁴³.

For scenarios with no antibiotic-use reduction, our model consistently predicted reductions of PM incidences, ranging from 0% to 30%, with reductions being higher for higher prevaccine vaccine-serotype coverage, regardless of the resistance frequency. Those findings might qualitatively explain the trends reported in countries that introduced PCV7 but, to our knowledge, did not lower antibiotic use⁷,¹⁶⁻¹⁸. While a systematic comparison across countries is beyond the scope of this study, we note that our model makes a testable prediction, supported by some epidemiological studies²³,³⁶. All else being equal, countries with high antibiotic use should have higher resistance but, paradoxically, lower infection rates than countries with low antibiotic use. Despite likely difficulties (e.g., confounding factors, variations in reporting across countries), testing this prediction will undoubtedly prove useful to confirm our findings.

Our model ignores several complexities associated with pneumococcal epidemiology. First, our model was not age-structured, even though children <2 years old, targeted by PCV7, play a large part in pneumococcal transmission and are at higher risk of developing PM. Notably, PCV7 serotype coverage was initially higher in this population, with ~70% of PM caused by vaccine serotypes in 2001 (Supplementary Fig. S1), in contrast to the ~50% value in the general population that was used in our model. In that age class, the PCV7 effect was initially more pronounced but was followed by a rebound in 2005–2006 that cancelled the benefit of vaccination by 2008–2009 (Supplementary Fig. S1 and refs 10,44,45). This rebound, despite higher vaccine coverage in 2005–2006, was previously difficult to account for, but fits our general interpretation. Again, the PCV7 effect initially contained the predominant VR PM, but, combined with reduced antibiotic-use and the fitness costs of resistance, gradually favored strong replacement by NVS PM (+260% from 2001–2002 to 2008–2009 versus +120% in the total population). Simulations from the extended age-structured model confirmed this interpretation (Supplementary Fig. S9); in addition, carriage studies in French young children provided evidence of a similar pattern in carriage²²,⁴⁶, which was also well-reproduced by our model’s carriage predictions (Supplementary Text S3 and Fig. S6). Therefore, our model also provides, at least qualitatively, a plausible explanation of the reported trends in this age class. Second, previous studies showed that pneumococcal serotypes differ in fitness²⁶,⁴⁷. For simplicity, we did not consider such differences in our model, and we grouped serotypes according to inclusion in PCV7 and antibiotic susceptibility. We note, however, that comparable models have successfully been applied to study serotype replacement after vaccination²⁷,²⁸,⁴⁸⁻⁵⁰. Finally, we did not consider multiple carriage, which might be frequent for pneumococcus⁵¹,⁵². Rather, we allowed for neutral super-acquisition between serotypes in the model, so that any carrier could acquire another strain at any given time, as in other modeling studies⁵³,⁵⁴.
Public health interventions used to limit the spread of infectious diseases are often responsible for side effects, or externalities, which can either be deleterious or advantageous⁵⁵. Bacterial resistance is a well-known and extensively studied negative externality associated with antibiotic use, and previous studies highlighted the doubtless benefits of reducing antibiotic use to reduce the spread of resistant strains and to increase the fraction of treatable infections⁵⁶,⁵⁷. Our findings merely suggest that interventions aimed at lowering pneumococcus antibiotic-resistance might also engender negative side effects, linked to more infections caused by antibiotic-susceptible strains. Although further studies will be required to confirm our findings, we conclude that this possible pneumococcus-resistance dilemma could become a key public health question.

## Methods

**Pneumococcal meningitis data.** Monthly PM-incidence data were obtained from the French National Center for Pneumococci (NRCP). Briefly, NRCP is a passive surveillance system that collects pneumococcal infection and carriage data through the Regional Pneumococcus Observatories, a network of 400 laboratories covering all regions of France. Each PM isolate was analyzed to determine its penicillin susceptibility and serotype, using standard procedures⁵⁸. The data were then decomposed according to inclusion in the 7-valent pneumococcal conjugate vaccine (PCV7, vaccine serotypes, V, or nonvaccine serotypes, NV) and penicillin resistance (penicillin-susceptible strains, S, with MIC ≤ 0.06 mg/L and penicillin-resistant strains, R, with MIC > 0.06 mg/L). To assess the exhaustiveness of NRCP data, we compared NRCP numbers with the French Medicalization Program of Information System, a database that records all hospitalizations with a PM diagnosis (Supplementary Text S2). This comparison did not reveal any trend in the notification rate to NRCP; there were, however, systematic differences in notification rates between odd years (mean 65%, range 62–68%) and even years (mean 53%, range 49–58%). These differences were taken into account in the model (Supplementary Text S1).

**Antibiotic- and vaccine-use data.** In France, the National Health Insurance (NHI, general fund) refunds medical care provided by physicians in private practice, community clinics, and hospitals. Data on antibiotic prescriptions—beta-lactams and macrolides—were extracted from the NHI database and corresponded to ambulatory prescriptions reimbursed by the two main NIH agencies (CNAM–TS and RSI, covering 85% of the French population). Because antibiotics are prescription drugs in France, these data are nearly exhaustive. Vaccine coverage in children <2 years old was obtained from yearly estimates for a permanent 1/97 sample of the NHI general fund (Supplementary Text S2 and Table S2).

**Demographic data.** Demographic data in France, i.e., population size, birth rate and death rate, were available from the French National Institute of Statistics and Economic Studies (INSEE, http://www.insee.fr/en/; date of access: 01/03/2013).

**Model formulation and parameterization.** All models were formulated as discrete, stochastic systems and simulated using Gillespie's τ-leap algorithm with a time step of 1 day. The models were fitted to monthly PM-incidence data using the maximum iterated filtering algorithm, a likelihood-based, sequential Monte-Carlo method allowing for measurement error, non-stationarity, and inclusion of covariates²⁰. All analyses were performed with the pomp package (http://pomp.r-forge.r-project.org/), operating in the R environment. The profile likelihood was used to compute approximate 99% confidence intervals⁵⁹. All model details are given in the Supplementary Text S1.

## References

1. Bogaert, D., De Groot, R. & Hermans, P. W. *Streptococcus pneumoniae* colonisation: the key to pneumococcal disease. *Lancet Infect. Dis.* **4**, 144–154, doi:10.1016/S1473-3099(04)00938-7 (2004).

2. O'Brien, K. L. et al. Burden of disease caused by *Streptococcus pneumoniae* in children younger than 5 years: global estimates. *Lancet* **374**, 893–902, doi:10.1016/S0140-6736(09)61204-6 (2009).

3. Black, S. et al. Efficacy, safety and immunogenicity of heptavalent pneumococcal conjugate vaccine in children. Northern California Kaiser Permanente Vaccine Study Center Group. *Pediatr. Infect. Dis. J.* **19**, 187–195 (2000).

4. Lipsitch, M. Bacterial vaccines and serotype replacement: lessons from *Haemophilus influenzae* and prospects for *Streptococcus pneumoniae*. *Emerg. Infect. Dis.* **5**, 336–345, doi:10.3201/eid0503.990304 (1999).

5. Black, S. et al. Surveillance for invasive pneumococcal disease during 2000–2005 in a population of children who received 7-valent pneumococcal conjugate vaccine. *Pediatr. Infect. Dis. J.* **26**, 771–777, doi:10.1097/INF.0b013e318124a494 (2007).

6. Hicks, L. A. et al. Incidence of pneumococcal disease due to non-pneumococcal conjugate vaccine (PCV7) serotypes in the United States during the era of widespread PCV7 vaccination, 1998–2004. *J. Infect. Dis.* **196**, 1346–1354, doi:10.1086/521626 (2007).

7. Whitney, C. G. et al. Decline in invasive pneumococcal disease after the introduction of protein-polysaccharide conjugate vaccine. *N. Engl. J. Med.* **348**, 1737–1746, doi:10.1056/NEJMoa022823 (2003).

8. Singleton, R. J. et al. Invasive pneumococcal disease caused by nonvaccine serotypes among Alaska native children with high levels of 7-valent pneumococcal conjugate vaccine coverage. *JAMA* **297**, 1784–1792, doi:10.1001/jama.297.16.1784 (2007).

9. Ardanuy, C. et al. Epidemiology of invasive pneumococcal disease among adult patients in Barcelona before and after pediatric 7-valent pneumococcal conjugate vaccine introduction, 1997–2007. *Clin. Infect. Dis.* **48**, 57–64, doi:10.1086/594125 (2009).

10. Alexandre, C. et al. Rebound in the incidence of pneumococcal meningitis in northern France: effect of serotype replacement. *Acta Paediatr.* **99**, 1686–1690, doi:10.1111/j.1651-2227.2010.01914.x (2010).

11. Weinberger, D. M., Malley, R. & Lipsitch, M. Serotype replacement in disease after pneumococcal vaccination. *Lancet* **378**, 1962–1973, doi:10.1016/S0140-6736(10)62225-8 (2011).
12. Feikin, D. R. et al. Serotype-specific changes in invasive pneumococcal disease after pneumococcal conjugate vaccine introduction: a pooled analysis of multiple surveillance sites. *PLoS Med.* **10**, e1001517, doi:10.1371/journal.pmed.10001517 (2013).

13. Link-Gelles, R. et al. Geographic and Temporal Trends in Antimicrobial Nonsusceptibility in *Streptococcus pneumoniae* in the Post-vaccine era in the United States. *J. Infect. Dis.* **208**, 1266–1273, doi:10.1093/infdis/jit315 (2013).

14. Sabuncu, E. et al. Significant reduction of antibiotic use in the community after a nationwide campaign in France, 2002-2007. *PLoS Med.* **6**, e1000084, doi:10.1371/journal.pmed.1000084 (2009).

15. Chaussade, H., Varon, E., Vibet, M.-A., Watier, L. & Guillemot, D. in *51st Interscience Conference on Antimicrobial Agents and Chemotherapy* (Chicago, 2011). Abstract available at: http://tinyurl.com/n6ll9w6; date of access: 15/07/2013.

16. Foster, D. et al. Reduction in invasive pneumococcal disease following implementation of the conjugate vaccine in the Oxfordshire region, England. *J. Med. Microbiol.* **60**, 91–97, doi:10.1099/jmm.0.023135-0 (2011).

17. Harboe, Z. B. et al. Early effectiveness of heptavalent conjugate pneumococcal vaccination on invasive pneumococcal disease after the introduction in the Danish Childhood Immunization Programme. *Vaccine* **28**, 2642–2647, doi:10.1016/j.vaccine.2010.01.017 (2010).

18. Vestrheim, D. F. et al. Indirect effect of conjugate pneumococcal vaccination in a 2+1 dose schedule. *Vaccine* **28**, 2214–2221, doi:10.1016/j.vaccine.2009.12.054 (2010).

19. Rinta-Kokko, H., Dagan, R., Givon-Lavi, N. & Auranen, K. Estimation of vaccine efficacy against acquisition of pneumococcal carriage. *Vaccine* **27**, 3831–3837, doi:10.1016/j.vaccine.2009.04.009 (2009).

20. Ionides, E. L., Breto, C. & King, A. A. Inference for nonlinear dynamical systems. *Proc. Natl. Acad. Sci. USA* **103**, 18438–18443, doi:10.1073/pnas.0603181103 (2006).

21. Cohen, R. et al. Nasopharyngeal flora in children with acute otitis media before and after implementation of 7 valent pneumococcal conjugate vaccine in France. *BMC Infect. Dis.* **12**, 52, doi:10.1186/1471-2334-12-52 (2012).

22. Cohen, R. et al. Dynamic of pneumococcal nasopharyngeal carriage in children with acute otitis media following PCV7 introduction in France. *Vaccine* **28**, 6114–6121, doi:10.1016/j.vaccine.2009.05.037 (2010).

23. Dagan, R. et al. Seasonality of antibiotic-resistant *Streptococcus pneumoniae* that causes acute otitis media: a clue for an antibiotic-restriction policy? *J. Infect. Dis.* **197**, 1094–1102, doi:10.1086/528995 (2008).

24. Opatowski, L. et al. Antibiotic innovation may contribute to slowing the dissemination of multiresistant *Streptococcus pneumoniae*: the example of ketolides. *PLoS One* **3**, e2089, doi:10.1371/journal.pone.0002089 (2008).

25. Mossong, J. et al. Social contacts and mixing patterns relevant to the spread of infectious diseases. *PLoS Med.* **5**, e74, doi:10.1371/journal.pmed.0050074 (2008).

26. Hogberg, L. et al. Age- and serogroup-related differences in observed durations of nasopharyngeal carriage of penicillin-resistant pneumococci. *J. Clin. Microbiol.* **45**, 948–952, doi:10.1128/JCM.01913-06 (2007).

27. Choi, Y. H. et al. 7-Valent pneumococcal conjugate vaccination in England and Wales: is it still beneficial despite high levels of serotype replacement? *PLoS One* **6**, e26190, doi:10.1371/journal.pone.0026190 (2011).

28. Melegaro, A. et al. Dynamic models of pneumococcal carriage and the impact of the Heptavalent Pneumococcal Conjugate Vaccine on invasive pneumococcal disease. *BMC Infect. Dis.* **10**, 90, doi:10.1186/1471-2334-10-90 (2010).

29. Zhang, Y., Auranen, K. & Eichner, M. The influence of competition and vaccination on the coexistence of two pneumococcal serotypes. *Epidemiol. Infect.* **132**, 1073–1081 (2004).

30. Temime, L., Boelle, P. Y., Opatowski, L. & Guillemot, D. Impact of capsular switch on invasive pneumococcal disease incidence in a vaccinated population. *PLoS One* **3**, e3244, doi:10.1371/journal.pone.0003244 (2008).

31. Lipsitch, M. Vaccination against colonizing bacteria with multiple serotypes. *Proc. Natl. Acad. Sci. U. S. A.* **94**, 6571–6576 (1997).

32. Andersson, D. I. & Hughes, D. Antibiotic resistance and its cost: is it possible to reverse resistance? *Nat Rev Microbiol* **8**, 260–271, doi:10.1038/nrmicro2319 (2010).

33. Lipsitch, M. The rise and fall of antimicrobial resistance. *Trends Microbiol.* **9**, 438–444 (2001).

34. Rozen, D. E., McGee, L., Levin, B. R. & Klugman, K. P. Fitness costs of fluoroquinolone resistance in *Streptococcus pneumoniae*. *Antimicrob. Agents Chemother.* **51**, 412–416, doi:10.1128/AAC.01161-06 (2007).

35. Trzcinski, K., Thompson, C. M., Gilbey, A. M., Dowson, C. G. & Lipsitch, M. Incremental increase in fitness cost with increased beta-lactam resistance in pneumococci evaluated by competition in an infant rat nasal colonization model. *J. Infect. Dis.* **193**, 1296–1303, doi:10.1086/501367 (2006).

36. Guillemot, D. et al. Reduction of antibiotic use in the community reduces the rate of colonization with penicillin G-nonsusceptible *Streptococcus pneumoniae*. *Clin. Infect. Dis.* **41**, 930–938, doi:10.1086/432721 (2005).

37. Opatowski, L. et al. Assessing pneumococcal meningitis association with viral respiratory infections and antibiotics: insights from statistical and mathematical models. *Proc. Biol. Sci.* **280**, 20130519, doi:10.1098/rspb.2013.0519 (2013).

38. Domenech de Celles, M. et al. Intrinsic epidemicity of *Streptococcus pneumoniae* depends on strain serotype and antibiotic susceptibility pattern. *Antimicrob. Agents Chemother.* **55**, 5255–5261, doi:10.1128/AAC.00249-11 (2011).

39. Maher, M. C. et al. The fitness cost of antibiotic resistance in *Streptococcus pneumoniae*: insight from the field. *PLoS One* **7**, e29407, doi:10.1371/journal.pone.0029407 (2012).

40. Azoulay-Dupuis, E. et al. Relationship between capsular type, penicillin susceptibility, and virulence of human *Streptococcus pneumoniae* isolates in mice. *Antimicrob. Agents Chemother.* **44**, 1575–1577 (2000).

41. Rieux, V., Carbon, C. & Azoulay-Dupuis, E. Complex relationship between acquisition of beta-lactam resistance and loss of virulence in *Streptococcus pneumoniae*. *J. Infect. Dis.* **184**, 66–72, doi:10.1086/320992 (2001).

42. Bedos, J. P., Chevret, S., Chastang, C., Geslin, P. & Regnier, B. Epidemiological features of and risk factors for infection by *Streptococcus pneumoniae* strains with diminished susceptibility to penicillin: findings of a French survey. *Clin. Infect. Dis.* **22**, 63–72 (1996).

43. Ho, P. L. et al. Emergence of fluoroquinolone resistance among multiply resistant strains of *Streptococcus pneumoniae* in Hong Kong. *Antimicrob. Agents Chemother.* **43**, 1310–1313 (1999).

44. Lepoutre, A., Varon, E., Georges, S., Gutmann, L. & Levy-Bruhl, D. Impact of infant pneumococcal vaccination on invasive pneumococcal diseases in France, 2001-2006. *Euro Surveill.* **13**, 367–372 (2008).

45. Levy, C. et al. Pneumococcal meningitis in French children before and after the introduction of pneumococcal conjugate vaccine. *Pediatr. Infect. Dis. J.* **30**, 168–170 (2011).

46. Cohen, R. et al. Impact of pneumococcal conjugate vaccine and of reduction of antibiotic use on nasopharyngeal carriage of nonsusceptible pneumococci in children with acute otitis media. *Pediatr. Infect. Dis. J.* **25**, 1001–1007, doi:10.1097/01.inf.0000243163.85163.a8 (2006).

47. Lipsitch, M. et al. Estimating rates of carriage acquisition and clearance and competitive ability for pneumococcal serotypes in Kenya with a Markov transition model. *Epidemiology* **23**, 510–519, doi:10.1097/EDE.0b013e31824f2f32 (2012).

48. Sutton, K. L., Banks, H. T. & Castillo-Chavez, C. Estimation of invasive pneumococcal disease dynamics parameters and the impact of conjugate vaccination in Australia. *Math Biosci Eng* **5**, 175–204 (2008).

49. Temime, L., Boelle, P. Y., Valleron, A. J. & Guillemot, D. Penicillin-resistant pneumococcal meningitis: high antibiotic exposure impedes new vaccine protection. *Epidemiol. Infect.* **133**, 493–501 (2005).
50. Temime, L., Guillemot, D. & Boelle, P. Y. Short- and long-term effects of pneumococcal conjugate vaccination of children on penicillin resistance. *Antimicrob. Agents Chemother.* **48**, 2206-2213, doi:10.1128/AAC.48.6.2206-2213.2004 (2004).

51. Auranen, K., Mehtala, J., Tanskanen, A. & M, S. K. Between-strain competition in acquisition and clearance of pneumococcal carriage--epidemiologic evidence from a longitudinal study of day-care children. *Am. J. Epidemiol.* **171**, 169–176, doi:10.1093/aje/kwp351 (2010).

52. Brugger, S. D., Frey, P., Aebi, S., Hinds, J. & Muhlemann, K. Multiple colonization with S. pneumoniae before and after introduction of the seven-valent conjugated pneumococcal polysaccharide vaccine. *PLoS One* **5**, e11638, doi:10.1371/journal. pone.0011638 (2010).

53. Granat, S. M. et al. Epidemiological evidence for serotype-independent acquired immunity to pneumococcal carriage. *J. Infect. Dis.* **200**, 99–106, doi:10.1086/599364 (2009).

54. Melegaro, A., Choi, Y., Pebody, R. & Gay, N. Pneumococcal carriage in United Kingdom families: estimating serotype-specific transmission parameters from longitudinal data. *Am. J. Epidemiol.* **166**, 228–235, doi:10.1093/aje/kwm076 (2007).

55. Althouse, B. M., Bergstrom, T. C. & Bergstrom, C. T. Evolution in health and medicine Sackler colloquium: a public choice framework for controlling transmissible and evolving diseases. *Proc. Natl. Acad. Sci. U. S. A.* **107 Suppl 1**, 1696–1701, doi:10.1073/ pnas.0906078107 (2010).

56. Albrich, W. C., Monnet, D. L. & Harbarth, S. Antibiotic selection pressure and resistance in *Streptococcus pneumoniae* and *Streptococcus pyogenes*. *Emerg. Infect. Dis.* **10**, 514–517, doi:10.3201/eid1003.030252 (2004).

57. Goossens, H., Ferech, M., Vander Stichele, R., Elseviers, M. & Group, E. P. Outpatient antibiotic use in Europe and association with resistance: a cross-national database study. *Lancet* **365**, 579–587, doi:10.1016/S0140-6736(05)17907-0 (2005).

58. Varon, E., Janoir, C. & Gutmann, L. Rapport d'activité 2010. (Centre National de Référence des Pneumocoques, Paris, 2010).

59. Raue, A. et al. Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood. *Bioinformatics* **25**, 1923–1929, doi:10.1093/bioinformatics/btp358 (2009).

## Acknowledgments

We would like to thank Nicolas Joly for computational support, Laurence Watier for helpful discussions, Anne Thiébaut and Lénaig Le Fouler for extracting antibiotic data, and Janet Jacobson for editorial assistance. We are grateful to Pejman Rohani for valuable comments and guidance in preparing the manuscript. This work was supported directly by internal resources of the French National Institute for Health and Medical Research, the Institut Pasteur, the University of Paris 11, the University of Versailles Saint Quentin and the Assistance Publique-Hôpitaux de Paris. This study has received funding from the French Government's Investissement d'Avenir program, Laboratoire d'Excellence "Integrative Biology of Emerging Infectious Diseases" (grant n°ANR-10-LABX-62-IBEID).

## Author Contributions

M.D.D.C. and M.P.S. conceived and implemented the model, performed the analysis, and drafted the manuscript. E.V. oversaw data collection and provided the incidence data. M.A.V. and C.L. extracted antibiotic data for use in the model. V.L. contributed content expertise and tools for modeling. L.O. and D.G. conceived the study design, oversaw the analysis, and helped draft the manuscript.

## Additional Information

**Supplementary information** accompanies this paper at http://www.nature.com/srep

**Competing financial interests:** Emmanuelle Varon received support for travel to meetings from Wyeth/Pfizer and Bayer, and was a speaker for Pfizer. Other authors involved in this study declare having no competing interests.

**How to cite this article:** Domenech de Cellès, M. *et al*. Interaction of Vaccination and Reduction of Antibiotic Use Drives Unexpected Increase of Pneumococcal Meningitis. *Sci. Rep.* **5**, 11293; doi: 10.1038/srep11293 (2015).

<figure><img src="image_9.png" /></figure>

This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third party material in this article are included in the article's Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/