# Quantitative Benefit-Risk Models Used for Rotavirus Vaccination: A Systematic Review

Hugo Arlegui, G. Nachbaur, N. Praet, Bernard Begaud

► To cite this version:

Hugo Arlegui, G. Nachbaur, N. Praet, Bernard Begaud. Quantitative Benefit-Risk Models Used for Rotavirus Vaccination: A Systematic Review. Open Forum Infectious Diseases, 2020, 7 (4), pp.ofaa087. 10.1093/ofid/ofaa087. hal-03167404

HAL Id: hal-03167404

https://hal.science/hal-03167404

Submitted on 12 Mar 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Quantitative Benefit-Risk Models Used for Rotavirus Vaccination: A Systematic Review

Hugo Arlegui,¹,² Gaëlle Nachbaur,² Nicolas Praet,³ and Bernard Bégaud¹

¹INSERM, Bordeaux Population Health Research Center, Team Pharmacoepidemiology, UMR 1219, University of Bordeaux, Bordeaux, France, ²Pharmaco-Epidemiology and Health Outcomes Research, GSK, Rueil-Malmaison, France, ³Clinical Research and Development, GSK, Wavre, Belgium

**Background.** Although rotavirus vaccines have proven to prevent the risk of rotavirus gastroenteritis (RVGE) in children under 5 years old, they are also associated with an increased transient risk of intussusception (IS). Several quantitative benefit-risk models (qBRm) are performed to measure this balance in hospitalizations and deaths prevented versus the ones induced.

**Method.** In this study, our objective was to provide a complete overview of qBRm used for rotavirus vaccination. We systematically searched 3 medical literature databases to identify relevant articles, in English, that were published between 2006 and 2019.

**Results.** Of the 276 publications screened, 14 studies using qBRm for rotavirus vaccination were retained, based on preselected criteria. Four were performed in low- and middle-income countries. Almost all (13 of 14) displayed the following characteristics: force of infection assumed to be constant over time (static model), indirect effect of rotavirus vaccination (herd effect) not considered, closed model (individuals not allowed to enter and/or exit the model over time), and aggregated level (no tracking of individual's behavior). Most of the models were probabilistic (9 of 14) and reported sensitivity and/or scenario analyses (12 of 14). Input parameter values varied across studies. Selected studies suggest that, depending on the models used, for every IS hospitalization and death induced, vaccination would prevent, respectively, 190-1624 and 71-743 RVGE-related hospitalizations and deaths.

**Conclusions.** The benefits of rotavirus vaccination were shown to largely exceed the increased risk of IS, across all studies. Future research aiming to harmonize qBRm for rotavirus vaccination should ensure the comparability of studies and provide additional information for regulatory authorities, physicians, and patients.

**Keywords.** benefit-risk; intussusception; rotavirus; vaccines and immunization; systematic review.

Infection with rotaviruses is the most common cause of severe diarrhea and dehydration in young children. Although spread worldwide, rotavirus infection induces a higher burden in low-income countries [1]. These highly contagious viruses virtually infect all children before they reach the age of 5 [2]. Rotavirus was responsible for an estimated 258 million (95% confidence interval [CI], 193-341 million) episodes of gastroenteritis and 128 000 (95% CI, 104 500-155 600) deaths in children under the age of 5 in 2016 [3].

Historically, 9 months after an oral rhesus-human reassortant rotavirus tetravalent vaccine (RotaShield; Wyeth) was licensed in the United States in October 1998, the immunization program was suspended because of a temporal association between rotavirus vaccination and occurrence of intussusception

(IS) [4]. The estimated relative risk (RR) of IS during the 3-7 days after RotaShield administration was 58.9 (95% CI, 31.7-109.6) postdose 1 and 11.0 (95% CI, 4.1-29.5) postdose 2 [5]. Intussusception is a rare but serious medical condition observed when a segment of the intestine invaginates into an adjacent distal segment [6, 7] resulting in blood vessel compression and leading to pain, bowel oedema, and—if arterial supply is compromised—intestinal ischemia, necrosis, and even perforation. If left untreated, IS can be fatal. Although rare, IS is the most common cause of acute intestinal obstruction in infants, occurring usually between 4 and 10 months of age [1]. Intussusception occurs without rotavirus vaccination with an average worldwide background incidence rate estimated at 74 cases per 100 000 children under 1 year of age, and it was shown to range between 9 and 328 per 100 000 across countries [7]. Surgical rates of IS are substantially higher in Africa (77%) and Central and South America (86%) compared with other regions (13%–50%) [7, 8].

Since 2006, 2 live-attenuated rotavirus vaccines have been licensed in more than 100 countries [9]: Rotarix (GlaxoSmithKline Biologicals), a 2-dose schedule oral human rotavirus vaccine, and RotaTeq (Merck & Co., Inc.), a 3-dose schedule oral human-bovine reassortant rotavirus vaccine [10–13]. The 2 established vaccines have proven to be effective, and

<footnote>Received 19 August 2019; editorial decision 5 March 2020; accepted 10 March 2020.
Correspondence: Hugo Arlegui, PharmD, MSc, PhD, GSK, 23 Rue François Jacob, 92500 Rueil-Malmaison, France (hugo.arlegui@gmail.com).</footnote>

<footnote>**Open Forum Infectious Diseases®**
© The Author(s) 2020. Published by Oxford University Press on behalf of Infectious Diseases Society of America. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited. DOI: 10.1093/ofid/ofaa087</footnote>
they have led to a significant decline in rotavirus gastroenteritis (RVGE)-related morbidity and mortality [10, 11]. Two new rotavirus vaccines have received prequalification from the World Health Organization (WHO) in 2018: ROTAVAC (Bharat Biotech International Limited) and ROTASIIL (Serum Institute of India Limited) [14]. Moreover, other rotavirus vaccines have been licensed for national markets in China (Lanzhou Lamb rotavirus vaccine; Lanzhou Institute of Biological Products) and in Vietnam (Rotavin-M1 rotavirus vaccine; Center for Research and Production of Vaccines) [15]. In 2009, the WHO recommended rotavirus vaccination to all children, especially in countries with high diarrhea-related mortality rates [1]. By the end of 2018, 92 countries had introduced rotavirus vaccination into their routine immunization program for children [16].

Several observational postlicense surveillance studies have been undertaken to assess the risk of IS after vaccination with Rotarix and RotaTeq in real-life settings [8, 17–24]. Data from epidemiological studies suggest that between 1 and 6 cases of IS per 100 000 vaccinated children may be attributable to rotavirus vaccination [25]. A meta-analysis has reported an overall estimate of RR of IS postdose 1 of 5.4 (95% CI, 3.9–7.4) and 5.5 (95% CI, 3.3–9.3) and postdose 2 of 1.8 (95% CI, 1.3–2.5) and 1.7 (95% CI, 1.1–2.6), after vaccination with Rotarix and RotaTeq, respectively [26]. These overall estimates were further confirmed by 2 recent meta-analyses [22, 27].

Given the increased risk of IS associated with rotavirus immunization, it is crucial to balance it with the benefits of vaccination in reducing RVGE-related hospitalizations and deaths [28, 29]. In this context, several studies have been conducted in various geographical settings to investigate the benefit-risk (BR) profile of rotavirus vaccination. These studies using quantitative BR models (qBRm) provided key information for regulatory authorities, physicians, and parents [30–33].

The aim of the present systematic literature review was as follows: (1) to provide a comprehensive overview of published qBRm focusing on rotavirus vaccination and their methodological approaches and (2) to characterize the BR profile of rotavirus vaccination on the basis of available scientific evidence.

# METHODS

## Search Strategy

We systematically searched Medline, Scopus, and the Institute for Scientific Information (ISI) Web of Knowledge databases to identify original studies on qBRm for rotavirus vaccination published from January 1, 2006 to December 13, 2019. The search strategy used prespecified terms (“benefit-risk” and “rotavirus vaccines”), as detailed in Appendix Table 1, and was limited to publications in English.

Two reviewers (H.A. and N.P.) independently screened all titles and abstracts using predefined criteria (Appendix Table 2). Subsequently, the assessment for eligibility of

identified publications was carried out by examining their full text. Disagreements between the 2 reviewers were resolved through discussion. In addition, reference lists of eligible articles were screened (ie, “snowballing”) to identify potential additional publications. Finally, a gray literature search of public health organization websites and Google was performed using the prespecified search terms. All citations were downloaded and imported in EndNote (version X7; Thomson Reuters Corp., New York, NY).

## Data Extraction and Analysis

The following data were extracted and summarized: the qBRm general information, the model characteristics, the input parameters, and the BR estimates. General information includes the studied vaccine(s), the alternative(s) to the studied vaccine(s), the vaccine funding sources, and the income category of the countries for which the BR was estimated. The model characteristics were classified according to the 8 attributes [34–36] as follows:

1. Simulation versus Nonsimulation model: the BR estimates were either derived from modeling approach using simulation techniques of various degrees of complexity (eg, cohort or microsimulation models) including as many components and interaction as possible (simulation) or from a simple computation, mathematical function, or statistical model (nonsimulation).

2. Dynamic versus Static model: the force of infection was assumed to change over time (dynamic) or not (static).

3. Model considering Herd effect or Not: a potential herd effect of rotavirus vaccination was considered (yes) or not (no).

4. Model considering Waning effect or Not: a potential waning effect (ie, vaccine efficacy/effectiveness decrease with time) was considered (yes) or not (no).

5. Open versus Closed model: an open model allows individuals to enter and exit the model over time (open), whereas a closed model does not allow for new entrances over time (closed).

6. Probabilistic versus Deterministic model: the model takes into account the uncertainty around the input parameters (probabilistic) or not (deterministic).

7. Model integrating Aggregate versus Individual-based data: The population's behavior in the model is simulated using population's averages (aggregate data) or considering each individual's attributes (individual-based data).

8. Model including Scenario/Sensitivity analyses or Not: scenario (using analyses to investigate different epidemiological or healthcare scenarios of interest) and/or sensitivity (using analyses to quantify the range of uncertainty) analyses were conducted or not. Sensitivity analyses were categorized between deterministic (using point estimates) and probabilistic (using probability distributions).
Input parameters used to perform qBRm along with benefit, risk, and BR ratio (BRR) after rotavirus vaccination were extracted from analyzed studies. The benefit of rotavirus vaccination was reported as the annual number or proportion of RVGE-related hospitalizations or deaths prevented by vaccination in children before 5 years of age. The risk of rotavirus vaccination was reported as the annual number or proportion of IS-related hospitalizations or deaths attributed to vaccination in children under 1 year of age. The BRR after rotavirus vaccination was expressed as the ratio of the annual number of RVGE-related hospitalizations or deaths prevented (benefit) and the annual number of IS-related hospitalizations or deaths attributed to vaccination (risk).

# RESULTS

## Study Selection

After removing duplicates, the search strategy yielded 276 unique records, from which 248 were excluded based on titles and/or abstracts that were not relevant to the present analysis. The full-text review of the 28 selected articles led to the consensual exclusion of 14 of them by both reviewers, leaving 14 publications for data extraction and analysis (Figure 1).

## General Information and Model Characteristics of Selected Studies

Quantitative BR models used for rotavirus vaccination were published from 2009 onwards (Table 1) [17, 18, 37–48]. Among the 14 selected studies, 8 investigated Rotarix (6 of 14) [18, 37,

41, 44] or RotaTeq (2 of 14) [38], whereas 5 assessed both rotavirus vaccines [17, 39, 40, 42, 43]. The last study [47] investigated all currently licensed vaccines (Rotarix, RotaTeq, ROTAVAC, ROTASIIL, and RV3-BB) and were assumed to be equivalent in terms of vaccine efficacy, effectiveness, or impact, or IS risks. All studies focused on rotavirus vaccines administered according to the national or WHO recommended vaccination schedule. Five of the 14 studies also considered rotavirus vaccination without age restriction [37, 42, 43, 46, 47], and 1 study [45] considered a targeted strategy with selective rotavirus vaccination of infants with medical risk conditions (prematurity, low birth weight, or congenital conditions). Only 2 studies were reported as funded by a pharmaceutical company (GlaxoSmithKline) [41, 48], whereas the others were classified as “other sources of funding” (such as academic institutions or health authorities). Nine studies were performed in high-income countries (HICs) [17, 37, 38, 40, 41, 44–46, 48], and 5 were performed in low- and/or middle-income countries (LMICs) [18, 39, 42, 43, 47]. Studies in HICs were country-specific and mainly used local data, whereas those in LMICs were conducted across several countries: in 2, 14, 117, 135, and 158 LMICs, respectively. In the studies that included 117, 135, and 158 LMICs, a generic model using data provided by geographic area (not country-specific) was used to calculate the different estimates [42, 43, 47].

Studies included in the review used simulation (9 of 14) or nonsimulation models (5 of 14) to estimate final BR outcomes. All simulation models used a cohort model as modeling

<figure><img src="image_2.png" /><figcaption>**Figure 1.** PRISMA flow diagram. ISI, Institute for Scientific Information.</figcaption></figure>
<table><caption>Table 1. General Information and Model Characteristics of Studies Using Quantitative Benefit-Risk Models for Rotavirus Vaccination</caption><thead><tr><th rowspan="2">Source</th><th colspan="2">General Information</th><th colspan="10">Model Characteristics</th></tr><tr><th>Vaccines(s)</th><th>Alternative(s)</th><th>Funding</th><th>Income Category</th><th>Simulation Model</th><th>Static/ Dynamic Model</th><th>Herd Effect</th><th>Waning Effect</th><th>Closed / Open Model</th><th>Deterministic/ Probabilistic Model</th><th>Aggregate/ Individual- Based Model</th><th>Scenario/ Sensitivity Analyses</th></tr></thead><tbody><tr><td>Patel et al [42]</td><td>Rotarix/RotaTeq</td><td>No vaccination<br/>No age restriction to vaccination</td><td>Other</td><td>LMICs</td><td>No</td><td>Static</td><td>No</td><td>No</td><td>Closed</td><td>Deterministic</td><td>Aggregate</td><td>Scenario</td></tr><tr><td>Patel et al [18]</td><td>Rotarix</td><td>No vaccination</td><td>Other</td><td>LMICs</td><td>No</td><td>Static</td><td>No</td><td>No</td><td>Closed</td><td>Deterministic</td><td>Aggregate</td><td>No</td></tr><tr><td>Desai et al [39]</td><td>Rotarix/RotaTeq</td><td>No vaccination</td><td>Other</td><td>LMICs</td><td>Yes</td><td>Static</td><td>No</td><td>No</td><td>Closed</td><td>Probabilistic</td><td>Aggregate</td><td>PSA</td></tr><tr><td>Patel et al [43]</td><td>Rotarix/RotaTeq</td><td>No vaccination<br/>No age restriction to vaccination</td><td>Other</td><td>LMICs</td><td>Yes</td><td>Static</td><td>No</td><td>No</td><td>Closed</td><td>Probabilistic</td><td>Aggregate</td><td>Scenario<br/>PSA</td></tr><tr><td>Carlin et al [17]</td><td>Rotarix/RotaTeq</td><td>No vaccination</td><td>Other</td><td>HIC</td><td>No</td><td>Static</td><td>No</td><td>No</td><td>Closed</td><td>Deterministic</td><td>Aggregate</td><td>No</td></tr><tr><td>Desai et al [38]</td><td>RotaTeq</td><td>No vaccination</td><td>Other</td><td>HIC</td><td>Yes</td><td>Static</td><td>No</td><td>No</td><td>Closed</td><td>Probabilistic</td><td>Aggregate</td><td>DSA/PSA</td></tr><tr><td>Clark et al [37]</td><td>Rotarix</td><td>No vaccination<br/>No age restriction to vaccination</td><td>Other</td><td>HIC</td><td>Yes</td><td>Static</td><td>No</td><td>Yes</td><td>Closed</td><td>Probabilistic</td><td>Aggregate</td><td>Scenario<br/>PSA</td></tr><tr><td>Yung et al [44]</td><td>Rotarix</td><td>No vaccination</td><td>Other</td><td>HIC</td><td>No</td><td>Static</td><td>No</td><td>No</td><td>Closed</td><td>Deterministic</td><td>Aggregate</td><td>Scenario</td></tr><tr><td>Ledent et al [41]</td><td>Rotarix</td><td>No vaccination</td><td>Pharma ceutical</td><td>HIC</td><td>Yes</td><td>Static</td><td>No</td><td>No</td><td>Closed</td><td>Probabilistic</td><td>Aggregate</td><td>DSA/PSA</td></tr><tr><td>Lamrani et al [40]</td><td>Rotarix/RotaTeq</td><td>No vaccination</td><td>Other</td><td>HIC</td><td>Yes</td><td>Static</td><td>No</td><td>Yes</td><td>Closed</td><td>Probabilistic</td><td>Aggregate</td><td>Scenario<br/>PSA</td></tr><tr><td>Ledent et al [48]</td><td>Rotarix</td><td>No vaccination</td><td>Pharma ceutical</td><td>HIC</td><td>Yes</td><td>Static</td><td>No</td><td>No</td><td>Closed</td><td>Probabilistic</td><td>Aggregate</td><td>DSA/PSA<br/>Scenario</td></tr><tr><td>Bruijing/Verhagen et al [45]</td><td>RotaTeq</td><td>No vaccination<br/>Targeted vaccination</td><td>Other</td><td>HIC</td><td>Yes</td><td>Static</td><td>Yes</td><td>Yes</td><td>Closed</td><td>Probabilistic</td><td>Aggregate</td><td>DSA/PSA<br/>Scenario</td></tr><tr><td>Bruun et al [46]</td><td>Rotarix</td><td>No vaccination<br/>No age restriction to vaccination</td><td>Other</td><td>HIC</td><td>No</td><td>Static</td><td>No</td><td>Yes</td><td>Closed</td><td>Probabilistic</td><td>Aggregate</td><td>Scenario</td></tr><tr><td>Clark et al [47]</td><td>Rotarix/RotaTeq/ RV3-BB/ROTA/AC/ ROTASIII</td><td>No vaccination<br/>No age restriction to vaccination</td><td>Other</td><td>LMICs</td><td>Yes</td><td>Static</td><td>No</td><td>Yes</td><td>Closed</td><td>Probabilistic</td><td>Aggregate</td><td>PSA<br/>Scenario</td></tr></tbody></table>

Abbreviations: DSA, deterministic sensitivity analyses; HIC, high-income country; LMICs, low- and middle-income countries; PSA, probabilistic sensitivity analyses.
<table><caption>Table 2. Input Parameters of Quantitative Benefit-Risk Models Used for Rotavirus Vaccination</caption><thead><tr><th>Source</th><th>Location</th><th>Vaccine(s)</th><th>Vaccine Efficacy/Effectiveness</th><th>Vaccine Coverage</th><th>IS Risk Period (Days): Relative Risk</th><th>Birth Cohort</th></tr></thead><tbody><tr><td>Patel et al [42]</td><td>LMIC (117)</td><td>Rotarix/RotaTeq</td><td>D1: 50%<br>D2 and D3: 75%</td><td>54%</td><td>D1 (1-7): 6.0<br>D2 (1-7): 3.0<br>D3 (1-7): 1.0</td><td>NR</td></tr><tr><td>Patel et al [18]</td><td>Brazil<br>Mexico</td><td>Rotarix<br>Rotarix</td><td>D1 and D2: 85%<br>D1 and D2: 85%</td><td>50%<br>50%</td><td>D1 (1-7): 1.1 [0.3-3.3]<br>D2 (1-7): 2.6 [1.3-5.2]<br>D1 (8-14): 1.3 [0.5-3.4]<br>D2 (8-14): 1.4 [0.7-3.0]<br>D1 (15-21): 0.2 [0.0-1.4]<br>D2 (15-21): 0.9 [0.4-2.0]</td><td>Brazil: 5.3 [3.0-9.3]<br>Mexico: 1.8 [0.9-3.8]<br>1.1 [0.5-2.7]<br>2.2 [1.1-4.2]<br>0.9 [0.3-2.2]<br>2.2 [1.2-4.0]</td><td>3 068 249<br>2 414 329</td></tr><tr><td>Desai et al [39]</td><td>Latin America (14)</td><td>Rotarix/RotaTeq</td><td>Hosp: 66% [31-83] to 85% [72-93]<br>Death: 80% [59-90] to 100% [74-100]</td><td>54%-92%</td><td>D1 (1-7): 5.3 [3.0-9.3]<br>D2 (1-7): 2.6 [1.3-5.2]</td><td>9 588 000</td></tr><tr><td>Patel et al [43]</td><td>LMIC (158)</td><td>Rotarix/RotaTeq</td><td>61% [44-73] to 97% [84-100]</td><td>Country-specific</td><td>D1 (1-7): 5.5 [4.1-7.5]<br>D2 (1-7): 1.7 [1.2-2.4]</td><td>123 600 000</td></tr><tr><td>Carlin et al [17]</td><td>Australia</td><td>Rotarix/RotaTeq</td><td>D1: 50%<br>D2 and D3: 80%</td><td>85%</td><td>D1 (1-7): 6.8 [2.4-19.0]<br>D1 (8-21): 3.5 [1.3-8.9]<br>D2 (1-7): 2.8 [1.1-7.3]</td><td>RotaTeq: 9.9 [3.7-26.4]<br>6.3 [2.8-14.4]<br>2.8 [1.2-6.8]</td><td>290 446</td></tr><tr><td>Desai et al [38]</td><td>United States</td><td>RotaTeq</td><td>D1: 66% [16-86]<br>D2: 90% [75-96]<br>D3: 92% [86-96]</td><td>D1: 96%<br>D2: 93%<br>D3: 82%</td><td>D1 (1-7): 5.3 [3.0-9.3]</td><td>4 261 494</td><td></td></tr><tr><td>Clark et al [37]</td><td>England</td><td>Rotarix</td><td>D1 >6 m: 96% [90.2-98.8]<br>D1 >12 m: 90.7% [85.6-94.3]<br>D2 >4 m: 100% [81.8-100]<br>D2 >10 m: 92.2% [65.6-99.1]</td><td>D1 >15 w:<br>96%<br>D2 >24 w:<br>94%</td><td>D1 (1-7): 6.8 [2.4-19.0]<br>D1 (8-21): 3.5 [1.3-8.9]<br>D2 (1-7): 2.8 [1.1-7.3]<br>D2 (8-21): 2.1 [1.0-4.6]</td><td>656 457</td><td></td></tr><tr><td>Yung et al [44]</td><td>Singapore</td><td>Rotarix</td><td>D1: 50%<br>D2: 80%</td><td>90%</td><td>D1 (1-7): 8.4 [2.4-29.0]<br>D2 (1-7): 3.1 [0.4-23.4]<br>D2 (8-21): 1.5 [0.2-11.7]</td><td>40 000</td><td></td></tr><tr><td>Ledent et al [41]</td><td>Japan</td><td>Rotarix</td><td>D1: 73.9% [50.1-83.7]<br>D2: 91.6% [62.4-99.1]</td><td>D1: 100%<br>D2: 98% [95-98]</td><td>D1 (1-7): 5.4 [3.9-7.4]<br>D2 (1-7): 1.8 [1.2-2.7]</td><td>1 018 400</td><td></td></tr><tr><td>Lamrani et al [40]</td><td>France</td><td>Rotarix/RotaTeq</td><td>Rotarix<br>D1 >6 m: 96% [90.2-98.8]<br>D1 >12 m: 90.7% [85.6-94.3]<br>D2 >4 m: 100% [81.8-100]<br>D2 >10 m: 92.2% [65.6-99.1]</td><td>RotaTeq<br>D1 <12 m: 58.9% [51.7-65.0]<br>D1 >24 m: 53.6% [46.4-59.7]<br>D2 <12 m: 77.4% [71.1-82.1]<br>D2 >24 m: 72.1% [65.8-76.8]<br>D3 <12 m: 95.8% [90.5-98.2]<br>D3 >24 m: 88.0% [82.7-90.4]</td><td>D1: 92%<br>D2: 88%<br>D3: 84%</td><td>D1 (1-7): 6.8 [2.4-19.0]<br>D1 (8-21): 3.5 [1.3-8.9]<br>D2 (1-7): 2.8 [1.1-7.3]<br>D2 (8-21): 2.1 [1.0-4.6]</td><td>9.9 [3.7-26.4]<br>6.3 [2.8-14.4]<br>2.8 [1.2-6.8]<br>1.8 [0.8-3.9]</td><td>765 550</td></tr><tr><td>Ledent et al [48]</td><td>France</td><td>Rotarix</td><td>D1: 75% [55-88]<br>D2: 90% [81-95]</td><td>D1: 100%<br>D2: 92% [72-100]</td><td>D1 (1-7): 5.4 [3.9-7.4]<br>D2 (1-7): 1.8 [1.3-2.5]</td><td>D1 (1-7): 5.4 [3.9-7.4]<br>D2 (1-7): 1.8 [1.3-2.5]</td><td>791 183</td></tr><tr><td>Bruijing-Verhagen et al [45]</td><td>Netherlands</td><td>RotaTeq</td><td>D1 and D2: 88%<br>D3: 94.8%</td><td>86%</td><td>NR</td><td></td><td>171 387</td></tr></tbody></table>
<table><caption>**Table 2. Continued**</caption><thead><tr><th>Source</th><th>Location</th><th>Vaccine(s)</th><th>Vaccine Efficacy/Effectiveness</th><th>Vaccine Coverage</th><th>IS Risk Period (Days): Relative Risk</th><th>Birth Cohort</th></tr></thead><tbody><tr><td>Bruun et al [46]</td><td>Norway</td><td>Rotarix</td><td>D1 and D2: 93% [87–98]</td><td>D1: 91%<br>D2: 86%</td><td>D1 (1–21): 2.4 [1.5–3.8]<br>D2 (1–21): 1.8 [1.3–2.4]</td><td>60 000</td></tr><tr><td>Clark et al [47]</td><td>LMIC (135)</td><td>Rotarix/RotaTeq/<br>ROTAVAC/ROTASIII/<br>RV3-BB</td><td>79% [75–82] to 100% [99–100]</td><td>Country-specific</td><td>D1 (1–7): 6.3 [4.3–9.2]<br>D1 (8–21): 1.7 [1.1–2.7]<br>D2 (1–7): 1.8 [1.4–2.3]<br>D2 (8–21): 1.4 [1.0–1.8]</td><td>60 000 000</td></tr></tbody></table>

Abbreviations: D1, dose 1; D2, dose 2; D3, dose 3; Hosp, hospitalization; IS, intussusception; LMICs, low- and middle-income countries; m, months; N, number; NR, not reported; w, weeks.

approach, ie, simulated a hypothetical cohort of individuals through a set of health states over time. A few studies (5 of 14) considered a waning effect over time after rotavirus vaccination. Three attributes were identical across all studies, ie, all models were static, closed, and reported results at an aggregate/ population average level. Only 1 study took herd effect into account. Models were probabilistic in 9 studies and deterministic in the remaining ones. Most studies reported results from additional analyses: scenario analyses (9 of 14), probabilistic sensitivity analyses (PSA) (9 of 14), and deterministic sensitivity analyses (4 of 14). General information for each study and a description of the different models are summarized in Table 1.

## Summary of Input Parameters

Almost all (13 of 14) analyses included the following input parameters: vaccine efficacy or effectiveness (VE), vaccine coverage (VC), RR of IS after vaccination during a given risk period (Table 2), and the baseline incidence of hospitalizations or deaths (related to RVGE or IS) in children under 5 years (for RVGE) or 1 year of age (for IS) in the absence of vaccination (Table 3). One study considered baseline incidence for RVGE under <15 years [45]. Vaccine effectiveness varied according to the number of doses administered (1 to 3), the age of immunization (eg, VE >6 months and >12 months after vaccination), the vaccine used (mainly Rotarix or RotaTeq), and the health outcome of interest (hospitalization or death). Vaccine coverage considered for a full vaccination schedule was low in LMICs (approximately 50%) and high in HICs (approximately 90%). All studies considered a 7-day risk period for the risk of IS after vaccination, whereas half of them also investigated an additional risk period of up to 21 days [17, 18, 37, 40, 44, 46, 47]. The RR of IS ranged between 1.1 (95% CI, 0.3–3.3) (Brazil) and 9.9 (95% CI, 3.7–26.4) (Australia and France) after the first dose with a 7-day risk period. The RR of IS ranged from 1.7 (95% CI, 1.2–2.4) (158 LMICs assessed in the study by Patel et al [43]) to 3.1 (95% CI, 0.4–23.4) (Singapore) after the second dose with a 7-day risk period. Only 1 study analyzed a RR of IS after the third dose with a risk period of 7 days [42]. Details on RRs used according to the different risk periods are available in Table 2. The baseline incidence of hospitalizations or deaths (number and rate) for RVGE and IS in the absence of vaccination were country or area specific and were higher in LMICs for deaths (Table 3 and Appendix Table 3).

## Benefit-Risk Estimates of Rotavirus Vaccination

Based on the 14 selected publications, vaccination would prevent 59.0% (Australia) to 89.9% (England) of RVGE-related hospitalizations and 32.1% (mean percentage in 135 LMICs assessed by Clark et al [47]) to 87.5% (Lamrani et al [40] in France) of RVGE deaths expected to occur in a no-vaccination scenario in children under 5 years of age. On the other hand, the IS-related hospitalization rate and the IS-related death rate
<table><caption>**Table 3.** Benefit-Risk Estimates of Rotavirus Vaccination in Analyzed Studies</caption><thead><tr><th>Source</th><th>Location</th><th>Vaccine(s)</th><th>Events</th><th>Baseline Incidence RVGE &lt;5 y (N)</th><th>Prevented RVGE &lt;5 y (N)</th><th>Prevented RVGE &lt;5 y (%)</th><th>Baseline Incidence IS &lt;1 y (N)</th><th>Caused IS &lt;1 y (N)</th><th>Caused IS &lt;1 y (%)</th><th>BRR (RVGE/IS)</th></tr></thead><tbody><tr><td>Patel et al [42]</td><td>LMIC (117)</td><td>Rotarix/RotaTeq</td><td>Hosp Death</td><td>NR 517 959</td><td>NR 194 564</td><td>NR 37.6<sup>a</sup></td><td>NR NR</td><td>NR NR</td><td>NR 176<sup>a</sup></td><td>NR</td></tr><tr><td rowspan="2">Patel et al [18]</td><td rowspan="2">Brazil</td><td rowspan="2">Rotarix</td><td>Hosp Death</td><td>92 453</td><td>69 572</td><td>75.3<sup>a</sup></td><td>2146</td><td>55</td><td>2.6<sup>a</sup></td><td>1265<sup>a</sup></td></tr><tr><td>Hosp Death</td><td>850</td><td>640</td><td>75.3<sup>a</sup></td><td>107</td><td>3</td><td>2.8<sup>a</sup></td><td>213<sup>a</sup></td></tr><tr><td rowspan="2">Mexico</td><td rowspan="2">Rotarix</td><td>Hosp Death</td><td>16 086</td><td>11 551</td><td>71.8<sup>a</sup></td><td>1215</td><td>41</td><td>3.4<sup>a</sup></td><td>282<sup>a</sup></td></tr><tr><td>Hosp Death</td><td>923</td><td>663</td><td>71.8<sup>a</sup></td><td>61</td><td>2</td><td>3.3<sup>a</sup></td><td>332<sup>a</sup></td></tr><tr><td rowspan="2">Desai et al [39]</td><td rowspan="2">Latin America (14)</td><td rowspan="2">Rotarix/RotaTeq</td><td>Hosp Death</td><td>229 656</td><td>144 746 [128 821–156 707]<sup>c</sup></td><td>63.0<sup>a</sup></td><td>5556</td><td>172 [126–293]<sup>c</sup></td><td>3.1<sup>a</sup></td><td>841 [479–1142]<sup>c</sup></td></tr><tr><td>Hosp Death</td><td>6302</td><td>4124 [3740–4239]<sup>c</sup></td><td>65.4<sup>a</sup></td><td>326</td><td>10 [6–17]<sup>c</sup></td><td>3.1<sup>a</sup></td><td>395 [207–526]<sup>c</sup></td></tr><tr><td rowspan="2">Patel et al [43]</td><td rowspan="2">LMIC (158)</td><td rowspan="2">Rotarix/RotaTeq</td><td>Hosp Death</td><td>NR</td><td>NR</td><td>NR</td><td>NR NR</td><td>NR NR</td><td>NR 615<sup>a</sup></td><td>NR</td></tr><tr><td>Hosp Death</td><td>452 800 [386 600–519 900]<sup>ab</sup></td><td>155 800 [83 300–217 700]<sup>b</sup></td><td>34.4<sup>a</sup></td><td>NR</td><td>253 (76–689)<sup>b</sup></td><td>NR NR</td><td>NR</td><td>NR</td></tr><tr><td>Carlin et al [17]</td><td>Australia</td><td>Rotarix/RotaTeq</td><td>Hosp Death</td><td>11 073</td><td>6528</td><td>NR</td><td>59.0<sup>a</sup></td><td>14</td><td>9.7<sup>a</sup></td><td>466<sup>a</sup></td><td>NR</td></tr><tr><td rowspan="2">Desai et al [38]</td><td rowspan="2">United States</td><td rowspan="2">RotaTeq</td><td>Hosp Death</td><td>71 175 [50 131–96 802]<sup>b</sup></td><td>53 444 [37 622–72 882]<sup>b</sup></td><td>75.1<sup>a</sup></td><td>NR</td><td>NR</td><td>45 [21–86]<sup>b</sup></td><td>NR NR</td><td>1093 [688–1902]<sup>b</sup></td></tr><tr><td>Hosp Death</td><td>33 [23–43]<sup>b</sup></td><td>14 [10–19]<sup>b</sup></td><td>42.4<sup>a</sup></td><td>NR</td><td>0.2 [0.1–0.3]<sup>b</sup></td><td>NR</td><td>71 [48–112]<sup>b</sup></td></tr><tr><td>Clark et al [37]</td><td>England</td><td>Rotarix</td><td>Hosp Death</td><td>14 770 [14 113–15 427]<sup>ab</sup></td><td>13 276 [12 255–14 181]<sup>b</sup></td><td>89.9<sup>a</sup></td><td>248</td><td>35.4 [7.0–9.76]<sup>cd</sup></td><td>14.3<sup>a</sup></td><td>375 [136–1900]<sup>ad</sup></td></tr><tr><td rowspan="2">Yung et al [44]</td><td rowspan="2">Singapore</td><td rowspan="2">Rotarix</td><td>Hosp Death</td><td>3.3 [1.7–4.9]<sup>b</sup></td><td>2.9 [1.7–4.1]<sup>b</sup></td><td>86.7<sup>a</sup></td><td>0.3<sup>a</sup></td><td>0.1 [0.0–0.2]<sup>bd</sup></td><td>10.6<sup>a</sup></td><td>88 [18–852]<sup>bd</sup></td></tr><tr><td>Hosp Death</td><td>808</td><td>570</td><td>70.5<sup>a</sup></td><td>22</td><td>3</td><td>13.6<sup>a</sup></td><td>190<sup>a</sup></td><td>NR</td></tr><tr><td>Ledent et al [41]</td><td>Japan</td><td>Rotarix</td><td>Hosp Death</td><td>20 829 [16 301–26 129]<sup>b</sup></td><td>17 925 [11 715–23 276]<sup>b</sup></td><td>86.1<sup>a</sup></td><td>1571 [1308–1868]<sup>b</sup></td><td>50 [7.2–237]<sup>b</sup></td><td>3.2<sup>a</sup></td><td>350 [69–251]<sup>b</sup></td></tr><tr><td rowspan="2">Lamrani et al [40]</td><td rowspan="2">France</td><td rowspan="2">Rotarix/RotaTeq</td><td>Hosp Death</td><td>7.3 [5.7–9.3]<sup>b</sup></td><td>6.3 [4.1–8.2]<sup>b</sup></td><td>86.3<sup>a</sup></td><td>0.5 [0.2–1.2]<sup>b</sup></td><td>0.1 [0.0–0.1]<sup>b</sup></td><td>3.1<sup>a</sup></td><td>366 [59–327]<sup>b</sup></td></tr><tr><td>Hosp Death</td><td>11 866<sup>a</sup></td><td>10 375 [7802–13 293]<sup>a</sup></td><td>87.4<sup>a</sup></td><td>214</td><td>47 [25–81]<sup>b</sup></td><td>21.9<sup>a</sup></td><td>214 [128–362]<sup>b</sup></td></tr><tr><td rowspan="2">Ledent et al [48]</td><td rowspan="2">France</td><td rowspan="2">Rotarix</td><td>Hosp Death</td><td>16 [15–18]</td><td>14 [12–15]<sup>a</sup></td><td>87.5<sup>a</sup></td><td>0.3</td><td>0.1 [0.0–0.2]<sup>b</sup></td><td>17.8<sup>a</sup></td><td>273 [89–1228]<sup>b</sup></td></tr><tr><td>Hosp Death</td><td>15 059 [12 100–18 476]<sup>b</sup></td><td>11 132 [7841–14 409]<sup>b</sup></td><td>73.9<sup>a</sup></td><td>323 [257–400]<sup>b</sup></td><td>6.9 [2.3–38.4]<sup>b</sup></td><td>2.1<sup>a</sup></td><td>1624 [240–5243]<sup>b</sup></td></tr><tr><td rowspan="2">Bruijing-Verhagen et al [45]</td><td rowspan="2">Netherlands</td><td rowspan="2">RotaTeq</td><td>Hosp Death</td><td>10.1 [4.6–19.5]<sup>b</sup></td><td>7.43 [3.27–14.68]<sup>b</sup></td><td>73.3<sup>a</sup></td><td>0.5 [0.2–0.9]<sup>b</sup></td><td>0.1 [0.0–0.1]<sup>b</sup></td><td>2.2<sup>a</sup></td><td>743 [93–3723]<sup>b</sup></td></tr><tr><td>Hosp Death</td><td>2700 [2400–3000]<sup>ab</sup></td><td>2000 [1800–2200]<sup>ab</sup></td><td>74.1<sup>a,e</sup></td><td>NR</td><td>2.9<sup>a</sup></td><td>NR</td><td>685 [603–767]<sup>a</sup></td></tr><tr><td rowspan="2">Bruun et al [46]</td><td rowspan="2">Norway</td><td rowspan="2">Rotarix</td><td>Hosp Death</td><td>5.5 [3.0–8.8]<sup>ab</sup></td><td>5.2 [2.8–8.3]<sup>ab</sup></td><td>93.6<sup>a,e</sup></td><td>22 [19–26]<sup>a</sup></td><td>1 [1–2]<sup>b</sup></td><td>5.7<sup>a</sup></td><td>1360</td></tr><tr><td>Hosp Death</td><td>NR</td><td>1788 [1761–1774]<sup>ab</sup></td><td>NR</td><td>NR</td><td>NR</td><td>NR</td><td>NR</td></tr><tr><td rowspan="2">Clark et al [47]</td><td rowspan="2">LMIC (135)</td><td rowspan="2">Rotarix/RotaTeq/ ROTAVAC/ ROTASIL/RV3-BB</td><td>Hosp Death</td><td>194 471 [158 603–257 080]<sup>b</sup></td><td>62 485 [47 895–83 238]<sup>b</sup></td><td>32.1<sup>a</sup></td><td>NR</td><td>NR</td><td>122 [44–322]<sup>b</sup></td><td>NR NR</td><td>512 [218–1338]<sup>b</sup></td></tr></tbody></table>

Abbreviations: BRR, benefit-risk ratio; Hosp, hospitalization; IS, intussusception; LMIC, low- and middle-income countries; N, number; NR, not reported; RVGE, rotavirus gastroenteritis; y, years.

<sup>a</sup>Using data from original publications.

<sup>b</sup>Median values.

<sup>c</sup>90% CI.

<sup>d</sup>IS risk period (0–2 years).

<sup>e</sup>Baseline incidence RVGE <15 years.
A

<figure><img src="image_3.png" /></figure>

B

<figure><img src="image_4.png" /><figcaption>**Figure 2.** Forest plot of benefit-risk ratios associated with rotavirus vaccination from selected studies. (a) Hospitalizations. (b) Deaths. Confidence intervals were not reported for all modeling studies.</figcaption></figure>

would increase by 2.1% (Ledent et al [48] in France) to 21.9% (Lamrani et al [40] in France) and 2.2% (Ledent et al [48] in France) to 17.8% (Lamrani et al [40] in France) as a result of vaccination in children under 1 year of age, respectively. Benefit-risk ratios ranged from 190 (Singapore) to 1624 (Ledent et al [48] in France) RVGE-related hospitalizations prevented for every additional vaccine-related IS hospitalization, whereas 71 (United States) to 743 (Ledent et al [48] in France) RVGE-related deaths would be prevented for every additional IS-death caused by the vaccine (Table 3 and Figure 2).

DISCUSSION

To our knowledge, the present review is the first to gather
available evidence on qBRm used for rotavirus vaccina-
tion. It had 2 main objectives: (1) to describe methodolog-
ical approaches used in the selected models (ie, their model

characteristics and input parameters) and (2) to characterize
the BR profile of rotavirus vaccination based on the available
scientific evidence.

Although a herd effect has been observed for rotavirus
vaccination [49–54], only 1 study considered it as a model
characteristic, among the 14 selected studies. Some authors
argued that this choice intended to make the approach more
conservative. In addition, more complex modeling tech-
niques such as transmission dynamic models were not used
at all, and PSA were only conducted in 9 of 14 of the qBRm.
Choosing for simpler approaches might be explained by the
fact that some studies were not conducting qBRm as primary
but as secondary objective. Although 5 studies considered a
waning effect of rotavirus vaccination [37, 40, 45–47], their
estimates of the proportion of RVGE-related hospitalizations
or deaths prevented by vaccination in children aged less than
5 years were similar to figures reported in the other studies.
This might be explained by the fact that the majority of severe RVGE cases occurs during infancy [2], before the protective effect of vaccination starts to wane.

Comparing input parameters across the different studies showed that lower VC figures were considered in LMIC than in HIC studies. This might be linked to the year of publication, ie, between 2009 and 2012, the WHO-recommended administration of the first dose of rotavirus vaccines with an upper age limit of 12-14 weeks to minimize the potential risk of IS. This strict age restriction may have reduced VC in some developing countries where the timeliness of pediatric vaccination varies widely [55]. In 2013, the WHO removed this age restriction to improve VC [56]. However, it is worth noting that the use of different VC figures had no impact on BRR estimates, because none of the selected qBRm considered transmission dynamic modeling.

The annual number of IS-related hospitalizations or deaths in children less than 1 year of age used as input parameter also varied across studies. The etiology of IS is not yet clearly understood. Differences in infant diet, breastfeeding, maternal antibody levels, and association with several pathogens including adenoviruses might all contribute to the variances in background rates of IS [7, 57, 58]. A higher number of IS-related deaths are observed in LMICs compared with HICs among selected studies. This finding might be due to differences in healthcare infrastructure or delays in care [7].

The present review systematically collected the published information on qBRm for rotavirus vaccination, which allowed further characterizing its BR profile. All selected studies concluded that vaccine-prevented RVGE-related hospitalizations and deaths outweigh vaccine-induced IS-related hospitalizations and deaths, with no marked difference between LMICs and HICs. Differences in BRR noted across studies included in this review can be explained by (1) the choice of model attributes (eg, simulation versus nonsimulation models), (2) varying epidemiology of RVGE and IS observed across countries and areas, (3) data availability at the time of the study, and (4) differences in the choice of input parameter values. Nevertheless, it is crucial to consider those differences when comparing models and their outputs. For example, despite the fact that they used similar modeling approaches, the 2 qBRm studies conducted in France showed differences with BRRs of 214 and 273 in Lamrani et al ([40]) and 1624 and 743 in Ledent et al ([48]), for hospitalizations and deaths, respectively. This might be explained by the value of some input parameters (eg, risk period duration and RR of IS). In this specific example, the use of scenario analysis by Ledent [48] et al considering the same risk period of 21 days as Lamrani et al [40] resulted in similar BRR between both studies [48].

This systematic literature review has some limitations. First, the search strategy may have not identified all relevant studies, notably due to the lack of limited specific keywords for qBRm.

In addition, some studies conducted by or for local governments or pharmaceutical companies may not have been made publicly available or indexed. Second, the data from included studies were not pooled in a meta-analysis to estimate an overall BRR for rotavirus vaccination, because 95% CIs were not available for all studies. Nevertheless, a forest plot allowing a visual assessment of differences between BRRs is depicted without providing overall BRR estimate (Figure 2).

## CONCLUSIONS

The present review provides a comprehensive overview of publications reporting on qBRm for rotavirus vaccination. This evidence confirms the favorable benefit-risk profile of rotavirus vaccines. The observed differences in qBRm approaches between studies complexified the comparison of their outputs and warrant the need for harmonization in such analysis to ensure comparability. In addition, because most studies focused on HICs, there is a need to increase BRR estimations in LMICs considering setting-specific input parameters and including sensitivity and/or scenario analyses to fully capture their effect.

## Supplementary Data

Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

## Acknowledgments

We thank Emmanuelle Ghys (XPE Pharma and Science) for editorial assistance and publication coordination on behalf of GSK.

*Financial support.* This study was conducted as part of a PhD program funded by GlaxoSmithKline Biologicals SA and the Association Nationale pour la Recherche et la Technologie (ANRT), Paris, France.

*Potential conflicts of interest.* H. A., G. N., and N. P. are employed by the GSK group of companies, which produces 1 of the vaccines of interest (Rotarix) in this review. G. N. and N. P. also hold shares in the GSK group of companies. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.

## References

1. Rotavirus vaccines. WHO position paper - January 2013. Wkly Epidemiol Rec **2013**; 88:49-64.

2. Sanderson C, Clark A, Taylor D, Bolanos B. Global review of rotavirus morbidity and mortality data by age and region. **2011**. Available at: https://www.who.int/immunization/sage/meetings/2012/april/Sanderson_et_al_SAGE_April Rotavirus. pdf. Accessed 28 March 2020.

3. Troeger C, Khalil IA, Rao PC, et al. Rotavirus vaccination and the global burden of rotavirus diarrhea among children younger than 5 years. JAMA Pediatr **2018**; 172:958-65.

4. Centers for Disease Control and Prevention. Withdrawal of rotavirus vaccine recommendation. MMWR Morb Mortal Wkly Rep **1999**; 48:1007.

5. Murphy TV, Gargiullo PM, Massoudi MS, et al.; Rotavirus Intussusception Investigation Team. Intussusception among infants given an oral rotavirus vaccine. N Engl J Med **2001**; 344:564-72.

6. Bines JE, Patel M, Parashar U. Assessment of postlicensure safety of rotavirus vaccines, with emphasis on intussusception. J Infect Dis **2009**; 200(Suppl 1):S282-90.
7. Jiang J, Jiang B, Parashar U, et al. Childhood intussusception: a literature review. PLoS One 2013; 8:e68482.

8. Oberle D, Jenke AC, von Kries R, et al. Rotavirus vaccination: a risk factor for intussusception? Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2014; 57:234-41.

9. Tate JE, Burton AH, Boschi-Pinto C, Parashar UD, World Health Organization-Coordinated Global Rotavirus Surveillance N. Global, regional, and national estimates of rotavirus mortality in children <5 years of age, 2000-2013. Clin Infect Dis 2016; 62(Suppl 2):S96-105.

10. Ruiz-Palacios GM, Pérez-Schael I, Velázquez FR, et al.; Human Rotavirus Vaccine Study Group. Safety and efficacy of an attenuated vaccine against severe rotavirus gastroenteritis. N Engl J Med 2006; 354:11-22.

11. Vesikari T, Matson DO, Dennehy P, et al.; Rotavirus Efficacy and Safety Trial (REST) Study Team. Safety and efficacy of a pentavalent human-bovine (WC3) reassortant rotavirus vaccine. N Engl J Med 2006; 354:23-33.

12. European Medicines Agency. Rotateq: Summary of Opinion (post authorisation). 19 January 2012. Available at: http://www.ema.europa.eu/docs/en_GB/document_library/Summary_of_opinion/human/000669/WC500120764.pdf. Accessed 28 March 2020.

13. European Medicines Agency. Rotarix: a summary of the European Public Assessment Report. 2015. Available at: http://www.ema.europa.eu/ema/index.jsp?curl=pages/medicines/human/medicines/000639/human_med_001043.jsp&mid=WC0b01ac058001d124. Accessed 28 March 2020.

14. World Health Organization. Summary of key characteristics of currently WHO-Pre-qualified rotavirus vaccines. 2019. Available at: https://www.who.int/immunization/diseases/rotavirus/WHO_Summary_xtcs_PQ'd_rotavirus_vaccines.PDF?ua=1. Accessed 28 March 2020.

15. Deen J, Lopez AL, Kanungo S, et al. Improving rotavirus vaccine coverage: can newer-generation and locally produced vaccines help? Hum Vaccin Immunother 2018; 14:495-9.

16. Burke RM, Tate JE, Kirkwood CD, et al. Current and new rotavirus vaccines. Curr Opin Infect Dis 2019; 32:435-44.

17. Carlin JB, Macartney KK, Lee KJ, et al. Intussusception risk and disease prevention associated with rotavirus vaccines in Australia's National Immunization Program. Clin Infect Dis 2013; 57:1427-34.

18. Patel MM, López-Collada VR, Bulhões MM, et al. Intussusception risk and health benefits of rotavirus vaccination in Mexico and Brazil. N Engl J Med 2011; 364:2283-92.

19. Velázquez FR, Colindres RE, Grajales C, et al. Postmarketing surveillance of intussusception following mass introduction of the attenuated human rotavirus vaccine in Mexico. Pediatr Infect Dis J 2012; 31:736-44.

20. Weintraub ES, Baggs J, Duffy J, et al. Risk of intussusception after monovalent rotavirus vaccination. N Engl J Med 2014; 370:513-9.

21. Yih WK, Lieu TA, Kuldorff M, et al. Intussusception risk after rotavirus vaccination in U.S. infants. N Engl J Med 2014; 370:503-12.

22. Stowe J, Andrews N, Ladhani S, Miller E. The risk of intussusception following monovalent rotavirus vaccination in England: a self-controlled case-series evaluation. Vaccine 2016; 34:3684-9.

23. Groome MJ, Tate JE, Arnold M, et al. Evaluation of intussusception after oral monovalent rotavirus vaccination in South Africa. Clin Infect Dis 2019. doi: 10.1093/cid/ciz431.

24. Tate JE, Mwenda JM, Armah G, et al.; African Intussusception Surveillance Network. Evaluation of intussusception after monovalent rotavirus vaccination in Africa. N Engl J Med 2018; 378:1521-8.

25. Rha B, Tate JE, Weintraub E, et al. Intussusception following rotavirus vaccination: an updated review of the available evidence. Expert Rev Vaccines 2014; 13:1339-48.

26. Rosillon D, Buyse H, Friedland LR, et al. Risk of intussusception after rotavirus vaccination: meta-analysis of postlicensure studies. Pediatr Infect Dis J 2015; 34:763-8.

27. Dong R, Yang Y-F, Chen G, Shen Z, Zheng S. Risk of intussusception after rotavirus vaccination: a meta-analysis. Int J Clin Exp Med 2016; 9:1306-13.

28. Parashar UD, Cortese MM, Payne DC, et al. Value of post-licensure data on benefits and risks of vaccination to inform vaccine policy: the example of rotavirus vaccines. Am J Prev Med 2015; 49:S377-82.

29. Parashar UD, Orenstein WA. Editorial commentary: intussusception and rotavirus vaccination-balancing risk against benefit. Clin Infect Dis 2013; 57:1435-7.

30. Bonhoeffer J, Black S, Izurieta H, et al. Current status and future directions of post-marketing vaccine safety monitoring with focus on USA and Europe. Biologicals 2012; 40:393-7.

31. Di Pasquale A, Bonanni P, Garçon N, et al. Vaccine safety evaluation: practical aspects in assessing benefits and risks. Vaccine 2016; 34:6672-80.

32. Greenberg M, Simondon F, Saadatian-Elahi M. Perspectives on benefit-risk decision-making in vaccinology: conference report. Hum Vaccin Immunother 2016; 12:176-81.

33. Schosser R. Risk/benefit evaluation of drugs: the role of the pharmaceutical industry in Germany. Eur Surg Res 2002; 34:203-7.

34. Kim SY, Goldie SJ. Cost-effectiveness analyses of vaccination programmes: a focused review of modelling approaches. Pharmacoeconomics 2008; 26:191-215.

35. Kuntz K, Sainfort F, Butler M, Taylor B, Kulasingam S, Gregory S, et al. Decision and Simulation Modeling in Systematic Reviews. Rockville, MD: AHRQ Methods for Effective Health Care; 2013.

36. York Health Economics Consortium. A Glossary of Health Economic Terms. Available at: http://www.yhec.co.uk/tools-resources/glossary/. Accessed 28 March 2020.

37. Clark A, Jit M, Andrews N, et al. Evaluating the potential risks and benefits of infant rotavirus vaccination in England. Vaccine 2014; 32:3604-10.

38. Desai R, Cortese MM, Meltzer MI, et al. Potential intussusception risk versus benefits of rotavirus vaccination in the United States. Pediatr Infect Dis J 2013; 32:1-7.

39. Desai R, Parashar UD, Lopman B, et al. Potential intussusception risk versus health benefits from rotavirus vaccination in Latin America. Clin Infect Dis 2012; 54:1397-405.

40. Lamrani A, Tubert-Bitter P, Hill C, Escolano S. A benefit-risk analysis of rotavirus vaccination, France, 2015. Euro Surveill 2017; 22. doi:10.2807/1560-7917. ES.2017.22.50.17-00041.

41. Ledent E, Lieftucht A, Buyse H, et al. Post-marketing benefit-risk assessment of rotavirus vaccination in Japan: a simulation and modelling analysis. Drug Saf 2016; 39:219-30.

42. Patel MM, Clark AD, Glass RI, et al. Broadening the age restriction for initiating rotavirus vaccination in regions with high rotavirus mortality: benefits of mortality reduction versus risk of fatal intussusception. Vaccine 2009; 27:2916-22.

43. Patel MM, Clark AD, Sanderson CF, et al. Removing the age restrictions for rotavirus vaccination: a benefit-risk modeling analysis. PLoS Med 2012; 9:e1001330.

44. Yung CF, Chan SP, Soh S, et al. Intussusception and monovalent rotavirus vaccination in Singapore: self-controlled case series and risk-benefit study. J Pediatr 2015; 167:163-8.e1.

45. Bruijning-Verhagen P, van Dongen JAP, Verberk JDM, et al. Updated cost-effectiveness and risk-benefit analysis of two infant rotavirus vaccination strategies in a high-income, low-endemic setting. BMC Med 2018; 16:168.

46. Bruun T, Watle SSV, Vteteraas IH, Flem E. Intussusception among Norwegian children: what to expect after introduction of rotavirus vaccination? Vaccine 2019; 37:5717-23.

47. Clark A, Tate J, Parashar U, et al. Mortality reduction benefits and intussusception risks of rotavirus vaccination in 135 low-income and middle-income countries: a modelling analysis of current and alternative schedules. Lancet Glob Health 2019; 7:e1541-52.

48. Ledent E, Arlegui H, Buyse H, et al. Benefit versus risk assessment of rotavirus vaccination in France: a simulation and modeling analysis. BioDrugs 2018; 32:139-52.

49. Atchison CJ, Stowe J, Andrews N, et al. Rapid declines in age group-specific rotavirus infection and acute gastroenteritis among vaccinated and unvaccinated individuals within 1 year of rotavirus vaccine introduction in England and Wales. J Infect Dis 2016; 213:243-9.

50. Buttery JP, Standish J, Bines JE. Intussusception and rotavirus vaccines: consensus on benefits outweighing recognized risk. Pediatr Infect Dis J 2014; 33: 772-3.

51. Cortese MM, Tate JE, Simonsen L, et al. Reduction in gastroenteritis in United States children and correlation with early rotavirus vaccine uptake from national medical claims databases. Pediatr Infect Dis J 2010; 29:489-94.

52. Lopman BA, Curns AT, Yen C, Parashar UD. Infant rotavirus vaccination may provide indirect protection to older children and adults in the United States. J Infect Dis 2011; 204:980-6.

53. Payne DC, Staat MA, Edwards KM, et al. Direct and indirect effects of rotavirus vaccination upon childhood hospitalizations in 3 US counties, 2006-2009. Clin Infect Dis 2011; 53:245-53.

54. Standaert B, Strens D, Alwan A, Raes M. Medium- to long-term impact of rotavirus vaccination on hospital care in Belgium: a 7-year follow-up of the rotavirus Belgium impact study (RotaBIS). Infect Dis Ther 2016; 5:31-44.

55. Clark A, Sanderson C. Timing of children's vaccinations in 45 low-income and middle-income countries: an analysis of survey data. Lancet 2009; 373:1543-9.

56. World Health Organization. Weekly Epidemiological Record. Rotavirus vaccines-WHO position paper 2013. Available at: https://apps.who.int/iris/bitstream/handle/10665/242024/WER8805_49-64.PDF?sequence=1&isAllowed=y. Accessed 28 March 2020.

57. Bines JE, Liem NT, Justice FA, et al.; Intussusception Study Group. Risk factors for intussusception in infants in Vietnam and Australia: adenovirus implicated, but not rotavirus. J Pediatr 2006; 149:452-60.

58. Tate JE, Simonsen L, Viboud C, et al. Trends in intussusception hospitalizations among US infants, 1993-2004: implications for monitoring the safety of the new rotavirus vaccination program. Pediatrics 2008; 121:e1125-32.