Epidemiological impact and effectiveness
of COVID-19 measures
RIVM/CIb/EPI/MOD

Version 23 January 2024

Contents
Verzoek van VWS aan RIVM.................................................................................................................... 4
Samenvatting .......................................................................................................................................... 5
Summary ................................................................................................................................................. 6
Introduction ............................................................................................................................................ 7
COVID-19 measures in the Netherlands ................................................................................................. 9
Non-pharmaceutical interventions ..................................................................................................... 9
Vaccination coverage .......................................................................................................................... 9
Effectiveness of COVID-19 measures in reducing transmission ........................................................... 10
Effectiveness in reducing transmission ............................................................................................. 10
Effective reproduction number Rt in the Netherlands ..................................................................... 10
Effectiveness of non-pharmaceutical interventions in reducing transmission in the Netherlands . 11
Effectiveness of control strategies in reducing severe outcomes ........................................................ 12
What if we had implemented COVID-19 control strategies as in Belgium, Denmark, Germany,
Sweden, or the UK?........................................................................................................................... 13
What if the Dutch control strategy would have started a few days later?....................................... 15
The effect of interventions in reducing number of contacts per day ................................................... 17
Non-pharmaceutical interventions (NPIs) ........................................................................................ 17
Contact surveys reveal that contacts were reduced ........................................................................ 17
Pienter Corona contact survey ..................................................................................................... 17
COMIX contact survey................................................................................................................... 18
SCONE contact survey ................................................................................................................... 19
Side-effect of COVID-19 NPIs: a decline in transmission of other respiratory infections................. 20
Contribution of individual interventions to overall effectiveness ........................................................ 21
Non-pharmaceutical interventions (NPI): evidence from literature ................................................ 21
Decomposing total effectiveness into contributions by individual measures: international
evaluation reports............................................................................................................................. 21
Decomposing total effectiveness into contributions by individual measures: estimates derived
from combined data from different countries ................................................................................. 22
The effect of compliance on effectiveness ................................................................................... 23
The effect of seasonality on estimates of effectiveness ............................................................... 24
The effectiveness of contact tracing ............................................................................................. 24
The complementarity of interventions ......................................................................................... 25
Mediating variables....................................................................................................................... 25
Vaccines: evidence from literature ................................................................................................... 25
Discussion.............................................................................................................................................. 27

Recommendations ................................................................................................................................ 29
Composing effective bundles of interventions ................................................................................. 29
Preparedness of data streams .......................................................................................................... 29
References ............................................................................................................................................ 31
Appendix A studies on which this report is based ................................................................................ 33
Publications in peer-reviewed scientific journals ............................................................................. 33
Manuscripts in preparation .............................................................................................................. 33
Appendix B Evaluation of the epidemiological effectiveness of COVID measures: a literature review
.............................................................................................................................................................. 34

Verzoek van VWS aan RIVM
Naar aanleiding van het tweede rapport van de OVV d.d. woensdag 12 oktober 2022 heeft VWS het
RIVM gevraagd te bezien op welke wijze de effectiviteit van (combinaties van) maatregelen
inzichtelijk gemaakt kunnen worden, zodat deze kennis beschikbaar is t.b.v. advisering en
besluitvorming in een volgende besmettingsgolf of pandemie. Het doel van deze opdracht is om
voorbereid te zijn op de toekomst. RIVM heeft hierin als kennisinstituut - zowel vanuit de wettelijk
taak van het RIVM gerelateerd aan de Wet publieke gezondheid (op het terrein van
infectieziektebestrijding, gezondheidsadvisering en psychosociale hulpverlening) als op basis van het
gezondheidsonderzoek bij crisis en rampen – een belangrijke rol waar het gezondheids- en
maatschappelijke (welzijns-) effecten en gedrag betreft in de vorm van advisering aan VWS, OMT en
MIT. De wens is om hierbij ook internationaal beschikbaar onderzoek te betrekken en waar mogelijk
de Nederlandse situatie te vergelijken met andere vergelijkbare landen.
Het gaat hierbij primair om het versterken van het inzicht in directe epidemiologische gevolgen van
maatregelen (effectiviteit). Dit levert kennis en inzichten op, die vooruitkijkend bruikbaar zijn voor
advisering en beleidsvorming t.b.v. COVID-19 en pandemische paraatheid.

Samenvatting
We leveren bewijs voor de effectiviteit van de genomen COVID-19 maatregelen in Nederland. We
schatten de effectiviteit van de pakketten van niet-farmaceutische interventies in het tegengaan van
transmissie van infectie. We laten zien dat eenzelfde effectiviteit tegen transmissie tot heel andere
uitkomsten van de pandemie zou hebben geleid als de interventies één of drie dagen later zouden
zijn ingegaan, en we vergelijken de effecten van de maatregelpakketten zoals gekwantificeerd voor
Nederland met die zoals gekwantificeerd voor andere landen (België, Duitsland, Denemarken,
Zweden, Verenigd Koninkrijk). We laten zien dat de effectiviteit van maatregelen toeneemt met
toenemende stringentie van de maatregelpakketten. Deze associatie wordt aangevuld met de
waarneming dat met toenemende stringentie van de maatregelpakketten het aantal gerapporteerde
contacten in contactonderzoeken afnam, en dat de transmissie afnam van andere infectieziekten
met dezelfde transmissieroute als SARS-CoV-2. In een literatuuroverzicht identificeren we wat er
bekend is over de bijdrage van individuele interventies aan de effectiviteit van maatregelpakketten,
en wat er bekend is over de effectiviteit van COVID-19 vaccins. We bespreken methoden om de
effectiviteit van maatregelpakketten te ontleden in bijdragen van individuele interventies met
behulp van gegevens uit meerdere landen, we verwijzen naar de huidige beste schattingen, en
identificeren de beperkingen van het gebruik van deze schattingen voor toekomstige
infectieziektebestrijding. We bevelen aan de complementariteit van interventies te onderzoeken en
een structuur voor gegevensstromen op te zetten zodat informatie geleverd kan worden voor
infectieziektebestrijding bij toekomstige pandemie.

Summary
We provide evidence for the effectiveness of COVID-19 measures in the Netherlands. We estimate
the effectiveness of the bundles of non-pharmaceutical interventions (NPIs) against transmission.
We show that the effectiveness against transmission would have resulted in a very different
outcome on the pandemic if the interventions would have been delayed by one or three days, and
we compare the effects of the bundles of interventions as quantified for the Netherlands with the
bundles of interventions as quantified for other countries (Belgium, Germany, Denmark, Sweden,
UK). We show that the effectiveness increases with increasing stringency of the control measures.
This association is complemented by an association with a decreasing number of contacts as
reported in contact surveys, and a reduction in transmission of other infectious diseases that have
the same transmission route as SARS-CoV-2. In a literature review we identify what is known about
the contribution of individual interventions and the vaccine effectiveness. We report on approaches
to disentangle the contribution of individual interventions to the overall effectiveness in reducing
transmission, using information from multiple countries, we point to current best estimates and
identify the limitations to using these estimates for future infection control. We recommend looking
into complementarity of interventions, and setting up data streams that inform infection control in a
future pandemic.

Introduction
Pandemics are a threat to society. At the peak of a pandemic wave there is a substantial risk of
increased absenteeism and problems might arise with, for example, supply chains to supermarkets,
garbage collection, and traffic control. This can lead to societal unrest and even to collapse of the
critical societal infrastructure. The healthcare demand can exceed the available healthcare capacity,
which can result in a collapse of the healthcare infrastructure. Critical healthcare personnel can
become infected, which reduces the available capacity, which increases the risk of collapse of the
healthcare infrastructure. A collapse of healthcare infrastructure, or societal infrastructure, could
result in withholding essential care from those who need it most, resulting in higher mortality rates.
The costs of such a collapse are huge.
The healthcare demand per day due to an infectious disease is determined by several factors. These
factors are the number of new infections per day, the probability of a severe outcome per infection,
and the number of days that healthcare is needed for a severe outcome of the disease. For SARSCoV-2, the virus that causes the disease COVID-19, the transmission potential was sufficiently high to
spreadamong a large number of individuals who were susceptible in the beginning of 2020, which
could cause a large number of new infections per day in absence of control. This, combined with a
relatively high probability of severe outcome and relatively long duration of required healthcare for
those with a severe outcome, provided COVID-19 the potential to cause a collapse of the healthcare
infrastructure in absence of control. In the Netherlands, the approach to control during the first
phase of the COVID-19 pandemic has been guided by monitoring the spread of the virus, ensuring
that healthcare demand does not exceed the available capacity, and protecting the vulnerable risk
groups in the population. Later, in 2021, weighing the economic and societal consequences was also
explicitly mentioned.
Infection control reduces the probability of infection per unit of time of being exposed to infection.
One way of interpreting this, is that infection control results in postponing the time of infection and
spreading the risk of infection over a longer period. By postponing and spreading the time of
infection, a wave of healthcare demand that would exceed the capacity can become a steady trickle
of demand that can be dealt with in the healthcare infrastructure. Infection control can also reduce
the transmissibility of infection. When infection control measures reduce transmissibility, they will
decrease the growth rate of the number of daily infections, hospitalizations, IC admissions, deaths.
The level of control required to stop this growth is determined by the reproduction number, defined
as the number of infections that are caused on average by a typical single infectious person. For
example, if an infection is transmissible such that each infectious person would on average infect
five others, we say the reproduction number is 5, and that the critical level of control is achieved
when 4 out of 5 (80%) of the secondary infections are prevented. If we prevent fewer secondary
cases, the number of daily new cases will increase. If we prevent more, the number of daily new
cases will decrease.
In a pandemic, policy makers have to base their decisions on incoming information from infectious
disease surveillance and monitoring. Decision making is a dynamic problem in the sense that, when
control is insufficient, the risk of collapse of the healthcare infrastructure increases exponentially
fast over time, whereas the costs of control tend to remain constant. When the objective is to avoid
healthcare collapse, the question is when to start control and how to achieve the required level of
control. Given the situational assessment, policy makers decide when to control, where to control,
and how targeted control measures are. The required level of control is determined by the
characteristics of the infection and the population. A key question for policy makers is which bundles
of interventions can achieve the required effectiveness. A secondary question is which sufficiently

effective bundles meet additional requirements of sufficient support, sufficiently high compliance, at
proportional and acceptable societal costs.
Here we address the following questions: How effective were bundles of COVID-19 measures in
reducing SARS-CoV-2 transmission in the Netherlands? How does this translate into effectiveness
against other outcome measures, such as hospitalisations and deaths? How did individual measures
contribute to this effectiveness? It is important to know whether measures were effective. It is also
important to understand the contribution of individual measures to the overall effectiveness to
support the application of similar measures in a future pandemic.
This report summarizes information from the international literature and from various studies that
focus on the COVID-19 pandemic in the Netherlands. These studies have been published in
international peer-reviewed journals, or they are being prepared for publication. The list with these
publications, manuscripts, and a review of the international literature, are provided as appendices to
this report. As these publications, manuscripts and review contain a detailed overview of the
national and international literature, we have chosen to provide only essential references in this
summary. We will make a distinction in the control measures and treat vaccines separately from
non-pharmaceutical interventions (NPIs) such as distancing and working from home.
We organize the text into chapters that each address a specific question. We will first indicate where
one can find information on vaccine coverage in the Netherlands and information on the timeline of
implemented NPIs. Second, we assess the effectiveness of bundles of NPIs and vaccination against
transmission of SARS-CoV-2 in the Netherlands. Third, we translate the effectiveness against
transmission into effectiveness of control strategies against severe outcomes such as death and
hospitalisations. We compare strategies from different neighbouring countries and highlight the
importance of timing of interventions in determining the effectiveness against severe outcomes.
Fourth, we check whether bundles of NPIs also resulted in lower number of contacts in the
population, and whether a lower number of contacts in the population resulted in less transmission
of pathogens that spread along the transmission route of SARS-CoV-2. Fifth, we describe a strategy
to decompose the effectiveness against transmission into individual contributions of measures and
highlight the underlying assumptions; we summarize results for individual measures from peerreviewed scientific publications. Sixth, we pull the different lines of argument together to indicate
what this means for the limitations on what we can do with the outcomes for a future pandemic.
Last, we recommend specific further steps for preparing for infection control in future pandemics.

COVID-19 measures in the Netherlands
Non-pharmaceutical interventions
An overview of the non-pharmaceutical interventions in the Netherlands is provided online
(https://www.rivm.nl/gedragsonderzoek/tijdlijn-van-coronamaatregelen-2020). As there are many
individual measures, we score the stringency of the entire bundle of individual measures at any day.
We use the Stringency Index, defined by the Oxford Coronavirus Government Response Tracker
(OxCGRT) project. This is a composite measure of nine of the response metrics: school closures;
workplace closures; cancellation of public events; restrictions on public gatherings; closures of public
transport; stay-at-home requirements; public information campaigns; restrictions on internal
movements; and international travel controls, see https://github.com/OxCGRT/covid-policytracker/blob/master/documentation/index_methodology.md. An increasing score implies that a
country’s government response is becoming more stringent, a decreasing score implies that the
response becomes less stringent. The absolute value of the stringency index is hard to interpret or
compare but changes in value are useful to indicate the periods when control becomes more
stringent, and the periods when control is relaxed. The stringency of the Dutch government
response as measured by the Oxford Stringency Index shows gradual build-up towards local peaks
followed by gradual relaxation, with localized peaks occurring by the end of March 2020; the end of
January 2021; and the start of January 2022 (Figure 1).

Figure 1: Stringency Index for the Dutch government response to the COVID-19 pandemic (1 January
2020 - 31 December 2021)

Vaccination coverage
An overview of vaccination coverage in the Dutch population is available by age-group/birth cohort
and by type of vaccine. RIVM routinely update their estimate of the current national-level
vaccination coverage (both aggregated over age, and by 5- or 10-year age-group). These estimates
are available online (https://coronadashboard.rijksoverheid.nl/landelijk/vaccinaties). For instance,
according to data as of 12 April 2023, 80.2% of persons aged 12 years or older had completed their
primary series of vaccinations (1 or 2 doses, depending on vaccine type and whether the person
already has confirmed SARS-CoV-2 infection). The per age-group coverage for the primary series
varied from 3% (5-11 year olds) to 93% (70-79 year olds).

Effectiveness of COVID-19 measures in reducing transmission
Effectiveness in reducing transmission
Effect of interventions can be quantified using various outcome measures, such as the reproduction
number, or numbers of cases, hospitalizations, or deaths. Of these, the effectiveness in reducing
reproduction numbers is an instantaneous effect, whereas other outcome measures are affected by
earlier interventions with large lag times and have to be defined over a specific time interval. The
primary rationale of non-pharmaceutical interventions (NPIs) is to reduce person-to-person
transmission by altering contact rates and contact patterns and by reducing the probability of
infection upon contact. This in turn will reduce the probability of infection per day of those who are
at risk of developing severe disease (hospitalization or death). This rationale motivates our choice to
focus first on the effectiveness of NPIs to reduce transmission and quantify reduced transmission by
the reduction in the effective reproduction number.

Effective reproduction number Rt in the Netherlands
The effective reproduction number Rt is defined as the average number of secondary cases infected
by a primary case. The epidemic grows when the value is larger than one, and the epidemic declines
when the value is smaller than one. The value of the reproduction number provides information on
the required control effort to prevent the epidemic from growing. Here, we use the reproduction
number for SARS-CoV-2 infections to assess the effect of various bundles of interventions. The
reproduction numbers for the Netherlands were calculated weekly or twice per week from March
2020 onwards, and the results are published in the open RIVM data (https://data.rivm.nl/covid-19/)
and on the COVID-19 dashboard (https://coronadashboard.rijksoverheid.nl/). The effective
reproduction number is calculated from the time series of reported symptom onset dates and the
distribution of the generation interval (i.e., the time between successive infections). The mean
generation interval was 4 days for all SARS-CoV-2 variants other than Omicron, and 3.5 days for
Omicron variants (Figure 2).

Figure 2 Time series of effective reproduction number Rt in the Netherlands (mean and 95%
confidence interval), based on data of 2 May 2023. The reproduction number was based on the
hospitalisations reported in Osiris from 17 Feb 2020 until 12 Jun 2020, on the test-positive cases

reported in Osiris from 13 Jun 2020 until 14 Mar 2023 and on the hospitalisations reported in NICE
from 15 Mar 2023 onwards. Of these, we consider the reproduction number based on test-positive
cases as the most accurate. As test-positive cases were not available at the start of the pandemic, we
use test-positive hospitalizations in the initial phase, as recorded in Osiris. As test-positive cases were
no longer available at the end, we use hospitalizations which were tested positive by NICE in the most
recent phase.

Effectiveness of non-pharmaceutical interventions in reducing transmission in the
Netherlands
We use the time series of the effective reproduction number Rt to explore the effectiveness of nonpharmaceutical interventions (NPIs). We use the basic reproduction number, R0(t), to express how
many secondary cases would have been infected on average by a primary case in a fully susceptible
population without any control measures. This coincides with the observed value of Rt at the very
start of an epidemic before interventions were implemented. We would like to quantify to what
extent the Rt is reduced from this value R0(t) due to interventions, and not due to a build-up of
immunity due to natural infection and vaccination. To this end we estimate the effective susceptible
fraction of the population (Seff). This leads to an estimator of the effectiveness of the NPIs:
R(t)

effectiveness(t) = 1- eff(t)R (t).
S

0

In this definition, the effectiveness of NPIs includes compliance.

The basic reproduction number R0(t) can change over time. Seasonal effects can increase R0 in
winter due to meteorological conditions that are favorable for virus survival and crowding. For SARSCoV-2 the basic reproduction number has also increased over time due to the emergence of new,
more transmissible variants. Here, we assume that the growth advantages of the Alpha and Delta
variants were completely attributable to a higher transmissibility (that is, to a higher R0). The
calculations do not account for the immune escape that allowed the Omicron variants to supersede
the previously circulating variants. Therefore, we limit the analysis to the period before the Omicron
variants became dominant.
Over the course of the pandemic, the proportion of immunes increased in the population by
infection and from early 2021 onwards by vaccination. We estimate the proportion immunes by
infection from the Pienter Corona serosurveys that were conducted two or three times per year. For
each age group, we impute the cumulative infection incidence for the days between consecutive
serosurveys from the reported case data. The proportion of immunes due to vaccination is
calculated from the vaccination uptake per age group, multiplied with the vaccine effectiveness. We
account for the waning of immunity over time, i.e., the loss of protection against infection, both
after infection and after vaccination. The effective susceptible fraction is calculated as a population
weighted average of the proportion susceptibles (those who are not immune or infected). In an
additional sensitivity study, we also accounted for the dynamic effect of heterogeneities in infection
attack rate between age groups (results not shown), but this did not change the results.
The results (Figure 3) indicate that effectiveness of NPIs in reducing transmission in the Netherlands
was high during the first wave (March – May 2020) and during the winter of 2021 when the Alpha
variant emerged. The negative values for effectiveness around July 2021 (‘dansen met Janssen’)
suggest that the number of at-risk contacts for transmission was higher than before implementation
of interventions. In the autumn of 2021, the effectiveness of NPIs in reducing transmission
decreased due to high vaccination levels, and the extent of this decrease strongly depends on the

assumed rate of waning. Localized peaks in effectiveness correspond to localized peaks the
stringency index.
In interpreting these results, it is important to acknowledge the assumptions made. For instance,
immunity is considered as an all-or-nothing response, where an individual whose immunity waned is
assumed to be as susceptible to infection as a naïve individual, and – when infected again – as
infectious as a naïve individual. Second, the calculations accounted for the uncertainty with respect
to the effective reproduction number, the serological survey results, and the waning rates, but not
for uncertainty with respect to the seasonality effect and the transmissibility of new variants.

Figure 3 Effectiveness of NPIs in reducing transmission from the start of the COVID-19 pandemic up
to the emergence of the Omicron variant in the Netherlands, for three different scenarios of waning
of immunity over time after infection and vaccination (mean and 95% confidence interval). For
comparison, we indicate the stringency index of the Dutch COVID-19 policy. Localized peaks of
stringency (dark red) correspond with localized peaks in effectiveness, and conversely, localized
throughs in stringency (blue) correspond with localized throughs in effectiveness. (10 February 2020 –
1 November 2021

Effectiveness of control strategies in reducing severe outcomes
Whereas implementing a bundle of interventions has an almost instantaneous effect on reducing
transmission, the effect of on other outcome measures, such as number of infections, or number of
cases, number of hospital admissions and mortality, will depend also on the timing of the
interventions. Here we illustrate how different interventions with different timing could have led to
alternative outcomes in the Netherlands. We focus on interventions as taken by other WesternEuropean countries, as these countries were in a comparable situation to the Netherlands at the

start of the COVID-19 pandemic. We focus on mortality as an outcome measure for the practical
reason that time series with mortality are available for multiple countries. We choose to use a
method and code that has been developed and published by another research team outside the
Netherlands, and that has not been previously applied to data on the Netherlands. We focus on the
first pandemic wave, specifically the period from February 2020 to July 2020, to highlight the impact
of timing of intervention measures; in subsequent waves the impact of timing of interventions
relative to the timing of the wave would suffer from interference with earlier measures.

What if we had implemented COVID-19 control strategies as in Belgium, Denmark,
Germany, Sweden, or the UK?
We explore how the impact of the first COVID-19 wave in the Netherlands may have differed if it had
adopted response strategies from other Western-European countries. We build upon a prior
modelling study by Mishra et al. (2020) and compare response strategies of six countries: the
Netherlands, Belgium, Denmark, Germany, Sweden, and the United Kingdom (UK). The timedependent reproduction number (Rt) during the first pandemic wave (February 2020 up to and
including June 2020) was estimated for each country using time-series of daily laboratory-confirmed
COVID-19 deaths by date of death. In this study, we use different data and a different method to
estimate the effective reproduction number Rt as compared to the previous section. Here, the basic
reproduction number R0 is obtained by fitting the model to the first week of time-series of deaths
after a country had observed a total of 10 cumulative death cases.
Time series of laboratory-confirmed COVID-19 mortality by date of death for the Netherlands were
extracted from the OSIRIS database, the national registry for laboratory-confirmed COVID-19 cases
of the RIVM. For Belgium, similar time series data were obtained from Sciensano, and for Germany
from the Robert Koch Institute (personal communication, M. an der Heiden, 1 December 2022). A
detailed description of the mortality time series for the other countries and on how Rt was
computed can be found in Mishra et al (2020). Infection-to-death delay distributions were assumed
to be the same between countries. We described the distribution of generation interval by a gamma
distribution with a mean of 4 days and a standard deviation of 2 days to maintain consistency with
the observed duration of the serial interval for SARS-CoV-2 transmission in the Netherlands and the
calculations in the previous section. For the counterfactual assessment, we simulated a growing
epidemic for the Netherlands with the reproduction number Rt equal to R0 until 13 March 2020.
From 13 March 2020 to 1 July 1 2020 we substituted the relative reduction in the reproduction
number (i.e., the ratio of the effective and the basic reproduction numbers Rt/R0) for the
Netherlands by that of another country (e.g. Belgium) on the same calendar day. This approach
allowed us to adopt the calendar time and reduction in transmission as observed in another country.
The median basic reproduction number R0 was highest for Belgium, the Netherlands, and the UK,
and lowest for Sweden and Denmark. Following the implementation of strict control measures after
13 March, the reproduction number Rt decreased rapidly in all countries, eventually dropping below
1, indicating the suppression of the epidemic. The pace of this decline in Rt during March varied
among countries. The Netherlands and Denmark were the first countries achieving an Rt below 1.
Sweden showed a less rapid decline in the reproduction number Rt as compared to the other
countries. In all countries, there was an increase in the reproduction number Rt in the second half of
April.
We obtain alternative, counterfactual control strategies for the Netherlands by transferring the
relative reduction in reproduction number from other countries to the Netherlands from 13 March
onwards. The subtle differences in the reproduction number Rt between countries lead to

substantial differences in absolute numbers of deaths (Figure 7 and Table 1). If Rt were to drop
below 1 only a few days later, as seen in the time series of reproduction numbers as observed in
Denmark, Belgium, Germany and the UK, the peak number of deaths in the Netherlands during the
first wave would have increased from approximately 170 per day to a range of 300 to 600 per day
(Figure 4, based on median estimates). In case of a response strategy from Sweden, the peak
number of daily deaths in the Netherlands would have reached almost a thousand deaths per day.
With any other response strategy, the total number of deaths throughout the first wave would have
significantly increased in the Netherlands (Table 1). Transferring the response strategies of Belgium
and Denmark to the Netherlands would have resulted in a two-fold increase in number of deaths
compared to the observed amount. The response strategies of Germany and UK would have led to
three-fold increase in deaths, and the response strategy of Sweden was even associated with a
seven-fold increase in deaths.

Figure 4: Estimated median number of daily deaths with 95% credible intervals for the Netherlands,
showing the fit to observed data (blue bars) and the counterfactual analyses, involving the transfer of
the relative reduction in reproduction number from Belgium (Bel), Denmark (Dmk), Germany (Ger),
Sweden (Swe), and the United Kingdom (UK) to the Netherlands (NL) in the period March 13, 2020, to
July 1, 2020.

Table 1: Estimated relative differences in cumulative deaths during the first COVID-19 wave,
compared to observed number of deaths, when the response strategy from another country would
have been transferred to the Netherlands. The analysis covers the period February 2020, up to and
including June 2020, with counterfactual scenarios involving the transfer of the relative reduction in
reproduction number from different countries to the Netherlands from March 13, 2020, onwards. If
the multiplication factor is above 1, the number of deaths of the Netherlands would have increased
with the counterfactual response strategy, and if below 1, the number of deaths would have
decreased.

Response strategy
The Netherlands
Belgium
Denmark
Germany
Sweden
United Kingdom

Multiplication factor of cumulative deaths,
median (95% credible interval)
Reference (=1)
2.4 (1.8-3.0)
2.5 (1.7-3.5)
3.4 (2.6-4.3)
7.1 (5.6-8.9)
3.0 (2.4-3.7)

What if the Dutch control strategy would have started a few days later?
To assess whether the outcomes might be due to differences in timing of implementation of the
strategies, rather than the difference in strategies per se, we also assessed how the impact of the
first COVID-19 wave may have differed if the response strategy of the Netherlands had been delayed
by one day or by three days. To assess the consequences of delaying the Dutch response strategy,
we shifted the time-series of deaths used for computing the reproduction number Rt to one day or
three days later and interchanged the values of the relative reduction in the reproduction number
(Rt/R0) as explained for the international comparison.
Slight changes in the reproduction number Rt lead to substantial differences in absolute deaths. A
one-day delay in response measures would increase the peak number deaths to more than 200 per
day (Figure 5), while a three-day delay would have more than doubled the peak number of daily
deaths to 400. The number of deaths throughout the first wave was estimated to increase by a
factor 1.2 (95% credible interval [CrI]: 0.9-1.6) for a one-day delay (Table 2). For a three-day delay,
the number of deaths throughout the first wave significantly increased by factor 2.3 (95% CrI: 1.73.1).

Figure 5: Estimated median number of daily deaths with 95% credible intervals for the Netherlands,
showing the fit to observed data (blue bars) and the counterfactual analyses involving response
measures taken one day later or three days later in the period March 13, 2020, to July 1, 2020
Table 2: Estimated relative differences in cumulative deaths during the first COVID-19 wave,
compared to observed number of deaths, if the response strategy of the Netherlands would have
been delayed by one day or by three days. The analysis covers the period February 2020, up to and
including June 2020, with counterfactual scenarios involving the transfer of the relative reduction in
reproduction number from different countries to the Netherlands from March 13, 2020, onwards. If
the multiplication factor is above 1, the number of deaths of the Netherlands would have increased

with the counterfactual response strategy, and if below 1, the number of deaths would have
decreased.
Response strategy
The Netherlands
The Netherlands, one-day delay of response
The Netherlands, three-day delay of response

Multiplication factor of cumulative deaths,
median (95% credible interval)
Reference (=1)
1.2 (0.9-1.6)
2.3 (1.7-3.1)

This analysis provides a quantitative understanding of the possible impact that alternative COVID-19
response strategies would have had on mortality in the Netherlands during the first pandemic wave
in the period February 2020, up to and including June 2020. The analysis illustrates that in a rapidly
growing epidemic, relatively slight differences in timing of measures can result in large differences in
absolute number of deaths. In the initial phase of the epidemic, without control measures, the
incidence of new infections doubled every 2-3 days. Delaying the response strategy by three days
could therefore lead to more than a doubling in mortality during a single epidemic wave. As the
Netherlands was slightly earlier to suppress the epidemic wave (in the sense of bringing the
reproduction number Rt below the critical value of 1), compared the Belgium, Denmark, Germany,
Sweden and the UK, adopting the response strategies from these other countries would have led to
at least a doubling in mortality in the Netherlands. As the response strategy from Sweden was also
less strict, such an approach would have led to a seven-fold increase in mortality in the Netherlands.
The question is whether this outcome is specific to the choice of mortality as outcome measure, and
whether a similar ordering of strategies would result if we would have taken time series of
hospitalization. When comparing weekly hospitalizations rates by admission date during the first
wave between the Netherlands and Belgium, the Netherlands was also slightly earlier in scaling the
peak and decreasing the incidence of hospitalizations. Even though there are variations in admission
criteria and hospital bed capacities, which might complicate the interpretation of incidence of
hospitalizations, the order for strategies in which they achieved the peak incidence of new
hospitalizations aligns with the order for strategies in which they achieved the peak incidence of
mortality. This suggests the outcome of the counterfactual assessment using mortality time series
would have been similar the outcome of a counterfactual assessment with hospitalizations time
series.
Great care must be taken in distinguishing the counterfactual assessment where the time series of
relative reduction in reproduction numbers are transferred from one country to another from the
actual implementation a different strategy in another country. For example, a high incidence of
infection may trigger policy makers to increase stringency of the measures, the perceived severity of
infection and fear may affect compliance; also, trust in the government may play a role in
determining compliance and thus affect the outcome. Public support for more stringent measures
might be higher in a country when neighboring countries are introducing or have already introduced
stringent measures, as compared to a situation where a country is the first to impose stringent
measures. The six countries differed with respect to the incidence of infection when more
information became available about the SARS-CoV-2, the required control effort, and the severity of
infection. Countries with a higher level of incidence of infection at the start date of this study will
have, all other aspects remaining equal, a higher number of infections and severe outcomes over the
entire study period.

The effect of interventions in reducing number of contacts per day
When we assess the effectiveness of interventions on transmission, this implicitly suggests that the
intervention is a cause, and that the reduced transmission is an effect. For vaccines, the randomized
clinical trials provide evidence that there is a causal relation. For non-pharmaceutical interventions
there are no randomized clinical trials; a trial with a NPI during a pandemic might be considered
unethical. Because the effectiveness of a bundle of NPIs is achieved in part by reducing contacts, we
can see if there is evidence for an association between the intervention and the reduction of
contacts, and between the reduction of contacts and the reduction of transmission.

Non-pharmaceutical interventions (NPIs)
Unlike vaccines, NPIs do not go through randomized controlled trials (RCTs) to prove their
effectiveness before they are implemented. The estimates of their effectiveness are derived from
observational studies. The primary rationale of NPIs is to reduce transmission through reducing
contacts between individuals and reducing the probability of transmission upon contact. It is
essential to test whether they have achieved the reduction in contacts and reduction in probability
of transmission upon contact. In this setting, the contacts per day are termed ‘mediating variables,’
as they mediate between the cause and effect. The RIVM has set up studies to observe the effects of
interventions on contacts for different age groups, as mediating variables, and we report the results
here. If the reduction in contacts affects the transmission of SARS-CoV-2 we should also expect it
would affect the transmission of other infections that have a similar transmission route as SARS-CoV2; we report here on the surveillance of such infections.

Contact surveys reveal that contacts were reduced
Pienter Corona contact survey
The effect of intervention bundles on contact patterns in the Dutch population was monitored in
regular intervals. The RIVM conducted the Pienter Corona studies in which a representative sample
of several thousand persons from the Dutch population indicated how many contacts were made on
the previous day. Intervals between successive survey rounds were a few months, with the first
round in April 2020 and at the time of writing the most recent round in autumn 2023.
The results of this study show that the intervention bundles affected the age groups differently.
During the initial stages of the first wave, contacts of all age groups were drastically reduced (on
average 76%, Backer et al. 2020), but whereas the youngest children quickly reverted to their
prepandemic behaviour, adults have remained below their prepandemic levels by the autumn 2023.
In the survey rounds of November 2022 and May 2023 the weighted average number of contacts in
the general population is well below the prepandemic level. This indicates a ‘new normal’ behaviour
may have established, which could affect the transmission potential of not only COVID-19 but all
respiratory infections (Figure 6).

Figure 6 Number of community contacts (i.e., with persons outside the household) per participant per
day (weighted by age, sex, and week/weekend) over the course of the COVID-19 pandemic,
compared to the prepandemic baseline (Pienter3, 2016/2017). Shown is the average number of
contacts (horizontal line) and 95% bias-corrected bootstrap interval (shaded area). Study rounds are
shown from the minimum to maximum survey date, with the median survey date (vertical line). As a
reference the weekly number of hospital admissions and the Oxford Stringency Index are depicted on
top with the same timeline. Localized peaks of stringency (dark red) correspond with localized dips in
the number of contacts in the community per capita, and conversely, localized throughs in stringency
(blue) correspond with localized peaks in the number of contacts in the community per capita. (1
January 2020 – 30 June 2023).

COMIX contact survey
The CoMix study was conducted as part of the EU Horizon project EpiPose, with the aim of
monitoring contact patterns using the same methodology in various European countries. A panel of
Dutch people reported how many contacts were made with an interval between consecutive
reporting moments of two weeks.
The survey included 1659 participants from April to August 2020 and 2514 participants from
December 2020 to September 2021. We categorized the number of unique contacted persons
excluding household members, reported per participant per day into six activity levels, defined as 0,
1, 2, 3-4, 5-9 and 10 or more reported contacts. After correcting for age, vaccination status, risk
status for severe outcome of infection, and frequency of participation, activity levels increased over
time, coinciding with relaxation of COVID-19 control measures.
Because of the European design of the CoMix study, we can make an international comparison of
contact patterns. The same protocol and questionnaire were used in 20 European countries, with

the longest time series of contact patterns for England, Belgium, and the Netherlands. A final survey
round was conducted in these three countries November and December 2022 with 7,477
participants. Despite the number of contacts being higher compared to pandemic levels, they were
not back to the levels seen prior to the pandemic (Figure 7).

Figure 7. Relative change in typical number of contacts per person in the CoMIx survey in 2022
compared to a baseline contact survey in 2007, Polymod study, which measured contacts in Western
European countries, using the dominant eigenvalues of contact matrices to estimate a relative
change in R0 (after Jarvis et al. 2023). If there would have been no change in contacts per person
since 2007, the value would have been one, the observed values lower than one indicate a decreased
number of contacts in 2022 relative to 2007.

SCONE contact survey
Another survey monitoring contact patterns was specifically focused on contacts of the elderly. The
SCONE study, a ZonMw project conducted by RIVM, looked at how frailty affected the number of
contacts of persons aged 70 years and older, in two periods in 2021. During the first period in April
2021 when stringent measures were in place, frail and non-frail participants had significantly lower
numbers of contacts compared to the second survey period in October 2021 with moderate
measures. During this second survey period, non-frail participants had significantly more contacts
outside their household than frail participants. The change in number of contacts between the first
and second survey period was largest for the eldest non-frail participants. As these eldest non-frail
individuals may interact closely with highly aged and highly frail persons, a reduction in the number
of their contacts might offer indirect protection of frail elderly from SARS-CoV-2 exposure (Figure 8).

Figure 8: Weekly number of community contacts (i.e., persons contacted outside the household) per
participant by age in survey periods 1 (April 2021) and 2 (October 2021). Plots show the data (one
point for each participant) and model results (mean as solid line and 95% confidence interval as
shaded area) by frailty (columns) and gender (rows).

Side-effect of COVID-19 NPIs: a decline in transmission of other respiratory infections
Circumstantial evidence for the effectiveness of NPIs is provided by a drop in reported infections
with other seasonal respiratory viruses while measures were in place during the COVID-19 pandemic
(Middeldorp et al. 2020; Annual report Surveillance of COVID-19, influenza and other respiratory
infections in the Netherlands: winter 2021/2022 https://www.rivm.nl/bibliotheek/rapporten/20220098.pdf). If the NPIs are effective in reducing transmission of SARS-CoV-2, they should be effective
in reducing transmission of any other pathogen that has a similar transmission route as SARS-CoV-2.
Substantive reductions in transmission are expected for pathogens with a transmission potential
that is lower than SARS-CoV-2. Moreover, the typical seasonal dynamics of such pathogens might be
shifted in presence of NPIs. Examples of other respiratory pathogens with a lower reported
incidence of cases, and with altered seasonal dynamics include influenza virus, RSV and
metapneumovirus. The incidence of cases with invasive pneumococcal infections was also lower.
There is hardly any underreporting of cases with an invasive pneumococcal infection since the
infection is severe and results in hospitalization. The drop in reported incidence cannot be attributed
to a changing health-seeking behaviour during the pandemic and reflects reduced transmission. This
was not only the case in the Netherlands. Many countries around the world reported much less
influenza and RSV during the winters of 2020-2021 and 2021-2022 than in previous years. It is
difficult to demonstrate that interventions did not affect infections with other transmission routes,
as interventions could also have affected other transmission routes and health-seeking behaviour.

Contribution of individual interventions to overall effectiveness
Non-pharmaceutical interventions (NPI): evidence from literature
We reviewed literature to evaluate the empirical effectiveness of COVID-19 measures. Even though
there are thousands of articles on the topic, there are only a handful of reviews with acceptable to
high quality, which indicates a lack of synthesis of results. The evidence in this literature review
suggests that COVID-19 measures did in fact reduce the risk of transmission and the risk of
acquisition of infection. A precise quantification of the reduction in risk is methodologically
complicated. Most NPIs were implemented around the same time, bundled in packages, making it
difficult to disentangle the effect of the different interventions, as interventions influence each
other. Furthermore, the effect of the different NPIs is dependent on the timing of implementation
regarding the COVID-19 epidemiology, as well as contextual factors such as the demographic
composition of the population (e.g., age structure, but also household size and population density),
the social and political-economic situation, and cultural factors, including trust in the government
and compliance.
We found the following results for the different NPIs:
•
•
•
•

•

•

Hand hygiene: Most included studies showed a protective effect of handwashing. However,
when and how frequently hand hygiene should be performed was not clear.
Social distancing: Measures such as physical distancing, stay at home measures, restrictions
on (mass) gatherings, and lockdown were found to be effective in reducing transmission.
School closure: This was predominantly effective in reducing transmission and disease, but
effectiveness varied.
Travel restrictions: The literature provides mixed evidence. If it is effective, it is in delaying
an epidemic in the country receiving infections from another country that has an outbreak
or a large epidemic. Symptom-based or exposure-based screening was likely not effective.
Face masks: Multiple studies found a protective effect of wearing face masks, while other
studies show a less pronounced effect to no effect of wearing face masks in the community
on COVID-19 incidence. Studies, mainly in the healthcare setting, suggest that higher-quality
masks (e.g., N95 masks) were more effective than surgical masks.
(Digital) Contact tracing: Some studies point towards a decreasing effect of contract tracing
apps on transmission (quantified by Rt), incidence and mortality, while others found no
effect on COVID-19 epidemic control.

Appendix B gives the full text of the review.

Decomposing total effectiveness into contributions by individual measures:
international evaluation reports
Reports from various national and international organizations that evaluated the COVID-19 response
mention the methodological difficulties and state that evaluation of the individual effects of
measures may not be possible. The UK Chief Medical Officers write in their Technical Report (Whitty
et al, 2022) in Chapter 8: “It may never be possible fully to disentangle some of the effects of
individual NPIs in this pandemic as many were used together”. The German
Sachverständigenausschuss states on page 12 and 13 of their report (2022) “Die genaue Wirksamkeit
von Schulschließungen auf die Eindämmung der Ausbreitung des Coronavirus ist trotz biologischer
Plausibilität und zahlreicher Studien weiterhin offen, auch, weil im schulischen Bereich eine Reihe
von Maßnahmen gleichzeitig eingesetzt wurden und damit der Effekt von Einzelmaßnahmen nicht
evaluiert werden kann.” The OECD report (2023) states on page 130: “Understanding the relative

impact of different NPIs is of great interest to policy makers, as it provides a basis for calibrating the
public health response throughout different stages of a shock such as a pandemic. Studies use
advanced modelling and statistical techniques to evaluate the relative contribution of NPIs to
containing the spread of COVID-19. However, such analyses are difficult for technical reasons – for
example, the simultaneous implementation of multiple measures makes it difficult to disentangle
the relative contribution of each.” Below we look into a study that presents an approach to
disentangling the effects of individual NPIs and focus on the technical difficulties in more detail.

Decomposing total effectiveness into contributions by individual measures: estimates
derived from combined data from different countries
Many control measures were introduced and lifted at the same time, which results in poor
identifiability of effectiveness of individual control measures. Combining information from different
countries where different bundles of interventions were used alleviates this identifiability problem
but requires measures to be grouped into categories across countries. Effectiveness of categorized
individual interventions in reducing the reproduction number can then be estimated using a
regression approach. For example, Brauner et al. 2021 provide estimates of effectiveness for the first
pandemic wave, and Sharma et al. 2021 provide estimates of effectiveness for the second pandemic
wave. The latter publication uses a set of European countries that includes the Netherlands, at a
regional level (for the Netherlands, these are the safety regions). Comparing the estimates over
different period reveals that the estimated effectiveness of individual measures is dynamic, in the
sense that estimated values change over time as the context change. Especially the estimated
effectiveness of school closure was higher in the first wave than in the second wave; the high
effectiveness in the first wave is considered an artefact rather than a real effect. We present the
estimates of effectiveness obtained in the second pandemic wave by Sharma et al. (2021) in Table 3.
These estimates of effectiveness obtained in the second wave performed better in predicting the
effectiveness of measures in the third wave, as compared to estimates obtained in the first wave,
but as argued by the authors who report these estimates, these estimates cannot be generalized for
future use. As lessons learned from this approach, the authors state that the practical value of the
existing evidence is limited due to methodological issues and absence of a standardized assessment
practice (Lison et al. 2023).
Table 3. Intervention effectiveness in a multinational dataset that includes the Netherlands, as
percentage reduction in the reproduction number Rt, after Sharma et al 2021, Figure 2A.
Effectiveness has been estimated of NPIs in 114 regions of 7 European countries, including the
Netherlands (in Safety regions) over the period 1 August 2020 – 9 January 2021. Uses daily public
data on daily reported cases and deaths at low geographical resolution. Note that estimates differ
from earlier estimates using a similar method for a different set of countries over the period between
January 2020 and the end of May 2020 (Brauner et al 2021). The authors stress that these estimates
outperformed earlier, different estimates in predicting the effectiveness in the third wave, and that
the results on NPI effectiveness are dynamic in time. This implies that to inform crucial policy decision
should depend on real-time modelling of evolving NPI effects.
Intervention

Reduction in Rt (%)

All non-essential business
closed
All gatherings banned
All educational institutions
closed

35

Uncertainty range (95%
credible interval)
29 - 41

26
7

18 - 32
4 - 10

Nighttime curfew
Stricter mask wearing policy

13
12

6 - 20
7 - 17

The effectiveness attributed to individual interventions, as in Table 3, is obtained using a regression
approach. The main assumptions in this regression approach are that the effect of each intervention
is multiplicative, that there are no interaction effects between the interventions, and that all
relevant factors that would affect the reproduction number are accounted for. However, the
regression model does not include effects of compliance and the stringency of an intervention, it
does not include effects of contact tracing or seasonal variation in transmission. The regression
framework does account for mediating variables such as the daily number of contacts or a proxy
measure such as observed mobility. The regression approach assumes that the interventions are
independent (there is no multi-collinearity). In the subsequent sections we assess to what extent
violation of these assumptions would affect the estimated effectiveness of individual control
measures.

The effect of compliance on effectiveness
The regression approach that was used to obtain the estimates of effectiveness of individual
interventions as in Table 3 does not account for the compliance or the intensity of the measures. In
this section we analyse how compliance could affect the resulting estimates of effectiveness. We
focus on “stay-at-home requirements” as an NPI, specifically those that restrict the number of
visitors per household. We study the impact of introducing the stay-at-home measure on the
effective reproduction number and the proportion of the population that becomes infected during
an epidemic wave (infection attack rate) at various intensities of the measure. We use a
mathematical transmission model that allows for differences in compliance: a part of the population
will comply to the measures and another part of the population will continue its contact behaviour
irrespective of the measure. A reduction in contacts for the compliant sub-population always results
in reduced average number of contacts in the entire population. As expected, an intervention of a
moderate intensity is more effective in reducing the reproduction number than no intervention.
However, counterintuitively, an intervention of high intensity can be less effective in reducing the
reproduction number than an intervention of a moderate intensity (Figure 9, left panel). This
reduced effectiveness at high intensity arises because non-compliant individuals will visit noncompliant households, such that most infections are transmitted among the individuals who do not
comply. Similar outcomes for the reproduction number are expected for other NPIs where contacts
between complying and non-complying change at increasing intensity. We observe a similar but less
pronounced for the proportion of the population that becomes infected (Figure 9, right panel).
Throughout, the compliant sub-population is relatively unaffected by the behaviour of the noncompliant sub-population. The results also reveal that the effectiveness of a measure in reducing the
number of infections in an epidemic wave can be much higher than the effectiveness in reducing
reproduction numbers. These findings highlight the importance of monitoring the actual compliance
to measures and including it into the regression analysis, as compliance has a modulating effect on
the epidemiological outcomes of interest.

1.00

Effectiveness

Effectiveness

1.00
0.75
0.50
0.25
0.00

0

25

75

Intensity of the measure

% non-compliant
1
5
10
15
20

50

100

0.75
0.50
0.25
0.00

reference
homogeneous
mixing
threshold
for control

0

25

50

75

Intensity of the measure

(sub)population
total
compliant
non-compliant

100

reference
homogeneous
mixing

Figure 9: Effectiveness of intensity of the stay-at-home measure on the reproduction number (left)
and the proportion of the population that becomes infected during an epidemic wave (right). Results
are obtained with an epidemic transmission model where individuals belong to a household, and all
the households are compliant or not. The effectiveness of intensity of the stay-at-home measure is
evaluated at various levels of compliance.

The effect of seasonality on estimates of effectiveness
The regression approach that was used to obtain the estimates of effectiveness of individual
interventions as in Table 3 did not account for a seasonal fluctuation of the reproduction number.
The effect of these seasonal fluctuations was not firmly established at the time of the analysis in
2020-2021. Since then, a sufficient number of publications have appeared that allows us to establish
the effect size. We express the seasonal effect as the reduction in the reproduction number in
Summer compared to Winter. Early in the pandemic, Xu et al (2020) studied the relation between Rt
and meteorological factors by comparing many regions in the world, resulting in an estimate of
about 20% reduction in Summer in London and Paris as compared to Winter. Later studies looked at
observations throughout a complete year. Gavenciak et al (2021) took data from 143 locations in
Europe and estimated a 42% reduction in Rt in Summer compared to Winter. Johnsen et al (2022)
estimated the relation between Rt and temperature in Denmark and estimated that Rt was reduced
by 27%. For the Netherlands, we estimated a 26% reduction in the effective reproduction number Rt
in Summer compared to Winter. The magnitude of this effect of seasonality on the reproduction
number Rt is large in comparison to the attributed effectiveness of individual interventions. This
implies that seasonality should be accounted for in a statistical regression approach to estimate
effectiveness, and that ignoring the effect of seasonality could result in substantially biased results.

The effectiveness of contact tracing
The regression approach that was used to obtain the estimates of effectiveness of individual
interventions as in Table 3 did not account for contact tracing. We estimate the effectiveness of
contact tracing by making a mathematical model that describes the control measure in detail. For
instance, we estimated the effectiveness of the contact tracing app CoronaMelder by a detailed
simulation model that combined all available knowledge and data on epidemiology and behaviour.
For instance, the model included epidemiological details on infectivity, incubation period, contact

frequencies, behavioural details on willingness to test, go into quarantine, or use the tracing app,
and technical details on the logistics of testing and tracing such as the time between making a test
appointment and receiving the test results. The effectiveness of CoronaMelder was measured as a
reduction in the reproduction number Rt, which is the mean number of secondary infections per
primary infected. It is the mean reduction, measured in a population where some people strictly
adhere to all measures and others not at all. According to this model, the CoronaMelder app
resulted in a 0.3% reduction in the reproduction number, up to 0.8% in subgroups with many app
users. Contact tracing, in the baseline scenario, resulted in a 6.5% reduction in the reproduction
number. The small magnitude of this effect of contact tracing, relative to the attributed
effectiveness of individual interventions, implies that not accounting for contact tracing in statistical
regression analysis could results in limited bias of the results.

The complementarity of interventions
The regression approach that was used to obtain the estimates of effectiveness of individual
interventions as in Table 3 did not account for interactions between effects of interventions. As such
interactions are to be expected, estimating the magnitude of these interactions would be helpful for
assembling individual interventions into effective bundles. However, it is statistically challenging to
estimate both the effectiveness and the interactions of individual interventions. The interactions
provide a way to quantify how the effectiveness of interventions is determined by the context (the
other interventions that are implemented at the same time). The variation over time in estimated
magnitude of effectiveness of individual interventions suggests that such interactions do exist, and
that their effect is relevant. That quantification is hard does mean that interactions are unimportant
or that context is irrelevant.

Mediating variables
The regression approach that was used to obtain the estimates of effectiveness of individual
interventions as in Table 3 did not account for the number of reported contacts per day. A statistical
approach that may be appropriate for assessing the impact of public health measures while
accounting for this variable could be along the lines of a method known as path analysis. This is a
regression-based approach that can be used to statistically separate explained variation and to
examine the causal chain between public health interventions, measured mediating variables (e.g.,
number of contacts per day), and the effective reproduction number, while adjusting for potential
confounding factors (e.g., seasonality or temperature). The working assumption of this approach is
that the impact of intervention measures on the course of the epidemic can be observed in the
effective reproduction number (as calculated from the time-series of reported positive tests and/or
hospital admissions for COVID-19). A major difficulty with this (and related modelling approaches)
remains the disentangling of the effects of measures that had been introduced simultaneously. In
other words, attribution of the potential impact on transmissibility from overlapping potential
causes remains highly challenging. This kind of analysis is only viable for analysis of time periods
before the launch of the vaccination programme, as effective vaccination can have large and
prolonged direct and indirect effects on the course of the epidemic, and before the arrival of virus
variants that (partially) escape natural immunity.

Vaccines: evidence from literature
Vaccines go through an extensive process of testing, including randomized controlled trials (RCTs) to
prove that protective effects are actually caused by the vaccine. For personal protection, the
effectiveness of vaccination can be quantified as effectiveness against infection, COVID-19 disease,
hospitalization, or death due to COVID-19. The rationale of vaccination in infection control is to
reduce person-to-person transmission by reducing the risk of infection and subsequent transmission

upon exposure. Estimates of the effectiveness of vaccination against infection and transmission tend
to be higher than effects of NPIs, and the effectiveness lasts for considerable duration (e.g., Wu et al
2023).
We reviewed evidence for vaccine effectiveness, and briefly sketch the qualitative patterns that
emerged from this review. Vaccine effectiveness (VE) estimates are different for the different virus
variants, vaccine types, populations, and outcomes assessed. In general, a very high VE (> 95%) was
found against severe disease, especially shortly after vaccination, for all variants, vaccines and
populations. Against mild disease or infection, the VE was a bit lower but still high (> 80%), although
the estimate for the Omicron variant was substantially lower (around 50%). The estimated VE
against transmission for non-Omicron variants was a bit lower than the VE against severe or mild
disease (50-70%). For all outcomes, the VE decreased within weeks or months after vaccination;
waning was slower for VE against severe disease as compared to VE against the other outcomes. The
estimates of VE were high again shortly after booster vaccination. Appendix B gives the full text of
the review.

Discussion
We address the following questions: How effective were bundles of COVID-19 measures in reducing
SARS-CoV-2 transmission? How did individual measures contribute to this effectiveness? How does
this translate into effectiveness against severe measures, such as hospitalisations or deaths? We
assessed the effectiveness and established increasing effectiveness against transmission was
associated with increasing stringency of control measures, and conversely, decreasing effectiveness
against transmission was associated with decreasing stringency of control measures. We established
that the effectiveness against severe outcomes such as death is determined by effectiveness against
transmission as well as the timing of measures: delaying the bundle of control measures in March
2020 would have resulted in more severe outcomes. We have shown that the total effectiveness
against transmission can be decomposed into contributions of individual control measures under
specific assumptions, but that these assumptions are not always met, which means that the
estimates are biased and can change over time.
We briefly discuss the effectiveness of measures against COVID-19.
•

•

Vaccination. This is the control measure for which we have randomized controlled trials
(RCTs) and good estimates of effectiveness, independent of other measures. Of these
estimates, the best documented is the effectiveness against severe outcomes such as death
and hospitalization; the effectiveness against transmission of SARS-COV-2 is harder to
measure in RCTs but seems reasonably high.
NPIs. We do not have RCTs but there are observational studies that reveal associations
between implementation of interventions and increased effectiveness in reducing
transmission. We verify that the interventions have affected the contact behaviour in the
population as intended, and reduced transmission of infections with a similar transmission
route as SARS-COV-2.

We have abstained from balancing effectiveness of measures versus unintended societal
consequences of these measures. The Royal Society report on the effectiveness of NPIs (2023)
asserts that balancing effectiveness versus unintended societal consequences is a political question,
not a scientific one. Science can assist in clarifying which decisions are to be made: If the objective is
to keep numbers of infection below a given threshold value set by a policy maker, science can
provide the average effectiveness that a bundle of measures should attain, as determined by
properties of the pathogen and the population. The policy question is how to compose the package
of measures such that the unintended consequences remain limited. Scientific expert groups (such
as the Health Council, the MIT (Maatschappelijk Impact Team) and the Outbreak Management Team
in the Netherlands) could assist in identifying tradeoffs and bringing consequences of measures to
the attention of policy makers. We agree with the Royal Society in the sense that scientists should
bring relevant expertise and evidence to the table and be careful to avoid being cast into a role of
weighing values that is up to policy makers.
We have not discussed the effectiveness of interventions in specific settings, including nursing
homes, hospitals, restaurants, pubs, night clubs, concert venues and shops. Here we faced an
absence of data to provide evidence for effectiveness. This does not imply absence of effectiveness
of those measures.
We have presented results of observational studies on effectiveness of bundles of control measures
in reducing transmission, the effectiveness in reducing contacts, and the breakdown by individual
measures. The take-home message of all of these, taken together over many different countries, is

that there is clear evidence that the bundles of NPIs and the combined effect of vaccination and NPIs
were effective in reducing SARS-CoV-2 transmission. The observational studies also show that
individual NPIs were effective in reducing transmission. The question how effective individual NPIs
were, and how effective they were in combination with others, is difficult to answer as the
effectiveness is determined by the context in which they were applied, such as their timing relative
to epidemic, the intensity of the measure, the compliance in the population, and the
complementarity to other interventions. Irrespective of such difficulties, the comprehensive
literature review and individual studies presented here provides additional information about the
effectiveness of individual measures that were taken in the COVID-19 pandemic.
We have stressed several methodological issues that limit the usefulness of existing estimates of
effectiveness of individuals interventions in a future pandemic. This is in line with the studies that
use multi-country data and regression approaches to obtain the existing estimates; most of these
studies are accompanied by extensive investigations into the substantial uncertainty surrounding
estimates of individual interventions, and sensitivity analyses form an essential part of these studies
(although often reported in supplementary information). Information of effectiveness in subgroups
in the population, based on ethnicity or socio-economic status or religion, is not available.
In summary, we provide evidence that the applied measures, both vaccines and bundles of NPIs,
were effective in reducing the transmission of SARS-CoV-2. How this reduction in transmission
translates into reduction of numbers of cases, hospitalizations and deaths will depend on the timing
of the intervention and the period over which the effectiveness is evaluated. The effectiveness of
individual interventions depends on the specific context in which they are implemented. Attributing
effectiveness to individual interventions, when possible, has limitations for future use.

Recommendations
Composing effective bundles of interventions
In trying to label an intervention with a single effectiveness estimate, one misses several other
important aspects of the interventions: communication about the interventions, complementarity of
interventions, and the objective of an intervention. We will highlight briefly three recommendations
that follow from this.
•

•

•

Communication about interventions is a relevant element of effectiveness that deserves
further attention. Clear messaging is essential to ensure proper compliance with NPIs.
Consistency in messaging may clash with the continuous updating of information based on
ongoing scientific research into a novel pathogen. An example is case isolation and contact
tracing: this generates information on the incubation period, which is an integral part of the
isolation procedure such that updated values for the incubation period can easily cause
confusion. Clear and consistent messaging should allow for updates in the information that
is communicated.
Complementarity of interventions deserves more attention. A first start at quantifying
complementarity of interventions would be to test for interactions between interventions in
the regression analysis for attributing effectiveness to individual interventions. Another
approach would be to map out the different at-risk contacts in a population by age groups
and settings and reconstruct from contact surveys, after introduction of a control measure,
which of the contacts were prevented or replaced by other contacts. Understanding the
complementarity is essential to composing effective bundles of interventions.
The objective of individual interventions deserves further attention. Interventions can be
designed for specific sub-populations or risk-groups and can target distinct parts of the
transmission chain. They could aim to reduce the risk of introduction of infection into a
population, reduce the risk of transmission of infection in that population toward individuals
who are at increased risk of a severe outcome, and to reduce the risk of infection of
individuals who are at increased risk of a severe outcome during a potential exposure.
Interventions such as screening and triage are relevant for preventing introduction, the use
of personal protection is relevant for reducing risk of exposure. Composing effective bundles
of interventions, aimed at specific populations, could benefit from recognizing the specific
role and objective of individual interventions.

Preparedness of data streams
Monitoring and surveillance of infectious disease in the Dutch population is needed in real time to
underpin effective responses and adaptations to changing conditions during a pandemic. Here it is
essential to collect information for different sub-populations (based on, for example, age, gender,
ethnicity). During the COVID-19 pandemic data on ICU admissions was used in the Netherlands
because all admitted patients were tested for SARS-CoV-2, surveillance was timely, and the number
of occupied ICU beds were perceived to be a bottleneck in healthcare. But such surveillance is not
standard, and for a different infection the surveillance could be focused on other groups.
•

Data for establishing the incubation period and the time scale of transmission of the new
pathogen are essential to assess the reproduction number and to determine the required
control effort in a new pandemic. Suitable data for estimating the incubation period requires
reporting for the same case the approximate moment of infection and symptom onset date.
Suitable data for estimating the time scales requires reporting for the same case the most
likely infector, the moment of infection or symptom onset date. During the COVID-19

•

•

pandemic these variables could be inferred from well-documented outbreaks, from contact
tracing reports and the OSIRIS registration system. This should be routinely built into
surveillance and monitoring systems for other pathogens.
Contact surveys in the population proved essential during the COVID-19 outbreak in the
Netherlands to check how NPIs worked out and were critical to inform policy decisions. We
need a baseline assessment how many at-risk contacts there are in the population,
preferably broken down by demographics of those involved in the contacts, the setting of
the contact, the duration of the contact, the location of the contact. This is helpful in finding
out which of these contacts can be prevented, or which of these contacts has a lower risk of
transmission, when a control measure would be implemented. Such contact surveys should
be structurally funded to provide a baseline measurement when a new pandemic starts.
Mobility, as measured by aggregated and anonymized mobile phone data, has been
identified as a useful proxy for the response of the population to NPIs in many European
countries. In the Netherlands, it has not been possible to collect and use this information. If
the concerns about possible infringement of privacy sensitive data of the anonymized
mobile phone data outweigh the expected societal benefits in the Netherlands, there should
be room for an alternative approach (for example, mobility as monitored in the National
Mobility Panel with an app such that a participant consents to sharing information). Such
monitoring instruments are not structurally available for infectious disease surveillance, this
should be ready before a new pandemic starts.

References
Backer JA, Mollema L, Vos ER, Klinkenberg D, van der Klis FR, de Melker HE, van den Hof S, Wallinga
J. Impact of physical distancing measures against COVID-19 on contacts and mixing patterns:
repeated cross-sectional surveys, the Netherlands, 2016-17, April 2020 and June 2020. Euro Surveill.
2021 Feb;26(8):2000994. doi: 10.2807/1560-7917.ES.2021.26.8.2000994. PMID: 33632374; PMCID:
PMC7908067.
Brauner JM, Mindermann S, Sharma M, Johnston D, Salvatier J, Gavenčiak T, Stephenson AB, Leech
G, Altman G, Mikulik V, Norman AJ, Monrad JT, Besiroglu T, Ge H, Hartwick MA, Teh YW,
Chindelevitch L, Gal Y, Kulveit J. Inferring the effectiveness of government interventions against
COVID-19. Science. 2021 Feb 19;371(6531):eabd9338. doi: 10.1126/science.abd9338. Epub 2020 Dec
15. PMID: 33323424; PMCID: PMC7877495.
Gavenčiak T, Monrad JT, Leech G, Sharma M, Mindermann S, Bhatt S, Brauner J, Kulveit J. Seasonal
variation in SARS-CoV-2 transmission in temperate climates: A Bayesian modelling study in 143
European regions. PLoS Comput Biol. 2022 Aug 26;18(8):e1010435. doi:
10.1371/journal.pcbi.1010435. PMID: 36026483; PMCID: PMC9455844.
Jarvis CI, Coletti P, Backer JA, Munday JD, Faes C, Beutels P, Althaus CL, Low N, Wallinga J, Hens N,
Edmunds WJ. Social contact patterns following the COVID-19 pandemic: a snapshot of postpandemic behaviour from the CoMix study. MedRxiv 2023.08.29.23294767
https://doi.org/10.1101/2023.08.29.23294767
Johnsen MG, Christiansen LE, Græsbøll K. Seasonal variation in the transmission rate of covid-19 in a
temperate climate can be implemented in epidemic population models by using daily average
temperature as a proxy for seasonal changes in transmission rate. Microb Risk Anal. 2022
Dec;22:100235. doi: 10.1016/j.mran.2022.100235. Epub 2022 Oct 8. PMID: 36248679; PMCID:
PMC9546506.
Lison A, Banholzer N, Sharma M, Mindermann S, Unwin HJT, Mishra S, Stadler T, Bhatt S, Ferguson
NM, Brauner J, Vach W. Effectiveness assessment of non-pharmaceutical interventions: lessons
learned from the COVID-19 pandemic. Lancet Public Health. 2023 Apr;8(4):e311-e317. doi:
10.1016/S2468-2667(23)00046-4. PMID: 36965985; PMCID: PMC10036127.
Middeldorp M, van Lier A, van der Maas N, Veldhuijzen I, Freudenburg W, van Sorge NM, Sanders
EAM, Knol MJ, de Melker HE. Short term impact of the COVID-19 pandemic on incidence of vaccine
preventable diseases and participation in routine infant vaccinations in the Netherlands in the period
March-September 2020. Vaccine. 2021 Feb 12;39(7):1039-1043. doi: 10.1016/j.vaccine.2020.12.080.
Epub 2021 Jan 6. PMID: 33478793; PMCID: PMC7787078.
Mishra S, Scott JA, Laydon DJ, Flaxman S, Gandy A, Mellan TA, Unwin HJT, Vollmer M, Coupland H,
Ratmann O, Monod M, Zhu HH, Cori A, Gaythorpe KAM, Whittles LK, Whittaker C, Donnelly CA,
Ferguson NM, Bhatt S. Comparing the responses of the UK, Sweden and Denmark to COVID-19 using
counterfactual modelling. Sci Rep. 2021 Aug 11;11(1):16342. doi: 10.1038/s41598-021-95699-9.
PMID: 34381102; PMCID: PMC8358009.
OECD (2023), Ready for the Next Crisis? Investing in Health System Resilience, OECD Health Policy
Studies, OECD Publishing, Paris, https://doi.org/10.1787/1e53cf80-en.

The Royal Society. COVID-19: examinnig the effectiveness of non-pharmaceutical interventions.
August 2023. https://royalsociety.org/topics-policy/projects/impact-non-pharmaceuticalinterventions-on-covid-19-transmission/
Sachverständigenausschuss. Evaluation der Rechtsgrundlagen und Maßnahmen Evaluation der
Pandemiepolitik. Published June 2022 https://www.tagesschau.de/gutachten-sachverstaendigenratcorona-101.pdf
Sharma M, Mindermann S, Rogers-Smith C, Leech G, Snodin B, Ahuja J, Sandbrink JB, Monrad JT,
Altman G, Dhaliwal G, Finnveden L, Norman AJ, Oehm SB, Sandkühler JF, Aitchison L, Gavenčiak T,
Mellan T, Kulveit J, Chindelevitch L, Flaxman S, Gal Y, Mishra S, Bhatt S, Brauner JM. Understanding
the effectiveness of government interventions against the resurgence of COVID-19 in Europe. Nat
Commun. 2021 Oct 5;12(1):5820. doi: 10.1038/s41467-021-26013-4. PMID: 34611158; PMCID:
PMC8492703.Lison Lancet Public Health
Whitty C et al. Technical report on the COVID-19 pandemic in the UK: A technical report for future
UK Chief Medical Officers, Government Chief Scientific Advisers, National Medical Directors and
public health leaders in a pandemic. Published 1 December 2022
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file
/1128099/Technical-report-on-the-COVID-19-pandemic-in-the-UK-PRINT.pdf
Wu N, Joyal-Desmarais K, Ribeiro PAB, Vieira AM, Stojanovic J, Sanuade C, Yip D, Bacon SL. Longterm effectiveness of COVID-19 vaccines against infections, hospitalisations, and mortality in adults:
findings from a rapid living systematic evidence synthesis and meta-analysis up to December, 2022.
Lancet Respir Med. 2023 May;11(5):439-452. doi: 10.1016/S2213-2600(23)00015-2. Epub 2023 Feb
10. PMID: 36780914; PMCID: PMC9917454.Middeldorp et al 2020
Xu R, Rahmandad H, Gupta M, DiGennaro C, Ghaffarzadegan N, Amini H, Jalali MS. Weather, air
pollution, and SARS-CoV-2 transmission: a global analysis. Lancet Planet Health. 2021
Oct;5(10):e671-e680. doi: 10.1016/S2542-5196(21)00202-3. PMID: 34627471; PMCID: PMC8497024.

Appendix A studies on which this report is based
Publications in peer-reviewed scientific journals
Backer JA, Bogaardt L, Beutels P, Coletti P, Edmunds WJ, Gimma A, van Hagen CCE, Hens N, Jarvis CI,
Vos ERA, Wambua J, Wong D, van Zandvoort K, Wallinga J. Dynamics of non-household contacts
during the COVID-19 pandemic in 2020 and 2021 in the Netherlands. Sci Rep. 2023 Mar
30;13(1):5166. doi: 10.1038/s41598-023-32031-7. PMID: 36997550; PMCID: PMC10060924.
https://www.nature.com/articles/s41598-023-32031-7
Backer JA, van de Kassteele J, El Fakiri F, Hens N, Wallinga J. Contact patterns of older adults with
and without frailty in the Netherlands during the COVID-19 pandemic. BMC Public Health. 2023 Sep
20;23(1):1829. doi: 10.1186/s12889-023-16725-1. PMID: 37730628; PMCID: PMC10510272
https://bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-023-16725-1
Leung KY, Metting E, Ebbers W, Veldhuijzen I, Andeweg SP, Luijben G, de Bruin M, Wallinga J,
Klinkenberg D. Effectiveness of a COVID-19 contact tracing app in a simulation model with indirect
and informal contact tracing. Epidemics, in press. medRxiv 2023.06.15.23291010; doi:
https://doi.org/10.1101/2023.06.15.23291010

Manuscripts in preparation
Jantien A Backer, Cheyenne C. E. van Hagen, Eric R. A. Vos, Hester E. de Melker, Fiona R. M. van der
Klis, Jacco Wallinga. Contact behaviour before, during and after the COVID-19 pandemic in the
Netherlands. Manuscript in preparation.
Pieter de Boer, Fuminari Miura, Jacco Wallinga. Counterfactual modelling of COVID-19 Response
Strategies: A comparison between the Netherlands and Five European Countries. Manuscript in
preparation.
Jantien A Backer. Effectiveness of non-pharmaceutical measures in reducing SARS-CoV-2
reproduction numbers in the Netherlands, 2020-2022. Manuscript in preparation.
Ka Yin Leung, Jacco Wallinga. Unintended outcomes of non-pharmaceutical interventions for
pandemic control: how they may arise, and how they can be prevented. Manuscript in preparation.

Appendix B Evaluation of the epidemiological effectiveness of COVID
measures: a literature review
RIVM/CIb/EPI
Version 19 December 2023

Literature search
Non-pharmaceutical interventions
An extensive amount of papers are published since the start of the COVID-19 pandemic. When
searching scientific medical literature for COVID-19 and non-pharmaceutical interventions (NPIs),
approximately 8,500 articles were found. When adding the pharmaceutical intervention
‘vaccination’ to the search terms, about 60,000 extra articles were found. To obtain an overview of
this immense field, our search was restricted to reviews and meta-analyses on NPIs, in English and
Dutch, focusing on Northern, Western and Southern European countries. NPIs and outcomes
included in the search can be found in the appendix C. A literature search was conducted in Embase
by an experienced library information specialist on June 8, 2023. The complete search is available on
request. Using strict search terms for the measures and outcomes to include, 140 review articles on
NPIs implemented during the COVID-19 pandemic in the Netherlands were identified. An updated
search on October 11, 2023 resulted in 7 extra review articles on NPIs. Next to that, the Cochrane
library was searched using the keyword ‘COVID’, resulting in 96 papers. Additionally, the six review
articles that formed the basis of the report on the effectiveness of NPIs of The Royal Society in the
UK1 were assessed. Selection criteria included the use of empirical data on COVID-19 and reporting
of quantitative results. No formal quality assessment was done, but quality of the retrieved articles
was deemed variable. After careful review of titles, abstracts and full texts of these 249 articles, 16
were included in the final selection, assessed to be of moderate to high quality and fulfilling the
selection criteria. Three more articles were added after reference checking, for a total of 19 review
articles (Table 1).

Vaccination
Vaccination is a pharmaceutical intervention. The vaccine effectiveness (VE) of COVID-19 vaccination
is extensively researched by the RIVM for the Dutch situation. VE is a measure indicating the
protection provided by vaccination; by comparing the occurrence of, e.g., infection, hospitalization
or death from COVID-19 in a vaccinated group to an unvaccinated group. RIVM uses vaccination
coverage data, surveillance data, and dedicated studies to monitor vaccine effectiveness against
SARS-CoV-2 infection, transmission, and severe COVID-19 (i.e., hospitalization and intensive care
unit (ICU) admission, and death). 2, 3 The RIVM also structurally keeps track of the scientific literature
regarding VE of COVID-19 vaccination. An ongoing weekly systematic search by the International
Vaccine Access Center of the Johns Hopkins Bloomberg School of Public Health, the World Health
Organization and the Coalition for Epidemic Preparedness Innovations was checked and summarized
for internal use. 4 Furthermore, online resources such as LitCovid (containing articles from PubMed
categorized by research topic) 5, EPPIcentre (a living map of research articles on COVID-19) 6 and the
COVID-19 rapid reviews conducted by the UK Health Security Agency (UKHSA) 7 were monitored on a
regular basis in the beginning of the pandemic. Next to that, Embase was searched weekly by

experienced library information specialists of the RIVM. A summary of the results of studies on VE in
the Netherlands, combined with results from international literature, is presented in this review.

Results
NPIs
Hand hygiene
When pooling results from 3 observational studies on the effectiveness of hand washing using a
random effects model and adjusting for heterogeneity, Talic et al. 8 found an non-statistically
significant reduction of 53% on incidence of SARS-CoV-2 infection (relative risk (RR) 0.47, 95% CI
0.19;1.12). i The unadjusted model showed a similar, but significant reduction (0.49, 95% CI
0.33;0.72). The included studies differed in study design (two case-control studies and one crosssectional survey) and definition of hand washing (often vs. sometimes, yes vs. no, hand washing
after specific activities), making it difficult to combine results. Risk of bias of the included studies
ranged from moderate to serious or critical. For example, other (risk) behaviour of included
individuals should be taken into account to avoid confounding. A review by Gozdzielewska et al. 9,
summarizing results from eight observational studies (mostly from the year 2020) on hand washing
and COVID-19, is in line with this, with most included studies showing a (statistically significant)
protective effect. However, similar limitations as described above hold, and all studies were assessed
to have an unclear or high risk of bias. Also, when and how frequently hand hygiene should be
performed was not clear. Nevertheless, the protective effect of hand washing in the community
setting is well established for other respiratory and gastrointestinal infectious diseases. 9-12

Social distancing
Social distancing is a term used to describe multiple measures. All measures that aim to limit
contacts between people fall under this definition, e.g., physical distancing, stay-at-home
requirements, lockdown, school closure, workplace closure, restrictions on gathering size, etc. Some
of the included studies focus on individual measures, while others evaluated social distancing in the
broad sense.
Murphy et al. 13 conclude in their review that social distancing measures were the most effective of
all NPIs, with more stringent measures having greater effects. Although no pooled measure is
available, Sun et al. 14 report a lower transmission risk related to physical distancing and prohibition
of mass gatherings by combining results from seven observational studies. Liu et al. 15 analysed data
from seven observational studies using contact patterns before and during initial mitigation (defined
as national and/or regional declaration of lockdown and the most stringent measures in spring 2020)
to extrapolate to changes in the effective reproduction number (Rt) ii. They found a 62-83% reduction
in Rt due to changed contact patterns. It should be noted that contact patterns used for the preCOVID period in several cases were from many years earlier.

Relative risk (RR): ratio of the probability of an outcome (e.g., SARS-CoV-2 infection) in an exposed group
(e.g., those who wash their hands according to the guidelines) to the probability of an outcome in an
unexposed group (e.g., those who do not wash their hands according to the guideline). A RR below 1 indicates
a protective effect of the ‘exposure’ or NPI. Thus, a RR of 0.47 means that the intervention (handwashing)
reduces the risk of SARS-CoV-2 infection with ((1 - 0.47) * 100%) 53%.
ii
Effective reproduction number (Rt): average number of secondary cases per index case on a given timepoint,
i.e., how many people are on average infected by someone who is infected with SARS-CoV-2.
i

Physical distancing
In the review and meta-analyses of Talic et al. 8, physical distancing was found to be effective in
reducing incidence of SARS-CoV-2 infection, with a pooled RR of 0.75 (95% CI 0.59;0.95) based on
five observational studies. It is not mentioned what physical distances were studied. Three studies
not included in the meta-analysis all reported a decreasing effect on either transmission, Rt and
mortality. Chu et al. 16 pooled data from 5 observational studies on physical distance and risk of
COVID-19 infection using a random effects model and found a RR of 0.15 (95% CI 0.03;0.73).
Distances differed between the different studies, ranging from 0 to 1.8 metres. In a further analysis
by Chu et al., also including papers on other pathogens (MERS, SARS, and COVID-19), protection was
increased as distance was increased. Murphy et al. 13 included 34 papers in their review on physical
distancing (of which 19 were observational), and all but one found it effective. However, no pooled
quantitative measure was available.
Restrictions on mass gatherings
Twenty-eight studies were included in Murphy et al. 13 that focused on mass gathering restrictions,
of which all but two reported a substantial reduction in the impact of COVID-19. The smaller the
maximum gathering size, the greater the reduction in transmission. Furthermore, the effectiveness
of the measure seemed to increase over time. However, the time period considered was relatively
short (first half of 2020), thus not taking into account that adherence could have waned if the
restrictions were implemented for longer periods, e.g., in subsequent waves.
Stay-at-home requirements
Stay at home or isolation measures were assessed in the review of Talic et al. 8 All four observational
studies included in the review reported reductions in transmission of SARS-CoV-2. For example, in an
observational study in the US these measures were found to have contributed to a 51% reduction in
Rt (95% CI 46;57). In a observational study in the UK Rt pre-intervention was 3.6, and decrease to 0.6
(95% CI 0.37;0.89) post-intervention. In Iran, similar results were found: 2.70 and 1.13 (95% CI
1.03;1.25), respectively. The effect of quarantine on incidence and transmission was assessed in two
included studies. 8 A 4.9% decrease in the incidence of COVID-19 eight weeks after the
implementation was reported in an observational study from Saudi Arabia. An observational study
from India reported a 14 times higher risk of SARS-CoV-2 transmission associated with no quarantine
compared with strict quarantine.
Lockdown
A universal lockdown was found to have a decreasing effect on incidence of COVID-19 in three
observational studies included in the review of Talic et al. 8 with reductions ranging from 11% to
14%. Three observational studies assessed the effect on mortality, all reporting a decrease, although
one was not statistically significant. In four observational studies looking at the effect on
transmission, the (absolute) decrease in Rt ranged from 1.27 to 3.97 in three studies, and a relative
decrease in Rt of 11% was found in the fourth. Sun et al. 14 give an overview of articles on the
effectiveness of lockdown and report similar reductions in Rt and incidence, but no pooled measure
is available. The same holds for Vardavas et al. 17. Murphy et al. 13 included 151 studies, of which 119
found a substantial reduction in Rt, incidence of SARS-CoV-2 and mortality of stay-at-home orders or
lockdowns. The included studies differed in design, population and definitions and stringency of
stay-at-home orders. Reductions in Rt were estimated around 50%, however, the range was wide
(6% to 81%). Also, several studies found that the additional effect of lockdown on top of other
measures (such as school and business closures, and restrictions on gathering size) was small, with

one study estimating Rt to drop by 13% (95% prediction interval 5;31). The effectiveness of stay-athome orders on incidence and mortality were more mixed, with over one third of the included
studies reporting no significant association.
Closure of schools, businesses and workplaces
School closure as an individual measure was assessed in only one paper from Sweden included in the
Cochrane review by Krishnaratne et al. 18 They found exposure to open rather than closed schools
resulted in a small to moderate increase in the number of infections among parents and teachers,
and their partners. Quantitative results are not reported in the review. Sun et al. 14 describe an
predominantly decreasing effect of school closure and workspace measures on COVID-19 incidence
and mortality. School closure was also assessed in five observational studies included in the review
of Talic et al. 8 A US study reported a 62% decrease on incidence of COVID-19 infections and a 58%
decrease in COVID-19 mortality after state-wide closure of primary and secondary schools, while in a
study in Japan no effect of school closure on incidence was seen. Two other studies in the US found
a 10-13% reduction in Rt after school closure. Murphy et al. 13 included 104 studies on school
closure, and over half of these studies showed evidence for the effectiveness. However, the impact
varied due to differences in study populations, study period, the timing of implementation and the
interaction with other NPIs. Thirteen studies examined the impact of reopening schools, of which
nine showed an increasing trend in the number of daily new confirmed COVID-19 cases, growth rate
or Rt. The other four studies observed no substantial increase in transmission when other measures
were in place.
Business closure was assessed in two observational studies from the US included in the review of
Talic et al. 8 and they observed a 12-16% reduction in transmission after business closure. Thirtyseven studies were included in the review by Murphy et al. 13 on workspace closure. Most studies
(92%) observed a beneficial effect of workplace closures alone or in combination with other
interventions to reduce incidence. However, the sizes of these effects are not reported.

Travel restrictions
Travel restrictions were studied in a Cochrane review by Burns et al. 19 Symptom/exposure-based
screening at borders was likely not effective, with eight out of nine observational studies reporting
that the proportion of cases detected would be less than 54%. Five studies observed that the
proportion of cases detected through test-based screening varied from 58% to 90%. One
observational study included in the review of Talic et al. 8 came to a similar conclusion that screening
for fever lacked sensitivity (ranging from 18% to 24%) in detecting people with SARS-CoV-2 infection.
Four observational studies assessed the impact of quarantine and screening at borders, and found
that the proportion of cases detected ranged from 68% to 92%.19 Another Cochrane review20
indicated that evidence of empirical studies is scarce, but the included (empirical and modelling)
studies point towards a reduction of incidence and mortality of COVID-19 due to travel restrictions,
especially for quarantine of individuals travelling from a country with a declared COVID-19 outbreak.
However, the magnitude of the effect is uncertain. Similar conclusions were drawn by Grepin et al. 21
Although no quantitative measures were mentioned, they conclude that symptom/exposure based
screening had no had no significant effect on reducing importation or transmission. Targeted
restrictions such as banning entry from specific countries likely had a moderate effect on
transmission, especially in the beginning of the pandemic, while quarantine at entry borders was
assessed to be the most effective.

Two observational studies included in the review of Talic et al. 8 had opposing results regarding the
impact of border closures on Rt: in African countries border closures had minimal effect on the
incidence of COVID-19, while in the US restrictions on travel between states were estimated to
reduce SARS-CoV-2 transmission by about 11%. Bou-Karroum et al. 22 included 15 observational
studies on effectiveness of travel restrictions. Travel restrictions included amongst others border
closure, screening of travellers, quarantine of travellers, and restriction on international or regional
movement. Although quantitative outcomes were hardly reported and it was unclear which results
were derived from empirical studies (as opposed to modelling studies), travel measures seemed
predominantly effective in controlling the COVID-19 pandemic. Overall, there is mixed evidence and
it will most likely be effective only in delaying the onset of an epidemic in the country receiving
infections from another country where the pathogen already is spread. The effect of travel
restrictions is influenced by many factors and depends on, amongst others, levels of community
transmission, travel volumes and duration, other public health measures in place, and the timing of
the measures.

Face masks
The effectiveness of wearing face masks in the community was assessed in the Cochrane review of
Jefferson et al. 12 Based on results from 9 randomized controlled trials (of which one was conducted
during the COVID-19 pandemic) they conclude that wearing medical/surgical masks in the
community probably makes little or no difference of the outcome of influenza-like illness/COVID-19
like illness compared to not wearing masks (RR 0.95, 95% CI 0.84;1.09). Also, little to no result was
seen for the outcome of laboratory-confirmed influenza/SARS-CoV-2 (RR 1.01, 95% CI 0.72;1.42)
when combining results from 6 RCTs (of which one was conducted during the COVID-19 pandemic).
Boulos et al. 23 included 24 studies on the effectiveness of masks versus no masks on SARS-CoV-2
transmission in community settings, of which two were RCTs and 22 were observational.
Furthermore, 23 observational studies in healthcare settings were included. Due to widely varying
study designs and timing during the pandemic no pooling was done, but of the observational studies,
39 (87%) studies found that mask wearing was significantly associated with a reduction in
transmission, with effect sizes (OR) ranging from 0.08 to 0.80. Five (11%) studies found no significant
association, and one (2%) study performed in the community setting favoured the control group.
One RCT found that mask wearing was significantly associated with a reduction in transmission (OR
0.88; 95% CI 0.85;0.90), while the other found no effect (OR 0.82, 95% CI 0.54;1.23). Of note,
multiple studies were also included in the other reviews included here. Using mostly the same
studies, a living review by Chou et al. 24 which had its final update in 2023, came to a similar
conclusion that in community settings mask use may be associated with a small reduced risk for
SARS-CoV-2 infection versus no mask use. Furthermore, Boulos et al. 23 found that wearing higher
quality compared to lower quality masks was favourable in those studies where a significant effect
was found, in line with Chu et al. 16 Also, the majority of included studies found that mask mandates
(compared to voluntary mask wearing) reduced transmission of SARS-CoV-2. 23 However, critical risks
of bias were present in most studies.
Talic et al. 8 combined data from 6 studies to estimate the effectiveness of face masks in the
community on incidence of SARS-CoV-2. The studies had different designs, including a randomized
controlled trial as well as observational studies. The pooled RR was estimated to be 0.47 (95% CI
0.29;0.75), indicating a protective effect. The type of mask was not specified. The estimate is in line
with the review of Chu et al. 16 who found a protective effect of face masks on infection, with a
pooled unadjusted RR of 0.34 (95% CI 0.26;0.45, 29 observational studies), and an adjusted RR of

0.18 (95% CI 0.08;0.38, 10 observational studies). They also found that there were stronger
associations with N95 or similar respirators compared with disposable surgical masks or similar.
However, most of the studies included the latter review were done in the healthcare setting and it
must be noted that studies on MERS and SARS were also included, next to COVID-19. Five other
observational studies included in the review of Talic et al. 8 (but not included in the meta-analysis) all
show a reduction in incidence, transmission, and/or mortality after mandatory mask wearing or
when comparing countries or regions with and without mandatory mask wearing. The two
observational studies included in the review of Vardavas et al. 17, both in the community setting,
show a less pronounced to no effect on COVID-19 incidence. However, they conclude that the timing
of implementation may have played a crucial role.

Digital contact tracing
Five observational studies included in a living review by Jenniskens et al. 25, 26 point towards a
decreasing effect of contract tracing apps on Rt, incidence of COVID-19 infections and mortality.
However, data were limited and risk of bias was judged to be unclear or high. Pozo-Martin et al. 27
provide an overview of twelve observational studies, of which two report results of contact tracing
combined with another NPI. Two studies were also included in Jenniskens et al. 26 Of the remaining
two studies focusing exclusively on contact tracing, one found a decreasing effect of contact tracing
on mortality, while the other study found no effect on the attack rate. The other six articles explored
the effectiveness of contact tracing in the context of the implementation of other NPIs and found a
marginal weak (1 study) or no effect (5 studies) on COVID-19 epidemic control. The review by
Littlecott et al. 28 included seven papers on the impact of contact tracing, of which three were also
included in Jenniskens et al. 26 Of the remainder, three studies concluded that contact tracing led to
a significant decline in incidence, Rt or mortality, with reductions in mortality of 48 to 68%, while
one study found no relationship.

Corona certificate; Self testing
No reviews of sufficient quality on these subjects were found in our search.

Vaccination
Here, we describe the vaccine effectiveness (VE) iii of COVID-19 vaccination for different outcomes.
First, an overview of results from the Netherlands is given, and at the end of each paragraph a
comparison is made with results from international literature. The text below is a summary of the
chapters on VE in the yearly reports on the National Immunization Program in the Netherlands. 2, 3 A
more detailed overview can be found there.

Vaccine effectiveness against hospitalization and ICU admission
By enriching the hospital register data from the Netherlands (NICE) with data from the central
COVID-19 vaccination Information and Monitoring System (CIMS), VE of COVID-19 vaccines against
hospital and ICU admission could be estimated. Estimates of VE of full primary vaccination against
hospitalization and ICU admission were very high during the period when the Delta variant
Vaccine effectiveness (VE) is expressed as (1-RR) * 100%. Thus a VE of 93% against hospitalization means that
vaccination reduces the risk of hospitalization after SARS-CoV-2 infection with 93%.
iii

dominated (July-November 2021): 93% (95% CI 93;94) and 97% (95% CI 96;97), respectively. 29, 30
After November 2021 a decrease in VE against hospitalization and ICU admission was seen with
increasing time since vaccination. 31
Omicron BA.1 and BA.2 dominated from February 2022 onwards, and VE was much lower for these
variants. VE of full primary vaccination against hospitalization was 35% (95% CI 30;39) in the period
February-March 2022, while it was 45% (95% CI 34;55) against ICU admission. However, VE
increased considerably after booster vaccination to 81% (95% CI 80;82) against hospitalization and
90% (95% CI 88;92) against ICU admission. Waning of VE by time since vaccination was also observed
after booster vaccination. 32 Therefore, a second booster dose was offered to vulnerable groups
(e.g., persons aged 60 years and older, and persons below the age of 60 with chronic diseases) since
the end of February 2022. After this booster, VE increased to 88% (95% CI 73;92) against
hospitalization in persons aged 70 years and over. 33
During the course of the pandemic, infection-induced immunity and hybrid immunity (due to both
infection and vaccination) became more important factors in determining the risk of severe
outcomes. VE estimates were increasingly impacted by previous infections. Therefore, since August
2022 the RIVM does not report on VE against hospitalization anymore, but on the RRD (relative risk
difference). During June-July 2022, when Omicron BA.5 dominated, the RRD for the first booster vs.
primary vaccination was -51% (95% CI -56;-46), e.g., persons who received the first booster had a
51% lower risk for hospitalization compared to persons who received the full primary series
vaccination only. 34 Persons aged 60 years and older who received a second booster, had a 25%
lower risk for hospitalization compared to persons who received the first booster only (RRD -25%,
95% CI -32;-18). The RRDs dropped to -47% (95% CI -53;-39) and -22% (95% CI -30;-12), respectively,
during July-September 2022. 35
Since September 2022 bivalent booster vaccinations have been offered. During October-November
2022 the RRD for a bivalent booster vaccination compared with receipt of at least one COVID-19
vaccination but not a bivalent vaccination for persons aged 60 years or over was -63% (95% CI -68;58) against hospitalization and -55% (95 CI -76;-14) against ICU admission, which reduced to -42%
(95% CI -46;-37) and -45% (95% CI -61;-22), respectively, during March-May 2023. 36
The above findings of the RIVM for the Dutch situation are in line with results from international
research, which also show that VE was lower during the period where Omicron variants were
dominant, compared to the period where the Delta variant dominated. 37, 38 Furthermore, waning of
VE with increasing time since vaccination was also observed. A booster vaccination increased VE to
80-90%, but again waning was visible after approximately four months. 37-39 International studies
showed that receiving a fourth monovalent COVID-19 vaccine dose restored the effectiveness
against hospitalization during the Omicron predominant period. 40-50 Studies on VE of a bivalent
booster against severe disease showed a protective effect, with estimates ranging from 50% to
81%.50-57 Waning of the effectiveness was shown 10+ weeks after receiving the bivalent booster. 58

Vaccine effectiveness against death
Through linkage of the CIMS vaccination registry to causes of death and other registry data of
Statistics Netherlands (CBS), VE against COVID-19 mortality was estimated for the period January
2021-January 2022. 59 A summary measure was calculated for all circulating SARS-CoV-2 variants that
year (wild type, Alpha, Delta, Omicron), as stratification into variant periods did not result in
informative estimates due to small numbers. VE against COVID-19 mortality – adjusted for amongst

others medical risk group, sex, year of birth and country of origin – was >90% for all age groups two
months after completion of the primary series. VE gradually decreased thereafter, to around 80% at
7-8 months post-primary series for most groups, and around 60% for elderly receiving a high level of
long-term care and for people aged 90+ years. Following a first booster dose, the VE increased to
>85% in all groups. 59
Similar results were found in international literature, with high VE against COVID-19 mortality after
completion of the primary series (>90%, although lower in the highest age groups) 60-62, but waning
was seen over time. 61-65 VE after booster vaccination was estimated to be around 90%.66, 67 In a
study by the WHO using data from 33 European counties, it was estimated that 51% of the expected
deaths in persons aged 60 years and older were averted by vaccination from December 2020 to
November 2021. The impact of vaccination on mortality by country ranged from 6% to 93%, with the
largest impact in countries with high early uptake. 68 It should be noted that no data from the
Netherlands were included in this study.

Vaccine effectiveness against mild disease/infection
VE against infections was estimated using community testing data in a test-negative case-control
design. During the Delta-dominant period, VE against infection was high (>80%), with lower VE
among elderly, close contacts of confirmed cases, and people having received vector vaccines.
Protection of primary vaccination against Omicron variants was much lower (35% at >=7 months
after vaccination). Booster vaccination increased VE against Omicron infection to around 65% at one
month post-vaccination. 69
In the VAccine Study COvid-19 (VASCO) cohort it was found that during the Delta period, VE
decreased from 80% (95% CI 69;87) <6 weeks after completing the primary series to 71% (95% CI
65;77) 18-23 weeks after completion of the primary series, and increased to 96% (95% CI 86;99) <6
weeks after booster vaccination. In the Omicron period, these estimates were 46% (95% CI 22;63),
25% (95% CI 8;39) and 57% (95% CI 52;62), respectively; VE decreased to 31% (95% CI 17;44) 18-23
weeks after booster vaccination. VE was lower among persons belonging to medical risk groups. 70
Furthermore, it was found that hybrid immunity was more protective against infection with SARS‐
CoV‐2 Omicron than vaccine‐induced immunity. 71
Effectiveness of bivalent original/Omicron BA.1 vaccination relative to receiving the primary
vaccination series and one or two monovalent booster vaccinations was 31% (95% CI 18;42) in 1859-year-olds and 14% (95% CI 3;24) in 60-85-year-olds during September-December 2022. In both
age groups, relative protection from a prior Omicron infection with or without bivalent vaccination
was substantially higher (80-83%).72
Similar results were found in international literature. VE against infection with the Omicron BA.1 and
BA.2 variant appeared to be lower compared to previous variants and protection waned over time. 73
Comparable results were found for VE against infection with the Omicron BA.4 and BA.5 variant. 58
The UK Health Security Agency provides VE consensus estimates by taking estimates from the UK as
well as international data into account. They found that 0-3 months after the first booster dose, VE
estimates against infection with Omicron BA.1 or BA.2 were around 50% (95% CI 40;60%). After 4-6
months, this was approximately 30% (95% CI 20;40%).74 VE estimates against infection with Omicron
BA.4, BA.5, BQ.1 and CH1.1 0-1 month after a monovalent or bivalent booster dose were around
30% (95% CI 20;40). After 2-3 months, the VE estimates decreased to around 20% (95% CI 10;30) and

after 4-6 months to approximately 10% (95% CI 0;20). 58 When specifically looking at bivalent booster
vaccines, VE estimates ranged between 8% and 29% 7 days or more after vaccination. 75-78

Vaccine effectiveness against transmission in case of infection
Estimation of VE against transmission in case of infection in the Dutch situation was possible due to
comprehensive source and contract tracing data. During the period that the Alpha variant
dominated (February-May 2021), VE against transmission was estimated to be 71% (95% CI 63;77). 79
During August and September 2021, when the Delta variant dominated, VE against transmission to
unvaccinated household contacts was 63% (95% CI 46;75), while VE against transmission to fully
vaccinated household contacts was 40% (95% CI 20;54). 80 With an alternative method, comparable
to the screening method, researchers at the RIVM found similar results: in all age groups, VE against
transmission dropped from 59-72% in April-June 2021 to 32% in October 2021.
In the VASCO cohort, the VE of primary vaccination against transmission was 64% (95% CI 15;84) in
the Delta-dominant period and 49% (95% CI -4;75) in the Omicron-dominant period. In the Omicron
period, VE against transmission was higher with each additional booster: 64% (95% CI 32-81) for the
first booster, 67% (95% CI 35;84) for the second booster and 70% (95% CI 33-86) for the third
booster. VE of the booster vaccination in the Delta-dominant period was very uncertain due to low
numbers. 70-72
In international literature, VE against transmission of the Alpha variant has been estimated between
35% and 88%.79, 81-84 VE estimates against transmission of the Delta variant vary between 24% and
82%.80-82, 85-87 Available data indicates a small, but significant effect of a booster vaccination against
Omicron-transmission, with VE estimates ranging from 1-12% for transmission to household
contacts85, 88, and 24-44% to close contacts in a non-household setting. 88-91 However, the UK Health
Security Agency stated in their COVID-19 vaccine surveillance report from January 2023 that there is
insufficient data for a consensus VE estimate against transmission for a booster dose. 52

Discussion
Most measures during the COVID-19 pandemic were suggested, advised and implemented because
of their biological or epidemiological plausibility to decrease the risk of transmission from one
individual to another. This literature study indicates that indeed the evidence suggests that these
measures reduced the risk of transmission or acquisition of infection. In this sense, they were
effective. The precise quantification of this reduction is methodologically much more complicated.
Most NPIs were implemented in bundles, making it methodologically difficult to disentangle the
effect of individual interventions. Interaction between effects of individual measures, missingness of
factors affecting outcomes (such as seasonality), and difficulties in including mediating variables
(such as mobility statistics) make it difficult to pinpoint a precise effectiveness for individual
measures. Countries surrounding the Netherlands, such as Germany92 and the UK1, 93 come to similar
conclusions in their reports evaluating the COVID-19 pandemic, stating that “It may never be
possible fully to disentangle some of the effects of individual NPIs in this pandemic as many were
used together”. 93 Similar conclusions were drawn by the OECD. 94
Furthermore, the effect of the different NPIs is dependent on the timing of implementation
regarding the COVID-19 epidemiological situation, as well as contextual factors such as the
demographic composition of the population (e.g., age structure, but also household size and
population density), the social and political-economic situation, and cultural factors, including trust

in the government and compliance or adherence. 1, 95 Similar limitations hold for the implementation
of COVID-19 vaccination.
Cross-country comparisons of the effectiveness of NPIs are even more difficult as differences in the
effectiveness of NPIs can be attributed to multiple factors. Combinations of NPIs implemented in
different countries can lead to different effect estimates. Next to the abovementioned aspects,
factors such as resilience of the healthcare system, health system characteristics, national per capita
expenditures for health, and prior experiences with novel epidemics influence the effectiveness. 1, 94
Therefore, the (diversity of) effect sizes mentioned in the paragraphs before for different countries
should be interpreted with caution and cannot be directly translated to other situations (e.g., the
Dutch situation in the same time period, or effects size in future waves of COVID-19, or effect size in
a future pandemic caused by a different pathogen).
Our initial (unrestricted) literature search showed that an enormous amount of scientific articles has
been published related to COVID-19. However, even with our restricted search, we ended up with a
large number of which many articles were of low quality. Only a handful of reviews with acceptable
to high quality were found, indicating a lack of synthesis of results. Next to the aforementioned
limitations regarding the quantification of the effect of NPIs and vaccination, there are other
methodological challenges, further complicating a review on effects of NPIs. 96 Many of the reviews
that we found and included were an inventory of individual studies, without a pooled measure or
meta-analysis. This is not always possible due to, for example, the low number of (good quality)
studies, or heterogeneity in populations included, outcomes studied, definition of NPIs or study
designs. 8, 96 However, many reviews still provide pooled measures, despite methodological barriers.
Their validity may, in some cases, be questionable. An innovative way to circumvent these issues
was used in the review of Mendez-Brito et al. 97 who display the results of the different included
studies using heatmaps by the ranking the NPIs on impact and not pooling quantitative measures,
because of the corresponding limitations. Multiple reviews state that there was a lack of empirical,
real-world data, e.g., there were very little data on the actual implementation of interventions. 18-20
Nevertheless, it is to be expected that more reviews will come available in the coming years.
Several issues should be kept in mind when assessing the included reviews. Publication bias could
have played a role, as it is likely that mostly studies that found an (statistically significant) effect
were published, while studies that found no effect were not. Furthermore, the definition of the
included NPIs differed per study. For example, ‘social distancing’ is interpreted in many different
ways. While for some the physical distance between persons was the main focus of interest, i.e., a
single intervention8, 16; in other studies multiple measures could fall under this definition. In those
studies ‘social distancing’ is actually studied as a package of interventions. 14, 15 It should also be
noted that most studies took place during the first year of the COVID-19 pandemic. This could be the
result of the delay in publication of reviews, but it is also likely that disentangling the effect of the
different interventions gets even more methodologically challenging later on in the pandemic.
Effects of NPIs are getting increasingly intertwined, including the effect of vaccination that started in
the end of 2020/beginning of 2021.
Vaccine effectiveness of COVID-19 vaccination could be studied relatively well for the Dutch
situation, as this is extensively researched by the RIVM for the Dutch situation. It is one of the few
interventions where it has been possible to quantify the effectiveness in a reliable way with small
bias. The results of this review show that COVID-19 vaccination is very effective in preventing SARSCoV-2 infection, transmission, and severe COVID-19. International literature is in line with this. The
VE estimates differ for the different virus variants, vaccine types, populations, and outcomes
assessed. Studies on the VE in the Dutch situation were complicated by the fact that consent from

the vaccinee is needed for registration in the Dutch national COVID-19 vaccination register (CIMS).
This causes misclassification of vaccinated persons that did not give consent as being unvaccinated.
An analysis by the RIVM shows that a modest non-consent for registration of vaccination records
could result in substantial bias in the VE (e.g., underestimating the VE), especially when there is a
high vaccination uptake. 98 These results stress the importance of vaccination registers with national
coverage and adequate privacy assurance that are complete and do not suffer from non-consent
bias, to facilitate adequate monitoring and evaluation of current and future vaccination
programmes.
Given the large number of publications already published and the rate at which new publications
appear, one cannot claim that a literature search is complete. The restrictions we applied to our
literature search may have left some relevant studies out, as our search terms may not have
captured all articles discussing relevant NPIs and outcomes. Furthermore, we only included reviews
and no individual studies. This led to some individual studies being included in multiple reviews. We
have tried to filter this out as much as possible. Also, it must be noted that some of the included
papers are pre-prints and are not yet peer-reviewed. Their results should be interpreted with
caution. New techniques, such as artificial intelligence and machine learning, can make the process
of conducting a systematic literature review much more efficient and transparent. 99 As a pilot we
imported the 8,500 articles on COVID-19 and NPIs that we found with our broad search into
ASReview. 100 We did not redo the complete selection process of the articles, but we experienced
that already soon after starting, mostly relevant-looking articles were presented to us by the
algorithm. These algorithms are promising tools for future literature reviews; not only by speeding
up the process but also by making it possible to include more papers (i.e., less restrictions) in the
initial search.
The focus of the review was on the effectiveness of measures, not on the costs. Most NPIs can have
a negative impact on the general well-being of people, the functioning of society, and the economy.
1, 94, 101, 102
In the future, based on lessons from the COVID-19 pandemic, the effectiveness of NPIs
should be carefully balanced with the costs for society.

Conclusion
We conclude that vaccination is an important and effective intervention to protect the population
from SARS-CoV-2 infection and sequelae, but vaccine effectiveness estimates are different for the
different virus variants, vaccine types, populations, and outcomes assessed. NPIs were also effective
in reducing transmission and (severe) disease, but effectiveness varied. Most included studies on
hand washing showed a protective effect. Yet, when and how frequently hand hygiene should be
performed was not clear. Social distancing measures such as physical distancing, stay at home
measures, restrictions on (mass) gatherings, and lockdown were found to be effective in reducing
transmission. School closure was predominantly effective on reducing transmission and disease, but
effectiveness varied. The literature provides mixed evidence on the effect of travel restrictions. If it is
effective, it is in delaying an epidemic in the country receiving infections from another country that
has an outbreak or a large epidemic, not in preventing it. This would require quarantine upon entry.
Symptom/exposure-based screening was likely not effective. Multiple studies found a protective
effect of wearing face masks, while other studies show a less pronounced to no effect of wearing
face masks in the community on COVID-19 incidence. Studies, mainly in the healthcare setting,
suggest that higher-quality masks (e.g., N95 masks) were more effective than surgical masks. Results
of some studies point towards a small reduction of contract tracing apps on Rt, incidence and

mortality, while others found no effect on COVID-19 epidemic control. Whereas precise
quantification of the effects is methodologically complicated, if not impossible, and caution is
warranted when translating the effects from one context to another, it is clear in retrospect that the
bundled interventions were effective in reducing SARS-Cov-2 transmission Also, high quality reviews
on the effectiveness of NPIs are scarce. Better synthesis of results is warranted.

Table 1. Included reviews
Author

Journal

Publication
year
2021
2023

End date literature
search
December, 2020
January 27, 2023

NPI studied

Number of studies included

Bou-Karroum et al. 22
Boulos et al. 23

Journal of Infection
Philosophical Transactions
of the Royal Society A

Travel restrictions
Face masks

2021

November 13, 2020

Travel restrictions

Chou et al. 24

Cochrane Database of
Systematic Reviews
Annals of Internal Medicine

69 of which 19 observational
75 of which 35 in the community
setting (3 RCTs and 32
observational)
62 of which 13 observational

Burns et al. 19

2023

June 2, 2022

Face masks

Chu et al. 16

Lancet

2020

March 26, 2020

Physical distancing
Face masks

Gozdzielewska et al. 9

BMC Public Health

2022

February 2022

Hand hygiene

Grepin et al. 21

Philosophical Transactions
of the Royal Society A
Cochrane Database of
Systematic Reviews
F1000Research (previous
version: BMJ Open)
Cochrane Database of
Systematic Reviews
Philosophical Transactions
of the Royal Society A
Epidemiology
Philosophical Transactions
of the Royal Society A

2023

Early 2023

Travel restrictions

2023

October, 2022

2022

June 9, 2021

Face masks
Hand hygiene
Contact tracing

2022

December 9, 2020

School closure

38 of which 4 observational

2023

January 6, 2023

Contact tracing

25 observational

2021
2023

February 15, 2021
December 1, 2022

Physical distancing
Social distancing

12 observational
338 (mostly observational)

Jefferson et al. 12
Jenniskens et al. 25, 26^
Krishnaratne et al. 18
Littlecott et al. 28
Liu et al. 15
Murphy et al. 13

24 of which 13 in the community
setting (2 RCTs and 11
observational)
172 observational, of which 44
included in meta-analysis*
22 (6 RCTs and 16
observational)*
5 reviews containing 43
observational studies
78 RCTs*
27 of which 5 observational

Nussbaumer-Streit et al.

2020

June 23, 2020

Quarantine

51 of which 8 observational*

2023

July 7, 2021

Contact tracing

78 of which 12 observational

Sun et al. 14

Cochrane Database of
Systematic Reviews
European Journal of
Epidemiology
BMJ Open

2022

September 30, 2020

Social distancing

Talic et al. 8

BMJ

2021

June 7, 2021

Vardavas et al. 17

medRxiv

2021

April 15, 2021

Hand washing
Face masks
Physical distancing
Stay at home/Isolation
Quarantine
School closure
Business closure
Lockdown
Travel restrictions
Lockdown
Face masks

41 (observational and modelling,
all based on empirical data)
72 observational

20

Pozo-Martin et al. 27

* also including other respiratory illnesses (e.g., SARS, MERS, influenza)
^ two versions of the paper (one peer reviewed, one updated preprint)

45 of which 15 observational

References
1.
The Royal Society. COVID-19: examining the effectiveness of non-pharmaceutical
interventions. 2023.
2.
Pluijmaekers AJM and de Melker HE. The National Immunisation Programme in the
Netherlands. Surveillance and developments in 2021-2022. 2022. DOI: 10.21945/RIVM-2022-0042
3.
Pluijmaekers AJM and de Melker HE. The National Immunisation Programme in the
Netherlands. Surveillance and developments in 2022-2023 (unpublished). 2023.
4.
International Vaccine Access Center, Johns Hopkins Bloomberg School of Public Health and
World Health Organization and Coalition for Epidemic Preparedness Innovations. Results of COVID19 Vaccine Effectiveness Studies. An Ongoing Systematic Review, https://view-hub.org/resources.
5.
National Center for Biotechnology Information, National Library of Medicine, National
Institutes of Health, US. LitCovid, https://www.ncbi.nlm.nih.gov/research/coronavirus/.
6.
EPPI Centre, University College London. COVID-19: a living systematic map of the evidence,
https://eppi.ioe.ac.uk/cms/Projects/DepartmentofHealthandSocialCare/%20Publishedreviews/COVI
D-19Livingsystematicmapoftheevidence/tabid/3765/Default.aspx.
7.
UK Health Security Agency. UKHSA COVID-19 Rapid Reviews, https://ukhsalibrary.kohaptfs.co.uk/covid19rapidreviews/.
8.
Talic S, Shah S, Wild H, et al. Effectiveness of public health measures in reducing the
incidence of covid-19, SARS-CoV-2 transmission, and covid-19 mortality: systematic review and
meta-analysis. BMJ 2021; 375: e068302. DOI: 10.1136/bmj-2021-068302.
9.
Gozdzielewska L, Kilpatrick C, Reilly J, et al. The effectiveness of hand hygiene interventions
for preventing community transmission or acquisition of novel coronavirus or influenza infections: a
systematic review. BMC Public Health 2022; 22: 1283. DOI: 10.1186/s12889-022-13667-y.
10.
MacLeod C, Braun L, Caruso BA, et al. Recommendations for hand hygiene in community
settings: a scoping review of current international guidelines. BMJ Open 2023; 13: e068887. DOI:
10.1136/bmjopen-2022-068887.
11.
Aiello AE, Coulborn RM, Perez V and Larson EL. Effect of hand hygiene on infectious disease
risk in the community setting: a meta-analysis. Am J Public Health 2008; 98: 1372-1381. DOI:
10.2105/AJPH.2007.124610.
12.
Jefferson T, Dooley L, Ferroni E, et al. Physical interventions to interrupt or reduce the
spread of respiratory viruses. Cochrane Database Syst Rev 2023; 1: CD006207. DOI:
10.1002/14651858.CD006207.pub6.
13.
Murphy C, Lim WW, Mills C, et al. Effectiveness of social distancing measures and lockdowns
for reducing transmission of COVID-19 in non-healthcare, community-based settings. Philos Trans A
Math Phys Eng Sci 2023; 381: 20230132. DOI: 10.1098/rsta.2023.0132.
14.
Sun KS, Lau TSM, Yeoh EK, et al. Effectiveness of different types and levels of social
distancing measures: a scoping review of global evidence from earlier stage of COVID-19 pandemic.
BMJ Open 2022; 12: e053938. DOI: 10.1136/bmjopen-2021-053938.
15.
Liu CY, Berlin J, Kiti MC, et al. Rapid Review of Social Contact Patterns During the COVID-19
Pandemic. Epidemiology 2021; 32: 781-791. DOI: 10.1097/EDE.0000000000001412.
16.
Chu DK, Akl EA, Duda S, et al. Physical distancing, face masks, and eye protection to prevent
person-to-person transmission of SARS-CoV-2 and COVID-19: a systematic review and meta-analysis.
Lancet 2020; 395: 1973-1987. DOI: 10.1016/S0140-6736(20)31142-9.
17.
[preprint] Vardavas CI, Nikitara K, Aslasnoglou K, et al. Effectiveness of non-pharmaceutical
measures (NPIs) on COVID-19 in Europe: A systematic literature review. medRxiv 2021. DOI:
10.1101/2021.11.11.21266216.
18.
Krishnaratne S, Littlecott H, Sell K, et al. Measures implemented in the school setting to
contain the COVID-19 pandemic. Cochrane Database Syst Rev 2022; 1: CD015029. DOI:
10.1002/14651858.CD015029.

48

19.
Burns J, Movsisyan A, Stratil JM, et al. Travel-related control measures to contain the COVID19 pandemic: a rapid review. Cochrane Database Syst Rev 2020; 10: CD013717. DOI:
10.1002/14651858.CD013717.
20.
Nussbaumer-Streit B, Mayr V, Dobrescu AI, et al. Quarantine alone or in combination with
other public health measures to control COVID-19: a rapid review. Cochrane Database Syst Rev 2020;
9: CD013574. DOI: 10.1002/14651858.CD013574.pub2.
21.
Grepin KA, Aston J and Burns J. Effectiveness of international border control measures
during the COVID-19 pandemic: a narrative synthesis of published systematic reviews. Philos Trans A
Math Phys Eng Sci 2023; 381: 20230134. DOI: 10.1098/rsta.2023.0134.
22.
Bou-Karroum L, Khabsa J, Jabbour M, et al. Public health effects of travel-related policies on
the COVID-19 pandemic: A mixed-methods systematic review. J Infect 2021; 83: 413-423. DOI:
10.1016/j.jinf.2021.07.017.
23.
Boulos L, Curran JA, Gallant A, et al. Effectiveness of face masks for reducing transmission of
SARS-CoV-2: a rapid systematic review. Philos Trans A Math Phys Eng Sci 2023; 381: 20230133. DOI:
10.1098/rsta.2023.0133.
24.
Chou R and Dana T. Major Update: Masks for Prevention of SARS-CoV-2 in Health Care and
Community Settings-Final Update of a Living, Rapid Review. Ann Intern Med 2023; 176: 827-835.
DOI: 10.7326/M23-0570.
25.
Jenniskens K, Bootsma MCJ, Damen J, et al. Effectiveness of contact tracing apps for SARSCoV-2: a rapid systematic review. BMJ Open 2021; 11: e050519. DOI: 10.1136/bmjopen-2021050519.
26.
[preprint] Jenniskens K, Bootsma MCJ, Damen JAAG, et al. Effectiveness of contact tracing
apps for SARS-CoV-2: an updated systematic review. F1000Research 2022. DOI:
10.12688/f1000research.110668.1.
27.
Pozo-Martin F, Beltran Sanchez MA, Muller SA, et al. Comparative effectiveness of contact
tracing interventions in the context of the COVID-19 pandemic: a systematic review. Eur J Epidemiol
2023; 38: 243-266. DOI: 10.1007/s10654-023-00963-z.
28.
Littlecott H, Herd C, O'Rourke J, et al. Effectiveness of testing, contact tracing and isolation
interventions among the general population on reducing transmission of SARS-CoV-2: a systematic
review. Philos Trans A Math Phys Eng Sci 2023; 381: 20230131. DOI: 10.1098/rsta.2023.0131.
29.
RIVM. COVID-19 ziekenhuisopnames per vaccinatiestatus, https://www.rivm.nl/covid-19vaccinatie/covid-19-ziekenhuisopnames-per-vaccinatiestatus (2023).
30.
RIVM. Effectiviteit van COVID-19 vaccinatie tegen ziekenhuis en intensive-care-opname in
Nederland (opnames 11 juli – 23 november 2021). 30 November 2021.
31.
RIVM. Effectiviteit van COVID-19 vaccinatie tegen ziekenhuis en intensive-care-opname in
Nederland (opnames 11 juli – 7 december 2021). 14 December 2021.
32.
RIVM. Effectiviteit van COVID-19 vaccinatie tegen ziekenhuis en intensive-care-opname in
Nederland (opnames 01 februari 2022 – 05 april 2022). 12 April 2022.
33.
RIVM. Effectiviteit van COVID-19 vaccinatie tegen ziekenhuis en intensive-care-opname in
Nederland (opnames 01 februari 2022 – 03 mei 2022). 10 May 2022.
34.
RIVM. COVID-19 ziekenhuis en intensive-care-opnames per vaccinatiestatus in Nederland
(opnames 15 maart 2022 – 26 juli 2022). 2 August 2022.
35.
RIVM. COVID-19 ziekenhuis en intensive-care-opnames per vaccinatiestatus in Nederland
(opnames 15 maart 2022 – 18 september 2022). 27 September 2022.
36.
RIVM. COVID-19 ziekenhuis en intensive-care-opnames per vaccinatiestatus in Nederland
(opnames 3 oktober 2022 – 2 november 2022). 8 November 2022.
37.
Ferdinands JM, Rao S, Dixon BE, et al. Waning 2-Dose and 3-Dose Effectiveness of mRNA
Vaccines Against COVID-19-Associated Emergency Department and Urgent Care Encounters and
Hospitalizations Among Adults During Periods of Delta and Omicron Variant Predominance - VISION
Network, 10 States, August 2021-January 2022. MMWR Morb Mortal Wkly Rep 2022; 71: 255-263.
DOI: 10.15585/mmwr.mm7107e2.
49

38.
Thompson MG, Natarajan K, Irving SA, et al. Effectiveness of a Third Dose of mRNA Vaccines
Against COVID-19-Associated Emergency Department and Urgent Care Encounters and
Hospitalizations Among Adults During Periods of Delta and Omicron Variant Predominance - VISION
Network, 10 States, August 2021-January 2022. MMWR Morb Mortal Wkly Rep 2022; 71: 139-145.
DOI: 10.15585/mmwr.mm7104e3.
39.
Tartof SY, Slezak JM, Puzniak L, et al. Durability of BNT162b2 vaccine against hospital and
emergency department admissions due to the omicron and delta variants in a large health system in
the USA: a test-negative case-control study. Lancet Respir Med 2022; 10: 689-699. DOI:
10.1016/S2213-2600(22)00101-1.
40.
UK Health Security Agency. COVID-19 vaccine quarterly surveillance reports (September 2021
to June 2023). 2023.
41.
Magen O, Waxman JG, Makov-Assif M, et al. Fourth Dose of BNT162b2 mRNA Covid-19
Vaccine in a Nationwide Setting. N Engl J Med 2022; 386: 1603-1614. DOI: 10.1056/NEJMoa2201688.
42.
Amir O, Goldberg Y, Mandel M, et al. Protection against Omicron BA.1/BA.2 severe disease
0-7 months after BNT162b2 booster. Commun Biol 2023; 6: 315. DOI: 10.1038/s42003-023-04669-6.
43.
Muhsen K, Maimon N, Mizrahi AY, et al. Association of Receipt of the Fourth BNT162b2 Dose
With Omicron Infection and COVID-19 Hospitalizations Among Residents of Long-term Care
Facilities. JAMA Intern Med 2022; 182: 859-867. DOI: 10.1001/jamainternmed.2022.2658.
44.
Link-Gelles R, Levy ME, Gaglani M, et al. Effectiveness of 2, 3, and 4 COVID-19 mRNA Vaccine
Doses Among Immunocompetent Adults During Periods when SARS-CoV-2 Omicron BA.1 and
BA.2/BA.2.12.1 Sublineages Predominated - VISION Network, 10 States, December 2021-June 2022.
MMWR Morb Mortal Wkly Rep 2022; 71: 931-939. DOI: 10.15585/mmwr.mm7129e1.
45.
Moller Kirsebom FC, Andrews N, Stowe J, et al. Effectiveness of the COVID-19 vaccines
against hospitalisation with Omicron sub-lineages BA.4 and BA.5 in England. Lancet Reg Health Eur
2022; 23: 100537. DOI: 10.1016/j.lanepe.2022.100537.
46.
Gazit S, Saciuk Y, Perez G, et al. Short term, relative effectiveness of four doses versus three
doses of BNT162b2 vaccine in people aged 60 years and older in Israel: retrospective, test negative,
case-control study. BMJ 2022; 377: e071113. DOI: 10.1136/bmj-2022-071113.
47.
Fabiani M, Mateo-Urdiales A, Sacco C, et al. Relative effectiveness of a 2nd booster dose of
COVID-19 mRNA vaccine up to four months post administration in individuals aged 80 years or more
in Italy: A retrospective matched cohort study. Vaccine 2023; 41: 76-84. DOI:
10.1016/j.vaccine.2022.11.013.
48.
Carazo S, Skowronski DM, Brisson M, et al. Effectiveness of previous infection-induced and
vaccine-induced protection against hospitalisation due to omicron BA subvariants in older adults: a
test-negative, case-control study in Quebec, Canada. Lancet Healthy Longev 2023; 4: e409-e420.
DOI: 10.1016/S2666-7568(23)00099-5.
49.
Tartof SY, Slezak JM, Puzniak L, et al. BNT162b2 vaccine effectiveness against SARS-CoV-2
omicron BA.4 and BA.5. Lancet Infect Dis 2022; 22: 1663-1665. wDOI: 10.1016/S14733099(22)00692-2.
50.
Grewal R, Nguyen L, Buchan SA, et al. Effectiveness of mRNA COVID-19 vaccine booster
doses against Omicron severe outcomes. Nat Commun 2023; 14: 1273. DOI: 10.1038/s41467-02336566-1.
51.
UK Health Security Agency. COVID-19 vaccine surveillance report - Week 48. 2022.
52.
UK Health Security Agency. COVID-19 vaccine surveillance report - Week 2. 2023.
53.
Andersson NW, Thiesson EM, Baum U, et al. Comparative effectiveness of bivalent BA.4-5
and BA.1 mRNA booster vaccines among adults aged >/=50 years in Nordic countries: nationwide
cohort study. BMJ 2023; 382: e075286. DOI: 10.1136/bmj-2022-075286.
54.
[preprint] Poukka E, Goebeler S, Nohynek H, et al. Bivalent booster effectiveness against
severe COVID-19 outcomes in Finland, September 2022 — January 2023. medRxiv 2023. DOI:
10.1101/2023.03.02.23286561.

50

55.
[preprint] Chatzilena A, Hyams C, Challen R, et al. Relative vaccine effectiveness (rVE) of
mRNA COVID-19 boosters in people aged at least 75 years in the UK vaccination programme, during
the Spring-Summer (monovalent vaccine) and Autumn-Winter 2022 (bivalent vaccine) booster
campaigns: a prospective test negative case-control study. medRxiv 2023. DOI:
10.1101/2023.03.16.23287360.
56.
Lin DY, Xu Y, Gu Y, et al. Effectiveness of Bivalent Boosters against Severe Omicron Infection.
N Engl J Med 2023; 388: 764-766. DOI: 10.1056/NEJMc2215471.
57.
Kirsebom FCM, Andrews N, Stowe J, et al. Duration of protection of ancestral-strain
monovalent vaccines and effectiveness of bivalent BA.1 boosters against COVID-19 hospitalisation in
England: a test-negative case-control study. Lancet Infect Dis 2023; 23: 1235-1243. DOI:
10.1016/S1473-3099(23)00365-1.
58.
UK Health Security Agency. COVID-19 vaccine surveillance report - Week 14. 2023.
59.
de Gier B, van Asten L, Boere TM, et al. Effect of COVID-19 vaccination on mortality by
COVID-19 and on mortality by other causes, the Netherlands, January 2021-January 2022. Vaccine
2023; 41: 4488-4496. DOI: 10.1016/j.vaccine.2023.06.005.
60.
Haas EJ, Angulo FJ, McLaughlin JM, et al. Impact and effectiveness of mRNA BNT162b2
vaccine against SARS-CoV-2 infections and COVID-19 cases, hospitalisations, and deaths following a
nationwide vaccination campaign in Israel: an observational study using national surveillance data.
Lancet 2021; 397: 1819-1829. DOI: 10.1016/S0140-6736(21)00947-8.
61.
Nunes B, Rodrigues AP, Kislaya I, et al. mRNA vaccine effectiveness against COVID-19-related
hospitalisations and deaths in older adults: a cohort study based on data linkage of national health
registries in Portugal, February to August 2021. Euro Surveill 2021; 26. DOI: 10.2807/15607917.ES.2021.26.38.2100833.
62.
Wu N, Joyal-Desmarais K, Ribeiro PAB, et al. Long-term effectiveness of COVID-19 vaccines
against infections, hospitalisations, and mortality in adults: findings from a rapid living systematic
evidence synthesis and meta-analysis up to December, 2022. Lancet Respir Med 2023; 11: 439-452.
DOI: 10.1016/S2213-2600(23)00015-2.
63.
Andrews N, Tessier E, Stowe J, et al. Duration of Protection against Mild and Severe Disease
by Covid-19 Vaccines. N Engl J Med 2022; 386: 340-350. DOI: 10.1056/NEJMoa2115481.
64.
Horne EMF, Hulme WJ, Keogh RH, et al. Waning effectiveness of BNT162b2 and ChAdOx1
covid-19 vaccines over six months since second dose: OpenSAFELY cohort study using linked
electronic health records. BMJ 2022; 378: e071249. DOI: 10.1136/bmj-2022-071249.
65.
Nordstrom P, Ballin M and Nordstrom A. Risk of infection, hospitalisation, and death up to 9
months after a second dose of COVID-19 vaccine: a retrospective, total population cohort study in
Sweden. Lancet 2022; 399: 814-823. DOI: 10.1016/S0140-6736(22)00089-7.
66.
Lin DY, Gu Y, Xu Y, et al. Association of Primary and Booster Vaccination and Prior Infection
With SARS-CoV-2 Infection and Severe COVID-19 Outcomes. JAMA 2022; 328: 1415-1426. DOI:
10.1001/jama.2022.17876.
67.
Arbel R, Hammerman A, Sergienko R, et al. BNT162b2 Vaccine Booster and Mortality Due to
Covid-19. N Engl J Med 2021; 385: 2413-2420. DOI: 10.1056/NEJMoa2115624.
68.
Mesle MM, Brown J, Mook P, et al. Estimated number of deaths directly averted in people
60 years and older as a result of COVID-19 vaccination in the WHO European Region, December 2020
to November 2021. Euro Surveill 2021; 26. DOI: 10.2807/1560-7917.ES.2021.26.47.2101021.
69.
Andeweg SP, de Gier B, Eggink D, et al. Protection of COVID-19 vaccination and previous
infection against Omicron BA.1, BA.2 and Delta SARS-CoV-2 infections. Nat Commun 2022; 13: 4738.
DOI: 10.1038/s41467-022-31838-8.
70.
Huiberts AJ, de Gier B, Hoeve CE, et al. Vaccine effectiveness of primary and booster COVID19 vaccinations against SARS-CoV-2 infection in the Netherlands from July 12, 2021 to June 6, 2022:
A prospective cohort study. Int J Infect Dis 2023; 133: 36-42. DOI: 10.1016/j.ijid.2023.04.401.

51

71.
de Gier B, Huiberts AJ, Hoeve CE, et al. Effects of COVID-19 vaccination and previous
infection on Omicron SARS-CoV-2 infection and relation with serology. Nat Commun 2023; 14: 4793.
DOI: 10.1038/s41467-023-40195-z.
72.
Huiberts AJ, de Gier B, Hoeve CE, et al. Effectiveness of bivalent mRNA booster vaccination
against SARS-CoV-2 Omicron infection, the Netherlands, September to December 2022. Euro Surveill
2023; 28. DOI: 10.2807/1560-7917.ES.2023.28.7.2300087.
73.
ECDC. Public health considerations and evidence to support decisions on the implementation
of a second mRNA COVID-19 vaccine booster dose. 28 April 2022.
74.
UK Health Security Agency. COVID-19 vaccine surveillance report - Week 17. 2022.
75.
Link-Gelles R, Ciesla AA, Fleming-Dutra KE, et al. Effectiveness of Bivalent mRNA Vaccines in
Preventing Symptomatic SARS-CoV-2 Infection - Increasing Community Access to Testing Program,
United States, September-November 2022. MMWR Morb Mortal Wkly Rep 2022; 71: 1526-1530.
DOI: 10.15585/mmwr.mm7148e1.
76.
Shrestha NK, Burke PC, Nowacki AS, et al. Effectiveness of the Coronavirus Disease 2019
Bivalent Vaccine. Open Forum Infect Dis 2023; 10: ofad209. DOI: 10.1093/ofid/ofad209.
77.
Auvigne V, Tamandjou Tchuem CR, Schaeffer J, et al. Protection against symptomatic SARSCoV-2 infection conferred by the Pfizer-BioNTech Original/BA.4-5 bivalent vaccine compared to the
mRNA Original monovalent vaccines - A matched cohort study in France. Vaccine 2023; 41: 54905493. DOI: 10.1016/j.vaccine.2023.07.071.
78.
Lin DY, Xu Y, Gu Y, et al. Durability of Bivalent Boosters against Omicron Subvariants. N Engl J
Med 2023; 388: 1818-1820. DOI: 10.1056/NEJMc2302462.
79.
de Gier B, Andeweg S, Joosten R, et al. Vaccine effectiveness against SARS-CoV-2
transmission and infections among household and other close contacts of confirmed cases, the
Netherlands, February to May 2021. Euro Surveill 2021; 26. DOI: 10.2807/15607917.ES.2021.26.31.2100640.
80.
de Gier B, Andeweg S, Backer JA, et al. Vaccine effectiveness against SARS-CoV-2
transmission to household contacts during dominance of Delta variant (B.1.617.2), the Netherlands,
August to September 2021. Euro Surveill 2021; 26. DOI: 10.2807/1560-7917.ES.2021.26.44.2100977.
81.
[preprint] Clifford S, Waight P, Hackman J, et al. Effectiveness of BNT162b2 and ChAdOx1
against SARS-CoV-2 household transmission: a prospective cohort study in England. medRxiv 2021.
DOI: 10.1101/2021.11.24.21266401.
82.
Eyre DW, Taylor D, Purver M, et al. Effect of Covid-19 Vaccination on Transmission of Alpha
and Delta Variants. N Engl J Med 2022; 386: 744-756. DOI: 10.1056/NEJMoa2116597.
83.
Braeye T, Cornelissen L, Catteau L, et al. Vaccine effectiveness against infection and onwards
transmission of COVID-19: Analysis of Belgian contact tracing data, January-June 2021. Vaccine 2021;
39: 5456-5460. DOI: 10.1016/j.vaccine.2021.08.060.
84.
Harris RJ, Hall JA, Zaidi A, et al. Effect of Vaccination on Household Transmission of SARSCoV-2 in England. N Engl J Med 2021; 385: 759-760. DOI: 10.1056/NEJMc2107717.
85.
Jalali N, Brustad HK, Frigessi A, et al. Increased household transmission and immune escape
of the SARS-CoV-2 Omicron compared to Delta variants. Nat Commun 2022; 13: 5706. DOI:
10.1038/s41467-022-33233-9.
86.
Ng OT, Koh V, Chiew CJ, et al. Impact of Severe Acute Respiratory Syndrome Coronavirus 2
(SARS-CoV-2) Vaccination and Pediatric Age on Delta Variant Household Transmission. Clin Infect Dis
2022; 75: e35-e43. DOI: 10.1093/cid/ciac219.
87.
Lyngse FP, Molbak K, Denwood M, et al. Effect of vaccination on household transmission of
SARS-CoV-2 Delta variant of concern. Nat Commun 2022; 13: 3764. DOI: 10.1038/s41467-022-31494y.
88.
Allen H, Tessier E, Turner C, et al. Comparative transmission of SARS-CoV-2 Omicron
(B.1.1.529) and Delta (B.1.617.2) variants and the impact of vaccination: national cohort study,
England. Epidemiol Infect 2023; 151: e58. DOI: 10.1017/S0950268823000420.

52

89.
Braeye T, Catteau L, Brondeel R, et al. Vaccine effectiveness against transmission of alpha,
delta and omicron SARS-COV-2-infection, Belgian contact tracing, 2021-2022. Vaccine 2023; 41:
3292-3300. DOI: 10.1016/j.vaccine.2023.03.069.
90.
Tan ST, Kwan AT, Rodriguez-Barraquer I, et al. Infectiousness of SARS-CoV-2 breakthrough
infections and reinfections during the Omicron wave. Nat Med 2023; 29: 358-365. DOI:
10.1038/s41591-022-02138-x.
91.
Wang K, Guo Z, Zeng T, et al. Transmission Characteristics and Inactivated Vaccine
Effectiveness Against Transmission of SARS-CoV-2 Omicron BA.5 Variants in Urumqi, China. JAMA
Netw Open 2023; 6: e235755. DOI: 10.1001/jamanetworkopen.2023.5755.
92.
Sachverständigenausschusses nach § 5 Absatz 9 Infektionsschutzgesetz. Evaluation der
Rechtsgrundlagen und Maßnahmen der Pandemiepolitik. 2022.
93.
Department of Health and Social Care. Technical report on the COVID-19 pandemic in the UK.
2022.
94.
OECD. Ready for the Next Crisis? Investing in Health System Resilience. 2023.
95.
Williams SN, Dienes K, Jaheed J, et al. Effectiveness of communications in enhancing
adherence to public health behavioural interventions: a COVID-19 evidence review. Philos Trans A
Math Phys Eng Sci 2023; 381: 20230129. DOI: 10.1098/rsta.2023.0129.
96.
Banholzer N, Lison A, Ozcelik D, et al. The methodologies to assess the effectiveness of nonpharmaceutical interventions during COVID-19: a systematic review. Eur J Epidemiol 2022; 37: 10031024. DOI: 10.1007/s10654-022-00908-y.
97.
Mendez-Brito A, El Bcheraoui C and Pozo-Martin F. Systematic review of empirical studies
comparing the effectiveness of non-pharmaceutical interventions against COVID-19. J Infect 2021;
83: 281-293. DOI: 10.1016/j.jinf.2021.06.018.
98.
[preprint] van Werkhoven CH, de Gier B, McDonald S, et al. Information bias of vaccine
effectiveness estimation due to informed consent for national registration of COVID-19 vaccination:
estimation and correction using a data augmentation model. medRxiv 2023. DOI:
10.1101/2023.05.23.23290384.
99.
van de Schoot R, de Bruin J, Schram R, et al. An open source machine learning framework for
efficient and transparent systematic reviews. Nature Machine Intelligence 2021; 3: 125-133. DOI:
10.1038/s42256-020-00287-7.
100. ASReview, https://asreview.nl/ (accessed November 13, 2023).
101. ECDC. Guidelines for non-pharmaceutical interventions to reduce the impact of COVID-19 in
the EU/EEA and the UK. 24 September 2020.
102. The European Observatory on Health Systems and Policies. Health systems resilience during
COVID-19: Lessons for building back better. 2021.

53

Appendix C
NPIs included in the literature search
Hand hygiene
Social distancing
-

-

-

-

Physical distancing
Stay at home requirements
o Isolation
o Quarantine
o Working from home
Closure of non-essential businesses
o Closure of hospitality industry
o Work ban for non-medical contact-based professions/Closure of close contact
service
o Closure of non-essential shops/retail
o Closure of night clubs
Closure of schools and child care
Restricting visitors per household
Restrictions on gatherings
o Restrictions on the number of visitors
o Closure of publicly accessible locations/public events
Night time curfew
Sport restrictions

Travel restrictions
Face mask
(Digital) Contact tracing
Corona certificate
-

Proof of vaccination
Proof of recovery
Negative test result

Self-testing

54

Outcomes included in the literature search
-

Reproduction number/Rt
Infection rate
Virus transmission
Mortality
Intensive care unit admission
Hospital admission
Life expectancy

Additional (broad) search terms:
-

Mathematical model
Quantitative analysis

55