Estimating the effectiveness of first dose of COVID-19 vaccine against mortality in England: a quasi-experimental study
Authors: Bermingham Charlotte , Morgan Jasper , Ayoubkhani Daniel , Glickman Myer , Islam Nazrul , Sheikh Aziz , Sterne Jonathan , Walker A. Sarah , Nafilyan Vahé 
Published: July 2021

Background

Estimating real-world vaccine effectiveness is vital to assess the impact of the vaccination programme on the pandemic and inform the ongoing policy response. However, estimating vaccine effectiveness using observational data is inherently challenging because of the non-randomised design and the potential for unmeasured confounding.

Methods

We used a Regression Discontinuity Design (RDD) to estimate vaccine effectiveness against COVID-19 mortality in England, exploiting the discontinuity in vaccination rates resulting from the UK’s age-based vaccination priority groups. We used the fact that people aged 80 or over were prioritised for the vaccine roll-out in the UK to compare the risk of COVID-19 and non-COVID-19 death in people aged 75–79 and 80–84.

Findings

The prioritisation of vaccination of people aged 80 or above led to a large discrepancy in vaccination rates in people 80–84 compared to those 75–79 at the beginning of the vaccination campaign. We found a corresponding difference in COVID-19 mortality, but not in non-COVID-19 mortality, suggesting that our approach appropriately addresses the issue of unmeasured confounding factors. Our results suggest that the first vaccine dose reduced the risk of COVID-19 death by 52.6% (95% Cl 26.6–84.2) in those aged 80.

Interpretations

Our results support existing evidence that a first dose of a COVID-19 vaccine has a strong protective effect against COVID-19 mortality in older adults. The RDD estimate of vaccine effectiveness is comparable to previously published studies using different methods, suggesting that unmeasured confounding factors are unlikely to substantially bias these studies.

Funding

Office for National Statistics.

Research in Context

Evidence before this study

We searched PubMed for studies reporting on the ‘real-world’ effectiveness of the COVID-19 vaccination on risk of death using terms such as “COVID-19”, “vaccine effectiveness”, “mortality” and “death”. The relevant published studies on this topic report vaccine effectiveness estimates against risk of death ranging from 64.2% to 98.7%, for varying times post-vaccination. All of these are observational studies and therefore potentially subject to bias from unmeasured confounding. We found no studies that used a quasi-experimental method such as regression discontinuity design, which is not subject to bias from unmeasured confounding, to calculate the effectiveness of the COVID-19 vaccination on risk of COVID-19 death, or on other outcomes such as hospitalisation or infection.

Added value of this study

The estimates of vaccine effectiveness based on observational data may be biased by unmeasured confounding. This study uses a regression discontinuity design to estimate vaccine effectiveness, exploiting the fact that the vaccination campaign in the UK was rolled out following age-based priority groups. This enables the calculation of an unbiased estimate of the effectiveness of the COVID-19 vaccine against risk of death.

The vaccine effectiveness estimate of 52.6% (95% Cl 26.6–84.2) is slightly lower but similar to previously published estimates, therefore suggesting that these estimates are not substantially affected by unmeasured confounding factors and confirming the effectiveness of the COVID-19 vaccine against risk of COVID-19 death.

Implications of all the available evidence

Obtaining an unbiased estimate of COVID-19 vaccine effectiveness is of vital importance in informing policy for lifting COVID-19 related measures. The regression discontinuity design provides confidence that the existing estimates from observational studies are unlikely to be substantially biased by unmeasured confounding.