# Spatial Variation in Excess Mortality Across Europe: A Cross-Sectional Study of 561 Regions in 21 Countries

Florian Bonnet, Pavel Grigoriev, Markus Sauerberg, Ina Alliger, Michael

Mühlichen, Carlo Giovanni Camarda

► To cite this version:

Florian Bonnet, Pavel Grigoriev, Markus Sauerberg, Ina Alliger, Michael Mühlichen, et al. Spatial Variation in Excess Mortality Across Europe: A Cross-Sectional Study of 561 Regions in 21 Countries. Journal of Epidemiology and Global Health, 2024, 10.1007/s44197-024-00200-0. hal-04582044

HAL Id: hal-04582044

https://hal.science/hal-04582044

Submitted on 21 May 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Spatial Variation in Excess Mortality Across Europe: A Cross-Sectional Study of 561 Regions in 21 Countries

Florian Bonnet, Pavel Grigoriev, Markus Sauerberg, Ina Alliger, Michael

Mühlichen, Carlo Giovanni Camarda

► To cite this version:

Florian Bonnet, Pavel Grigoriev, Markus Sauerberg, Ina Alliger, Michael Mühlichen, et al. Spatial Variation in Excess Mortality Across Europe: A Cross-Sectional Study of 561 Regions in 21 Countries. Journal of Epidemiology and Global Health, 2024, 10.1007/s44197-024-00200-0. hal-04582044

HAL Id: hal-04582044

https://hal.science/hal-04582044v1

Submitted on 21 May 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Spatial Variation in Excess Mortality Across Europe: A Cross-Sectional Study of 561 Regions in 21 Countries

Florian Bonnet<fnref n="1" /> ID · Pavel Grigoriev<fnref n="1" /> ID · Markus Sauerberg<fnref n="2" /> ID · Ina Alliger<fnref n="2" /> · Michael Mühlichen<fnref n="2" /> ID · Carlo-Giovanni Camarda<fnref n="1" /> ID

Received: 31 August 2023 / Accepted: 24 January 2024
© The Author(s) 2024

## Abstract

**Objective** To measure the burden of the COVID-19 pandemic in 2020 at the subnational level by estimating excess mortality, defined as the increase in all-cause mortality relative to an expected baseline mortality level.

**Methods** Statistical and demographic analyses of regional all-cause mortality data provided by the vital statistics systems of 21 European countries for 561 regions in Central and Western Europe. Life expectancy losses at ages 0 and 60 for males and females were estimated.

**Results** We found evidence of a loss in life expectancy in 391 regions, whilst only three regions exhibit notable gains in life expectancy in 2020. For 12 regions, losses of life expectancy amounted to more than 2 years and three regions showed losses greater than 3 years. We highlight geographical clusters of high mortality in Northern Italy, Spain and Poland, whilst clusters of low mortality were found in Western France, Germany/Denmark and Norway/Sweden.

**Conclusions** Regional differences of loss of life expectancy are impressive, ranging from a loss of more than 4 years to a gain of 8 months. These findings provide a strong rationale for regional analysis, as national estimates hide significant regional disparities.

**Keywords** COVID-19 · Excess mortality · European regions · Life expectancy · Geographical clusters

## Abbreviations

**NUTS** Nomenclature of Territorial Units for Statistics.

**IHME** Institute for Health Metrics and Evaluation

## 1 Introduction

The outbreak of SARS-CoV-2 triggered a strong reaction from public authorities, who responded in accordance with the threat that this new infectious disease entailed for the population. Despite having adopted social distancing measures that had never been seen in living memory in the twenty-first century, this pandemic led to most countries seeing significantly increased mortality [1], which the national surveillance authorities in Europe highlighted daily with

their published reports [2, 3]. Nowadays, the daily monitoring of the pandemic's evolution has been partially replaced by the quest to globally assess its mortality burden.

However, because these various national surveillance authorities were quickly established ad hoc to monitor the pandemic, use of the data they provide is broadly questioned for a wide range of reasons: different definitions of data amongst countries; time-varying collection methods; reporting delays; and diverse coverage by place of death [4, 5]. In contrast, the standard official statistics systems provide all-cause deaths, which are essential for computing excess mortality, defined as the difference between the deaths observed and the deaths that would have been observed under normal circumstances. In the context of the SARS-CoV-2 pandemic, excess mortality is defined as “the difference between the number of deaths (from any cause) that occur during the pandemic and the number of deaths that would have occurred in the absence of the pandemic” [6]. Excess mortality is considered by scholars to be the gold standard for estimating the global impact of SARS-CoV-2 [7, 8], which is why numerous studies have estimated excess mortality for a large number of countries [9–18].

<figure><img src="image_2.png" /></figure>

<footnote>Florian Bonnet
florian.bonnet@ined.fr</footnote>

<footnote n="1">¹ French Institute for Demographic Studies (INED), 9 cours des Humanités, 93300 Aubervilliers, France</footnote>

<footnote n="2">² Federal Institute for Population Research (BiB), Wiesbaden, Germany</footnote>
# Spatial Variation in Excess Mortality Across Europe: A Cross-Sectional Study of 561 Regions in 21 Countries

Florian Bonnet¹ID • Pavel Grigoriev¹ID • Markus Sauerberg²ID • Ina Alliger<fnref n="2" /> • Michael Mühlichen²ID • Carlo-Giovanni Camarda¹ID

Received: 31 August 2023 / Accepted: 24 January 2024
© The Author(s) 2024

## Abstract

**Objective** To measure the burden of the COVID-19 pandemic in 2020 at the subnational level by estimating excess mortality, defined as the increase in all-cause mortality relative to an expected baseline mortality level.

**Methods** Statistical and demographic analyses of regional all-cause mortality data provided by the vital statistics systems of 21 European countries for 561 regions in Central and Western Europe. Life expectancy losses at ages 0 and 60 for males and females were estimated.

**Results** We found evidence of a loss in life expectancy in 391 regions, whilst only three regions exhibit notable gains in life expectancy in 2020. For 12 regions, losses of life expectancy amounted to more than 2 years and three regions showed losses greater than 3 years. We highlight geographical clusters of high mortality in Northern Italy, Spain and Poland, whilst clusters of low mortality were found in Western France, Germany/Denmark and Norway/Sweden.

**Conclusions** Regional differences of loss of life expectancy are impressive, ranging from a loss of more than 4 years to a gain of 8 months. These findings provide a strong rationale for regional analysis, as national estimates hide significant regional disparities.

**Keywords** COVID-19 · Excess mortality · European regions · Life expectancy · Geographical clusters

## Abbreviations

**NUTS** Nomenclature of Territorial Units for Statistics.

**IHME** Institute for Health Metrics and Evaluation

## 1 Introduction

The outbreak of SARS-CoV-2 triggered a strong reaction from public authorities, who responded in accordance with the threat that this new infectious disease entailed for the population. Despite having adopted social distancing measures that had never been seen in living memory in the twenty-first century, this pandemic led to most countries seeing significantly increased mortality [1], which the national surveillance authorities in Europe highlighted daily with

their published reports [2, 3]. Nowadays, the daily monitoring of the pandemic's evolution has been partially replaced by the quest to globally assess its mortality burden.

However, because these various national surveillance authorities were quickly established ad hoc to monitor the pandemic, use of the data they provide is broadly questioned for a wide range of reasons: different definitions of data amongst countries; time-varying collection methods; reporting delays; and diverse coverage by place of death [4, 5]. In contrast, the standard official statistics systems provide all-cause deaths, which are essential for computing excess mortality, defined as the difference between the deaths observed and the deaths that would have been observed under normal circumstances. In the context of the SARS-CoV-2 pandemic, excess mortality is defined as “the difference between the number of deaths (from any cause) that occur during the pandemic and the number of deaths that would have occurred in the absence of the pandemic” [6]. Excess mortality is considered by scholars to be the gold standard for estimating the global impact of SARS-CoV-2 [7, 8], which is why numerous studies have estimated excess mortality for a large number of countries [9–18].

<figure><img src="image_2.png" /></figure>

<footnote>Florian Bonnet
florian.bonnet@ined.fr</footnote>

<footnote n="1">¹ French Institute for Demographic Studies (INED), 9 cours des Humanités, 93300 Aubervilliers, France</footnote>

<footnote n="2">² Federal Institute for Population Research (BiB), Wiesbaden, Germany</footnote>
Whilst some of these studies evaluate the pandemic's impact on mortality by estimating the absolute or relative numbers of excess deaths, these figures are generally insufficient because they fail to consider changes in a population's age structure. For this reason, scholars prefer looking at age-specific mortality rates instead, as they can be collapsed into a summary measure such as the age-standardised death rate or period life expectancy at birth (hereinafter, shortened to simply "life expectancy"). Especially, the latter measure is a popular tool for quantifying period shocks in mortality because life expectancy readily expresses changes in mortality in terms of a longer or shorter average life span and are thus, comprehensible for a broad audience. The impact on changes in age-specific mortality rates on life expectancy trends is, however, more complex than often assumed (see e.g. [19–21]). Recently, [22] suggested interpreting a drop in life expectancy due to a mortality shock such as those incurred by the COVID-19 pandemic rather as a measure of premature mortality as compared to differences in mean longevity. This new interpretation might be suitable for our analysis because the mortality conditions observed in 2020 at the subnational level are unlikely to prevail in the future and thus, it is difficult to conceive the presented mortality changes as differences in average life spans [23].

To date, most of these studies estimate excess mortality for countries as a whole, with the undeniable risk of hiding intranational differences and hindering effective health policies within a pandemic context. Hence, in recent months, many papers have estimated regional excess mortality for specific countries [24–32]. However, comparing these regional patterns is relatively problematic because these studies take different approaches to compute the mortality levels that would have occurred without the pandemic. Specifically, they either use pre-pandemic levels or employ forecasting techniques. Moreover, these papers rely on different indicators to assess excess mortality, namely life expectancy or death toll. To our knowledge, only one study [33] allows for a simultaneous comparison of regional excess deaths in five European countries (Italy, Switzerland, Spain, Greece, England, and Wales) in 2020.

Our paper aims to fill this research gap by presenting life expectancy losses at birth in 2020 for 561 regions from 21 countries in Central and Western Europe. In dealing with small geographical areas, we focus on point-estimates as much as on uncertainty quantification. To obtain the mortality levels that would have been observed in 2020 in the absence of the pandemic, we took a robust forecasting approach that accounts for regional diversity and delivers analytic confidence intervals that account for all sources of uncertainty [34, 35]. Furthermore, this study also identifies geographical clusters of high and low losses of life expectancy.

# 2 Data and Methods

## 2.1 Data Preparation

We collected regional death and population counts for 21 European countries by age and sex from the national statistical offices, Eurostat, and the Human Mortality Database [36]. Since these data show varying age classes, we harmonised them into single-age intervals up to 95+ for all spatial units [37]. The lowest number of age groups in our input data is 16 (for Slovakia) and the largest age group that we ungrouped into single ages is 14 (for Germany, deaths at age 1 to 14). To ensure comparability in area size when selecting spatial units amongst the 21 countries, we relied mostly on level 3 (Czechia, Denmark, France, Italy, Luxembourg, Slovakia, Spain, Sweden) and level 2 (Austria, Belgium, England and Wales, Hungary, Iceland, Netherlands, Norway, Portugal, Switzerland, Slovenia) of the Nomenclature of Territorial Units for Statistics (NUTS 3 and NUTS 2). As exceptions, we used NUTS 1 for Ireland, Northern Ireland, and Scotland; and, for Germany, a national classification (“Raumordnungsregionen”) [38]. Minor adjustments were required due to territorial changes over time and data availability issues (see Table A1 in online Supplementary Information for details). In total, we analysed 561 harmonised spatial units containing populations ranging from 40,000 (Bornholm, Denmark) to 6,760,000 (Madrid, Spain).

Finally, we verified the data quality by comparing our regional data (aggregated at the national level) with those available in the Human Mortality Database. The differences between age-specific death rates calculated from the two data sources are almost nil for ages below 90, and very small for ages upper this limit. Thus, summary measures such as life expectancy at birth correspond to each other.

## 2.2 Methodology

When dealing with excess mortality, a central methodological issue concerns estimating the baseline mortality level, which is what would have been the expected level in the absence of the pandemic. Pre-pandemic mortality levels are often used as the baseline because they are easy to obtain and compute. However, because such a simplistic approach frequently ignores temporal trends, it is necessary to derive a more suitable expected mortality level in the absence of COVID-19 by forecasting pre-pandemic historical trends for the year affected by the pandemic (here, 2020). Among the numerous methodologies currently available (e.g. [39]), we opted for a CP-spline approach [35], which combines two-dimensional P-splines with prior demographical information derived from population-specific historical patterns.
Whilst some of these studies evaluate the pandemic's impact on mortality by estimating the absolute or relative numbers of excess deaths, these figures are generally insufficient because they fail to consider changes in a population's age structure. For this reason, scholars prefer looking at age-specific mortality rates instead, as they can be collapsed into a summary measure such as the age-standardised death rate or period life expectancy at birth (hereinafter, shortened to simply "life expectancy"). Especially, the latter measure is a popular tool for quantifying period shocks in mortality because life expectancy readily expresses changes in mortality in terms of a longer or shorter average life span and are thus, comprehensible for a broad audience. The impact on changes in age-specific mortality rates on life expectancy trends is, however, more complex than often assumed (see e.g. [19–21]). Recently, [22] suggested interpreting a drop in life expectancy due to a mortality shock such as those incurred by the COVID-19 pandemic rather as a measure of premature mortality as compared to differences in mean longevity. This new interpretation might be suitable for our analysis because the mortality conditions observed in 2020 at the subnational level are unlikely to prevail in the future and thus, it is difficult to conceive the presented mortality changes as differences in average life spans [23].

To date, most of these studies estimate excess mortality for countries as a whole, with the undeniable risk of hiding intranational differences and hindering effective health policies within a pandemic context. Hence, in recent months, many papers have estimated regional excess mortality for specific countries [24–32]. However, comparing these regional patterns is relatively problematic because these studies take different approaches to compute the mortality levels that would have occurred without the pandemic. Specifically, they either use pre-pandemic levels or employ forecasting techniques. Moreover, these papers rely on different indicators to assess excess mortality, namely life expectancy or death toll. To our knowledge, only one study [33] allows for a simultaneous comparison of regional excess deaths in five European countries (Italy, Switzerland, Spain, Greece, England, and Wales) in 2020.

Our paper aims to fill this research gap by presenting life expectancy losses at birth in 2020 for 561 regions from 21 countries in Central and Western Europe. In dealing with small geographical areas, we focus on point-estimates as much as on uncertainty quantification. To obtain the mortality levels that would have been observed in 2020 in the absence of the pandemic, we took a robust forecasting approach that accounts for regional diversity and delivers analytic confidence intervals that account for all sources of uncertainty [34, 35]. Furthermore, this study also identifies geographical clusters of high and low losses of life expectancy.

# 2 Data and Methods

## 2.1 Data Preparation

We collected regional death and population counts for 21 European countries by age and sex from the national statistical offices, Eurostat, and the Human Mortality Database [36]. Since these data show varying age classes, we harmonised them into single-age intervals up to 95+ for all spatial units [37]. The lowest number of age groups in our input data is 16 (for Slovakia) and the largest age group that we ungrouped into single ages is 14 (for Germany, deaths at age 1 to 14). To ensure comparability in area size when selecting spatial units amongst the 21 countries, we relied mostly on level 3 (Czechia, Denmark, France, Italy, Luxembourg, Slovakia, Spain, Sweden) and level 2 (Austria, Belgium, England and Wales, Hungary, Iceland, Netherlands, Norway, Portugal, Switzerland, Slovenia) of the Nomenclature of Territorial Units for Statistics (NUTS 3 and NUTS 2). As exceptions, we used NUTS 1 for Ireland, Northern Ireland, and Scotland; and, for Germany, a national classification (“Raumordnungsregionen”) [38]. Minor adjustments were required due to territorial changes over time and data availability issues (see Table A1 in online Supplementary Information for details). In total, we analysed 561 harmonised spatial units containing populations ranging from 40,000 (Bornholm, Denmark) to 6,760,000 (Madrid, Spain).

Finally, we verified the data quality by comparing our regional data (aggregated at the national level) with those available in the Human Mortality Database. The differences between age-specific death rates calculated from the two data sources are almost nil for ages below 90, and very small for ages upper this limit. Thus, summary measures such as life expectancy at birth correspond to each other.

## 2.2 Methodology

When dealing with excess mortality, a central methodological issue concerns estimating the baseline mortality level, which is what would have been the expected level in the absence of the pandemic. Pre-pandemic mortality levels are often used as the baseline because they are easy to obtain and compute. However, because such a simplistic approach frequently ignores temporal trends, it is necessary to derive a more suitable expected mortality level in the absence of COVID-19 by forecasting pre-pandemic historical trends for the year affected by the pandemic (here, 2020). Among the numerous methodologies currently available (e.g. [39]), we opted for a CP-spline approach [35], which combines two-dimensional P-splines with prior demographical information derived from population-specific historical patterns.

<figure><img src="image_3.png" /></figure>
A significant advantage to taking a non-parametric approach like that of CP-splines lies in its great flexibility for describing various mortality scenarios, which becomes particularly relevant when dealing with 561 diverse sub-populations across 21 European countries. Furthermore, smooth, plausible age profiles and time trends are ensured whilst we gain the added advantage of robustness when analysing small populations at risk. Moreover, the relatively low computation costs of CP-splines allow us to optimise region-specific time windows for our 2020 forecast values.

It is essential to quantify uncertainty before drawing any conclusions on excess mortality levels, particularly when dealing with regional mortality data. The error associated with the forecast of the life expectancy in 2020 must be added to the uncertainty around the observed 2020 mortality levels. Whereas the variance-covariance structure from CP-splines allows us to compute confidence intervals for our forecast values, a fully analytic procedure is used to compute uncertainty around the observed 2020 mortality levels. These two sources of uncertainty are then combined without performing any simulation or bootstrap procedures, thereby considerably reducing computational costs and time. A detailed description of the analytic procedure is available in online Supplementary Information C which includes a validation of our approach through a comparison with the traditional bootstrap procedure. To replicate our methods, the codes we applied in R as well as the population and death counts for France's 95 study regions are available in [34]. All calculations are performed in R version 4.0.3 [40].

It is noteworthy that the whole procedure can be performed for any given age and regardless of the mortality indicator selected for estimating excess mortality (e.g. life expectancy or age-standardised death rate). Our study focuses on losses in life expectancy at birth and at age 60, rather than on total excess deaths. A key advantage of our two indicators is that they remain unaffected by the age structure of the population, which differs greatly from one region to another in Europe.

To identify any spatial clustering of losses of life expectancy, we rely on the Getis-Ord Gi* statistic [41], which allows us to locate hot and cold spots, that is, neighbouring areas of high concentration of spatial units with either large or small loss of life expectancy. In practice, Gi* is calculated for all estimated losses of life expectancy in our dataset, and the resultant z-scores and p-values are evaluated within a defined neighbourhood. Here, we used the contiguity edges corners definition of neighbourhood, which specifically considers neighbours to be units that share a boundary, share a node, or overlap. To be a statistically significant hot spot, a spatial unit should not only have a high value but also be surrounded by other units with high values. The local sum for a given unit and its neighbours is evaluated against the sum of all spatial units. If the observed local sum is very

different from the expected local sum, then this difference is not the result of random chance, and the corresponding z-score is statistically significant. The same logic is applied for identifying cold spots. This well-established methodology has been implemented and thoroughly documented in ArcMap software (module Spatial Statistics Tools) [42], which we used to perform the respective analysis as well as to produce mortality maps. Required shapefiles were collected from Eurostat [43] and were re-projected using the Winkel tripel projection.

## 3 Results

Figure 1 shows the spatial distribution of losses of life expectancy at birth across 561 territorial units in 21 European countries during 2020. Light blue represents gains in life expectancy compared to expected values, whereas the remaining colours indicate life expectancy losses. Among all the countries considered in our analysis, those most affected by the pandemic appear to be Italy, Spain, and Poland. In most Polish regions as well as in many Spanish and Italian regions, losses in both male and female life expectancy amounted to more than 1 year. This fall in life expectancy observed within just one calendar year is unprecedented in twenty-first century Europe [9, 10]. Interestingly, they are amongst the areas with both the highest longevity (Spanish and Italian regions) and the lowest longevity (Polish regions) in Europe (see Figures A1 and A2 in the online Supplementary Information A). In contrast, most regions in Denmark, Germany, Norway, and western France, as well as some rural areas in Sweden, showed only moderate declines or even slight increases in life expectancy in 2020 compared to their expected values.

Although previous works could only show national differences in excess mortality associated with the COVID-19 pandemic, our regional analysis is able to highlight substantial within-country variations at the subnational level. Italy represents a notable example, with the whole country showing a loss of life expectancy at birth of 1.16 years for both sexes combined. However, this national value conceals both a loss of 4.18 years in the Bergamo province in the north (CI: -3.95 to -4.4) and a gain of 0.4 years in the Sicilian province of Caltanissetta, even if non-significant (CI: -0.17 to 0.92). Moreover, we can see in Fig. 3 that 95% confidence intervals are plotted along estimated excess mortality in life expectancy at birth. Other examples of remarkable subnational differences can be found in France and the United Kingdom. In the former case, higher excess mortality is visible in the north-east compared to the south-west regions. In the latter, south-west shows a moderate gain, unlike the central parts of England that experience high losses. Spatial variation is also evident in Germany, a country with
A significant advantage to taking a non-parametric approach like that of CP-splines lies in its great flexibility for describing various mortality scenarios, which becomes particularly relevant when dealing with 561 diverse sub-populations across 21 European countries. Furthermore, smooth, plausible age profiles and time trends are ensured whilst we gain the added advantage of robustness when analysing small populations at risk. Moreover, the relatively low computation costs of CP-splines allow us to optimise region-specific time windows for our 2020 forecast values.

It is essential to quantify uncertainty before drawing any conclusions on excess mortality levels, particularly when dealing with regional mortality data. The error associated with the forecast of the life expectancy in 2020 must be added to the uncertainty around the observed 2020 mortality levels. Whereas the variance-covariance structure from CP-splines allows us to compute confidence intervals for our forecast values, a fully analytic procedure is used to compute uncertainty around the observed 2020 mortality levels. These two sources of uncertainty are then combined without performing any simulation or bootstrap procedures, thereby considerably reducing computational costs and time. A detailed description of the analytic procedure is available in online Supplementary Information C which includes a validation of our approach through a comparison with the traditional bootstrap procedure. To replicate our methods, the codes we applied in R as well as the population and death counts for France's 95 study regions are available in [34]. All calculations are performed in R version 4.0.3 [40].

It is noteworthy that the whole procedure can be performed for any given age and regardless of the mortality indicator selected for estimating excess mortality (e.g. life expectancy or age-standardised death rate). Our study focuses on losses in life expectancy at birth and at age 60, rather than on total excess deaths. A key advantage of our two indicators is that they remain unaffected by the age structure of the population, which differs greatly from one region to another in Europe.

To identify any spatial clustering of losses of life expectancy, we rely on the Getis-Ord Gi* statistic [41], which allows us to locate hot and cold spots, that is, neighbouring areas of high concentration of spatial units with either large or small loss of life expectancy. In practice, Gi* is calculated for all estimated losses of life expectancy in our dataset, and the resultant z-scores and p-values are evaluated within a defined neighbourhood. Here, we used the contiguity edges corners definition of neighbourhood, which specifically considers neighbours to be units that share a boundary, share a node, or overlap. To be a statistically significant hot spot, a spatial unit should not only have a high value but also be surrounded by other units with high values. The local sum for a given unit and its neighbours is evaluated against the sum of all spatial units. If the observed local sum is very

different from the expected local sum, then this difference is not the result of random chance, and the corresponding z-score is statistically significant. The same logic is applied for identifying cold spots. This well-established methodology has been implemented and thoroughly documented in ArcMap software (module Spatial Statistics Tools) [42], which we used to perform the respective analysis as well as to produce mortality maps. Required shapefiles were collected from Eurostat [43] and were re-projected using the Winkel tripel projection.

## 3 Results

Figure 1 shows the spatial distribution of losses of life expectancy at birth across 561 territorial units in 21 European countries during 2020. Light blue represents gains in life expectancy compared to expected values, whereas the remaining colours indicate life expectancy losses. Among all the countries considered in our analysis, those most affected by the pandemic appear to be Italy, Spain, and Poland. In most Polish regions as well as in many Spanish and Italian regions, losses in both male and female life expectancy amounted to more than 1 year. This fall in life expectancy observed within just one calendar year is unprecedented in twenty-first century Europe [9, 10]. Interestingly, they are amongst the areas with both the highest longevity (Spanish and Italian regions) and the lowest longevity (Polish regions) in Europe (see Figures A1 and A2 in the online Supplementary Information A). In contrast, most regions in Denmark, Germany, Norway, and western France, as well as some rural areas in Sweden, showed only moderate declines or even slight increases in life expectancy in 2020 compared to their expected values.

Although previous works could only show national differences in excess mortality associated with the COVID-19 pandemic, our regional analysis is able to highlight substantial within-country variations at the subnational level. Italy represents a notable example, with the whole country showing a loss of life expectancy at birth of 1.16 years for both sexes combined. However, this national value conceals both a loss of 4.18 years in the Bergamo province in the north (CI: -3.95 to -4.4) and a gain of 0.4 years in the Sicilian province of Caltanissetta, even if non-significant (CI: -0.17 to 0.92). Moreover, we can see in Fig. 3 that 95% confidence intervals are plotted along estimated excess mortality in life expectancy at birth. Other examples of remarkable subnational differences can be found in France and the United Kingdom. In the former case, higher excess mortality is visible in the north-east compared to the south-west regions. In the latter, south-west shows a moderate gain, unlike the central parts of England that experience high losses. Spatial variation is also evident in Germany, a country with
<figure><img src="image_3.png" /><figcaption>**Fig. 1** Spatial distribution of losses of life expectancy at birth (years) across 21 European countries in 2020. Source: As shown in Table A1</figcaption></figure>

relatively better performance during the first pandemic wave in 2020. Specifically, its eastern and south-eastern regions bordering Poland, Czechia, and Austria show higher life expectancy losses than the rest of the country.

In contrast, although Poland shows life expectancy losses similar to Italy (1.32 year), there is considerably less sub-national variation between the highest and lowest affected regions, with Jeleniogorski in the south showing a loss of 0.7 years (CI: -0.16 to -1.09) and Przemyski in the east a loss of 2.17 years (CI: -1.7 to -2.65).

The analysis of spatial distribution of losses of life expectancy at age 60 has revealed very similar results (see Figure A3 in the online Supplementary Information A).

Our next step was to analyse the hot and cold spots for losses of life expectancy at birth (Fig. 2) and at age 60 (Fig. A4 in the online Supplementary Information A). This analysis statistically confirms our previous visual inspection: hot spots of life expectancy losses were indeed located in northern Italy and covered most of the Polish and Spanish territory; whereas cold spots were located in Denmark, Germany, western France, and Norway. Furthermore, our analysis identifies several small clusters of elevated excess mortality expanding across national borders. One example can be seen in an area covering parts of northern Italy and southern Switzerland, and—at least amongst women—spreading into eastern France. Another cross-national hot

spot expands from southern Poland into eastern Czechia and—again, amongst women—into northern Slovakia. These observations may well indicate possible spill-over effects of the pandemic burden across European countries.

Figure 3 ranks the life expectancy losses and shows the associated 95% confidence intervals (CI) for all 561 spatial units. This figure also allows us to visually assess the magnitude of uncertainty around point estimates. The Panel A colours correspond to the colours and cut points presented in Fig. 1. Panels B and C depict the top and bottom 30 regions, respectively. Figures A5 and A6 in the Supplementary Information A separately show the same distributions for men and women.

Figure 3 also reveals the large regional variability in excess mortality across Europe. The aforementioned Italian Bergamo province experienced the highest loss of life expectancy, amounting to 4.18 years (CI: -3.95 to -4.4), whereas the Ariège department in France benefited from the highest gain of 0.65 years, even if non-significant (CI: -0.13 to 1.45). Only a few regions experienced life expectancy gains for which the 95% confidence intervals do not include 0: Devon, Cornwall, and Scilly Island in the UK and Nordjylland in Denmark. Overall, 391 out of 561 regions suffered consequential life expectancy losses, meaning that the associated 95% confidence intervals do not contain 0. The tail of the distribution (Panel C) is dominated by northern Italian
<figure><img src="image_4.png" /><figcaption>**Fig. 1** Spatial distribution of losses of life expectancy at birth (years) across 21 European countries in 2020. Source: As shown in Table A1</figcaption></figure>

relatively better performance during the first pandemic wave in 2020. Specifically, its eastern and south-eastern regions bordering Poland, Czechia, and Austria show higher life expectancy losses than the rest of the country.

In contrast, although Poland shows life expectancy losses similar to Italy (1.32 year), there is considerably less sub-national variation between the highest and lowest affected regions, with Jeleniogorski in the south showing a loss of 0.7 years (CI: -0.16 to -1.09) and Przemyski in the east a loss of 2.17 years (CI: -1.7 to -2.65).

The analysis of spatial distribution of losses of life expectancy at age 60 has revealed very similar results (see Figure A3 in the online Supplementary Information A).

Our next step was to analyse the hot and cold spots for losses of life expectancy at birth (Fig. 2) and at age 60 (Fig. A4 in the online Supplementary Information A). This analysis statistically confirms our previous visual inspection: hot spots of life expectancy losses were indeed located in northern Italy and covered most of the Polish and Spanish territory; whereas cold spots were located in Denmark, Germany, western France, and Norway. Furthermore, our analysis identifies several small clusters of elevated excess mortality expanding across national borders. One example can be seen in an area covering parts of northern Italy and southern Switzerland, and—at least amongst women—spreading into eastern France. Another cross-national hot

spot expands from southern Poland into eastern Czechia and—again, amongst women—into northern Slovakia. These observations may well indicate possible spill-over effects of the pandemic burden across European countries.

Figure 3 ranks the life expectancy losses and shows the associated 95% confidence intervals (CI) for all 561 spatial units. This figure also allows us to visually assess the magnitude of uncertainty around point estimates. The Panel A colours correspond to the colours and cut points presented in Fig. 1. Panels B and C depict the top and bottom 30 regions, respectively. Figures A5 and A6 in the Supplementary Information A separately show the same distributions for men and women.

Figure 3 also reveals the large regional variability in excess mortality across Europe. The aforementioned Italian Bergamo province experienced the highest loss of life expectancy, amounting to 4.18 years (CI: -3.95 to -4.4), whereas the Ariège department in France benefited from the highest gain of 0.65 years, even if non-significant (CI: -0.13 to 1.45). Only a few regions experienced life expectancy gains for which the 95% confidence intervals do not include 0: Devon, Cornwall, and Scilly Island in the UK and Nordjylland in Denmark. Overall, 391 out of 561 regions suffered consequential life expectancy losses, meaning that the associated 95% confidence intervals do not contain 0. The tail of the distribution (Panel C) is dominated by northern Italian

<figure><img src="image_5.png" /></figure>
<figure><img src="image_4.png" /><figcaption>**Fig. 2** Hot and cold spots of losses of life expectancy at birth across 21 European countries in 2020. Source: as shown in Table A1</figcaption></figure>

and central Spanish regions. Among those harshly hit by the pandemic, 12 regions experienced losses of life expectancy greater than 2 years, with the Italian provinces of Piacenza, Cremona, and Bergamo experiencing more than 3 years of losses. As expected, the group of regions least affected by the pandemic (Panel B) is dominated by German, French, and Danish spatial units. However, this successful group also includes some Italian and UK regions. The numerical values of losses of life expectancy at birth and at age 60 for each spatial unit and for both females and males are provided in the online Supplementary Information B.

In addition to revealing intranational heterogeneity and cross-national patterns, using subnational data has the advantage of enabling a deeper analysis of excess mortality. Moreover, the uncommonly large number of spatial units also increases statistical power in these supplementary analyses. For instance, we can easily assess the association between sex differences in life expectancy loss and the overall regional loss. Figure 4 displays the results of this closer examination using a modified version of a Bland-Altman plot, where the differences between female and male changes in life expectancy are plotted against the corresponding overall regional change. Each dot in the plot is sized proportionally to the population of its respective region.

As already clear from previous analysis, the majority of regions present an overall loss in life expectancy and this

is indicated by the large number of dots in the red areas of Fig. 4. The average difference in excess mortality between males and females is clearly indicated by the mean line, showing that males have experienced over three months of additional loss on average. Regions with higher losses amongst males are represented by dots below the horizontal zero line (lighter red), whilst regions with greater losses amongst females are indicated by dots above this line (darker red).

The identification of outliers is particularly noteworthy. Here, we identify outliers by depicting spatial units lying beyond the 95% limits of agreement of the Bland-Altman plot estimated as prediction intervals of a mean model. Notably, nearly all outliers amongst regions with greater losses of males belong to areas identified as hot spots (Italy, Poland, and Spain). Nevertheless, some outliers also show greater female losses, and these are located predominantly in France.

# 4 Discussion

## 4.1 Statement of Principal Findings

This large study of 561 territorial units in 21 countries uses loss of life expectancy at birth and at age 60 to
<figure><img src="image_6.png" /><figcaption>**Fig. 2** Hot and cold spots of losses of life expectancy at birth across 21 European countries in 2020. Source: as shown in Table A1</figcaption></figure>

and central Spanish regions. Among those harshly hit by the pandemic, 12 regions experienced losses of life expectancy greater than 2 years, with the Italian provinces of Piacenza, Cremona, and Bergamo experiencing more than 3 years of losses. As expected, the group of regions least affected by the pandemic (Panel B) is dominated by German, French, and Danish spatial units. However, this successful group also includes some Italian and UK regions. The numerical values of losses of life expectancy at birth and at age 60 for each spatial unit and for both females and males are provided in the online Supplementary Information B.

In addition to revealing intranational heterogeneity and cross-national patterns, using subnational data has the advantage of enabling a deeper analysis of excess mortality. Moreover, the uncommonly large number of spatial units also increases statistical power in these supplementary analyses. For instance, we can easily assess the association between sex differences in life expectancy loss and the overall regional loss. Figure 4 displays the results of this closer examination using a modified version of a Bland–Altman plot, where the differences between female and male changes in life expectancy are plotted against the corresponding overall regional change. Each dot in the plot is sized proportionally to the population of its respective region.

As already clear from previous analysis, the majority of regions present an overall loss in life expectancy and this

is indicated by the large number of dots in the red areas of Fig. 4. The average difference in excess mortality between males and females is clearly indicated by the mean line, showing that males have experienced over three months of additional loss on average. Regions with higher losses amongst males are represented by dots below the horizontal zero line (lighter red), whilst regions with greater losses amongst females are indicated by dots above this line (darker red).

The identification of outliers is particularly noteworthy. Here, we identify outliers by depicting spatial units lying beyond the 95% limits of agreement of the Bland–Altman plot estimated as prediction intervals of a mean model. Notably, nearly all outliers amongst regions with greater losses of males belong to areas identified as hot spots (Italy, Poland, and Spain). Nevertheless, some outliers also show greater female losses, and these are located predominantly in France.

# 4 Discussion

## 4.1 Statement of Principal Findings

This large study of 561 territorial units in 21 countries uses loss of life expectancy at birth and at age 60 to
<figure><img src="image_5.png" /><figcaption>**Fig. 3** Losses or gains of life expectancy at birth (e0) associated with COVID-19 pandemic across 561 spatial units in 21 European countries, 2020, both sexes combined. Source: As shown in Table A1.</figcaption></figure>

Notes: (1) Excess mortality is estimated as the difference between observed and expected life expectancy at birth (2) Horizontal bars represent 95% confidence intervals

measure the burden of COVID-19 in 2020 throughout Western and Central Europe. Our findings focus on life expectancy at birth and show evidence of losses in 391 regions, whilst only three regions exhibit notable gains during the first year of pandemic. Overall, those most affected were central Spain, northern Italy, and eastern Poland, whilst Denmark, western France, Germany, and Norway were somewhat spared from the COVID-19

burden. Regional differences in life expectancy at birth changes were impressive, ranging from a loss of more than 4 years to a gain of 8 months. For 12 regions, losses amounted to more than 2 years, and three regions showed losses greater than 3 years. Our results regarding losses in life expectancy at age 60 are similar in magnitude since the deaths were mainly observed above this age.

<figure><img src="image_6.png" /></figure>
<figure><img src="image_7.png" /><figcaption>**Fig. 3** Losses or gains of life expectancy at birth (e0) associated with COVID-19 pandemic across 561 spatial units in 21 European countries, 2020, both sexes combined. Source: As shown in Table A1.</figcaption></figure>

Notes: (1) Excess mortality is estimated as the difference between observed and expected life expectancy at birth (2) Horizontal bars represent 95% confidence intervals

measure the burden of COVID-19 in 2020 throughout Western and Central Europe. Our findings focus on life expectancy at birth and show evidence of losses in 391 regions, whilst only three regions exhibit notable gains during the first year of pandemic. Overall, those most affected were central Spain, northern Italy, and eastern Poland, whilst Denmark, western France, Germany, and Norway were somewhat spared from the COVID-19

burden. Regional differences in life expectancy at birth changes were impressive, ranging from a loss of more than 4 years to a gain of 8 months. For 12 regions, losses amounted to more than 2 years, and three regions showed losses greater than 3 years. Our results regarding losses in life expectancy at age 60 are similar in magnitude since the deaths were mainly observed above this age.

<figure><img src="image_8.png" /></figure>
<figure><img src="image_7.png" /><figcaption>**Fig. 4** A modified version of a Bland-Altman plot to portray association between differences between female and male losses of life expectancy at birth and overall regional changes across 561 spatial units in 21 European countries, 2020 Source: As shown in Table A1</figcaption></figure>

This study further reveals that regional differences in excess mortality were also highly substantial within some countries. In Italy, for example, we estimated a 4-year life expectancy loss in the Bergamo province but no loss in the Caltanissetta province. However, the regional differences in one country can differ greatly from those in another. For example, although Italy and Poland both have similar losses of life expectancy at birth at the national level, the gap between the highest and lowest afflicted regions is three times larger in Italy than in Poland.

Geographical clusters identify where life expectancy was much lower than expected (hot spot), and others where life expectancy was in line with expectations or even higher (cold spot). Notwithstanding, these analyses pinpoint clusters where excess mortality expanded beyond national borders in certain European areas: The hot spots observed in northern Italy spilled over into southern Switzerland and eastern France, and other hot spots observed in Poland spilled over into northern Slovakia and eastern Czechia.

Meanwhile, this study singles out a cold spot located in two nearby countries (Germany and Denmark).

We observe differences between women and men in their life expectancy losses in 2020. In most analysed regions, reduced life expectancy was greater amongst men. For example, in the worst hit regions in northern Italy and central Spain, men's losses were more than 1 year higher as compared to their female counterparts. Widening sex differentials in mortality during the COVID-19 pandemic have been reported previously [1, 44, 45]. Yet, we also find more pronounced life expectancy losses for women in many European regions where drops in life expectancy were more moderate. This is relevant for the discussion on causes for male excess deaths, i.e. biological mechanisms vs. social mechanism [46]. As we observe substantial variability in the magnitude and even direction of sex differences in life expectancy losses, our results indicate that the degree of mortality deterioration is not necessarily linked to biological sex.
<figure><img src="image_9.png" /><figcaption>**Fig. 4** A modified version of a Bland-Altman plot to portray association between differences between female and male losses of life expectancy at birth and overall regional changes across 561 spatial units in 21 European countries, 2020 Source: As shown in Table A1</figcaption></figure>

This study further reveals that regional differences in excess mortality were also highly substantial within some countries. In Italy, for example, we estimated a 4-year life expectancy loss in the Bergamo province but no loss in the Caltanissetta province. However, the regional differences in one country can differ greatly from those in another. For example, although Italy and Poland both have similar losses of life expectancy at birth at the national level, the gap between the highest and lowest afflicted regions is three times larger in Italy than in Poland.

Geographical clusters identify where life expectancy was much lower than expected (hot spot), and others where life expectancy was in line with expectations or even higher (cold spot). Notwithstanding, these analyses pinpoint clusters where excess mortality expanded beyond national borders in certain European areas: The hot spots observed in northern Italy spilled over into southern Switzerland and eastern France, and other hot spots observed in Poland spilled over into northern Slovakia and eastern Czechia.

Meanwhile, this study singles out a cold spot located in two nearby countries (Germany and Denmark).

We observe differences between women and men in their life expectancy losses in 2020. In most analysed regions, reduced life expectancy was greater amongst men. For example, in the worst hit regions in northern Italy and central Spain, men's losses were more than 1 year higher as compared to their female counterparts. Widening sex differentials in mortality during the COVID-19 pandemic have been reported previously [1, 44, 45]. Yet, we also find more pronounced life expectancy losses for women in many European regions where drops in life expectancy were more moderate. This is relevant for the discussion on causes for male excess deaths, i.e. biological mechanisms vs. social mechanism [46]. As we observe substantial variability in the magnitude and even direction of sex differences in life expectancy losses, our results indicate that the degree of mortality deterioration is not necessarily linked to biological sex.
## 4.2 Strengths and Limitations

This study is the first to use a large set of small territorial units to provide estimates of loss of life expectancy during the COVID-19 pandemic in 2020. The contiguity of the regions investigated has allowed us to accurately identify geographical clusters of elevated mortality. This approach is novel because most previous studies have focussed primarily on single-country regions or on regions in various countries with no common borders.

Contrary to many other studies on COVID-19-related deaths, we estimated losses of life expectancy using official mortality data routinely collected by vital registration systems, which are less sensitive to reporting delays and misclassification. To verify that each country's sums of its regional data were consistent for all ages, we verified them with data from the Human Mortality Database. This is especially crucial, not only for calculating outcomes at older ages, which require statistical techniques to obtain single-year-of-age data, but also considering how COVID-19 was particularly dangerous for older individuals. Finally, we computed baseline mortality using an up-to-date statistical methodology that optimises the time windows in our models for forecasting regional trends in 2020. We compared our country-specific estimates with results from previous studies to evaluate our findings in terms of reliability. Our estimates for losses in life expectancy are consistent with results from Aburto et al. [9] and Islam et al. [15] (see table A2).

Our study also has several limitations. It covers 21 Central and Western European countries and still cannot be extended to Eastern European countries. However, comparisons with these countries would have been interesting insofar as this global health crisis triggered different political responses. Furthermore, this study covers only the year 2020. Given the experience gained by policymakers during the first waves of the pandemic and the emergence of new variants, the results for 2021 and 2022 should be different. As soon as more up-to-date data become available, the analyses we performed can be easily extended along both the spatial and temporal dimensions.

Whilst our adopted forecast approach is robust and flexible enough to be adapted to different demographic scenarios and relatively small populations, we model each spatial unit independently and without considering spatial autocorrelation. By including a spatial structure in modeling and forecasting mortality, this may eventually reduce uncertainty around excess mortality estimates and thus lead to more explicit outcomes. Nonetheless, it remains essential to account for temporal trends when estimating mortality levels in the absence of a pandemic and, consequently, compute excess mortality more accurately [15, 17, 33]. However, other approaches in computing baseline mortality levels

have been proposed [47–49], and these could be adopted in order to obtain alternative perspectives.

## 4.3 Policy Implications and Future Research

Our study extends the existing literature on excess mortality during the COVID-19 pandemic in 2020 by reporting results at the regional level for many European countries, which has not been done so far. The findings provide a strong rationale for this regional analysis, as we show that national estimates would have hidden significant regional disparities. Policy-makers should be made aware of this intranational heterogeneity to fully assess the burden of the pandemic in their own countries and adopt differentiated health policy responses.

Whist our study quantifies regional differences in excess mortality, it cannot provide avenues for explaining them. The first step towards doing so would be to link these estimates to regional contextual variables (such as income level or occupational structure) as well as to public policies of social distancing and international isolation, which were implemented at both regional and national levels. Eventually, ecological analyses should be complemented with carefully designed epidemiological studies. In this way, one will be able to highlight the decisive factors explaining these regional differences and ultimately gain a better understanding of how to deal with the spread of a new infectious disease. For example, the exceptional mortality observed in northern Italy might be due to the early onset of the pandemic, which triggered a strong public response from the Italian government that limited the spread of the pandemic to the southern parts of the country [50].

It should be emphasised that this study estimates the burden of the pandemic by calculating excess mortality, which is the overall net assessment of the pandemic's impact on society and must be distinguished from the mortality count due to COVID-19. More specifically, the Institute for Health Metrics and Evaluation (IHME) considers excess mortality to be influenced by six drivers related to the pandemic and the response to it, which are namely: (i) deaths directly related to SARS-CoV-2 infection; (ii) increased mortality due to cancelling or postponing care; (iii) increased mortality due to mental health problems, including depression, increased alcohol use, and increased opiate use; (iv) reduced mortality from decreases in accidents because of reduced mobility; (v) reduced mortality due to reduced transmission of other viruses, including influenza; and (vi) reduced mortality from many chronic diseases such as cardiovascular and respiratory diseases that would have killed frail individuals had COVID-19 not killed them instead. When detailed cause-of-death statistics become available, an accurate assessment must be made of COVID-19 mortality, as well as the distinction between direct and indirect COVID-19 mortality.

<figure><img src="image_8.png" /></figure>
## 4.2 Strengths and Limitations

This study is the first to use a large set of small territorial units to provide estimates of loss of life expectancy during the COVID-19 pandemic in 2020. The contiguity of the regions investigated has allowed us to accurately identify geographical clusters of elevated mortality. This approach is novel because most previous studies have focussed primarily on single-country regions or on regions in various countries with no common borders.

Contrary to many other studies on COVID-19-related deaths, we estimated losses of life expectancy using official mortality data routinely collected by vital registration systems, which are less sensitive to reporting delays and misclassification. To verify that each country's sums of its regional data were consistent for all ages, we verified them with data from the Human Mortality Database. This is especially crucial, not only for calculating outcomes at older ages, which require statistical techniques to obtain single-year-of-age data, but also considering how COVID-19 was particularly dangerous for older individuals. Finally, we computed baseline mortality using an up-to-date statistical methodology that optimises the time windows in our models for forecasting regional trends in 2020. We compared our country-specific estimates with results from previous studies to evaluate our findings in terms of reliability. Our estimates for losses in life expectancy are consistent with results from Aburto et al. [9] and Islam et al. [15] (see table A2).

Our study also has several limitations. It covers 21 Central and Western European countries and still cannot be extended to Eastern European countries. However, comparisons with these countries would have been interesting insofar as this global health crisis triggered different political responses. Furthermore, this study covers only the year 2020. Given the experience gained by policymakers during the first waves of the pandemic and the emergence of new variants, the results for 2021 and 2022 should be different. As soon as more up-to-date data become available, the analyses we performed can be easily extended along both the spatial and temporal dimensions.

Whilst our adopted forecast approach is robust and flexible enough to be adapted to different demographic scenarios and relatively small populations, we model each spatial unit independently and without considering spatial autocorrelation. By including a spatial structure in modeling and forecasting mortality, this may eventually reduce uncertainty around excess mortality estimates and thus lead to more explicit outcomes. Nonetheless, it remains essential to account for temporal trends when estimating mortality levels in the absence of a pandemic and, consequently, compute excess mortality more accurately [15, 17, 33]. However, other approaches in computing baseline mortality levels

have been proposed [47–49], and these could be adopted in order to obtain alternative perspectives.

## 4.3 Policy Implications and Future Research

Our study extends the existing literature on excess mortality during the COVID-19 pandemic in 2020 by reporting results at the regional level for many European countries, which has not been done so far. The findings provide a strong rationale for this regional analysis, as we show that national estimates would have hidden significant regional disparities. Policy-makers should be made aware of this intranational heterogeneity to fully assess the burden of the pandemic in their own countries and adopt differentiated health policy responses.

Whist our study quantifies regional differences in excess mortality, it cannot provide avenues for explaining them. The first step towards doing so would be to link these estimates to regional contextual variables (such as income level or occupational structure) as well as to public policies of social distancing and international isolation, which were implemented at both regional and national levels. Eventually, ecological analyses should be complemented with carefully designed epidemiological studies. In this way, one will be able to highlight the decisive factors explaining these regional differences and ultimately gain a better understanding of how to deal with the spread of a new infectious disease. For example, the exceptional mortality observed in northern Italy might be due to the early onset of the pandemic, which triggered a strong public response from the Italian government that limited the spread of the pandemic to the southern parts of the country [50].

It should be emphasised that this study estimates the burden of the pandemic by calculating excess mortality, which is the overall net assessment of the pandemic's impact on society and must be distinguished from the mortality count due to COVID-19. More specifically, the Institute for Health Metrics and Evaluation (IHME) considers excess mortality to be influenced by six drivers related to the pandemic and the response to it, which are namely: (i) deaths directly related to SARS-CoV-2 infection; (ii) increased mortality due to cancelling or postponing care; (iii) increased mortality due to mental health problems, including depression, increased alcohol use, and increased opiate use; (iv) reduced mortality from decreases in accidents because of reduced mobility; (v) reduced mortality due to reduced transmission of other viruses, including influenza; and (vi) reduced mortality from many chronic diseases such as cardiovascular and respiratory diseases that would have killed frail individuals had COVID-19 not killed them instead. When detailed cause-of-death statistics become available, an accurate assessment must be made of COVID-19 mortality, as well as the distinction between direct and indirect COVID-19 mortality.

<figure><img src="image_10.png" /></figure>
Finally, this study critically depends on reliable data being made available from the national statistical institutes of the different European countries analysed here. The magnitude of our presented results should encourage policymakers to promptly provide regional data to better measure the burden of future epidemics in a timely fashion. To produce accurate analyses in times of demographical ageing, it is of utmost importance to harmonise European population and mortality data with consistent age groups and an oldest age class of at least 100 years upwards.

**Supplementary Information** The online version contains supplementary material available at https://doi.org/10.1007/s44197-024-00200-0.

**Acknowledgements** We are very grateful to Markéta Majerová, Rok Hrzic, Magdalena Muszyńska-Spielauer, and Mathias Lerch for providing, respectively, Czech, Slovenian, Austrian, and Swiss data.

**Author Contributions** Conceptualization: FB, MS, PG, CGC, MM; methodology: CGC, FB, PG; data collection and curation: MS, FB, IA, MM, PG, CGC; formal analysis: FB, PG; writing—Original draft: PG, FB, CGC, MM, MS; writing—review and editing: PG, FB, CGC, MS, MM, IA; visualisation: MS, FB, PG, CGC.

**Funding** IA, PG, MM, and MS were supported by funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreement No 851485).

**Data Availability** Detailed values of our estimates and data visualisation tool are available at: https://osf.io/h68wz/?view_only=47353f6f4b2e41cab3c761246e59d615.

## Declarations

**Conflict of Interest** The authors have no relevant financial or non-financial interests to disclose.

**Ethics Approval and Consent to Participate** No individual participated to this study.

**Consent for Publication** No individual participated to this study.

**Open Access** This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

## References

1. Schöley J, Aburto JM, Kashnitsky I, Kniffka MS, Zhang L, Jaadla H, et al. Life expectancy changes since COVID-19. Nat Hum Behav. 2022;6:1649–59.

2. Caporali A, Garcia J, Couppié É, Poniakina S, Barbieri M, Bonnet F, et al. The demography of COVID-19 deaths database, a gateway to well-documented international data. Scientific Data. 2022;9:1–9.

3. Riffe T, Acosta E, The COVerAGE-DB Team. Data resource profile: COVerAGE-DB: a global demographic database of COVID-19 cases and deaths. Int J Epidemiol. 2021;50:390–390f.

4. Garcia J, Torres C, Barbieri M, Cambois E, Camarda CG, Caporali A, et al. Differences in COVID-19 mortality: the implications of imperfect and diverse data collection systems. Population. 2021;76:35–72.

5. Pathak EB, Garcia RB, Menard JM, Salemi JL. Out-of-hospital COVID-19 deaths: consequences for quality of medical care and accuracy of cause of death coding. Am J Public Health. 2021;111:S101–6.

6. Helleringer S, Queiroz BL. Commentary: measuring excess mortality due to the COVID-19 pandemic: progress and persistent challenges. Int J Epidemiol. 2022;51:85–7.

7. Beaney T, Clarke JM, Jain V, Golestaneh AK, Lyons G, Salman D, et al. Excess mortality: the gold standard in measuring the impact of COVID-19 worldwide? J R Soc Med. 2020;113:329–34.

8. Karlinsky A, Kobak D. Tracking excess mortality across countries during the COVID-19 pandemic with the World Mortality Dataset. Elife. 2021;10:e69336.

9. Aburto JM, Schöley J, Kashnitsky I, Zhang L, Rahal C, Missov TI, et al. Quantifying impacts of the COVID-19 pandemic through life-expectancy losses: a population-level study of 29 countries. Int J Epidemiol. 2022;51:63–74.

10. Mazzuco S, Campostrini S. Life expectancy drop in 2020 Estimates based on Human Mortality Database. PLoS ONE. 2022;17:e0262846.

11. Aburto J, Schöley J, Kashnitsky I, Kashyap R. Life expectancy declines in Russia during the Covid-19 pandemic in 2020. Int J Epidemiol. 2022;51:1695–7.

12. Aburto JM, Kashyap R, Schöley J, Angus C, Ermisch J, Mills MC, et al. Estimating the burden of the COVID-19 pandemic on mortality, life expectancy and lifespan inequality in England and Wales: a population-level analysis. J Epidemiol Community Health. 2021;75:735–40.

13. Peretz C, Rotem N, Keinan-Boker L, Furshpan A, Green M, Bitan M, et al. Excess mortality in Israel associated with COVID-19 in 2020–2021 by age group and with estimates based on daily mortality patterns in 2000–2019. Int J Epidemiol. 2022;51:727–36.

14. Kontopantelis E, Mamas MA, Deanfield J, Asaria M, Doran T. Excess mortality in England and Wales during the first wave of the COVID-19 pandemic. J Epidemiol Community Health. 2021;75:213–23.

15. Islam N, Jdanov DA, Shkolnikov VM, Khunti K, Kawachi I, White M, et al. Effects of COVID-19 pandemic on life expectancy and premature mortality in 2020: time series analysis in 37 countries. BMJ. 2021;375:e066768.

16. Kowall B, Standl F, Oesterling F, Brune B, Brinkmann M, Dudda M, et al. Excess mortality due to COVID-19? A comparison of total mortality in 2020 with total mortality in 2016 to 2019 in Germany Sweden and Spain. Plos One. 2021;16:e0255540.

17. Msemburi W, Karlinsky A, Knutson V, et al. The WHO estimates of excess mortality associated with the COVID-19 pandemic. Nature. 2023;613:130–7. https://doi.org/10.1038/s41586-022-05522-2.

18. Heuveline P. Global and national declines in life expectancy: an end-of-2021 assessment. Popul Dev Rev. 2022;48:31–50.

19. Keyfitz N, Caswell H. Applied mathematical demography. Springer; 2005.

20. Vaupel JW, Romo VC. Decomposing change in life expectancy: a bouquet of formulas in honor of Nathan Keyfitz’s 90th birthday. Demography. 2003;40:201–16.
Finally, this study critically depends on reliable data being made available from the national statistical institutes of the different European countries analysed here. The magnitude of our presented results should encourage policymakers to promptly provide regional data to better measure the burden of future epidemics in a timely fashion. To produce accurate analyses in times of demographical ageing, it is of utmost importance to harmonise European population and mortality data with consistent age groups and an oldest age class of at least 100 years upwards.

**Supplementary Information** The online version contains supplementary material available at https://doi.org/10.1007/s44197-024-00200-0.

**Acknowledgements** We are very grateful to Markéta Majerová, Rok Hrzic, Magdalena Muszyńska-Spielauer, and Mathias Lerch for providing, respectively, Czech, Slovenian, Austrian, and Swiss data.

**Author Contributions** Conceptualization: FB, MS, PG, CGC, MM; methodology: CGC, FB, PG; data collection and curation: MS, FB, IA, MM, PG, CGC; formal analysis: FB, PG; writing—Original draft: PG, FB, CGC, MM, MS; writing—review and editing: PG, FB, CGC, MS, MM, IA; visualisation: MS, FB, PG, CGC.

**Funding** IA, PG, MM, and MS were supported by funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreement No 851485).

**Data Availability** Detailed values of our estimates and data visualisation tool are available at: https://osf.io/h68wz/?view_only=47353f6f4b2e41cab3c761246e59d615.

## Declarations

**Conflict of Interest** The authors have no relevant financial or non-financial interests to disclose.

**Ethics Approval and Consent to Participate** No individual participated to this study.

**Consent for Publication** No individual participated to this study.

**Open Access** This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

## References

1. Schöley J, Aburto JM, Kashnitsky I, Kniffka MS, Zhang L, Jaadla H, et al. Life expectancy changes since COVID-19. Nat Hum Behav. 2022;6:1649–59.

2. Caporali A, Garcia J, Couppié É, Poniakina S, Barbieri M, Bonnet F, et al. The demography of COVID-19 deaths database, a gateway to well-documented international data. Scientific Data. 2022;9:1–9.

3. Riffe T, Acosta E, The COVerAGE-DB Team. Data resource profile: COVerAGE-DB: a global demographic database of COVID-19 cases and deaths. Int J Epidemiol. 2021;50:390–390f.

4. Garcia J, Torres C, Barbieri M, Cambois E, Camarda CG, Caporali A, et al. Differences in COVID-19 mortality: the implications of imperfect and diverse data collection systems. Population. 2021;76:35–72.

5. Pathak EB, Garcia RB, Menard JM, Salemi JL. Out-of-hospital COVID-19 deaths: consequences for quality of medical care and accuracy of cause of death coding. Am J Public Health. 2021;111:S101–6.

6. Helleringer S, Queiroz BL. Commentary: measuring excess mortality due to the COVID-19 pandemic: progress and persistent challenges. Int J Epidemiol. 2022;51:85–7.

7. Beaney T, Clarke JM, Jain V, Golestaneh AK, Lyons G, Salman D, et al. Excess mortality: the gold standard in measuring the impact of COVID-19 worldwide? J R Soc Med. 2020;113:329–34.

8. Karlinsky A, Kobak D. Tracking excess mortality across countries during the COVID-19 pandemic with the World Mortality Dataset. Elife. 2021;10:e69336.

9. Aburto JM, Schöley J, Kashnitsky I, Zhang L, Rahal C, Missov TI, et al. Quantifying impacts of the COVID-19 pandemic through life-expectancy losses: a population-level study of 29 countries. Int J Epidemiol. 2022;51:63–74.

10. Mazzuco S, Campostrini S. Life expectancy drop in 2020 Estimates based on Human Mortality Database. PLoS ONE. 2022;17:e0262846.

11. Aburto J, Schöley J, Kashnitsky I, Kashyap R. Life expectancy declines in Russia during the Covid-19 pandemic in 2020. Int J Epidemiol. 2022;51:1695–7.

12. Aburto JM, Kashyap R, Schöley J, Angus C, Ermisch J, Mills MC, et al. Estimating the burden of the COVID-19 pandemic on mortality, life expectancy and lifespan inequality in England and Wales: a population-level analysis. J Epidemiol Community Health. 2021;75:735–40.

13. Peretz C, Rotem N, Keinan-Boker L, Furshpan A, Green M, Bitan M, et al. Excess mortality in Israel associated with COVID-19 in 2020–2021 by age group and with estimates based on daily mortality patterns in 2000–2019. Int J Epidemiol. 2022;51:727–36.

14. Kontopantelis E, Mamas MA, Deanfield J, Asaria M, Doran T. Excess mortality in England and Wales during the first wave of the COVID-19 pandemic. J Epidemiol Community Health. 2021;75:213–23.

15. Islam N, Jdanov DA, Shkolnikov VM, Khunti K, Kawachi I, White M, et al. Effects of COVID-19 pandemic on life expectancy and premature mortality in 2020: time series analysis in 37 countries. BMJ. 2021;375:e066768.

16. Kowall B, Standl F, Oesterling F, Brune B, Brinkmann M, Dudda M, et al. Excess mortality due to COVID-19? A comparison of total mortality in 2020 with total mortality in 2016 to 2019 in Germany Sweden and Spain. Plos One. 2021;16:e0255540.

17. Msemburi W, Karlinsky A, Knutson V, et al. The WHO estimates of excess mortality associated with the COVID-19 pandemic. Nature. 2023;613:130–7. https://doi.org/10.1038/s41586-022-05522-2.

18. Heuveline P. Global and national declines in life expectancy: an end-of-2021 assessment. Popul Dev Rev. 2022;48:31–50.

19. Keyfitz N, Caswell H. Applied mathematical demography. Springer: 2005.

20. Vaupel JW, Romo VC. Decomposing change in life expectancy: a bouquet of formulas in honor of Nathan Keyfitz’s 90th birthday. Demography. 2003;40:201–16.
21. Goldstein JR, Lee RD. Demographic perspectives on the mortality of COVID-19 and other epidemics. Proc Natl Acad Sci. 2020;117:22035-41.

22. Heuveline P. Interpreting changes in life expectancy during temporary mortality shocks. Demogr Res. 2023;48:1-18.

23. Heuveline P. The Mean Unfulfilled Lifespan (MUL): a new indicator of the impact of mortality shocks on the individual lifespan, with application to mortality reversals induced by COVID-19. PLoS ONE. 2021;16:e0254925.

24. Blangiardo M, Cameletti M, Pirani M, Corsetti G, Battaglini M, Baio G. Estimating weekly excess mortality at sub-national level in Italy during the COVID-19 pandemic. PLoS ONE. 2020;15:e0240286.

25. Trias-Llimós S, Riffe T, Bilal U. Monitoring life expectancy levels during the COVID-19 pandemic: Example of the unequal impact of the first wave on Spanish regions. PLoS ONE. 2020;15:e0241952.

26. Gianicolo EAL, Russo A, Büchler B, Taylor K, Stang A, Blettner M. Gender specific excess mortality in Italy during the COVID-19 pandemic accounting for age. Eur J Epidemiol. 2021;36:213-8.

27. Locatelli I, Rousson V. A first analysis of excess mortality in Switzerland in 2020. PLoS ONE. 2021;16:e0253505.

28. Weinberger DM, Chen J, Cohen T, Crawford W, Mostashari F, Olson D, et al. Estimation of excess deaths associated with the COVID-19 pandemic in the United States, March to May 2020. JAMA Intern Med. 2020;180:1336-44.

29. Cuéllar L, Torres I, Romero-Severson E, Mahesh R, Ortega N, Pungitore S, et al. Excess deaths reveal the true spatial, temporal and demographic impact of COVID-19 on mortality in Ecuador. Int J Epidemiol. 2022;51:54-62.

30. Basellini U, Alburez-Gutierrez D, Del Fava E, Perrotta D, Bonetti M, Camarda CG, et al. Linking excess mortality to mobility data during the first wave of COVID-19 in England and Wales. Soc Sci Medi Popul Health. 2021;14:100799.

31. Scortichini M, Schneider dos Santos R, De'Donato F, De Sario M, Michelozzi P, Davoli M, et al. Excess mortality during the COVID-19 outbreak in Italy: a two-stage interrupted time-series analysis. Int J Epidemiol. 2021;49:1909-17.

32. Iuliano AD, Chang HH, Patel NN, Threlkel R, Kniss K, Reich J, et al. Estimating under-recognized COVID-19 deaths, United States, March 2020 - May 2021 using an excess mortality modeling approach. Lancet Reg Health-Am. 2021;1:100019.

33. Konstantinoudis G, Cameletti M, Gómez-Rubio V, Gómez IL, Pirani M, Baio G, et al. Regional excess mortality during the 2020 COVID-19 pandemic in five European countries. Nat Commun. 2022;13:1-11.

34. Bonnet F, Camarda CG. Estimating subnational excess mortality in times of pandemic. An application to French départements in 2020. Plos One. 2024;19(1):e0293752. Medrxiv.

35. Camarda CG. Smooth constrained mortality forecasting. Demogr Res. 2019;41:1091-130.

36. Barbieri M, Wilmoth JR, Shkolnikov VM, Glei D, Jasilionis D, Jdanov D, et al. Data resource profile: the human mortality database (HMD). Int J Epidemiol. 2015;44:1549-56.

37. Pascari MD, Dańko MJ, Schöley J, Rizzi S. Ungroup: An R package for efficient estimation of smooth distributions from coarsely binned data. J Open Source Softw. 2018;3:937.

38. Mühlichen M, Lerch M, Sauerberg M, Grigoriev P. Different health systems–Different mortality outcomes? Regional disparities in avoidable mortality across German-speaking Europe, 1992–2019. Soc Sci Med. 2023;329:115976.

39. Lee RD, Carter LR. Modeling and forecasting US mortality. J Am Stat Assoc. 1992;87:659-71.

40. R Development Core Team. R: A Language and Environment for Statistical Computing. Vienna. 2022.

41. Ord JK, Getis A. Local spatial autocorrelation statistics: distributional issues and an application. Geogr Anal. 1995;27:286-306.

42. Redlands CESRI. ArcGIS Desktop: Release 10. 2011.

43. Eurostat. NUTS geographical information. The Geographic Information System of the Commission (2021). Accessed Nov 2022.

44. Nielsen J, Nørgaard SK, Lanzieri G, Vestergaard LS, Moelbak K. Sex-differences in COVID-19 associated excess mortality is not exceptional for the COVID-19 pandemic. Sci Rep. 2021. https://doi.org/10.1038/s41598-021-00213-w.

45. Luy M, Sauerberg M, Muszyńska-Spielauer M, Lego VD. Decrease in life expectancy in Germany in 2020: men from Eastern Germany most affected. Comp Popul Stud. 2021. https://doi.org/10.12765/CPoS-2021-20.

46. Hossin MZ. The male disadvantage in life expectancy: can we close the gender gap? Int Health. 2021;13:482-4.

47. Nepomuceno MR, Klimkin I, Jdanov DA, Alustiza-Galarza A, Shkolnikov VM. Sensitivity analysis of excess mortality due to the COVID-19 pandemic. Popul Dev Rev. 2022. https://doi.org/10.1111/padr.12475.

48. Shkolnikov VM, Klimkin I, McKee M, Jdanov DA, Alustiza-Galarza A, Németh L, et al. What should be the baseline when calculating excess mortality? New approaches suggest that we have underestimated the impact of the COVID-19 pandemic and previous winter peaks. SSM Popul Health. 2022;18:101118.

49. Schöley J. Robustness and bias of European excess death estimates in 2020 under varying model specifications. MedRxiv. 2021.

50. Basellini U, Camarda CG. Explaining regional differences in mortality during the first wave of COVID-19 in Italy. Popul Stud. 2022;76:99-118.
21. Goldstein JR, Lee RD. Demographic perspectives on the mortality of COVID-19 and other epidemics. Proc Natl Acad Sci. 2020;117:22035-41.

22. Heuveline P. Interpreting changes in life expectancy during temporary mortality shocks. Demogr Res. 2023;48:1-18.

23. Heuveline P. The Mean Unfulfilled Lifespan (MUL): a new indicator of the impact of mortality shocks on the individual lifespan, with application to mortality reversals induced by COVID-19. PLoS ONE. 2021;16:e0254925.

24. Blangiardo M, Cameletti M, Pirani M, Corsetti G, Battaglini M, Baio G. Estimating weekly excess mortality at sub-national level in Italy during the COVID-19 pandemic. PLoS ONE. 2020;15:e0240286.

25. Trias-Llimós S, Riffe T, Bilal U. Monitoring life expectancy levels during the COVID-19 pandemic: Example of the unequal impact of the first wave on Spanish regions. PLoS ONE. 2020;15:e0241952.

26. Gianicolo EAL, Russo A, Büchler B, Taylor K, Stang A, Blettner M. Gender specific excess mortality in Italy during the COVID-19 pandemic accounting for age. Eur J Epidemiol. 2021;36:213-8.

27. Locatelli I, Rousson V. A first analysis of excess mortality in Switzerland in 2020. PLoS ONE. 2021;16:e0253505.

28. Weinberger DM, Chen J, Cohen T, Crawford W, Mostashari F, Olson D, et al. Estimation of excess deaths associated with the COVID-19 pandemic in the United States, March to May 2020. JAMA Intern Med. 2020;180:1336-44.

29. Cuéllar L, Torres I, Romero-Severson E, Mahesh R, Ortega N, Pungitore S, et al. Excess deaths reveal the true spatial, temporal and demographic impact of COVID-19 on mortality in Ecuador. Int J Epidemiol. 2022;51:54-62.

30. Basellini U, Alburez-Gutierrez D, Del Fava E, Perrotta D, Bonetti M, Camarda CG, et al. Linking excess mortality to mobility data during the first wave of COVID-19 in England and Wales. Soc Sci Medi Popul Health. 2021;14:100799.

31. Scortichini M, Schneider dos Santos R, De'Donato F, De Sario M, Michelozzi P, Davoli M, et al. Excess mortality during the COVID-19 outbreak in Italy: a two-stage interrupted time-series analysis. Int J Epidemiol. 2021;49:1909-17.

32. Iuliano AD, Chang HH, Patel NN, Threlkel R, Kniss K, Reich J, et al. Estimating under-recognized COVID-19 deaths, United States, March 2020 - May 2021 using an excess mortality modeling approach. Lancet Reg Health-Am. 2021;1:100019.

33. Konstantinoudis G, Cameletti M, Gómez-Rubio V, Gómez IL, Pirani M, Baio G, et al. Regional excess mortality during the 2020 COVID-19 pandemic in five European countries. Nat Commun. 2022;13:1-11.

34. Bonnet F, Camarda CG. Estimating subnational excess mortality in times of pandemic. An application to French départements in 2020. Plos One. 2024;19(1):e0293752. Medrxiv.

35. Camarda CG. Smooth constrained mortality forecasting. Demogr Res. 2019;41:1091-130.

36. Barbieri M, Wilmoth JR, Shkolnikov VM, Glei D, Jasilionis D, Jdanov D, et al. Data resource profile: the human mortality database (HMD). Int J Epidemiol. 2015;44:1549-56.

37. Pascari MD, Dańko MJ, Schöley J, Rizzi S. Ungroup: An R package for efficient estimation of smooth distributions from coarsely binned data. J Open Source Softw. 2018;3:937.

38. Mühlichen M, Lerch M, Sauerberg M, Grigoriev P. Different health systems–Different mortality outcomes? Regional disparities in avoidable mortality across German-speaking Europe, 1992–2019. Soc Sci Med. 2023;329:115976.

39. Lee RD, Carter LR. Modeling and forecasting US mortality. J Am Stat Assoc. 1992;87:659-71.

40. R Development Core Team. R: A Language and Environment for Statistical Computing. Vienna. 2022.

41. Ord JK, Getis A. Local spatial autocorrelation statistics: distributional issues and an application. Geogr Anal. 1995;27:286-306.

42. Redlands CESRI. ArcGIS Desktop: Release 10. 2011.

43. Eurostat. NUTS geographical information. The Geographic Information System of the Commission (2021). Accessed Nov 2022.

44. Nielsen J, Nørgaard SK, Lanzieri G, Vestergaard LS, Moelbak K. Sex-differences in COVID-19 associated excess mortality is not exceptional for the COVID-19 pandemic. Sci Rep. 2021. https://doi.org/10.1038/s41598-021-00213-w.

45. Luy M, Sauerberg M, Muszyńska-Spielauer M, Lego VD. Decrease in life expectancy in Germany in 2020: men from Eastern Germany most affected. Comp Popul Stud. 2021. https://doi.org/10.12765/CPoS-2021-20.

46. Hossin MZ. The male disadvantage in life expectancy: can we close the gender gap? Int Health. 2021;13:482-4.

47. Nepomuceno MR, Klimkin I, Jdanov DA, Alustiza-Galarza A, Shkolnikov VM. Sensitivity analysis of excess mortality due to the COVID-19 pandemic. Popul Dev Rev. 2022. https://doi.org/10.1111/padr.12475.

48. Shkolnikov VM, Klimkin I, McKee M, Jdanov DA, Alustiza-Galarza A, Németh L, et al. What should be the baseline when calculating excess mortality? New approaches suggest that we have underestimated the impact of the COVID-19 pandemic and previous winter peaks. SSM Popul Health. 2022;18:101118.

49. Schöley J. Robustness and bias of European excess death estimates in 2020 under varying model specifications. MedRxiv. 2021.

50. Basellini U, Camarda CG. Explaining regional differences in mortality during the first wave of COVID-19 in Italy. Popul Stud. 2022;76:99-118.