Author response: Tracking excess mortality across countries during the COVID-19 pandemic with the World Mortality Dataset
2021-06-08

Article Figures and data Abstract eLife digest Introduction Results Discussion Materials and methods Data availability References Decision letter Author response Article and author information Metrics Abstract Comparing the impact of the COVID-19 pandemic between countries or across time is difficult because the reported numbers of cases and deaths can be strongly affected by testing capacity and reporting policy. Excess mortality, defined as the increase in all-cause mortality relative to the expected mortality, is widely considered as a more objective indicator of the COVID-19 death toll. However, there has been no global, frequently updated repository of the all-cause mortality data across countries. To fill this gap, we have collected weekly, monthly, or quarterly all-cause mortality data from 103 countries and territories, openly available as the regularly updated World Mortality Dataset. We used this dataset to compute the excess mortality in each country during the COVID-19 pandemic. We found that in several worst-affected countries (Peru, Ecuador, Bolivia, Mexico) the excess mortality was above 50% of the expected annual mortality (Peru, Ecuador, Bolivia, Mexico) or above 400 excess deaths per 100,000 population (Peru, Bulgaria, North Macedonia, Serbia). At the same time, in several other countries (e.g. Australia and New Zealand) mortality during the pandemic was below the usual level, presumably due to social distancing measures decreasing the non-COVID infectious mortality. Furthermore, we found that while many countries have been reporting the COVID-19 deaths very accurately, some countries have been substantially underreporting their COVID-19 deaths (e.g. Nicaragua, Russia, Uzbekistan), by up to two orders of magnitude (Tajikistan). Our results highlight the importance of open and rapid all-cause mortality reporting for pandemic monitoring. eLife digest Countries around the world reported 4.2 million deaths from SARS-CoV-2 (the virus that causes COVID-19) from the beginning of pandemic until the end of July 2021, but the actual number of deaths is likely higher. While some countries may have imperfect systems for counting deaths, others may have intentionally underreported them. To get a better estimate of deaths from an event such as a pandemic, scientists often compare the total number of deaths in a country during the event to the expected number of deaths based on data from previous years. This tells them how many excess deaths occurred during the event. To provide a more accurate count of deaths caused by COVID-19, Karlinsky and Kobak built a database called the World Mortality Dataset. It includes information on deaths from all causes from 103 countries. Karlinsky and Kobak used the database to compare the number of reported COVID-19 deaths reported to the excess deaths from all causes during the pandemic. Some of the hardest hit countries, including Peru, Ecuador, Bolivia, and Mexico, experienced over 50% more deaths than expected during the pandemic. Meanwhile, other countries like Australia and New Zealand, reported fewer deaths than normal. This is likely because social distancing measures reduced deaths from infections like influenza. Many countries reported their COVID-19 deaths accurately, but Karlinsky and Kobak argue that other countries, including Nicaragua, Russia, and Uzbekistan, underreported COVID-19 deaths. Using their database, Karlinsky and Kobak estimate that, in those countries, there have been at least 1.4 times more deaths due to COVID-19 than reported – adding over 1 million extra deaths in total. But they note that the actual number is likely much higher because data from more than 100 countries were not available to include in the database. The World Mortality Dataset provides a more accurate picture of the number of people who died because of the COVID-19 pandemic, and it is available online and updated daily. The database may help scientists develop better mitigation strategies for this pandemic or future ones. Introduction The impact of COVID-19 on a given country is usually assessed via the number of cases and the number of deaths, two statistics that have been reported daily by each country and put together into international dashboards such as the ones maintained by the World Health Organization (https://covid19.who.int) or by the Johns Hopkins University (https://coronavirus.jhu.edu) (Dong et al., 2020). However, both metrics can be heavily affected by limited testing availability and by different definitions of 'COVID-19 death' used by different countries (Riffe et al., 2021): for example, some countries count only PCR-confirmed COVID-19 deaths, while others include suspected COVID-19 deaths as well. Excess mortality, defined as the increase of the all-cause mortality over the mortality expected based on historic trends, has long been used to estimate the death toll of pandemics and other extreme events—from the Great Plague of London in 1665 (as described in Boka and Wainer, 2020), to the influenza epidemic in London in 1875 (Farr, 1885; Langmuir, 1976), the XX–XXI century influenza pandemics of 1918, 1957, 1968, 2009 (Murray et al., 2006; Viboud et al., 2005; Viboud et al., 2016; Simonsen et al., 2013) as well as seasonal influenza epidemics (Housworth and Langmuir, 1974), and more recently for example Hurricane Maria in Puerto-Rico in 2016 (Milken Institute, 2018). Even though the excess mortality does not exactly equal the mortality from COVID-19 infections, the consensus is that for many countries it is the most objective possible indicator of the COVID-19 death toll (Beaney et al., 2020; Leon et al., 2020). Excess mortality has already been used to estimate the COVID-19 impact in different countries, both in academic literature (e.g. Kontis et al., 2020; Alicandro et al., 2020; Ghafari et al., 2021; Woolf et al., 2020a; Woolf et al., 2020b; Weinberger et al., 2020; Blangiardo et al., 2020; Kobak, 2021a; Modi et al., 2021; Bradshaw et al., 2021; Islam et al., 2021, among many others) and by major media outlets. It has also been used to compare COVID-19 impact to the impact of major influenza pandemics (Faust et al., 2020; Petersen et al., 2020). Measuring and monitoring excess mortality across different countries requires, first and foremost, a comprehensive and regularly-updated dataset on all-cause mortality. However, there has been no single resource where such data would be collected from all over the world. The World Mortality Dataset presented here aims to fill this gap by combining publicly available information on country-level mortality, culled and harmonized from various sources. Several teams have already started to collect such data. In April 2020, EuroStat (http://ec.europa.eu/eurostat) began collecting total weekly deaths across European countries, 'in order to support the policy and research efforts related to COVID-19'. At the time of writing, this dataset covers 36 European countries and also contains sub-national (NUTS1–3 regions) data as well as data disaggregated by age groups and by sex for some countries. In May 2020, the Human Mortality Database (http://mortality.org), a joint effort by the University of California, Berkeley, and Max Planck Institute for Demographic Research (Barbieri et al., 2015), started compiling the Short Term Mortality Fluctuations (STMF) dataset (STMF, 2021; Islam et al., 2021; Németh et al., 2021). This dataset consists of weekly data, disaggregated by five age groups and by sex, and currently contains 35 countries with 2020 data. STMF only includes countries with complete high-quality vital registration data in all age groups. Both datasets are regularly updated and have considerable overlap, covering together 44 countries. In parallel, the EuroMOMO project (https://www.euromomo.eu), existing since 2008, has been displaying weekly excess mortality in 23 European countries, but without giving access to the underlying data. Another source of data is the UNDATA initiative (http://data.un.org; search for 'Deaths by month of death') by the United Nations, collecting monthly mortality data across a large number of countries. However, information there is updated very slowly, with January–June 2020 data currently available for only four countries. Media outlets such as the Financial Times, The Economist, the New York Times, and the Wall Street Journal have been compiling and openly sharing their own datasets in order to report on the all-cause mortality in 2020. However, these datasets are infrequently updated and their future is unclear. For example, the New York Times announced in early 2021 that they would stop tracking excess deaths due to staffing changes. Here, we present the World Mortality Dataset that aims to provide regularly-updated all-cause mortality numbers from all over the world. The dataset is openly available at https://github.com/akarlinsky/world_mortality and is updated almost daily. Our dataset builds upon the EuroStat and the STMF datasets, adding 59 additional countries — many more than any previous media or academic effort. At the time of writing, our dataset comprises 103 countries and territories. After the initial release of our manuscript, the dataset has been incorporated into the excess mortality trackers by Our World in Data (Giattino et al., 2020), The Economist, and the Financial Times. While not all countries provide equally detailed and reliable data, we believe that information from all 103 countries is reliable enough to allow computation of excess mortality (see Discussion). Our analysis (updated almost daily at https://github.com/dkobak/excess-mortality) showed statistically significant positive excess mortality in 69 out of 103 countries. Moreover, it suggests that the true COVID-19 death toll in several countries is over an order of magnitude larger than the official COVID-19 death count. Results Excess mortality We collected the all-cause mortality data from 103 countries and territories from 2015 onward into the openly available World Mortality Dataset. This includes 50 countries with weekly data, 51 countries with monthly data, and two countries with quarterly data (Figure 1). See Materials and methods for our data collection strategy. Briefly, we obtained the data from the websites of National Statistics Offices (NSOs). If we were unable to locate the data ourselves, we contacted the NSO for guidance. The data from EuroStat and STMF were included as is, with few exceptions (see Materials and methods). An important caveat is that recent (2020 and 2021) data are often preliminary and subject to backwards revisions, which we incorporate into our dataset. Other caveats and limitations are listed in the Materials and methods section. Figure 1 Download asset Open asset Countries in the World Mortality Dataset are shown in blue. Small countries and territories are shown with circles. For each country, we predicted the 'baseline' mortality in 2020 based on the 2015–2019 data (accounting for linear trend and seasonal variation; see Materials and methods). We then obtained excess mortality as the difference between the actual 2020–2021 all-cause mortality and our baseline (Figure 2, Figure 2—figure supplement 1). For each country, we computed the total excess mortality from the beginning of the COVID-19 pandemic (from March 2020) (Table 1). The total excess mortality was positive and significantly different from zero in 69 countries; negative and significantly different from zero in seven countries; not significantly different from zero (z<2) in 25 countries. For South Africa and Argentina, there was no historic data available in order to assess the significance, but the increase in mortality was very large and clearly associated with COVID-19 (Bradshaw et al., 2021; Rearte et al., 2021). Figure 2 with 1 supplement see all Download asset Open asset Excess mortality time series. Each subplot shows baseline mortality (black), mortality in 2015–2019 (gray), in 2020 (red) and in 2021 (blue). Excess mortality is shown in red/blue shading. The numbers in each subplot are: total excess mortality (red), excess mortality per 100,000 population (black), excess mortality as a percentage of annual baseline mortality (gray), and undercount ratio of COVID-19 deaths (blue). See text for the exact definitions. All numbers were rounded to two significant digits; numbers below 100 to one significant digit. The y-axis in each subplot starts at 0 and goes until 200% where 100% corresponds to the average baseline mortality. The x-axis covers the entire year. Asterisks mark excess mortality estimates that were downwards corrected (see Materials and methods). Countries are sorted by the excess mortality as a percentage of annual baseline mortality (gray number). Undercount estimates are not shown for countries with negative total excess deaths and for selected countries where excess deaths were likely not related to the COVID-19 pandemic (Hong Kong, Thailand, Cuba); see Materials and methods. Table 1 Excess mortality metrics for all countries in the dataset. Abbreviations: 'w' – weekly data, 'm' – monthly data, 'q' – quarterly data. All numbers were rounded to two significant digits; numbers below 100 — to one significant digit. See text for the exact definitions of all reported metrics. 'Official' means the official daily reported number of COVID-19 deaths. Undercount estimates are not shown for countries with negative total excess deaths and for selected countries where excess deaths were likely not related to the COVID-19 pandemic (Hong Kong, Thailand, Cuba); see Materials and methods. CountryData untilTypeOfficialExcessstdzUndercountPer 100kIncreaseAlbaniaMar 31, 2021m2,2009,300±81011.44.232043AndorraDec 31, 2020m8080±303.11.011025ArgentinaDec 31, 2020m43,00041,000±nannan1.09012ArmeniaApr 30, 2021m4,1008,300±84010.02.028033ArubaDec 31, 2020m5050±301.51.0507AustraliaMar 28, 2021w910−3,700±1,0003.6–−10−2AustriaJun 13, 2021w10,0009,800±1,4007.00.911012AzerbaijanFeb 28, 2021m3,20018,000±1,40013.05.618032BelarusJun 30, 2020m3905,700±9306.114.5605BelgiumJun 13, 2021w25,00016,000±1,8008.70.614014BoliviaMay 31, 2021m14,00036,000±77046.42.531068BosniaMar 31, 2021m6,6008,900±9909.01.427025BrazilMay 31, 2021m460,000500,000±14,00035.01.124037BulgariaJun 20, 2021w18,00032,000±1,80017.51.846029CanadaMar 07, 2021w22,00015,000±1,7008.60.7405ChileJun 13, 2021w31,00030,000±1,10026.71.016027ColombiaMay 09, 2021w77,00092,000±1,50061.21.218036Costa RicaDec 31, 2020m2,200940±3702.50.4204CroatiaMay 30, 2021w8,0008,800±1,0008.81.121017CubaDec 31, 2020m150580±2,1000.3–101CyprusMay 09, 2021w330340±1602.11.0305CzechiaMay 23, 2021w30,00035,000±1,80018.81.232030DenmarkJun 20, 2021w2,500−630±6101.0–−10−1EcuadorJun 20, 2021w21,00062,000±96064.42.935080EgyptNov 30, 2020m6,60087,000±13,0006.913.19016El SalvadorAug 31, 2020m7204,700±8905.36.67011EstoniaJun 27, 2021w1,3001,800±3006.01.414012FinlandJun 13, 2021w960410±6800.60.4101FranceJun 13, 2021w110,00072,000±8,0008.90.711012French GuianaJun 13, 2021w130−20±600.3–−10−2French PolynesiaDec 31, 2020m110120±901.41.1408GeorgiaDec 31, 2020m2,5004,800±1,0004.71.912011GermanyJun 20, 2021w90,00039,000±17,0002.30.4504GibraltarJan 31, 2021m8020±201.10.3707GreeceMay 02, 2021w10,0007,500±2,0003.80.7706GreenlandDec 31, 2020m0−20±300.5–−30−3GuadeloupeJun 13, 2021w260220±1102.00.8606GuatemalaDec 27, 2020w4,80010,000±70014.52.16012Hong KongMar 31, 2021m2002,100±1,1001.9–304HungaryMay 30, 2021w30,00024,000±2,30010.20.824018IcelandMar 21, 2021w30−20±700.2–−0−1IranSep 21, 2020q24,00058,000±7,9007.32.47015IrelandMay 31, 2021m5,0001,400±7301.90.3304IsraelMay 30, 2021w6,4004,800±5508.90.86010ItalyApr 04, 2021w110,000120,000±9,00013.91.121019JamaicaNov 30, 2020m260−320±3101.0–−10−2JapanApr 30, 2021m10,000−15,000±12,0001.3–−10−1KazakhstanApr 30, 2021m6,60035,000±3,40010.35.318026KosovoMar 31, 2021m1,9002,800±3108.91.515030KyrgyzstanApr 30, 2021m1,6007,900±67011.84.912024LatviaJun 13, 2021w2,5003,000±4406.81.216010LebanonApr 30, 2021m7,3008,900±9709.21.213036LiechtensteinApr 30, 2021m6050±301.70.812017LithuaniaJun 20, 2021w4,4009,500±60015.92.235025LuxembourgJun 06, 2021w820190±1401.40.2304MacaoApr 30, 2021m0−40±1100.3–−10−2MalaysiaMar 31, 2021m1,300−6,600±1,9003.5–−20−4MaltaMay 16, 2021w420360±1203.00.9809MartiniqueJun 13, 2021w10030±1100.30.3101MauritiusApr 30, 2021m20−440±2401.8–−30−4MayotteJun 13, 2021w170320±506.51.812040MexicoMay 23, 2021w220,000470,000±6,60070.12.136061MoldovaMar 31, 2021m5,0008,000±8809.01.620022MonacoMay 31, 2021m30120±502.53.730024MongoliaMay 31, 2021m280−1,900±4903.9–−60−11MontenegroMar 28, 2021w1,2001,400±1708.41.223021NetherlandsJun 20, 2021w18,00019,000±1,9009.81.111012New ZealandJun 06, 2021w30−1,900±4104.7–−40−5NicaraguaAug 31, 2020m1407,000±27026.050.810027North MacedoniaApr 30, 2021m4,9008,600±77011.31.842043NorwayJun 20, 2021w790−1,500±5302.9–−30−4OmanMay 31, 2021m2,3002,200±3306.70.94024PanamaApr 30, 2021m6,2006,500±42015.71.015031ParaguayMay 31, 2021m9,1009,600±92010.31.113028PeruJun 27, 2021w190,000190,000±2,00095.91.0590153PhilippinesDec 31, 2020m9,200−7,700±5,9001.3–−10−1PolandJun 13, 2021w75,000120,000±5,50021.11.631028PortugalJun 06, 2021w17,00019,000±2,1009.01.118016QatarApr 30, 2021m460650±709.21.42029RomaniaApr 25, 2021w31,00054,000±3,50015.31.728020RussiaApr 30, 2021m110,000500,000±33,00015.24.534028RéunionJun 13, 2021w210190±1301.50.9204San MarinoMay 31, 2021m90110±303.41.232042SerbiaMay 31, 2021m6,90028,000±3,6007.74.040027SeychellesDec 31, 2020m0−170±404.1–−170−20SingaporeMar 31, 2021m30−160±3800.4–−0−1SlovakiaMay 16, 2021w12,00017,000±92018.11.431030SloveniaMay 23, 2021w4,7003,700±37010.00.818017South AfricaJun 27, 2021w60,000160,000±nannan2.727032South KoreaMay 02, 2021w1,800−3,300±2,9001.1–−10−1SpainJun 20, 2021w81,00087,000±6,30013.91.119020SwedenJun 06, 2021w15,0008,900±1,1008.50.69010SwitzerlandJun 06, 2021w10,0008,600±1,1008.00.810013TaiwanMay 31, 2021m140−6,600±5,7001.2–−30−4TajikistanDec 31, 2020q909,000±1,4006.6100.09027ThailandJun 30, 2021m2,10014,000±13,0001.1–203TransnistriaMay 31, 2021m1,2001,600±2406.41.334023TunisiaFeb 14, 2021w7,5004,600±1,1004.30.6406UkraineApr 30, 2021m44,00081,000±13,0006.41.820014United KingdomJun 13, 2021w130,000110,000±9,20011.90.916018United StatesJun 06, 2021w590,000640,000±16,00038.91.119022UruguayDec 31, 2020m170−2,200±7103.2–−60−6UzbekistanMar 31, 2021m63020,000±3,9005.031.56013 In terms of the absolute numbers, the largest excess mortality in our dataset was observed in the United States (640,000 by June 6, 2021; all reported numbers here and below have been rounded to two significant digits), Brazil (500,000 by May 31, 2021), Russia (500,000 by April 30, 2021), and Mexico (470,000 by May 23, 2021) (Figure 3). Note that these estimates correspond to different time points as the reporting lags differ between countries (Table 1). See Figure 3—figure supplement 1 for the same analysis using the 2020 data alone. Figure 3 with 1 supplement see all Download asset Open asset Top 10 countries in the World Mortality Dataset by various excess mortality measures. Each subplot shows the top 10 countries for each of our four excess mortality measures: total number of excess deaths; excess deaths per 100,000 population; excess deaths as a percentage of baseline annual mortality; undercount ratio (ratio of excess deaths to reported COVID-19 deaths by the same date). Error bars denote 95% confidence intervals corresponding to the uncertainty of the excess deaths estimate. Countries with population below 500,000 are not shown. Different countries have different reporting lags, so the estimates shown here correspond to different time points, as indicated. Excess mortality estimates in Armenia and Azerbaijan were downwards corrected by 4000 to account for the war casualties (see Materials and methods). Some countries showed statistically significant negative excess mortality, likely due to lockdown measures and social distancing decreasing the prevalence of influenza (Kung et al., 2021), as we discuss further below. For example, Australia had −3,700 excess deaths, Uruguay had −2,200 deaths, and New Zealand had −1,900 deaths. In these three cases, the decrease in mortality happened during the southern hemisphere winter season (Figure 2). Similarly, Norway had −1,500 excess deaths, with most of this decrease happening during the 2020/21 winter season. Note that Uruguay had a large COVID outbreak in 2021, but we only have 2020 data available at the time of writing. The statistically significant mortality decrease in Malaysia, Mongolia, and Seychelles may also be related to the lockdown and social distancing measures but does not show clear seasonality, so may possibly also be due to other factors. As the raw number of excess deaths can be strongly affected by the country's population size, we normalized the excess mortality estimates by the population size (Table 1). The highest excess mortality per 100,000 inhabitants was observed in Peru (590), followed by some Eastern European and then Latin American countries: North Mexico Russia (Figure 3). Note that many countries with that international media such as and United had (Table 1). The of COVID-19 is strongly et al., 2020; et al., 2021). As the countries differ in their age the expected between countries. To account for the age we also normalized the excess mortality estimates by the annual of the baseline mortality, that is the expected number of deaths per without a pandemic event (Table 1). This relative also as a and 2020), was by the highest in Latin Peru and Mexico (Figure 3). Latin American countries have much to the European and North American countries, which is the excess mortality per 100,000 inhabitants there was than in several Eastern European countries, but the relative increase in mortality was higher COVID-19 the highest relative mortality increase was observed in Peru, is in with some of Peru the highest in the world et al., 2021). Undercount of COVID deaths For each country, we computed the ratio of the excess mortality to the reported COVID-19 death count by the same This ratio very strongly between countries (Table 1). Some countries had ratio below for example in and in where reporting of COVID deaths is to be very accurate et al., 2020). The likely is that the non-COVID mortality has due to the influenza (see to the excess mortality the true number of COVID deaths. many countries had above an undercount of COVID-19 deaths (Beaney et al., 2020). At the same time, between weekly reported COVID-19 deaths and weekly excess deaths was often very (Figure in Mexico ratio was and in South Africa ratio it was that excess mortality during a COVID outbreak can be by COVID-19 mortality, the is strongly Peru a until early the undercount ratio in Peru was with Peru the of reported to be more and backwards to of as a the undercount ratio to and to (Figure This example clearly that undercount above from deaths from COVID See Discussion for additional Figure Download asset Open asset between weekly excess deaths and weekly reported COVID-19 deaths. selected countries are shown together with the between the two time from 10 of 2020. Note the in excess mortality not in the reported COVID-19 associated with the 2020 in and in many countries, the undercount ratio was not across For example, the undercount ratio in and United was during the first (Figure but to during the This decrease of the undercount ratio may be due to COVID death and due to the excess mortality the true COVID mortality in winter due to influenza the other several countries showed very accurate reporting of the COVID-19 deaths with the undercount ratio to et al., 2020) from the beginning of the pandemic and up until the of the (e.g. Figure However, from 2020 and up until 2021 the excess deaths were the COVID-19 deaths in all these countries. The difference between the reported COVID-19 deaths and the excess deaths may correspond to the number of deaths caused by influenza and other infectious in winter This difference from of 2020 and until of 2021), as a of baseline annual deaths, was in the This is in with the total negative excess deaths observed in New Zealand, and Norway and from the and hemisphere winter (Kung et al., 2021). 100,000 the same difference was in the The undercount ratio for most countries was below (Table but some countries showed much larger We found the highest undercount in and (Figure 3). large undercount strongly or underreporting of COVID-19 deaths, as by Kobak, for the of Russia ratio Discussion We presented the World Mortality Dataset — the largest international dataset of all-cause mortality, currently 103 countries. The dataset is openly available and regularly We are to this dataset for the entire of the COVID-19 pandemic. The and of the data across countries, and some of the countries in our dataset may possibly report mortality numbers (e.g. covering only of the see caveats in the Data limitations and caveats of Materials and methods. This would the excess mortality estimate during the COVID-19 outbreak (as an example, et al., 2021 estimate that the true excess mortality in Peru may be higher than excess mortality computed here due to death registration in the early 2020 data for all countries in our dataset the baseline obtained from the historic 2015–2019 data, that the data are and the excess mortality estimates are not Another important caveat is that the most recent data points in many countries to be and can Both that some of the excess mortality estimates reported here may be Some of the countries in our dataset have excess death estimates available in the literature on excess deaths during the COVID-19 pandemic from official and The largest efforts include the analysis of STMF data et al., 2020; Islam et al., 2021) and excess mortality trackers by The and Financial Times. While the analysis is and the estimates there are many possible (the and the end of the total excess including or historic influenza the trend over or all the estimates to excess mortality excess mortality during the COVID-19 pandemic can be as the of several We account for and argue that for most countries, the of is in to in with the that excess mortality during an epidemic outbreak can be as a for COVID-19 mortality (Beaney et al., 2020). we discuss each of the listed factors. It is possible that a country a COVID deaths from non-COVID causes also increase due to the — Our data show that this not in