Measuring the true toll of the pandemic
By: James Tozer
Tags: Coronavirus, Data Visualization, Covid 19, Data Journalism, Journalism

How I calculated The Economist’s estimates

For each country or region, we have compared excess deaths with official covid-19 figures

One of the biggest challenges when it comes to measuring the spread of the virus is that every country collects its data differently. This is particularly true for the official covid-19 figures. For example, France has conducted a considerable amount of testing in care-homes, and includes fatalities from such institutions in its daily tolls, whereas almost all of the Netherlands’ reported deaths have occurred in hospitals.

Yet the data about deaths from all causes, which might seem more straightforward to collect, also vary in quality and frequency across countries. Spain’s Carlos III Institute, its national epidemiology centre, is publishing daily figures for each region — but these have a large lag, and are increased retrospectively when newer data come in (see chart below). By contrast, Britain’s Office for National Statistics produces a weekly total of the number of death certificates registered in its system, which are not revised in future. So the comparisons between countries are not entirely apples-to-apples. We have also had to use a simple method for our baseline of “expected deaths”: an average for the same week in previous years (ideally the past five years, but sometimes two or three, depending on availability).

Spain’s daily data on total deaths are often revised retrospectively

Moreover, some countries have reported only partial mortality data, which cover a non-random selection of regions or victims. In Istanbul and Jakarta, the two non-Western places currently in our tracker, I am relying on tallies of the number of people who have been buried, as reported by local departments. These figures will not pick up everybody who has died, but they are still much higher than the official covid-19 tolls. Turkey does not even give regular breakdowns of covid-19 deaths by region, so we had to estimate that Istanbul had half of the national share, based on a report from April 1st. Given how slowly many developing countries publish their all-cause mortality data, I will probably have to use a similar approach to find numbers in South America, Africa, Asia and the Middle East.

Even some rich nations do not have complete data. Italy’s National Institute of Statistics (ISTAT) has released figures on all deaths for just 1,700 of the country’s 7,900 municipalities (or comuni) — only the ones that have both shown at least a 20% increase this year and claim to have accurate records. This makes it impossible to come up with a national tally of excess deaths in the European country that was hit hardest. However, because these comuni cover 7.2m of the 10m people in Lombardy, the worst-affected Italian region, I was able to produce an estimate for that area. I found a snapshot map of official covid-19 deaths for those comuni at the end of March, and used that to predict what share of Lombardy’s deaths these 7.2m people suffered each week.

Our estimate of total and covid-19 deaths in Lombardy, based on the available comuni

All of this data-cleaning is fiddly and time-consuming, especially because the underlying spreadsheets are in different formats and languages. Whenever new data are published, I check them carefully, in case the definitions have changed. New York’s official death toll, for example, has started to include people who “probably” died of covid-19, according to the symptoms on their death certificates. To make this process more organised, I wrote a script in R, the programming language used most often by data journalists at The Economist, to do the cleaning for me. It is about 1,000 lines long, and spits out CSV files in the same format for each country. Those go into a Github repository, which Martín González, one of our brilliant interactive journalists, uses to make the dynamic charts for our tracker. We hope that, at some point, once we have made the code presentable, to make this resource publicly available — keep checking our GitHub.