National and subnational short-term forecasting of COVID-19 in Germany and Poland during early 2021
Authors: Bracher Johannes , Wolffram Daniel , Deuschel Jannik , Görgen Konstantin , Ketterer Jakob L. , Ullrich Alexander , Abbott Sam , Barbarossa Maria V. , Bertsimas Dimitris , Bhatia Sangeeta , Bodych Marcin , Bosse Nikos I. , Burgard Jan Pablo , Castro Lauren , Fairchild Geoffrey , Fiedler Jochen , Fuhrmann Jan , Funk Sebastian , Gambin Anna , Gogolewski Krzysztof , Heyder Stefan , Hotz Thomas , Kheifetz Yuri , Kirsten Holger , Krueger Tyll , Krymova Elena , Leithäuser Neele , Li Michael L. , Meinke Jan H. , Miasojedow Błażej , Michaud Isaac J. , Mohring Jan , Nouvellet Pierre , Nowosielski Jedrzej M. , Ozanski Tomasz , Radwan Maciej , Rakowski Franciszek , Scholz Markus , Soni Saksham , Srivastava Ajitesh , Gneiting Tilmann , Schienle Melanie 
Published: November 2021

Background

During the COVID-19 pandemic there has been a strong interest in forecasts of the short-term development of epidemiological indicators to inform decision makers. In this study we evaluate probabilistic real-time predictions of confirmed cases and deaths from COVID-19 in Germany and Poland for the period from January through April 2021.

Methods

We evaluate probabilistic real-time predictions of confirmed cases and deaths from COVID-19 in Germany and Poland. These were issued by 15 different forecasting models, run by independent research teams. Moreover, we study the performance of combined ensemble forecasts. Evaluation of probabilistic forecasts is based on proper scoring rules, along with interval coverage proportions to assess forecast calibration. The presented work is part of a pre-registered evaluation study and covers the period from January through April 2021.

Results

We find that many, though not all, models outperform a simple baseline model up to four weeks ahead for the considered targets. Ensemble methods (i.e., combinations of different available forecasts) show very good relative performance. The addressed time period is characterized by rather stable non-pharmaceutical interventions in both countries, making short-term predictions more straightforward than in previous periods. However, major trend changes in reported cases, like the rebound in cases due to the rise of the B.1.1.7 (alpha) variant in March 2021, prove challenging to predict.

Conclusions

Multi-model approaches can help to improve the performance of epidemiological forecasts. However, while death numbers can be predicted with some success based on current case and hospitalization data, predictability of case numbers remains low beyond quite short time horizons. Additional data sources including sequencing and mobility data, which were not extensively used in the present study, may help to improve performance.

Plain language summary

The goal of this study is to assess the quality of forecasts of weekly case and death numbers of COVID-19 in Germany and Poland during the period of January through April 2021. We focus on real-time forecasts at time horizons of one and two weeks ahead created by fourteen independent teams. Forecasts are systematically evaluated taking uncertainty ranges of predictions into account. We find that combining different forecasts into ensembles can improve the quality of predictions, but especially case numbers proved very challenging to predict beyond quite short time windows. Additional data sources, in particular genetic sequencing data, may help to improve forecasts in the future.