Informing pandemic response in the face of uncertainty.An evaluation of the U.S. COVID-19 Scenario Modeling Hub
Authors: Howerton Emily , Contamin Lucie , Mullany Luke C , Qin Michelle , Reich Nicholas G. , Bents Samantha , Borchering Rebecca K. , Jung Sung-mok , Loo Sara L. , Smith Claire P. , Levander John , Kerr Jessica , Espino J. , van Panhuis Willem G. , Hochheiser Harry , Galanti Marta , Yamana Teresa , Pei Sen , Shaman Jeffrey , Rainwater-Lovett Kaitlin , Kinsey Matt , Tallaksen Kate , Wilson Shelby , Shin Lauren , Lemaitre Joseph C. , Kaminsky Joshua , Hulse Juan Dent , Lee Elizabeth C. , McKee Clif , Hill Alison , Karlen Dean , Chinazzi Matteo , Davis Jessica T. , Mu Kunpeng , Xiong Xinyue , Pastore y Piontti Ana , Vespignani Alessandro , Rosenstrom Erik T. , Ivy Julie S. , Mayorga Maria E. , Swann Julie L. , España Guido , Cavany Sean , Moore Sean , Perkins Alex , Hladish Thomas , Pillai Alexander , Toh Kok Ben , Longini Ira , Chen Shi , Paul Rajib , Janies Daniel , Thill Jean-Claude , Bouchnita Anass , Bi Kaiming , Lachmann Michael , Fox Spencer , Meyers Lauren Ancel ,   UT COVID-19 Modeling Consortium, Srivastava Ajitesh , Porebski Przemyslaw , Venkatramanan Srini , Adiga Aniruddha , Lewis Bryan , Klahn Brian , Outten Joseph , Hurt Benjamin , Chen Jiangzhuo , Mortveit Henning , Wilson Amanda , Marathe Madhav , Hoops Stefan , Bhattacharya Parantapa , Machi Dustin , Cadwell Betsy L. , Healy Jessica M. , Slayton Rachel B. , Johansson Michael A. , Biggerstaff Matthew , Truelove Shaun , Runge Michael C. , Shea Katriona , Viboud Cécile , Lessler Justin 
Published: July 2023

Our ability to forecast epidemics more than a few weeks into the future is constrained by the complexity of disease systems, our limited ability to measure the current state of an epidemic, and uncertainties in how human action will affect transmission. Realistic longer-term projections (spanning more than a few weeks) may, however, be possible under defined scenarios that specify the future state of critical epidemic drivers, with the additional benefit that such scenarios can be used to anticipate the comparative effect of control measures. Since December 2020, the U.S. COVID-19 Scenario Modeling Hub (SMH) has convened multiple modeling teams to make 6-month ahead projections of the number of SARS-CoV-2 cases, hospitalizations and deaths. The SMH released nearly 1.8 million national and state-level projections between February 2021 and November 2022. SMH performance varied widely as a function of both scenario validity and model calibration. Scenario assumptions were periodically invalidated by the arrival of unanticipated SARS-CoV-2 variants, but SMH still provided projections on average 22 weeks before changes in assumptions (such as virus transmissibility) invalidated scenarios and their corresponding projections. During these periods, before emergence of a novel variant, a linear opinion pool ensemble of contributed models was consistently more reliable than any single model, and projection interval coverage was near target levels for the most plausible scenarios (e.g., 79% coverage for 95% projection interval). SMH projections were used operationally to guide planning and policy at different stages of the pandemic, illustrating the value of the hub approach for long-term scenario projections.