SEEDS: Emulation of Weather Forecast Ensembles with Diffusion Models
Authors: name: Lizao Li; name: Rob Carver; name: Ignacio Lopez-Gomez; name: Fei Sha; name: John Anderson
Published: 2023-06-24 22:00:06

Abstract: Probabilistic forecasting is crucial to decision-making under uncertainty
about future weather. The dominant approach is to use an ensemble of forecasts
to represent and quantify uncertainty in operational numerical weather
prediction. However, generating ensembles is computationally costly. In this
paper, we propose to generate ensemble forecasts at scale by leveraging recent
advances in generative artificial intelligence. Our approach learns a
data-driven probabilistic diffusion model from the 5-member ensemble GEFS
reforecast dataset. The model can then be sampled efficiently to produce
realistic weather forecasts, conditioned on a few members of the operational
GEFS forecasting system. The generated ensembles have similar predictive skill
as the full GEFS 31-member ensemble, evaluated against ERA5 reanalysis, and
emulate well the statistics of large physics-based ensembles. We also apply the
same methodology to developing a diffusion model for generative
post-processing: the model directly learns to correct biases present in the
emulated forecasting system by leveraging reanalysis data as labels during
training. Ensembles from this generative post-processing model show greater
reliability and accuracy, particularly in extreme event classification. In
general, they are more reliable and forecast the probability of extreme weather
more accurately than the GEFS operational ensemble. Our models achieve these
results at less than 1/10th of the computational cost incurred by the
operational GEFS system.