Using synthetic case studies to explore the spread and calibration of ensemble atmospheric dispersion forecasts
Authors: Jones Andrew R. , Leadbetter Susan J. , Hort Matthew C. 
Published: October 2023

Abstract. Ensemble predictions of atmospheric dispersion that account for the meteorological uncertainties in a weather forecast are constructed by
propagating the individual members of an ensemble numerical weather prediction forecast through an atmospheric dispersion model. Two event
scenarios involving hypothetical atmospheric releases are considered: a near-surface radiological release from a nuclear power plant accident and a
large eruption of an Icelandic volcano releasing volcanic ash into the upper air. Simulations were run twice-daily in real time over a 4-month
period to create a large dataset of cases for this study. The performance of the ensemble predictions is measured against retrospective simulations
using a sequence of meteorological fields analysed against observations. The focus of this paper is on comparing the spread of the ensemble members
against forecast errors and on the calibration of probabilistic forecasts derived from the ensemble distribution. Results show good overall performance by the dispersion ensembles in both studies but with simulations for the upper-air ash release generally
performing better than those for the near-surface release of radiological material. The near-surface results demonstrate a sensitivity to the
release location, with good performance in areas dominated by the synoptic-scale meteorology and generally poorer performance at some other sites
where, we speculate, the global-scale meteorological ensemble used in this study has difficulty in adequately capturing the uncertainty from local-
and regional-scale influences on the boundary layer. The ensemble tends to be under-spread, or over-confident, for the radiological case in
general, especially at earlier forecast steps. The limited ensemble size of 18 members may also affect its ability to fully resolve peak values or
adequately sample outlier regions. Probability forecasts of threshold exceedances show a reasonable degree of calibration, though the
over-confident nature of the ensemble means that it tends to be too keen on using the extreme forecast probabilities. Ensemble forecasts for the volcanic ash study demonstrate an appropriate degree of spread and are generally well-calibrated, particularly for ash
concentration forecasts in the troposphere. The ensemble is slightly over-spread, or under-confident, within the troposphere at the first output
time step T + 6, thought to be attributable to a known deficiency in the ensemble perturbation scheme in use at the time of this study, but improves
with probability forecasts becoming well-calibrated here by the end of the period. Conversely, an increasing tendency towards over-confident
forecasts is seen in the stratosphere, which again mirrors an expectation for ensemble spread to fall away at higher altitudes in the meteorological ensemble. Results in the volcanic ash case are also broadly similar between the three different eruption scenarios considered in the study, suggesting that
good ensemble performance might apply to a wide range of eruptions with different heights and mass eruption rates.