Continuous Ensemble Weather Forecasting with Diffusion models
Authors: Martin Andrae
Tomas Landelius
Joel Oskarsson
Fredrik Lindsten
Venue: ICLR 2025 Poster (2025)
Keywords: weather forecasting
diffusion
ensemble forecasting
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We introduce a novel method for probabilistic weather forecasting based on diffusion models, that significantly increases the efficiency and accuracy of high time-resolution forecasts.

Abstract: Weather forecasting has seen a shift in methods from numerical simulations to data-driven systems. While initial research in the area focused on deterministic forecasting, recent works have used diffusion models to produce skillful ensemble forecasts. These models are trained on a single forecasting step and rolled out autoregressively. However, they are computationally expensive and accumulate errors for high temporal resolution due to the many rollout steps. We address these limitations with Continuous Ensemble Forecasting, a novel and flexible method for sampling ensemble forecasts in diffusion models. The method can generate temporally consistent ensemble trajectories completely in parallel, with no autoregressive steps. Continuous Ensemble Forecasting can also be combined with autoregressive rollouts to yield forecasts at an arbitrary fine temporal resolution without sacrificing accuracy. We demonstrate that the method achieves competitive results for global weather forecasting with good probabilistic properties.

Published as a conference paper at ICLR 2025
CONTINUOUS ENSEMBLE WEATHER FORECASTING WITH DIFFUSION MODELS
Martin Andrae' Tomas Landelius² Joel Oskarsson'
Fredrik Lindsten 'Linköping University, Sweden SWeddish Metcorological and Hydrological Institute {martin. andrae, joel. oskarsson, fredrik. lindstenne iu tomas. landel u smhi.
wussssh........m se
ABSTRACT
Weather forecasting has seen shift methods from numerical simulations to data-driven systems. While initial research in the area focused on deterministic forecasting, recent works have used diffusion models to produce skillful ensemble forecasts. These models are trained on a single forecasting step and rolled out au- toregressively. However, they are computationally expensive and accumulate er- for high temporal resolution due to the many rollout steps. We address these limitations with Continuous Ensemble Forecasting, a novel and flexible method for sampling ensemble forecasts in diffusion models. The method can generate temporally consistent ensemble trajectories completely in parallel, with no au- toregressive steps. Continuous Ensemble Forecasting can also be combined with autoregressive rollouts to yield forecasts at an arbitrary fine temporal resolution without sacrificing accuracy. We demonstrate that the method achieves competi- tive results for global weather forecasting with good probabilistic properties.
INTRODUCTION
Forecasting of physical systems over both space and time a crucial problem with plenty of real- world applications, including in earth sciences, transportation, and energy systems. A prime exam- ple of this is weather forecasting, which billions of people depend on daily to plan their activities. Weather forecasting also crucial for making informed decisions in areas such as agriculture, re- newable energy production, and safcguarding communities against extreme weather events. Current Numerical weather prediction (NWP) systems predict the weather using complex physical mod- els and large supercomputers (Bauer et al. 2015). Recently Machine learning weather prediction (MLWP) models have emerged, rivaling the performance of existing NWP systems. These models not physics-based but data-driven and have been made possible thanks to advancements in deep learning. By analyzing patterns from vast amounts of meteorological data (Hersbach et al. 2020), MLWP models predict the weather with the same accuracy global operational NWP models in fraction of the time (Kurth et al. 2023 Lam et al., 2023 Bi et al. 2003).
Following the success of deterministic MLWP models, the focus of the field has increasingly shifted towards probabilistic modeling. The probabilistic models generate samples of possible future weather trajectories. By drawing many such samples, referred to ensemble members, is possible to generate a set of possible forecasts, referred to as an ensemble forecast, for quantifying forecast uncertainty and detecting extreme events (Leutbecher & Palmer, 2008). Sampling forecasts from deep generative models also address the blurriness often observed in predictions from deterministic MLWP, yielding forecasts that better preserve the variability of the modeled entities. A popular class of deep generative models used for probabilistic MLWP are diffusion models (Price et al.) 2025 Lang al., 2024. Larsson et al., (025). While these models generate accurate and realistic looking forecasts, they are computationally expensive due to requiring multiple sequential forward passes through the neural network to generate a sample. Moreover, they are often applied iteratively to roll out longer forecasts (Price et al. 2025), which exacerbates the computational issue. Naively switching out this iterative rollout to directly forecasting each future timestep does not result in tra- jectories that consistent over time. The regressege rollout additionally puts some limitations on the temporal resolution of the forecast. Taking too small timesteps results in large accumulation of error over time (Bi al. 2023), which forces existing models to resort temporal resolution

---

generate 10 day forecasts with hour resolution without Published as a conference paper at ICLR 2025
Stochastic noise process
Diffusion
Time t
Initial condition
hour
6 hours
24 hours
Our proposed framework, Continuous Ensemble Forecasting, generates ensemble weather forecasts using a conditional diffusion model. The model takes lead time as input and forecasts the future weather state in single step, e.g. the forecast at 24 hours is generated directly from the initial condition, without seeing the intermediate predictions. To ensure temporal consistency, we correlate the driving noises for the different lead times. This can be done by fixing the noise, by defining a stochastic noise process. Repeating the procedure for different starting noise gives an ensemble of forecasts. Using this framework sacrificing performance.
Figure 1:
of 12h Price et al. (2025). However, in many situations it is crucial to obtain probabilistic forecasts much higher temporal resolution. This is not least when the forecasts used decision support in extreme weather situations and for capturing rapidly changing weather events.
We propose a continuous forecasting diffusion model that takes lead time as input and forecasts the future weather state in single step, while maintaining a temporally consistent trajectory for each ensemble member. This enables both autoregressive and direct forecasting within single model, improves the accuracy compared to purely autoregressive models at high temporal resolutions, and enables forecasting arbitrary non-equuiistant) lead times throughout the forecast trajectory. To generate ensemble forecasts, the model solves the had-time-dependent probability flow ODE start- ing different pure noise samples. ensure temporal consistency, we correlate driving noises for the different lead times, c.g., using single noise sample for all timesteps, illustrated in fig. [] which enables generating continuous trajectory for each member. This enables parallel sam- pling of individual ensemble members, bypassing the need for multi-step loss functions, and thus accelerating ensemble forecasting.
Contributions. We propose a novel method for ensemble weather forecasting built on diffusion that: 1) can generate ensemble member trajectories without iteration, 2) can forecast arbitrary lead- times, 3) can be used together with iteration to improve performance on long rollouts, 4) achieves competitive performance in global weather forecasting.
RELATED WORK
Ensembles from perturbations. NWP, ensemble forecasts are created by perturbing initial conditions model parameterizations, The same idea has been applied in MLWP using initial state perturbations (Bi et al.) 2023; Kurth et al. 2023 Chen et al. 2023b ei al. 2024), model parameter perturbations (Hu et al. 2023 Weyn et al. (021) and Monte Carlo dropout (cheer & Messori] [021] Hu et al. 2023). all these cases the underlying model is still deterministic, trained with a mean squared error loss function predict the average future weather. These ensembles thus be viewed as a mixture of means, inheriting the spatial oversmoothing characteristic of deterministic forecasts.
'The code is available at httpr///tiicccmiimmttttooooooooe

---

Published as a conference paper at ICLR 2025
Generative models. Another way of getting ensembles is by modeling the distribution directly through generative models. Price et al. (2025) train a graph- based diffusion model to produce 15- day global forecasts, at 12-hour steps and 0.25 resouuuuot Their GenCast model generates realistic forecasts but is slow to sample from, requiring sequential forward passes through the network both for sampling each timestep and to roll out the forecast over time.
sartttt-vrraabe-baseeee MLWP models (Hu et al.) 2023] Oskarsson et al. 2024] ZZong et al. 2024) can perform inference in a single forward pass, but have been shown to suffer from more blurriness compared diffusion models and require delicate hyperparameter tuning (Oskarsson et al. 2024). Apart from forecasting, diffusion models have also been successfully applied to other weather-related tasks, including downscaling (Chen et al., 2023a Mardani et al. 2024, Pathak et al. 2024), enlarging physics-based ensembles (Li al. 2024) and generating realistic weather from climate scenarios (Bassetti et al., 2023). However, it is important to note that since the output from these generative models does not directly influence the forecasting process, there are no guarantees that the resulting dynamics will remain continuous
over time. Hua et al. (2024) explore the possibility to incorporate prior information in MLWP diffusion forecasting, by guidance from existing NWP forecasts or climatology, but they do not consider ensemble forecasting.
Temporal resolution. Unlike in NWP, where stability conditions dictate the timestep, MLWP models are free to predict at any temporal resolution. Still, the most common approach to learn to forecast single short timestep (6h) and iterate this process until the desired lead time (Lam) ict al., 2023; Chen et al., 2023b). Although intuitive, this process can lead to error accumulation and is impossible to parallelize due to its sequential nature (Bi et al., 2023). Multi-step losses have been shown to reduce the error accumulation for deterministic (Lam et al. (2023) and latenl-variable based models (Oskarsson et al. 2024), but are not trivially implemented in diffusion models. Taking longer timesteps (24h) has been shown to give better results (Couairon et al.] 2024 Bi et al.) 2023), > but comes at the loss of temporal resolution. Bi et al. (2023) resolves this by training multiple models to forecast different lead times, which are then combined in different ways to reach the lead times of interest. Nguyen et al. (2023] 2025) use similar setup, but train single model taking the forecast lead time as an imput. This approach, called continuous forecasting, parallelizes the prediction of the fine temporal scales, but has so far only been applied to deterministic models. Other approaches have tried to learn fully time-continuous dynamics by using an ODE to generate forecasts (Verma
ct al. 2024. Saleem et al. 2024 Rühling Cachay et al. 2023 Kochkov et al. 2024).
Spatio-temporal forecasting with diffusion models. Outside of MLWP, diffusion models have also been applied to forecasting other spatio-temporal processes (Yang et al., 2024). Notable ex- amples include turbulent flow simulation (Kohl al., 2024; Rühling Cachay al. (2023) and PDE solving (Lippe et al. (003)). Yang & Sommer (023) apply diffusion models conditioned on the pre- diction lead time specific floating-smet -moke Huid field, but do not consider ensemble forecasting. Another alternative to autoregressive rollouts is the DYffusion framework (Rühling Cachay et al.) 2023). where stochastic interpolation and deterministic forecasting is combined into diffusion- like model. The method allows for forecasting at arbitrary temporal resolution, but still requires sequen- tial computations for sampling the prediction. To get a probabilistic model, they introduce a layer dropout term in the interpolator that they keep during inference. This makes the performance sensitive to the dropout rate, which can not be changed without retraining both the interpolator and forecasting networks. Further, the interpolation gives no guarantee of temporal continuity of trajectories, and since its trained with a mean squared error loss, is prone to blurring similar to deterministic forecasting models. DYffusion has been successfully applied to climate modeling [Rühling Cachay et al. 2025). but not weather forecasting.
3 BACKGROUND
Problem statement. This paper targets the global weather forecasting problem, an initial-value problem with intrinsic uncertainty. Consider weather state space X' containing the grid of target
variables. Given information about previous weather states X(2) C X' at times Ω C (-o,, O]. the aim is to forecast a trajectory X(T) of future weather states X (-oc, o0) + X' at times T C (0, T) for some time horizon T. In particular, the task is to learn and sample from the conditional distribution p (X)(
3

---

Published as a conference paper at ICLR 2025
Autoregressive forecasting. To simplify the problem. T is often chosen set of discrete equally
spaced times {ko] for some timestep 6. By choosing f = {-kō) ) - k= and assuming Mth order
Markovian dynamics, the joint distribution of [[k] X x(k) can be factorized over successive states,
N-1
II k=
and the forecasts can be sampled autoregressivelly. This way, the network only has to learn to sample a single step x(X [K+1]|X [K--ilIXX-MMMM M:ke).
Conditional Diffuusion Models. Similarly Price et al. (2025), we model p k ++]+111 (K=
using a conditional diffusion model (Ho et al. 2 Song et al. 2021] Karras et al. 2022). Diffusion models generate samples by iteratively transforming noise into data. To forecast a future weather state [K++]] given X [k] we sample random noise from Proise and iteratively transform it until resembles sample from p(
We consider the SDE formulation of diffusion models presented by Karras et al. (2022), 2). but remark that our framework generalizes to any diffusion or flow matching framework based on stochastic differential equations. Sampling can then be done by solving the probability flow ODE
=(0) =00)= L. S, (z(1), ( ds, x(1) Praise
(2)
starting in pure noise and ending in our forecast 20)
MA&]. The ODE-solver
can be chosen freely, we employ the second-order Heun's 's method. Here Sp is the neural network trained to match the score function V. log Ps through the denoising training objective as presented by Karras et al. (202)). Similar to (Karras et al. 2022), we choose o, = s and feed the noise level o to Se in each layer Fourier embedding. Repeating this process for different noise samples 211) gives an ensemble of forecasts. For a more detailed description of diffusion models see appendix B
4 CONTINUOUS ENSEMBLE FORECASTING
Autoregressive forecasting models are simple to train but can suffer from error accumulation at long horizons. Consider again the general problem of sampling from (T) forecasting, a single-step prediction is given by conditioning on the lead time t E T and training conditional score network Sg(2, ((2,, t) to simulate directly from the marginal distribution (9)), l). This does not require setting a fixed ó, making the setup more flexible. The lead time l is added to the conditioning arguments for clarity and can be passed to the network in the same way as the noise level CT While this allows sampling a distribution of states X Xtt at each time t E T, combining these naively does not result in trajectory X(T). This is because X(t) are samples from the marginal distributions p( (t)( ()))) t) and not the joint trajectory distribution XX We propose Continuous Ensemble Forecasting, a novel method of combining samples from ((X( t) into forecast trajectories X(T) without resorting to autoregressive predictions.
In continuous
The core idea in our method is to control the source of randomness. To sample from p( (X)))) !), we sample some noise Z ~ Paoise and feed to an ODE-solver that solves the forecast X(t) for lead time t. Since the ODE-solver
probability flow ODE in eq. (2), giving is deterministic, the randomness is limited to the noise initialization Z. If we freeze the noise, the
ODE-solver becomes deterministic map f? : X X X' parameterized by the neural network So- Applying this map to previous states (22 and a time E 'T gives a forecast X().. Repeat- ing this for several t,,...,yy IN T gives a sequence of forecasts (((t)) ). Extending this to all t T. we can construct a trajectory X(T) (X))), T). We treat this as sample from P(X (X) and propose to use Algorithm to sample such trajectories.
4.1 MATHEMATICAL MOTIVATION
In deterministic system, the dynamics can be described by a forecasting function f X x T X' that maps previous states X(2) E X19|, to future states ((). While weather is in principle
4

---

Published as a conference paper at ICLR 2025
Algorithm 1 The Continuous Ensemble Forecasting algorithm
1: input: Initial conditions x([2), times {1,)/= ensemble size Nenss network So
2: sample {2 )mm
N(,, I) 1j=1
Can be done fully in parallel for all j and 4: x! +- PROBABILIY-FLLO--SOLLEEE (1 (12),
3: for all E { N j.j E {L...., ] do
return { N==1:
governed by deterministic equations, its chaotic nature, lack of information, and our inability to resolve the dynamics sufficiently high spatio-temporal resolution gives an intrinsic uncertainty. This motivates the formulation of weather as stochastic dynamical system. In such a system, no function can describe the entire dynamics. At each instance, there might be range of functions f2... that all describe some possible evolution, some more likely than others. If we consider the space of all such functions, we can formalize this by defining a probability density j over this space, describing the likelihood of each function. To create an ensemble of possible functions, from μ.. Given previous states X (22), evaluating each function gives an
sample several fi f f.N, ensemble of possible trajectories (T) ('(X T).
The key insight in motivating our method is the identification of the latent noise space (X' Puoise) with the solution space (F, 1.). In our method, the sampling algorithm can be represented by parame- terized function f/ where we have frozen the noise Z Pnoise. Under the regularity conditions specified in appendix [ this func- tion is uniquely defined by 0 and the ODE-solver for any given Z. Thus, as illustrated in fig. 2 our setting mirrors the theoretical set- ting. Given sufficient data and model capacity, the neural network So matches the score function S. Consequently, the distribution of the functions l should mirror that of the solutions I². Since I describes a possible evolution of weather, it has to be continuous a function of time. To ensure that the generated trajectories are also continuous in t, regularity conditions need to be imposed on Sp to ensure that it depends smoothly on the lead time. In appendix E we provide sufficient conditions and proof of this property. We also show empirically in sec. 5.1 that this is satisfied in practice.
(X; (roise) f----> (F,P)
(x(0)))) f-)(((0))))
X
Figure 2: Diagram depicting the connection between the latent noise space and the so- lution space.
4.2 AUTOCORRELATED NOISE
As mentioned above, under regularity conditions on Se, expect the solution map f (X(),, t) to be continuous in f. A shortcoming of this approach, however, is that it also constrains the trajectories to be conditionally deterministic, conditionally on the solution fo (X(S), t*) at any fixed time point
t" E T. Specifically, consider the conditional distribution p (t) (t*),, ( t) for t,t" E T under the proposed model. Conditionally on X(()) L (9), (*) we can, conceptually, invert the ODE which generated X(t)) recover the driving noise Z. Now, if X = 1 (X) is generated using the exact same noise variable, we find that (() XX t) is a Dirac point mass concentrated at f (X()),, This violates the assumed stochasticity of the dynamical process that we are modelling.
To address this shortcoming, simple extension of the proposed model is to replace the fixed Z with a stochastic process Z(T) with continuous sample trajectories. The stochastic process is chosen be stationary with marginal distribution Z(t) ~ Proise: Vt E T. This ensures that the genera- tive model at any fixed is probabilistically equivalent with the fixed noise setting. Specifically, no changes to the training algorithm are needed since the training is based solely on time marginals. Al- lowing temporal stochasticity in the driving noise process results in non-deterministic relationship between the states at different lead times, while we can keep the temporal consistency by ensur- ing that the driving process is sufficiently autocorrelated. A simple choice is to select Z(T) as an Ornstein-Uhlenbeck. (OU) process (Gillespie 1996). dt( = p((t)tt + /2pdW (t), for some correlation parameter 0 and W (t) being Brownian motion. This process can be easily simulated from to generate a noise sequence { (Z {t, for lead-times (t, J. -== as done in Algorithm 2 The initial noise sample (()) (O, I) is perturbed according to the OU process by sampling new
5

---

Published as a conference paper at ICLR 2025
noise I) in each timestep. The update step in line 5 comes from the choice of an OU process for Zt) and corresponds to an exact simulation of the process at the specified lead times (tt) le.. it guarantees the correct marginal distributions = [(t)) (O, I) for all lead times. An alternative to the OU process is to use a Gaussian process with stronger smoothness properties, e-g., with squared exponential kernel. This could prove useful for formally proving time continuity of the resulting trajectories also under a stochastic noise process, but we leave such an analysis for future work.
Algorithm The Extended Continuous Ensemble Forecasting algorithm
1: input: Initial conditions x(2), times {t,)/1 ensemble size Nenss network So, correlation pa- rameter
2: sample {=m ~ (0,1 I)
3: for 2 do
4: sample {v] 1-=1 ~ ().
5:
x expl expl(pp(tt p(t: 4-11))=-11 v1 V1- exp(x-2ppp -t,-1))
6: for all i {1..... Ni. E fl..... nmms do b Can be done fully in parallel for all j and i
7:
4 BROBAULLYY-FLOWWSLVEERCC ((;
return K=E
4.3 AUTOREGRESSIVE ROLL-OUTS CONTINUOUS INTERPOLATION
Continuous forecasting is effective for forecasting hours days but can struggle to forecast longer lead times where the correlation weaker. Autoregressive forecasting excels at long lead times when used with longer (24h) timesteps (Bi al. 2023), but comes the loss of temporal resolu- tion. In our framework, it becomes possible to sample both autoregressive and continuous forecasts with the ing the intermediate timesteps using Continuous Ensemble Forecasting, as outlined in Alg. 3 We refer to this combined method Autoregressive Rollouts with Continuous Interpolation (ARCI). Our method limits the eтror accumulation without sacrificing temporal resolution. This allows for producing forecasts an arbitrary fine temporal resolution, while retaining the accuracy of the best autoregressive methods throughout the whole forecast. By limiting the number of autoregressive steps, more of the forecast also becomes parallelizable, allowing for rapidly generating forecasts on large compute clusters. Furthermore, we can straightforwardly use different time resolutions during different parts of the forecast trajectories, by for instance forecasting with steps for the first few days and then switching to longer timesteps for long lead times. Note that ARCI can be used with either fixed (with Alg. I or stochastic (with Alg. 2) driving noise. However, we emphasize that these two algorithms are probabilistically equivalent for all time marginals, and only differ in the autocorrelation of forecast trajectories.
model. We propose to leverage this by iterating longer timestep and forecast-
Algorithm ARCI (Autoregressive roll-outs with continuous interpolation)
1: input: Initial conditions T-LL:0, interpolation times {t, N ensemble size nenss autoregressive steps M, network Sg
2: for m 0 to M - do
3: { fo 1==1
Alg. JmN LamN (t.) Denss So)
Also possible to use Alg.
//=11/m return {x/ (=MMM
5 EXPERIMENTS
Data. We evaluate our method on global weather forecasting up to 10 days hour timesteps. We use the downsampled ERA5 reanalysis dataset (Hersbach et al. (020) at resoluuttt and l-hour increments provided by WeatherBench (Rasp al. 2020). The models
6, and 24

---

Published as a conference paper at ICLR 2025
trained forecast 5 variables from the ERA5 dataset: geopotential 500hPa (z500), temper- ature at 850hPa (t (t85)), ground temperature 2m) and the ground wind components (u10, v10). The atmospheric fields 2500 and 8850 offer comprehensive view of atmospheric dynamics and thermodynamics, while the surface fields t.2m and u1 0, v1 0 are important for day-to-day activi- ties. We also evaluate the forecast of ground wind speed ws 1 D, computed from the model outputs as ws10 Vul v10². This is useful for evaluating how well the methods model cross-variable dependencies. All variables are standardized by subtracting their mean and dividing by their stan- dard deviation. Together with the previous states we also feed the models with static fields. These include the land-sea mask and orography, both rescaled to [0, 1]. All models are trained on the period 1979-2015, validated on 2016-2017 and tested on 2018. We consider every hour of each year as forecast initialization times, except for the first 24h and last 10 days in each subset. This guarantees that all times forecasted or conditioned on lie within the specific years.
Metries. We evaluate the skill of the forecasting models by computing the Root Mean Squared Error (RMSE) of the ensemble mean. As a probabilistic metric we also consider Continuous Ranked Probability Score (CRPS) GGeeiiigg& & Raftery, 2007). which measures how well the the predicted marginal distributions capture the ground truth. We also evaluate the Spread/Skill-Ratio (SSR), which is a common measure of calibration for ensemble forecasts. For model with well calibrated uncertainty estimates the SSR should be close to (Fortin et al. 2014). Detailed definitions of all metrics are given in appendix A
Models. We propose to use the ARCI model described in rreferrd as ARCI-24/6h.
We train it to forecast t E {6, 12, 18, 24} (hours) and roll it out autoregressively with 24h steps, hence the name. Training is done on 40GB NVIDIA A100 GPU and takes roughly 2 days. We emphasize again that using fixed, correlated or uncorrelated noise results in probabilistically equiv- alent forecasts for all time marginals, and only differ in the autocorrelation of forecast trajectories. Hence the choice of algorithm inside ARCI does not matter, and for all metrics below that are com- puted for specific lead times we only report results for one version of the algorithm. We return to the difference between Alg. T and Alg. 2 when studying the temporal difference below.
To evaluate the effectiveness of our approach, we compare it to other MLWP baselines. Determin- istic is deterministic model trained using MSE-loss
single hour timestep, and unrolled up to 10 days. AR-(6, 24)h are diffusion models trained only on forecasting a single fixed 6 = {6. 24)h ahead, and then autoregressively unrolled up 10 days. This is the exact forecasting setup of (Price) et al. 2025) and the AR- models can thus be seen as a reimplementation of GenCast with a U-Net architecture. Since
interested in forecasts with high temporal resolution, the AR-24h model operating at a coarser temporal resolution represents an upper limit on performance for long rollouts rather than a competing model C1-6h is a diffusion model performing continuous forecasting con- ditioned on a specific lead time. It is trained on uniformly sampled lead times from {k:5) k==1 with 6h. This is the method proposed in alg. I Sampling 10-day forecast with 6h resolution for a single member from AR-6h takes 32 seconds, but by parallelizing the 6h timesteps in ARCI-24/6h this reduces to seconds.
To compare against another family of ensemble forecasting models from the literature we retrain the Graph-EFM model (Oskarsson et al. 2024) on our exact data setup. Graph-EFM a graph- based variale model that produces forecasts by 6h iterative rollout steps. We also present new results from the DYffussin model Rühling Cachay et al. (2023), trained on ERA5 data at 1h resolution. All models except Graph-EFM use the same U-net with 3.5M parameters. The architecture is based on the U-net in Karras et al. (2022) and presented in appendix [B] Due to its specific
process, DYffusion only supports conditioning on the current timestep. Ω = {0}. For all other models we condition on the two previous timesteps f = {0, -5).
5.1 RESULTS
Qualitative results. Figure [egureBshowe shows example forecast from ARCI-24/6h for temperature at 850 hPa 850) at 10 days lead time. The forecasts are rich in detail, resembling the true state more than the ensemble mean. Examples of other variables are given in appendix
7

---

Published as a conference paper at ICLR 2025
Truth
Deterministic
Ensemble mean
Ensemble Std
Sia Dex
0
Sample #1
Sample #2
Sample #3
Sample #4
25
0
Figure 3: Example forecasts for temperature at 850hPa (:850) lead time 10 days. The forecasts are generated using ARCI-24/6h except for Deterministic which sampled using the deterministic model. The bottom shows 4 ensemble members, randomly chosen out of the 50.
Table 1: Selection of results for 5 and 10 day forecasting using models with 6h resolution for geopotential at 500 hPa (z500) and temperature at 850 hPa 850))). For RMSE and CRPS, lower values better, and SSR should be close to 1. The best values marked with bold and second best underlined. The AR-24h model included for reference, but is not considered for best model since it operates at coarser 24h resolution.
Lead time days
Lead time 10 days
Variable
Model
RMSE
CRPS
SSR
RMSE
CRPS
SSR
z500
Deterministic
766.7
483.9
1042
661.5
Graph-EFM
699.1
317.5
1.13
817.1
373.6
1.1
AR-6h
602.3
287.8
0.75
811.8
391.9
0.88
C1-6h
707.8
321.2
0.59
885.7
406.6
0.6
ARCI-24/6h
560.9
256.7
0.86
765.6
355.2
0.93
AR-24h
544.2
242.7
0.84
750.6
335.2
0.94
t850
Deterministic
3.48
2.36
4.54
3.17
Graph-EFM
3.12
1.56
1.11
3.51
1.77
1.12
AR-6h
2.72
1.34
0.82
3.39
1.69
0.92
C1-6h
3.06
1.5
0.74
3.68
1.85
0.71
ARCI-24/6h
2.6
1.27
0.9
3.29
1.63
0.95
AR-24h
2.55
1.24
0.89
3.25
1.6
0.96
Quantitative results. Table and figure 4a show metrics for selection of lead times and vari- ables. Scores for the remaining variables are listed in appendix [ For probabilistic models on 6h resolution, we sample 50 ensemble members at each initialization time. All probabilistic models show a clear improvement over deterministic model. C1-6h performs well on short-term fore- casting but struggles at longer horizons. This is likely due to the challenge of learning any useful relationships between initial states and later lead times, which are weakly correlated. ARCI-24+61 outperforms all models 6h resolution, including the external baseline Graph-EFM and the GenCast setup AR-6h, and matches AR-24h in almost all scores. All diffusion- -hase models have SSR < indicating some systematic underdispersion. In tables 6 in appendix we present error bars calculated for the ARCI-24/6h model, which shows that the model is robust to network initialization.
High Temporal Resolution. Our ARCI method allows for producing forecasts at high temporal resolution while retaining the accuracy of methods taking longer autoregressive steps. here demonstrate this by producing hourly forecasts. Figure 4b/ the scores of selection of models for t850 for 10-day forecasts at Ih resolution. Since the 1h forecasts requires more computational effort, we use only 10 ensemble members instead of 50. The AR-1h 1h model has the same GenCast- setup as AR- but on resolution and ARCI-24/11 hh is trained in the same way as ARCI-24/6h but 1h resolution and with Ω (0, 224. The autoregressive AR-1h 1h model performs much worse
8

---

Published as a conference paper at ICLR 2025
RMSE
CRPS
SSR
Graph-EFM
3
1.5
1.0
AR-6h
AR-24h
2
1.0
0.8
CI-6h
1
0.5
ARCI-24/6h
1
5
9
5
9
3
7
Lead Time (days)
Lead Time (days)
Lead Time (days)
(a) Selection of models at 6h resolution.
RMSE
CRPS
SSR
ARCI-24/1h
1.0
4
ARCI-24/2h*
2
AR-1h
2
0.5
AR-24h
DYffusion
1
1
5
1
5
9
Lead Time (days)
Lead Time (days)
Lead Time (days)
(b) Selection of models at Ih resolution.
Figure 4: RMSE, CRPS and SSR for temperature at 850 hPa (t.850).
than 6 or 24 hours. The continuous model, however, does not lose performance by increasing temporal resolution, making the Ih timestep forecasts as skillful as the 24 hour ones. Although DYffusion achieves much better results than AR-Ih Ih, it is beaten by ARCI-24/Ih in all variables and lead times.
4X
AX
Forecast continuity. By correlating the noise across timesteps, our continuous ensemble fore-
0.25
1.0
casting method can produce temporally consis- tent ensemble trajectories without resorting to autoregressive predictions. Here give em- pirical justification for this claim. Since autore- gressively sampled forecast trajectories are essarily continuous, there is no standard way of measuring the continuity of forecast. We propose using the mean temporal difference AX (X())))) X(t 1)] as measure of fore- cast continuity. Figure [ shows AX for a CI- 1h continuous model trained up to hours with 1-hour timesteps. Compared to using different noise each step (p + x.). the temporal dif- ference of our model (p = 0) stays close to the temporal difference of the data. This supports our claim that continuous ensemble forecasting pro- duces continuous trajectories. When the noise is fixed (p 0) the temporal difference decreases with lead time, corresponding to predictions with smaller temporal variations. Letting the noise vary with noise factor (p In10) as in alg. stops this from happening. The bias between AX of the data and our model likely due to the model producing slightly blurrier forecasts, making the differences smaller.
0.20
0.5
0.15
12
24
24
Lead Time (hours)
Lead Time (hours)
0=
p=
In10
p
Figure 5: Temporal difference for temperature 850 hPa (t850) for different values of p in algorithm 2 Choosing
0 fixes the noise, p o= In10 allows it vary and p - oo gives com- pletely uncorrelated noise. The black line refers to the temporal difference of the data.
Continuous Time Forecasting. One the key advantages of continuous ensemble forecasting is its ability to generate forecasts arbitrarily fine temporal resolutions. To test this, we train a model at a coarser temporal resolution and evaluate it at a higher resolution. Specifically, we consider the ARCI-24/2h* model, which is trained on lead times spaced hours apart but evaluated at every 1-hour timestep. To ensure a fair comparison with ARCI-24/1h, ARCI-24/2h# is trained with Ω {0, 224), providing access the same information each timestep. As shown in fig. 4b

---

Published as a conference paper at ICLR 2025
ARCI-24/2h* performs similarly to ARCI-24/Ih, demoonsratnn its ability to generalize beyond the temporal resolution seen during training.
For highly time-dependent fields such as t2m, ARC1-24/2h performs worse at the first forecast in each iteration (1h, 25h....) ), as seen figure T0 in appendix C For lead-times t not considered during training, the model has trained forecasting t- and f+ thus only having interpolate to However, since we do not train on forecasting Oh, the model instead has to extrapolate to t = Ih what was learned for 2h forecasts. This issue could possibly be fixed by letting the network also train on Oh forecasts.
An alternative directly producing forecasts fine temporal resolution would be to linearly interpolate the forecasts sampled using an autoregressive model. In fig. 8 in appendix [ we show that linearly interpolated forecasts behave much worse than the ARCI model both and 6-hour resolution.
CONCLUSION
We present Continuous Ensemble Forecasting, novel framework for probabilistic MLWP that in- creases the efficiency, accuracy, and flexibility of weather forecasts at high temporal resolution. When combined with autoregressive prediction, our ARCI method can produce accurate 10-day global ensemble weather forecasts with 1-hour resolution. It achieves the same accuracy as purely autoregressive model with 24-hour steps and surpasses models like GenCast and DYffusion when reimplemented using the same neural network architecture. With this work, we hope to show that the possibilities with generative modeling for spatio-tempont predictions are still largely unex- plored and fruitful area of research.
Limitations. While our proposed framework achieves good results on 5.625 Weatherbench (Rasp al. 2020) data, we have yet to show that the method scales to problems with higher spatial res- olution. Additionally, as the lead time increases, the correlation between initial and future states becomes weaker, limiting the application of continuous forecasting. While method parallelizes more of the sampling than previous autoregressive models, sampling from the diffusion model still requires many sequential forward passes through the network. Sampling is thus still slower than for latent variable models, but the predicted distribution more accurate.
Future work. One interesting direction for future work is further investigation of autocorrelated noise, in particular, how the choice of stochastic process can aid in producing continuous trajectories with stationary temporal difference. This includes correlating the noise in the autoregressive steps with the continuous steps, which could help ease the transition between them. Another idea is to take the direction DYffusion (Rühling Cachay et al. (023) and directly adjust the diffusion objective to better suit temporal data. While we have demonstrated continuous ensemble forecasting for weather, the idea is generally applicable and it would also be of interest apply other spatio-temporal forecasting problems.
ACKNOWLEDGMENTS
This work was financially supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation, the Swedish Research Council (project no: 2020-04122, 2024-0501 1), and the Excellence Center at Linköping in Information Technology (ELLIIT), Our computations were enabled by the Berzelius resource the National Supercomputer Centre, provided by the Knut and Alice Wallenberg Foundation.
REFERENCES
Michael Samuel Albergo and Eric Vanden-Eijnden. Building normalizing flows with stochastic interpolants. In The Eleventh International Conference on Learning Representations, 2023.
Seth Bassetti, Brian Hutchinson. Claudia Tebaldi, and Ben Kravitz. DiffESM: Conditional emula- tion of earth system models with diffusion models. In ICLR 2023 Workshop on Tackling Climate Change with Machine Learning, 2023.
10

---

Published as a conference paper at ICLR 2025
Peter Bauer, Alan Thorpe, and Gilbert Brunet. The quiet revolution of numerical weather prediction. Nature, 525(7567):47-55, 2015.
Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. Accurate medium- range global weather forecasting with 3D neural networks. Nature, 619(7970):533 538, 2023.
Lei Chen, Fei Du, Yuan Hu, Zhibin Wang, and Fan Wang. SwinRDM: integrate SwimRNN with diffusion model towards high-resolution and high-quality weather forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 322-330, 2023a.
Lei Chen, Xiaohui Zhong, Feng Zhang, Yuan Cheng, Yinghui Xu, Yuan Qi, and Hao Li. FuXi: cascade machine learning forecasting system for 15-day global weather forecast. npj Climate and Atmospheric Science, 6(1):1-11, 2023b.
Guillaume Couairon, Renu Singh, Anastase Charantonis, Christian Lessig, and Claire Monteleoni. Archesweather & archesweathergen: a deterministic and generative model for efficient ml weather forecasting. arXiv preprint drXiv:24/2.129711 2024.
Vincent Fortin, Mabrouk Abaza, Francois Anctil, and Raphael Turcotte. Why should ensemble spread match the RMSE of the ensemble mean? Journal of Hydrometeorology, 15(4):1708-1713, 2014.
Daniel Т. Gillespic. Exact numerical simulation of the cunstein-uhlenber process and its integral. Phys. Rev. E, 1996.
Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, 2010.
Tilmann Gneiting and Adrian E. Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477):359-377 2007.
Jack K. Hale. Ordinary Differential Equations. Courier Corporation, 2009. ISBN 978-0-486-47211
9.
Hans Hersbach, Bill Bell, Paul Berrisford, Shoji Hirahara, András Horányi, Joaquín Muñoz-Sabater, Julien Nicolas, Carole Peubey, Raluca Radu, Dinand Schepers, Adrian Simmons, Cornel Soci, Saleh Abdalla, Xavier Abellan, Gianpaolo Balsamo, Peter Bechtold, Gionata Biavati, Jean Bid- lot, Massimo Bonavita, Giovanna De Chiara, Per Dahlgren, Dick Dee, Michail Diamantakis, Rossana Dragani, Johannes Flemming, Richard Forbes, Manuel Fuentes, Alan Geer, Leo Haim- berger, Sean Healy, Robin J. Hogan, Elías Hólm, Marta Janisková, Sarah Keeley, Patrick Laloy- aux, Philippe Lopez. Cristina Lupu, Gabor Radnoti, Patricia De Rosnay, Iryna Rozum, Freja Vamborg, Sebastien Villaume, and Jean-Noël Thépaut. The ERAS global reanalysis. Quarterly Journal of the Royal Meteorological Society, 146(730):1999 2049, 2020.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Diffusion Probabilistic Models. Advances in Neural Information Processing Systems, 33, 2020.
Yuan Hu, Lei Chen, Zhibin Wang, and Hao Li. SwinVRNN: A data-driven ensemble forecasting model via learned distribution perturbation. Journal of Advances in Modeling Earth Systems, 15 (2), 2023.
Zhanxiang Hua. Yutong He, Chengqian Ma, and Alexandra Anderson-Frey. Weather prediction with diffusion guided by realistic forecast processes, 2024. arXiv:2402.06666
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Advances in Neural Information Processing Systems, 35, 2022.
Dmitrii Kochkov, Janni Yuval, Ian Langmore, Peter Norgaard, Jamie Smith, Griffin Mooers, Mi- lan Klöwer, James Lottes, Stephan Rasp, Peter Düben, Sam Hatfield, Peter Battaglia, Alvaro Manchez-Gonzale Matthew Willson, Michael P. Brenner, and Stephan Hoyer. Neural general circulation models for weather and climate. Nature, 1060022):000-106666 August 2024. ISSN 1476-4687. doi: 10.1038/s41 586-024-07744.. Publisher: Nature Publishing Group.

---

Published as a conference paper at ICLR 2025
Georg Kohl, Liwei Chen, and Nils Thuerey. Benchmarking autoregressive conditional diffusion models for turbulent flow simulation. In JCML 2024 for Science Workshop, 2024.
Thorsten Kurth, Shashank Subramanian, Peter Harrington, Jaideep Pathak, Morteza Mardani, David Hall, Andrea Miele, Karthik Kashinath, and Anima Anandkumar. FourCastNet: Accelerating global high-resolution weather forecasting using adaptive fourier neural operators. In Proceedings of the Platform for Advanced Scientific Computing Conference, 2023.
Remi Lam, Alvaro Sanchez-Gonzalez Matthew Willson, Peter Wirnsberger, Meire Fortunato, Fer- ran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton-Rosen, Weihua Hu, Alexander Merose, Stephan Hoyer, George Holland, Oriol Vinyals, Jacklynn Stott, Alexander Pritzel, Shakir Mo- hamed, and Peter Battaglia. Learning skillful medium-range global weather forecasting. Science, (92(6677)11416-142) 2023.
Simon Lang, Mihai Alexe, Mariana CA Clare, Christopher Roberts, Rilwan Adewoyin, Zied Ben Bouallègue, Matthew Chantry, Jesper Dramsch, Peter D Dueben, Sara Hahner, et al. Aifs-crps: Ensemble forecasting using a model trained with loss function based on the continuous ranked probability score. arXiv preprint arXiv:2412.0 158222...183.,... 2024.
Erik Larsson, Joel Oskarsson, Tomas Landelius, and Fredrik Lindsten. Diffusion-lamy Probabilistic limited area weather forecasting with diffusion. arXiv preprint arX1v:2502.05332, 2025.
Martin Leutbecher and Tim N Palmer. Ensemble forecasting. Journal of Computational Physies, 227(7)33155-339, 2008.
Lizao Li, Robert Carver, Ignacio Lopez-Gomez, Fei Sha, and John Anderson. Generative emulation of weather forecast ensembles with diffusion models. Science Advances, 10(13), 2024.
Phillip Lippe. Bas Veeling, Paris Perdikaris, Richard Turner, and Johannes Brandstetter. PDE- Refiner: Achieving accurate long rollouts with neural pde solvers. Advances in Neural Informa- fion Processing Systems, 36, 2023.
Ilya Loshchilov and Frank Hutter. Fixing weight decay regularization in adam. abs/1711.05101,, 2017a.
CoRR,
Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts. In Inter- national Conference on Learning Representations, 2017b.
Morteza Mardani, Noah Brenowitz, Yair Cohen, Jaideep Pathak, Chich-Yu Yu Chen, Cheng-Chin Liu, Arash Vahdat, Mohammad Amin Nabian, Tao Ge. Akshay Subramaniam, Karthik Kashinath, Jan Kautz, and Mike Pritchard. Residual corrective diffusion modeling for km-scale atmospheric downscaling, 2024. arXiv:2309.152 15214.
Tung Johanees Brandstetter, Ashish Kapoor, Jayesh K. Gupta, and Aditya Grover. ClimaX: Johannes
A foundation model for weather and climate. In Proceedings of the 40th International Conference on Machine Learning, 2023.
Tung Nguyen, Rohan Shah, Hritik Bansal, Troy Arcomano, Romit Maulik, Rao Kotamarthi, Ian Foster, Sandeep Madireddy, and Aditya Grover. Scaling transformer neural networks for skill- ful and reliable medium-range weather forecasting. Advances in Neural Information Processing Systems, 37:68740-68771, 2025.
Joel Oskarsson, Tomas Landelius, Mare Peter Deisenroth, and Fredrik Lindsten. Probabilistic weather forecasting with hierarchical graph neural networks. In The Thirty-eighth Annual Con- ference on Neural Information Processing Systems, 2024.
Jaideep Pathak, Yair Cohen, Piyush Garg, Peter Harrington, Noah Brenowitz, Dale Durran, Morteza Mardani, Arash Vahdat, Shaoming Xu, Karthik Kashinath, and Michael Pritchard. Kilometer-seal convection allowing model emulation using generative diffusion modeling. 2024. arXiv:2408.1099
Ilan Price, Alvaro Sanchez-Gonzalez. Ferran Alet, Tom R Andersson, Andrew El-Kadi, Dominic Masters, Timo Ewalds, Jacklynn Stott, Shakir Mohamed. Peter Battaglia, et al. Probabilistic weather forecasting with machine learning. Nature, 637(8044):84-90, 2025.
12

---

Published as a conference paper at ICLR 2025
Stephan Rasp, Peter D. Dueben, Sebastian Scher, Jonathan A. Weyn, Soukayna Mouatadid, and Nils Thuerey. WeatherBench: A benchmark dataset for data-driven weather forecasting. Journal of Advances in Modeling Earth Systems, 12(11), 2020.
Salva Rühling Cachay, Brian Henn, Oliver Watt-Meyer, Christopher S Bretherton, and Rose Yu. Probablistic emulation of a global climate model with spherical dyffusion. Advances in Neural Information Processing Systems, 37:127610-127644 2025.
Salva Rühling Cachay, Bo Zhao, Hailey Joren, and Rose Yu. DYffusion: A dymamics-informee diffusion model for spatiotemporal forecasting, Advances in Neural Information Processing Sys- tems, 36, 2023.
Hira Saleem, Flora Salim, and Cormac Purcell. Ste-vit: Spatio temporal continuous vision trans- former for weather forecasting, 2024. arXiv:2402.1796.
Sebastian Scher and Gabriele Messori. Ensemble Methods for Neural Network-Basse Weather Forecasts. Journal of Advances in Modeling Earth Systems, 13(2), 2021.
George F. Simmons. Differential Equations with Applications and Historical Notes. Chapman and Hall/CRC, 3rd edition edition, 2016. ISBN 978-1 1-9987-025--
Yang Song, Sahaj Garg, Jiaxin Shi, and Stefano Ermon. Sliced score matching: scalable approach to density and score estimation. In Uncertainty in artificial intelligence, pp. 574-584 PMLR, 2020.
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Interna- tional Conference on Learning Representations, 2021.
Yogesh Verma, Markus Heinonen, and Vikas Garg. ClimODE: Climate and weather forecasting with physics-informed neural odes. International Conference on Learning Representations, 2024.
Pascal Vincent. A connection between matching and denoising autoencoders. Neural compu- tation,
2011. Publisher: MIT Press.
Aladin Virmaux and Kevin Scaman, Lipschitz regularity of deep neural networks: analysis and efficient estimation. Advances Neural Information Processing Systems, 31, 2018
Jonathan A. Weyn, Dale R. Durran, Rich Caruana, and Nathaniel Cresswell-Clay. Sub-seasonal fore- casting with a large ensemble of deep-learning weather prediction models. Journal of Advances in Modeling Earth Systems, 13(7). 2021.
Gefan Yang and Stefan Sommer. A denoising diffusion model for fluid field prediction, 2023. arXiv:2301.11661.
Yiyuan Yang, Ming Jin, Haomin Wen, Chaoli Zhang, Yuxuan Liang, Lintao Ma, Yi Wang, Chenghao Liu, Bin Yang, Zenglin Xu, Jiang Bian, Shirui Pan, and Qingsong Wen. A survey on diffusion models for time series and spatio-temporal data, 2024. arXiv:2404. 18886.
Xiaohui Zhong, Lei Chen, Hao Li, Jun Liu, Xu Fan, Jie Feng, Kan Dai, Jing-Jia Luo, Jie Wu, and Bo Lu. FuXi-ENS: A machine learning model for medium-range ensemble weather forecasting, 2024. arXiv:2405.05925,
13

---

Published as a conference paper at ICLR 2025
A METRICS
We consider the following evaluation metrics used to assess the probabilistic forecasts produced by the diffusion model. Metrics from meteorology and general uncertainty quantification, such as RMSE, Spread/Skill ratio (SSR), and Continuous Ranked Probability Score (CRPS) are employed measure the effectiveness and reliability of the model outputs. All of our metrics are weighted by latitude-dependent weights. For a particular variable and lead time,
z i.a
represents the value of the k-th ensemble member initialization time indexed by 1...N N for grid cell in the latitude and longitude grid indexed by i I.
a denotes the corresponding ground truth.
S denotes the ensemble mean, defined by žt.
Σ
Pha k= "i,""
Tlane
a, denotes the area of the latitude-longituee grid cell, which varies by latitude and is nor- malized to unit mean over the grid (Rasp et al., 2020).
RMSE skill measures the accuracy of the forecast. Following Rasp et al. (2020) define the RMSE as the mean square root of the ensemble mean:
N
RMSE ==
Σ iET
a, - z2,n)².
the case deterministic predictions, the ensemble mean taken as the deterministic prediction. Spread represents the variability within the ensemble and is calculated as the root mean square of the ensemble variance:
N
Spread
Σ iE!
aa in - Jz,n)
(4)
T
Ylens
Ideally, the forecast achieves balance where skill and spread are proportional, leading to an optimal spread/skill ratio (SSR) close to indicating effective uncertainty estimation:
nens + Spread
SSR
(5)
nenss
RMSE
Continuous Ranked Probability Score (CRPS) (Gneiting Raftery 2007) measures the accuracy of probabilistic forecasts by comparing the cumulative distribution functions (CDFs) of the predicted and observed values. It integrates the squared differences between these CDFs, providing a single score that penalizes differences in location, spread, and shape of the distributions. An estimator of the CRPS is given by:
Them
Thaes
Thens
CRPS
ai
1 1.9
Vi.n
7lens
2nd
IEI
k= ==1
Temporal Difference measures the mean absolute difference between states consecutive times. It's 's used to measure the continuity of forecast. For forecasts z. ak at consecutive lead times, given by:
i,n
AX
aslar
(6)
N
Aenss
1=1
1=1
SET
B MODEL.
Score-based generative models uses a parameterized score So to sample from the target distribution. In practice, it tumss out to be easier to learn denoising network D9 using the denoising score matching objective Ho et al. (2020). By a result shown in Vincent (20IT), the score can then be retrieved using Sn = (D) 2)/0²
14

---

Published as a conference paper at ICLR 2025
Table 2: Scaling functions.
Skip scaling
Caip (o)
data/ (o² o ema)
Output scaling
Cout
a data /002 Jaan
Input scaling
Gin
1/v 0² + 0 data
Noise scaling
Cnoise (0)
In(()
The sampling process is based on the denoising neural network Do that takes a noisy residual and tries to denoise it. To help in this, it is also given the noise level the previous state (S2) and the lead time t. To simplify learning, Dg is parameterized by a different network F6 defined by
Preconditioning
Do(z, ((0), co) Cour (cin(a) Z, Croise (o): ((?),, (I)),),
where Cskip- Cout, Gin and Cnoise are scaling functions taken from (Karras et al. 2022) defined in Tab. 2 These scaling functions Cskip- Cout: Gn and Cuoise. specifically chosen to handle the influence of the noise level within the network, allowing Dg to adapt dynamically to different noise intensities without the need for adjusting the scale of a externally. Consequently, for consistency with the normalization of the data where daat is set to 1, the lead time is also scaled to within the range [0, 1]. This normalization ensures that the network inputs uniformly scaled, enhancing the efficiency and effectiveness of the denoising process.
Conditioning To condition on the initial conditions X ( and static fields, these are concatenated along the channel dimension with the input to the denoiser, increasing the dimension of the input. To condition on the noise level a and lead time I, we use Fourier embedding as specified in (Karras et al.] 2022). Fourier embedding captures periodic patterns in noise and time, enhancing the model's ability to handle complex time-series dependencies effectively. They work by transforming the time/noise into vector of sinc/cosine features at 32 frequencies with period 16. These vectors added and then passed through two fully connected layers with SiLU activation to obtain 128- dimensional encoding.
Architecture The backhone of the diffusion model is U-Net architecture, Our model is based on the one used in (Karras et al. 2022), reconfigured for our purposes with 32 filters the base multiplier. It is built up by blocks configured as in fig. 6 The blocks consist of two convolutional layers and are constructed in fig. 7 If the block is a down-/up-sample filters is different from the number of output filters, there is an additional skip layer from the input to the output. The blocks 16x32 resolution additionally has attention with a single head. The time/noise embedding is fed directly into each block and not passed through the network. Unlike the network in Karras et al. (2022), our convolutions uses zero padding at the poles and periodic padding at the left/right edges. This periodic padding ensures periodicity over longitudes. The model has 3.5M parameters.
if the number of input
Sampling To generate forecasts using our diffusion model, solve the probability flow ODE as defined in Karras et al. 2022)
d - G, V log (2) ds.
We employ the second-order Heun's method, deterministic ODE solver, as outlined in Algorithm For the noise parameters, we define the noise level function as as = s. Additionally, we set noise level schedule lower the noise during sampling from max Jnin over N steps:
1
)),
8;
Tmax
+
_ min
Qmax
i (0, N
N
The relevant parameters for training and sampling given in tab.
The dataset is partitioned into three subsets: training, validation, and testing. The train- ing subset is used for model training, the validation subset for evaluating generalization, and the testing subset to determine final accuracy. The diffusion model is trained using the following train- ing objective
Training
15

---

Published as a conference paper at ICLR 2025
Block Connectons
52:64
Down Connecions
Up Connections
Residual Connecionn
16-22
18x32
0x18
8x18
Figure 6: Overview of the U-Net Architecture, detailing layer configurations and the flow of infor- mation through convolutional blocks and skip connections.
Norm SiLU Conv 3x3 (Down/Up)
Linear layer
input
output
0 Add connection
embedding
Norm SiLU Conv Dropout
Conv 1x1 (Down/Up)
Figure 7: Construction of a Diffusion Model Block, showing the sequence of operations and the integration of embeddings with add connections.
Algorithm Deterministic sampling using Heun's 2nd order method.
1: procedure HEUNS X(Q), (), $tE(o....N), Z)
2: Za f- d²( Z
Generate initial sample sp
3: for = to N - 1 do
b> Solve ODE N timesteps
4: d; d. (z: Do(25) X(I),()
Evaluate da / da es
oxc
5:
2:+1 t 2; + (8i=1 - d;
D Take Euler step from lo
6: if 8:41 then
Apply 2nd order correction unless goos
7:
d' + a+s1
dost1 (zi+1 - Du(zi4150a, ,, ())))
Evaluate da ds at #
8:
Zi+1 f- + [ - 8;) (d: d
D. Explicit trapezoidal rule at
return 22
b Return noise free sample
Table 3: Parameters used for sampling and training.
Name
Notation
Value, sampling
Value, training
Maximum noise level
max Tmin
80
88
Minimum noise level
0.03
0.02
Shape of noise distribution
P
Number of noise levels
N
20
16

---

Published as a conference paper at ICLR 2025
Table Optimizer Hyperparameters.
Optimizer hyperparameters
Optimiser
Adam (Loshchilov Hutter 7017a
Initialization
Xavier Uniform (Glorot & Bengio 20TO
LR decay schedule
Cosine (Loshchilov Hutter 20
Peak LR
5e-4
Weight decay
0.1
Warmup steps
le3
Epochs
300
Batch size
256
Dropout probability
0.1
Table 5: Training schedule for Graph-EFM, using the notation from Oskarsson et al. 2024).
Epochs
Learning Rate
Unrolling steps
AKL
ACRPS
20
10-3
1
0
75
10-3
1
0.1
0
20
4
4
0.1
8
10-4
8
0.1
10
1 ΣΣ @2
1 (1 [T|J] (8()s- X(0css
Ex Boon. P)) (1
with X(1)= = Do(X(() + o,; ((D)) and J being the set of variables. Here, p² represents uniform distribution over the lead times. We have also included scaling term st -1 which scales the loss by the precomputed standard deviation sy(t) based on lead time for each variable J. This normalization process is designed to weigh short and longer times equally. The noise level distribution Pa is chosen be consistent with the sampling noise level described above. Specifically, its inverse CDF is:
F F-)(w)= = Tmax + ( mmin Tmax
))
and we sample from it by drawing U10, 1]. The training process is executed in Pytorch, with setup and parameters detailed in Tab.
Graph-EFM Baseline For the Graph-EFM baseline we use the same data setup as for the other models. Since are working on a coarser resolution than Oskarsson et (2024) some adaptation was necessary to the exact graph structure used in the model. We construct the graph by splitting global icosahedron times. resulting schedule as in Oskarsson al. (2024), with pre-training single step 6h prediction and fine-tuning on rollouts. Details the training schedule are given in tab.
hierarchical graph levels. The training follows the same
DYffusion Baseline We have adapted the DYffusion model to work with weather forecasting us- ing ERA5 data. This application was not considered in the original DYffusion papers which focused on sea-surface forecasting Rühling Cachay et al. (2023) and climate modeling Rühling Cachay et al. 2025). To ensure fair evaluation, we used the same backbone U-Net and a lat-lon weighted rrean-squaredeeroos loss. We have trained it with a horizon of 24h, with Ih timesteps, making it directly comparable to ARCI-24/Ih. The dropout rate set to 0.2 and used no artificial diffusion steps. For the training hyperparameters we used values similar to those proposed in the original paper.
diffusion models,
17

---

Published as a conference paper at ICLR 2025
z500
z500
ws10
ws10
150
1.0
RMSE:
100
S5R
RMSE 2
SSR
0.5
50
6
12
18
12 18
1
3
5
9
5 7 9
Lead Time (hours)
Lead Time (hours)
Lead Time (days)
Lead Time (days)
AR-6h
Linear-1h
ARCI-24/1h
Linear-6h
AR-24h
Figure 8: RMSE and SSR for linear interpolations of geopotential at 500 hPa (2500) and ground wind speed ()s Linear- Ih/Lincar-6h are linear interpolations of AR-6h/AR-244
C ADDITIONAL RESULTS
Interpolation An alternative to directly producing forecasts at fine temporal resolution would be to simply interpolate the forecasts sampled using an autoregressive model. We argue that our ARCI method produces more accurate and realistic predictions than simple linear interpolation. Figure shows the RMSE and Spread/Skill for linear interpolation of AR-24h, compared to ARCI- 24/6h. The interpolated forecasts behave much worse than the ARCI model on the same resolution, dismissing this argument.
Table 6 and show the results 5 and 10 day forecasting for all variables. The error bars given for ARCI-24/6h is standard deviation calculated by retraining the model five times and evaluating each model 10 ensemble members. This shows that the model is robust to network initialization. Figure / show the results on forecasting at 6h resolution for all variables. Figure [0 shows the same results forecasting with Ih resolution. Figure I shows the temporal difference for different
values of p.
D EXAMPLE FORECASTS
Figures 12
17
show example forecasts for the remaining variables.
18

---

Published as a conference paper at ICLR 2025
Table 6: Results day forecasting for all variables. For RMSE and CRPS, lower values are better, and SSR should be close to
Lead time days
Variable
Model
RMSE
CRPS
SSR
2500
Deterministic
766.7
483.9
Graph-EFM
699.1
317.5
1.13
AR-6h
602.3
287.8
0.75
AR-24h
544.2
242.7
0.84
CI-6h
707.8
321.2
0.59
ARCI-24/6h
560.9 + 15.6
256.7 + 20.2
0.86 + 0.015
t850
Deterministic
3.48
2.36
Graph-EFM
3.12
1.56
1.11
AR-6h
2.72
1.34
0.82
AR-24h
2.55
1.24
0.89
CI-6h
3,06
1.5
0.74
ARCI-24/6h
2.6
0.031
1.27 + 0.021
0.9 = 0.012
t2m
Deterministic
2.71
1.72
Graph-EFM
2.51
1.1
1.09
AR-6h
2.13
0.96
0.83
AR-24h
1.98
0.87
0.9
CI-6h
2.29
1.0
0.79
ARCI-24/6h
2.02 + 0.036
0.9 + 0.035
0.9 = 0.012
u10
Deterministic
4.37
2.93
Graph-EFM
3.81
1.93
0.97
AR-6h
3.47
1.71
0.86
AR-24h
3.32
1.62
0.92
CI-6h
3.87
1.91
0.77
ARCI-24/6h
3.35 + 0.032
1.64 + 0.019
0.93 + 0.007
v10
Deterministic
4.48
3.0
Graph-EFM
3.88
1.96
0.94
AR-6h
3.55
1.76
0.86
AR-24h
3,42
1.68
0.93
CI-6h
4.0
1.99
0.77
ARCI-24/6h
3,45 + 0.026
1.69 + 0.014
0.94 + 0.006
ws 1 10
Deterministic
3.09
2.2
Graph-EFM
2.56
1.35
1.01
AR-6h
2.38
1.23
0.9
AR-24h
2.3
1.18
0.96
CI-6h
2.54
1.32
0.88
ARCI-24/6h
2.31 0.016
1.19 + 0.01
0.96 - 0.006
19

---

Published as a conference paper at ICLR 2025
Table 7: Results on 10 day forecasting for all variables. For RMSE and CRPS, lower values are better, and SSR should be close to
Lead time 10 days
Variable
Model
RMSE
CRPS
SSR
2500
Deterministic
1042
661.5
Graph-EFM
817.1
373.6
1.1
AR-6h
811.8
391.9
0.88
AR-24h
750.6
335.2
0.94
CI-6h
885.7
406.6
0.6
ARCI-24/6h
765.6 + 21.4
355.2 + 33.0
0.93 + 0.018
t850
Deterministic
4.54
3.17
Graph-EFM
3.51
1.77
1.12
AR-6h
3.39
1.69
0.92
AR-24h
3.25
1.6
0.96
CI-6h
3,68
1.85
0.71
ARCI-24/6h
3.29 4: 0.05
1.63 + 0.041
0.95 + 0.007
t2m
Deterministic
3.56
2.28
Graph-EFM
2.88
1.32
1.14
AR-6h
2.62
1.18
0.89
AR-24h
2.48
1.09
0.95
CI-6h
2.75
1.21
0.75
ARCI-24/6h
2.51 + 0.068
1.11 + 0.076
0.94 + 0.017
u10
Deterministic
5.14
3.53
Graph-EFM
4.08
2.07
0.97
AR-6h
3.95
1.97
0.94
AR-24h
3.85
1.9
0.97
CI-6h
4.27
2.14
0.78
ARCI-24/6h
3.87 + 0.03
1.92 + 0.018
0.97 + 0.01
v10
Deterministic
5.23
3.58
Graph-EFM
4.11
2.08
0.93
AR-6h
4.02
2.0
0.96
AR-24h
3,96
1.96
0.99
CI-6h
4.39
2.2
0.8
ARCI-24/6h
3.97 + 0.018
1.97 + 0.011
0.99 + 0.01
ws 1 10
Deterministic
3.44
2.49
Graph-EFM
2.65
1.4
1.01
AR-6h
2.57
1.34
0.96
AR-24h
2.53
1.31
0.99
CI-6h
2.68
1.4
0.89
ARCI-24/6h
2.54 0.011
1.32 + 0.008
0.99 + 0.008
20

---

Published as a conference paper at ICLR 2025
t850
u10
v10
wsl0
RMSE
2
250
9 Lead Time (days)
1
7
1 3
9
1
3
5
7
9
3
5
9
5
7
7
9
Lead Time (days)
Lead Time (days)
Lead Time (days)
Lead Time (days)
Lead Time (days)
Graph-EFM
AR-6h
AR-24h
Cl-6h
ARCI-24/6h
z500
t850
t2m
u10
v10
ws10
1.00
1.0
1.0
1.0
0.75
0.8
0.8
Lead Time (days)
Lead Time (days)
Lead Time (days)
Lead Time (days)
Lead Time (days)
Lead Time (days)
Graph-EFM
AR-6h
AR-24h
Cl-6h
ARCI-24/6h
z500
t850
t2m
u10
v10
ws10
400
2
1.5
1.0
200
1.0
0.5
0.5
0.5
7
1
3 9
3
3 5
3
(days)
(days)
(days)
(days)
(days)
(days)
Graph-EFM
AR-6h
AR-24h
CI-6h
ARCI-24/6h
Figure 9: RMSE, SSR, and CRPS for selection of models at 6h resolution.
21

---

Published as a conference paper at ICLR 2025
1000
RMSE
2
2
2
1 3
1
3
5
7
1
3
5 7 9 (days)
1 5 7
5 9 Lead (days)
1
1
5 7
9
(days)
(days)
(days)
Lead (days)
ARCI-24/1h
ARCI-24/2h'
AR-1h
AR-24h
DYffusion
2500
t850
t2m
u10
v10
ws10
1.00
1.0
1.0
0.5
0.5
0.5
Lead Time (days)
Lead Time (days)
Lead Time (days)
Lead Time (days)
Lead Time (days)
Lead Time (days)
ARCI-24/1h
ARCI-24/2h*
AR-1h
AR-24h
DYffusion
2500
t850
t2m
u10
v10
ws10
GRPS
500
2
2
2
1.0
3 7
1 3357
979
3 5 7 9
3 9
1
3
5 7
3 5
Lead Time (days)
Lead Time (days)
Lead Time (days)
Lead Time (days)
Lead Time (days)
Lead Time (days)
ARCI-24/1h
ARCI-2A/2h*
AR-1h
AR-24h
DYffusion
Figure 10: RMSE, SSR and CRPS for selection of models at Ih resolution.
2500
t850
t2m
u10
v10
ws10
1.0
0.5
50
0-
12
24
12
24
Lead (hours)
Lead (hours)
(hours)
Lead Time
Lead (hours)
Time (hours)
p
2500
1850
t2m
u10
v10
ws10
0.25
0.4
0.3
0,20
0.30
0.3
0.3
0.15
0.25
0.2
0.2
12
24
12
24
12
24
12
24
12
24
0
12
24
Lead (hours)
Lead Time (hours)
Time (hours)
Lead (hours)
Lead Time (hours)
Time (hours)
p- 0
ln10
Figure 11: Temporal difference for different values of in algorithm 2 Choosing 0 fixes the noise, p = ln(10) allows it to vary slightly and p -> co gives completely uncorrelated noise. The black line refers to the temporal difference of the data.
22

---

Published as a conference paper at ICLR 2025
Truth
Deterministic
Ensemble mean
Ensemble Std
Std Dev.
1000
Sample #1
Sample #2
Sample #3
Sample #4
55000
m²s "
50000
Figure 12: Example forecasts for geopotential at 500 hPa (z500) at lead time 10 days. The forecasts are generated using ARCI-24/6h except for Deterministic which is sampled using the deterministic model. The bottom row shows 4 ensemble members, randomly chosen out of the 50.
Truth
Deterministic
Ensemble mean
Ensemble Std
Std
Sample #1
Sample #2
Sample #3
Sample #4
25
0
-25
Figure 13: Example forecasts for temperature at 850hPa (t5) at lead time days. The forecasts generated using ARCI-24/6h except for Deterministic which is sampled using the deterministic model. The bottom row shows ensemble members, randomly chosen out the 50.
Truth
Deterministic
Ensemble mean
Ensemble Std
10
Std Dev.
Sample #1
Sample #2
Sample #3
Sample #4
-50
Figure 14: Example forecasts for ground temperature (t2mm at lead time 10 days. The forecasts are generated using ARCI-24/6h except for Deterministic which sampled using the deterministic model. The bottom row shows 4 ensemble members, randomly chosen out of the 50.
23

---

Published as a conference paper at ICLR 2025
Truth
Deterministic
Ensemble mean
Ensemble Std
Std Dev.
5
0
Sample #1
Sample #2
Sample #3
Sample
0
Figure 15: Example forecasts for u-component of wind at 10m (u10) at lead time 10 days. The
forecasts are generated using ARCI-24/6h except for Deterministic which is sampled using the de- terministic model. The bottom row shows 4 ensemble members, randomly chosen out of the 50.
Truth
Deterministic
Ensemble mean
Ensemble Std
Std Dev.
5
Sample #1
Sample #2
Sample #3
Sample #4
20
ms
20
Figure 16: Example forecasts for v-component of wind at 10m (v10) at lead time 10 days. The forecasts are generated using ARCI-24/6h except for Deterministic which is sampled using the de- terministic model. The bottom shows 4 ensemble members, randomly chosen out of the 50.
Truth
Deterministic
Ensemble mean
Ensemble Std
Std Dev.
2.5
Sample
Sample
Sample
Sample #4
E
Figure 17: Example forecasts for wind speed at 10m (wa10) at lead time 10 days. The forecasts are generated using ARCI-24/6h except for Deterministic which sampled using the deterministic model. The bottom row shows 4 ensemble members, randomly chosen out of the 50.
24

---

Published as a conference paper at ICLR 2025
E CONTINUITY OF SOLUTIONS TO PROBABILITY FLOW ODE
The proposed method in section 4 generates continuous ensemble trajectories by introducing tem- poral correlations in the noise. Sampling is performed by solving the probability flow ODE:
dz(s) - -So(z(s) [ M:.), t)ds,
2 2(1) 20:
(8)
starting in pure noise ~ Proise and ending in a forecast z(0) ~ p(X [K- In practice, the integration is only done up to some time < to overcome instability caused by the singularity of the score function, V log Ps, at O.
To ensure that the trajectories are continuous, we must show that solutions to eq. (8) are continuous functions of the lead time 1. Here, we consider the case where the initial noise 2g remains fixed across different I. The more general scenario, where 2g follows a stochastic noise process, left for future work.
Let =,) == -sSa(2, 5; X defined for (s,z) := [e, 1] x R" and E []. The dependence the previous states X has been suppressed in the notation since all forecasts are conditioned on fixed X k: To phrase the result, we first make an assumption on the regularity of f.
Assumption 1. Let f(s,2, t) be continuous in (s, 2, t) and locally Lipschitz in 2, i.e. for any closed bounded set U D there a Ky such that
[f 21,t) f(s, 22, t)1 S Kula1 z2l
for any (s, 21), (s, 22) U and t [0,
If f(, 2) is continuous for (s, 2) E D, then by the fundamental existence theorem for ODEs (Theorem 1.1 in Hale (2009),, there exists least one solution eq. (8) passing through any given point (No, =0) E D). Suppose further this solution is unique and denote it as 2(5, 80, 2n,4). t). For any (8o; 20) E and E [0, T), let (a(sa, 20, 1), b(s,, 20. be the maximal interval of existence of 2(8, 80, 20. t). We define the set
E ( 50, 20, 1) a(50, =o, 1) < < b(5o, zo, (), (80. #0) E D, E [0,
as the domain of definition of 2(s, 8g, t). Since our primary interest lies in the continuity of solutions, assume access some domain of definition E without explicitly constructing it. For a proof of global existence under stronger assumptions, see theorem B in Section 70, Chapter 13 of Siimmons 2016). With this, we can state the main result.
Theorem 1 (Theorem 3.2 in Hale (2009)). Assume that satisfies assumption 1 and consider the initial value problem
(9) Then, for every (so, 20) E D and [0. there unique solution 2(s, su, 20, t) with 2(Sp, So 2n, t) = 2g, and this solution is continuous in (s, Sp, 2o, t) within its domain of definition.
d=(s) f(s, 2,t)ds 2(50)= = 20
Theorem shows that the solutions to the probability flow ODE eq. (8) are continuous as functions of lead-time if -Spp(2,, 8; neural network Sg is composition of continuous functions, it necessarily continuous function. The remaining assumption Sp being locally Lipschitz in 2 is not particularly strong one, and commonly assumed when proving results about neural networks (Karras et al., 2022] Song et al. 2020 Albergo & Handen-Eijndes 2023). To calculate the Lipschitz constant could for example use the method proposed Virmaux & Scaman (018)).
/) is continuous in (z, 8, () and locally Lipschitz in Since the
25


Reviews:
[{"review_id": "9mjzHbeq4R", "reviewer": "Reviewer_koDm", "summary": "This paper uses the diffusion model for continuous ensemble weather forecasting without requiring autoregressive steps to produce future values. As introduced in Section 4.2, the main idea is to replace the fixed Z with a stochastic process Z(T) with continuous sample trajectories.", "questions": "In the weakness section.", "limitations": "", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The proposed method is presented clearly.", "weaknesses": "W1: there have been exsiting works for nonautoregressive sequence forecasting by diffusion models. However, realted investigation and discussions are missing, e.g., \n\n[1] NeurIPS'21 CSDI Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation\n\n[2] ICML'23 Non-autoregressive Conditional Diffusion Models for Time Series Prediction \n\nW2: moreover, some questions and details are not provided.\n- it would be better to explain what ensemble forecasting is formally\n- it is still not very convincing why a stochastic process is better than random Gaussian noises for nonautoregressive generation. Because we can simply represent the forecast outputs as a vector of a length N. Then we can use a Gaussian vector to yield forecasts. \n- is there a significant difference between weather forecasting and general time series forecasting?\n- what is the input size (the number of variables) of the model in the experiments? It seems there are many variables in the weather forecasting task. Did you perform univariate forecasting tasks for each variable?   \n- how to fuse $n_{ens}$ trajectories to get the final predictions?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GrMlyc0mni", "reviewer": "Reviewer_c3dx", "summary": "The \"Continuous Ensemble Weather Forecasting with Diffusion Models\" paper introduces an innovative framework for probabilistic weather forecasting using diffusion models. It focuses on enhancing computational efficiency by removing the autoregressive nature of sampling from diffusion models.", "questions": "Could you clarify the rationale behind using fixed noise across time steps? Additionally, has any sensitivity analysis or ablation study been conducted to validate this design choice? If so, please provide these findings.\n\nCould you further explain how interpolation using an ODE solver contributes to the model’s performance, and how it compares to alternative interpolation methods?\n\nOnly two of the five forecasting variables are included in Table 2. Could you explain the rationale for this selective presentation and consider including results for all five variables to enable a comprehensive evaluation?\n\nCould you provide formal proofs or derivations for key claims, particularly those regarding trajectory continuity and temporal stability?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The model developed in this paper presents an approach that eliminates traditional autoregressive steps during sampling from diffusion models, allowing it to achieve high temporal resolution and efficiency in forecasting. This is an exciting approach for improving computational efficiency, a feature that has been challenging to achieve in weather prediction. \n\nThe paper's application is exciting, as diffusion models are typically used in areas like image generation rather than meteorological forecasting.", "weaknesses": "The paper lacks references to closely related methods, such as Zheng et al. (2023) on fast sampling of diffusion models via operator learning, which shares similarities with the proposed model’s sampling approach. It would be beneficial to clarify how the proposed sampling method diverges from these approaches.\n\nAlthough not directly used for weather data, recent models like Dyfusion, Alternator, and Mamba (with s-mamba used specifically for weather forecasting) have shown promise in similar domains, such as sea surface temperature forecasting. Including these models as baselines or discussing their relevance could offer a more comprehensive comparison, illustrating where this model stands relative to other contemporary approaches.\n\nThe paper mentions “regularity conditions” (e.g., Line 234) without specifying them rigorously. These conditions are fundamental to claims about trajectory continuity and temporal stability of forecasts. A formal mathematical description of these conditions, alongside proof or empirical validation, would be necessary to substantiate these claims.\n\nThe claims regarding fixed noise across time steps and ODE-based interpolation are under-supported. No mathematical evidence or ablation study is presented to show the impact of fixing noise on forecasting performance. Including such a study could clarify the advantages of this design choice.\n\nThe proposed Algorithm 2 seems tailored to a specific diffusion model setup. It would be valuable to discuss whether this algorithm generalizes to other diffusion models or is fundamentally limited to the framework outlined in the paper. A broader discussion on this topic could enhance the paper’s applicability to related models.\n\nThe results lack completeness, as only two of the five variables used for forecasting are presented in Table 2. Including comparisons for all five variables would provide a clearer picture of the model's strengths and weaknesses across different atmospheric variables.\n\nThe absence of error bars due to computational limitations weakens the robustness of the reported results. Given the probabilistic nature of the model, showcasing variability over multiple runs would offer greater confidence in the findings. Including error bars or confidence intervals for at least a subset of key results would be helpful, even if computational constraints prevent a complete assessment.\n\nNumerous claims about model performance, such as trajectory continuity, lack rigorous mathematical proof and appear to rely on empirical observations. Addressing this would lend greater theoretical robustness to the proposed model.\n\nThere is no mathematical derivation for the training objective and loss function. A step-by-step derivation of these, with explanations of any modifications relative to standard diffusion model objectives and the rationale for these modifications in the weather forecasting context, would clarify the theoretical foundation of the approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kdI8EbDAxY", "reviewer": "Reviewer_1nBA", "summary": "This work proposes an ODE-based method to address the limitations of previous a single forecasting step and rolled out autoregressively methods for weather prediction. It can generate long-range weather predictions without iteration and forecast arbitrary lead times.", "questions": "**Major:**\n- It is a good idea to use the probability flow ODE from the work proposed from https://arxiv.org/pdf/2206.00364 in diffusion models. But what is the assumption behind the customized ODE in Eq. (2)? Why is there a linear noise level $sigma$ in Eq. (2)? What if $\\sigma^2$? More explanation would help. \n- From the paper title and section 4 name, \"CONTINUOUS\" seems to be used to describe $ensemble forecasting$. However, what the authors really want to convey is the temporal continuity, i.e., any arbitrary fine temporal resolution. You may need to think over the rephrasing. Otherwise, it is confusing for readers (at least for me).\n- In Figure 1, do you need to include the 1-hour prediction as the input for the 6-hour prediction? I saw you have explanations in sections 3 and 4. But Figure 1 deserves a more precise description within its title since it is the main framework of your method.\n- It would be better to provide more comparisons with previous SOTA ML models. Even though I know not all models did not release source codes to the public, but some did, e.g., FourCastNet and ClimaX. \n- The authors mentioned the limitation of diffusion models: computationally expensive. I suggest adding more analysis of computational efficiency.\n\n**Minor:**\n- You are using the letter 'k' for different representations (see Eq. 1 and Alg. 1).\n- It is customary to denote a time point with the superscript or subscript. It looks like a function if using $X[k+1]$ or $X(\\tau)$.\n- In the line under Eq. (2), why is $z(1)$ for $X[k+1]$? Isn't it the number of diffusion steps or the noise levels?\n- What is $v_i^{k}$ in Algorithm 2?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- It is interesting to use probability flow ODE of the noise in diffusion models. Leveraging a lead-time-dependent deterministic ODE-solver to generate a smooth and continuous trajectory is meaningful compared to autoregressive methods.\n- Authors provide intensive experiments on their method.", "weaknesses": "- Need more descriptions on how probability flow ODE works.\n- Need more comparisons with other SOTAs. \n- No available codes to help readers better understand the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CXhGRWPr68", "reviewer": "Reviewer_PYFz", "summary": "This paper introduces Continuous Ensemble Forecasting for global weather forecasting.", "questions": "1.Were the limitations of previous methods that they were computationally expensive? How does the method proposed in the paper address the limitations of previous approaches?\n\n2.What specific contributions does the paper make that differentiate it from other weather forecasting models?", "limitations": "", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Weather forecasting is an important problem.\n\n2.The method introduced in the paper seems to exhibit a certain level of novelty.\n\n3.Some of the figures in the paper look good.", "weaknesses": "1.The paper's presentation seems somewhat disorganized. The introduction does not clearly articulate the limitations of current weather forecasting methods or how the proposed method addresses these limitations. Figure 1, being the first figure, lacks sufficient information; ideally, a single figure should clearly highlight the differences between the proposed method and previous methods, as well as explain how the proposed method overcomes the limitations of prior approaches. In the Methods section, there is a tendency to mix descriptions of existing methods with the new method, which can obscure the exact nature of the contribution made by the proposed method. Figure 2 should also provide a clearer depiction of the methodology.\n\n2.Based on the unclear description provided in the paper, the contributions appear insufficient, and the contributions listed in the introduction seem to be something that any weather forecasting model could achieve.\n\n3.The baseline models compared in the paper are somewhat limited and do not appear to be the latest SOTA models. Moreover, according to Table 1, the ARCI-24/6h model proposed in the paper (if I understand correctly) is not the optimal model under most settings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)