Elucidated Rolling Diffusion Models for Probabilistic Forecasting of Complex Dynamics
Authors: Salva Rühling Cachay
Miika Aittala
Karsten Kreis
Noah D Brenowitz
Arash Vahdat
Morteza Mardani
Rose Yu
Venue: NeurIPS 2025 poster ()
Keywords: diffusion models
AI for science
probabilistic forecasting
weather forecasting
Primary area: machine_learning_for_sciences
TL;DR: A sequence-level EDM-derived diffusion model using progressively increasing noise levels is designed for probabilistic forecasting of complex dynamics like weather.

Abstract: Diffusion models are a powerful tool for probabilistic forecasting, yet most applications in high-dimensional complex systems predict future states individually. This approach struggles to model complex temporal dependencies and fails to explicitly account for the progressive growth of uncertainty inherent to the systems. While rolling diffusion frameworks, which apply increasing noise to forecasts at longer lead times, have been proposed to address this, their integration with state-of-the-art, high-fidelity diffusion techniques remains a significant challenge. We tackle this problem by introducing Elucidated Rolling Diffusion Models (ERDM), the first framework to successfully unify a rolling forecast structure with the principled, performant design of Elucidated Diffusion Models (EDM). 
To do this, we adapt the core EDM components--its noise schedule, network preconditioning, and Heun sampler--to the rolling forecast setting. The success of this integration is driven by three key contributions: $(i)$ a novel loss weighting scheme that focuses model capacity on the mid-range forecast horizons where determinism gives way to stochasticity; $(ii)$ an efficient initialization strategy using a pre-trained EDM for the initial window; and $(iii)$ a bespoke hybrid sequence architecture for robust spatiotemporal feature extraction under progressive denoising. 
On 2D Navier–Stokes simulations and ERA5 global weather forecasting at $1.5^\circ$ resolution, ERDM consistently outperforms key diffusion-based baselines, including conditional autoregressive EDM.
ERDM offers a flexible and powerful general framework for tackling diffusion-based dynamics forecasting problems where modeling  uncertainty propagation is paramount.

Elucidated Rolling Diffusion Models for Probabilistic Forecasting of Complex Dynamics
Salva Rühling Cachay UC San Diego
Miika Aittala NVIDIA
Karsten Kreis NVIDIA
Noah Brenowitz NVIDIA
Arash Vahdat NVIDIA
Morteza Mardani
Rose Yu UC San Diego
NVIDIA
Abstract
Diffusion models are powerful tool for probabilistic forecasting, yet most ap- plications high-dimensional complex systems predict future states individually. This approach struggles to model complex temporal dependencies and fails to explicitly account for the progressive growth of uncertainty inherent the sys- tems. While rolling diffusion frameworks, which apply increasing noise to fore- casts longer lead times, have been proposed to address this, their integration with state-of-the
high-fidelity diffusion techniques remains significant chal- lenge. We tackle this problem by introducing Elucidated Rolling Diffusion Mod- els (ERDM), the first framework to successfully unify rolling forecast structure with the principled, performant design of Elucidated Diffusion Models (EDM). To do this, we adapt the core EDM components-its noise schedule, network pre- conditioning, and Heun sampler-to the rolling forecast setting. The success of this integration is driven by three key contributions: (+) novel loss weighting scheme that focuses model capacity on the mid-range forecast horizons where (ti) an efficient initialization strategy us-
determinism gives way to stochasticity; ing pre-trained EDM for the initial window; and (iti) a bespoke hybrid sequence architecture for robust spatiotemporal feature extraction under progressive denois- simulations and ERA5 global weather forecasting at
ing. On 2D Navier-Stokk 1.5 resolution, ERDM consistently outperforms key diffusion based baselines, including conditional autoregressive EDM. ERDM offers a flexible and powerful general framework for tackling diffusion-based dynamics forecasting problems where modeling uncertainty propagation is paramount."
Introduction
Probabilistic forecasting of complex dynamical systems is essential for realistically assessing future states ("snapshots") and their inherent uncertainties [2, 11]. For example, medium-range weather forecasting (& 15 days) grapples with the inherent chaotic nature of the atmosphere and high sen- sitívities to the initial conditions. Ensembles of numerical weather prediction (NWP) models are therefore standard practice for estimating forecast uncertainty and the likelihood of high-impact events [42, 3]. However, each ensemble run of a flagship system such IFS ENS [15] is computa- tionally expensive, motivating data-driver alternatives [7].
'Code available https://github..om/// wow//Vaabss//DDMMM
"Equal advising. Corresponding authors listed alphabetically.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

200
Start noise
102
End noise
Nez 2 denolmp Stys
10
Noise
10
10
Oree
Snapshot
New naise - A/O .i.
Figure I: ERDM sampling with window, highlighted in bold purple, of size W = 4. Top row: ERDM starts at diffusion time t 0 with snapshots, 21, 21,... Dw. corrupted by progressively larger noise levels, joint denoising steps, the sequence reaches lower
(t) < .....-- < dw (t) Oax- Middle row: After N noise levels at t such that (1) = miin and Fw (1) = 5 (0) for > 1, as illustrated in the right-hand panel. The now fully denoised first snapshot, æı, is returned. Bottom row: The rest of the sequence is shifted one slot to the right, and a fresh pure-noise snapshot is appended to the new window. The cycle then repeats.
Diffusion models have shown notable success in generative modeling, particularly for high- dimensional data such as images and videos [52, 13]. The EDM framework [26), for example, enables stable high-fidelity sample generation due to its principled design that generalizes and im- proves on the foundational denoising diffusion probabilistic model (DDPM) [21] and related diffu- sion paradigms [53, 54]. This principled approach makes them especially promising for modeling time-evolving dynamical systems. In this domain, the ability to generate sharp, diverse ensembles is crucial for robust uncertainty quantification, and producing high-fidelity, physically consistent sam- ples is a key ad advantage over deterministic models that often yield blurry, averaged-out predictions.
Autoregressive conditional diffusion models have been successfully applied to fluid dynamics [29] and weather [45]. These approaches often use limited temporal context and predict future snap- shots one by one. To explicitly model the progressively increasing uncertainty common sequence generation tasks, Rolling Sequence Diffusion Models (RSDM) [49, 57] employ snapshot-dependee noise schedules that escalate noise for distant future snapshots. However, existing RSDM's derived from DDPM, rather than leveraging improved practical design choices, such as network preconditioning and second-order sampling, that were key EDM's success.
Our work addresses these limitations by building upon the successful EDM framework [26]. We adapt and extend EDM's core design elements-irn noise schedule formulation, denoiser network parameterization, network preconditioning, loss weighting, and sampling algorithms- the spe- eific demands of RSDM-like sequence modeling. In developing ERDM, we identified and addressed key design choices for progressive noise schedules and temporal loss weighting within the EDM paradigm, aspects that received limited attention in prior work. Our work thus provides novel solu- tions for these crucial components. All in all, our key contributions are summarized as follows:
We develop Elucidated Rolling Diffusion Models (ERDM), integrating snapshot- dependent progressive noise levels into EDM, by adapting EDM's noise schedule, loss weighting, preconditioning, and sampling, complemented by a simple, but effective first- window initialization technique and a hybrid 3D denoiser architecture.
Extensive experiments on ERA5 weather data and Navier-Stokes flows show that ERDM consistently improves on the autoregressive, conditional EDM baseline in both predictive skill and calibration. On ERA5, ERDM rivals tiite-of-thh-are weather forecasters (IFS ENS and NeuralGCM ENS) for mid- to long-range lead times, while being computationally more efficient-requiring only H200 GPUs and days for training-and exhibiting high physical fidelity in its power spectra.

---

2 Related Works
Diffusion models for spatiotemporal data. Diffusion models, initially transformative in image and audio generation, are increasingly being adapted for spatiotemporal modeling [58, 20, 17, 50]. Latent-diffusion variants [56, 47] provide the computational advantages integral to state-of-the-an video generators [22, 5]. Recent work refines the diffusion process itself to strengthen temporal structure and boost efficiency [50, 32]. Rolling sequence diffusion models (RSDM) advance this idea by assigning progressively larger noise levels to later timesteps, mirroring the growth of pre- dictive uncertainty [57,4] 49, 27]. We embrace the same intuition but depart from these DDPM-based RSDM's in two crucial ways. (i) Our approach is derived from the EDM framework, which gen- eralises and improves upon DDPM. (ii) We re-examine often-overlooked design choices- noise schedule, loss weighting, and network architecture-ann and show that their coordination is critical for forecasting complex dynamics. Progressive schedules like ours form a specific instance of the completely randomized schedules proposed in Chen et al. [9], corresponding to its 'pyramid sam- pling' scheme. However, our ablations indicate that such training-time randomization degrades performance-possibyy because it misaligns with the temporally increasing uncertainty inherent to the chaotic systems that we study.
Data-driven medium-range weather forecasting. Initial machine learning (ML)-based attempts in this area were limited to deterministic forecasting [43, 4, 30, 38. 39, 6], often differentiated by their underlying neural network architectures [25]. More recently diffusion and flow match- ing models have shown promise for probabilistic weather forecasting [45, 12, 1], and other tasks such as downscaling [36, 34, 10, 55], data assimilation [23, 48, 35], and emulation [31, 51, 44]. Latent-variable models offer an alternative probabilistic route [40], and physics-ML hybrids such as NeuralGCM show promise but face scalability constraints [28]. Of particular relevance to our work is GenCast [45], which successfully applied the EDM framework, using the graph architecture from [30], for next-step probabilistic weather forecasting. Our primary baseline in this paper, EDM, corresponds to the same approach but using our experimental setup (e.g., same U-Net architecture and dataset resolution; see Appendix D.4 for more details). This design ensures a controlled and direct comparison between ERDM and an EDM-based next-step forecasting approach.
3 Background
In the following, we refer to clean data as y, noisy data as æ, and we use T to make clear when the data is corrupted according to progressively increasing noise levels. We abbreviate sequences by V1.W yI..... yw. Probabilistic forecasting is concerned with predicting future sequence (y=) l'iquce given initial condition yo, where the horizon Troreasst can be arbitrarily large. This requires learning the joint conditional distribution, which can be approximated with, e.g., diffusion models, by ps ( 10). Directly modeling the nigh-dimensional joint distribution is often intractable. A common alternative is modeling a smaller window of size W by lcarning a short-term transition, Pe (( y [45, 20J. Long sequences are then generated autoregressively.
Conditional diffusion models can be applied to this task by learning a denoiser network Do :::: yo). The denoiser is trained to predict the target sequence, y/.W, given the initial condition, yo. and data F1:W corrupted with Gaussian noise of standard deviation o [[min mmax] The maximum noise level needs to be chosen such that Tmix ddata. where datta is the standard devi- ation of the data y ~ Pdata and y E RRP. The trained network can then be used jointly with an ODE or SDE solver to iteratively generate the clean data starting from pure noise, ~ N/(0, o2 max Iw x D), and Yoo See Appendix B for an additional background on diffusion. Often, W = 1 [45, 29], and the diffusion model becomes a next-step forecasting model. However, the short window size models limited temporal interactions and generating each step requires a full reverse diffusion process, in- curring substantial computational cost. Standard sequence diffusion models with W > typically apply noise uniformly across the prediction window, failing to capture the increasing uncertainty inherent to future predictions.
Rolling Sequence Diffusion Models (RSDMs) [49, 9] address prior limitations by explicitly incor- porating progressive forecasting uncertainty. The core innovation is a progressive noise schedule with W > 1. Instead of uniform noise, RSDMs use a monotonically increasing schedule of noise standard deviations ( x (!) < (t) < Tw (f) R maxx where / denotes the global diffusion
3

---

time. A noisy future state is thus modeled as Eu ~ (y o (t) This structure reflects higher uncertainty for predictions further into the future. The denoiser Do(01.w: 1:w (t)) is trained analo- gously to standard diffusion model to predict the clean sequence, yı.w- In our implementation, we omit direct conditioning of the denoiser on the last known clean state, yo- This is feasible because we initialize the first window using an external forecaster (itself conditioned on yo). The minimally noisy start of the window provides sufficient conditioning to predict subsequent snapshots. When requiring the RSDM to self-initialize its first window, this is not possible [49].
4 Forecasting with Elucidated Rolling Diffusion Models
Our work, ERDM, integrates the rolling diffusion concept with the Elucidated Diffusion Model (EDM) framework [26]. While RSDMs introduce the idea of progressive noise, they usually do not follow EDM's principled design choices for preconditioning, loss weighting, and sampler design, which we hypothesize would similarly improve RSDM performance. ERDM aims to systematically port and adapt these EDM principles to the rolling, progressive noise setting, thereby providing a robust and theoretically grounded approach sequential data generation.
Rolling EDM noise schedule. In contrast to existing RSDMs, which use cosine or linear noise schedules, propose adapting EDM's sampling-ting time noise schedule to capture progressively in- creasing noise levels, by defining
o (t) = (oI (t)))
(t) (a 3 + (Pmin mmax +))²
(1)
where w 1 - w-t is the local diffusion time of the snapshot. The 1- is important so that near-future snapshots receive minimal amounts of noise, while far-future snapshots receive high levels of noise. This ensures the correct inductive bias that distant snap- shots are more uncertain than near-future ones. The continuous noise schedule is effectively divided into
W
200
10'
p-7
p 11
10
Noise level
10a
10
W segments such that the noise levels at snapshot satisfy (w) E [ōw-1(0), ow (1)], where t E 1), := Omin, and + (1) := 0max- Our
10
Gnin 1 0.002
formulation thus satisfies mmin on (1) (0)
Snapshot segment w
o2(1) < ... < (0) m Two example noise schedules trained to see any noise schedule level within the noise segments (color gradients in Fig. 2) by random- izing E U(O,, 1)). During sampling, each (t) is evolved from (0) (1). We found it important to tune the curvature parameter, p. Our proposed de- fault value, p 10, effectively causes snapshots to be under less noise compared to the default EDM choice, p = 7, which provides more information for accurate denoising. Importantly, these noise schedule parameters mmii maxx and p need be chosen training time.
Figure 2: Noise schedule comparison for se- quence length W = 6. yielding 6 visualized seg- ments [f (1). (0)]. The schedules differ by their curvature parameter p one using p (default EDM) and the other = -10 (ERDM). Color gradients illustrate segment progression.
illustrated in Fig. 2. ERDM is
Probability flow ODE.
Having defined our windowed noise schedule ot)) (#1(t).... ... .a.tt)....yyy. Fw (t)),
we can formulate the associated probability flow ODE [54], which describes the addition and re- moval of noise when moving the diffusion 'time', t, forward (1 - 0) and backward (0 1), respectively. For noisy sequence = Z1.w, this ODE, as derived in Appendix C.I, is:
dæ = diag(ā (t) (t)l((tt)........... Fw (t) (t) V p(z; (t))
(2)
(t)) the score func- tion [24] that we will approximate with a denoiser network. This ODE governs W coupled diffusion processes evolving simultaneously within the window. During inference (i.e., the backward process), an ODE solver is used, typically for N discrete steps. Upon completion of one such integration to
where (t) denotes the diffusion time derivative of a.. (t), and V log p( log
l 1, the first snapshot, & (1), is fully denoised (its noise level (1) = amin) and output as the forecast of yı- Concurrently, the subsequent snapshots ø2.w (1) are partially denoised. Key to
the rolling mechanism is the design of the per-snapshot noise schedules (.,( which ensure the
noise level of T.. (1) matches the initial noise level of the (w - 1)-th slot (i.e., (1) (w))

---

This specific noise structure at t = 1 enables the window to be advanced efficiently. For the next forecasting step. the sequence Z2.w (1) forms the initial state for the first W 1 slots at the next iteration's & = 0. A new, fully-noised snapshot, <w+1 No0, o [D), is then appended as the W-th element of this new window. The ODE solving process from
then repeated for this updated window. This iterative sampling procedure is visualized in Fig. 1.
4.1 Training Objective
We now describe how to train a denoiser network, (z (t)). that can be used to solve the proba- bility flow ODE above. Following EDM, it comprises raw neural network Fg and preconditioning To account for snapshot-dependent noise levels, we vectorize the preconditioning functions:
(0 (t)) (s ē Cour (f (t)) Cnoise (o))))
(3)
The preconditioning functions Cakip; Cout: Gin; Canise are adapted from EDM and applied per-snapshot based on the corresponding (t) in o (t). The denoiser network is trained to predict the clean window V1-W from its noisy version, I (y1.w. II) During training, we randomize t ~ U[[O, 1)) such that the network learns deal with the full extent of the segments of Fig. 2.
Uncertainty-aware loss reweighting, EDM employs a loss weighting X(o) = (o² + d (ord ensure their core network F6 targets signal with unit variance, promoting train- ing stability. We apply \(ō per frame as a direct extension. However, the rolling mechanism im- plies that certain noise levels within the o (t) window are more critical for learning the "de- tie-miiing" of temporal information. EDM itself indirectly emphasizes intermediate noise levels by sampling o values from a lognormal distribution Purain (a) during training, whose PDF corresponds
exp (-
(In(o) Pamca)) )2
[a; Pmeann Psdd) o Paav22
(4)
2P2
Since ERDM conditions on a fixed (t) per training instance, always covering a wide range of noise levels (rather than sampling each ã. from (o)), propose distinct loss weighting for ERDM that combines the EDM unit-variance objective with an emphasis on these critical intermediate noise levels within the progressive schedule. The effective loss weighting for snapshot in ERDM is \( f(ā Pmeam: Pau), which maintains EDM's target normalization via \(ō,,) while using f(o..) to upweight snapshots at the most informative noise levels. All things considered, we can write out our proposed score matching objective as
W (ow)f(oo))lDDo + Elw;;o) - yu yw
min Ey Byraw Pyaw Paata
-N E Σ
where we shorten a : (t). The full training algorithm described in Algorithm
Algorithm Elucidated Rolling Diffusion: Training
1: Require: Training data Dirain, network Fg, Omin: mmax; P2 P Pmnaan Pad
2: repeat
3: Sample y (y1, .y::...,>> yw) E RDWD from Dtrains N'O0, (w0,,, 1(),
4: a ol. (t))
5:
I y E
Add rolling noise to data
6:
ÿ - ( + Cour cou( F
Cnoise(or)
7:
Update 0 using Lg + ΣW (f Puncip Pincany Pad) 1yy tw
8: until Converged
Remark (temporal noise correlations). both training and sampling algorithms, we draw i.i.d. Gaussian noise e ~ (O, for each snapshot w independently. Recent studies show that tem- porally correlated noise improves video diffusion [8, 18, 1, 33]. Such priors are orthogonal to our sliding-window framework and can replace the i.i.d. draws in Algorithms 1-2. In our experiments,
we use the progressive noise model with a 1 proposed by Ge et al. [18], which we found to slightly improve long-range forecasts (see Appendix E). More details are provided in Appendix C.4.
5

---

4.2 Sampling
The trained denoiser, Da. can be used to estimate the score function, log p(;; ā ((), in Eq. (2) with (D, (D) (()))=z)/0(() for any E [0, 1) [26]. Based on this identity and the described probability flow ODE intuition, a mathematical formulation of our proposed sampling algorithm is provided in Algorithm 2. For simplicity, we omit the second-order Heun step and sampling stochasticity, which are detailed in Appendix C.2. Note, however, that in our main experiments, we always use the deterministic Heun sampler.
First-window initialization. The backward ODE requires a noisy window H.Ww whose clean la- tent yı-w is unknown. Ruhe et al. [49] addresses this issue by co-training the rolling model to generate & from clean context snapshots and a sequence of pure noise. This approach introduces extra hyperparameters and diverts the denoiser's capacity between
distinct denoising tasks. As an alternative, we propose drawing V1:w from an external forecaster Pe f1:W (80). We then sample Tw ~ N(u.. 2 ()) for w 1,... W, from where we can start our sliding-window windoww sampling algorithm. This choice reuses mature short-range models and injects any existing forecaster (diffusion-basse or not) can supply y..w. In our experiments, we rely on the
uncertainty;
EDM baselines for initializing ERDM. We ablate this choice in Appendix E.I.
4.3 Denoiser architecture for temporal dynamics
B1:W (b, c, b, b, W)
Our proposed diffusion model operates on temporal se- quences, necessitating an architecture capable of cap- turing temporal dependencies. Naive adaptations of 2D architectures, such as stacking the time dimension into channels, disrupt the inherent temporal structure and led to suboptimal results in our experiments (see Section 5.4). Alternatively, extending to 3D convolutions, while pre- serving temporality, often incurs significant computa- tional overhead and potentially higher sample complexity. Instead, we adopt prevalent strategy from latent video diffusion models [5]: augmenting well-established 2D U-Net architecture with explicit temporal processing lay- ers. Specifically, we integrate causal temporal attention layers into the 2D ADM U-Net [13] used EDM, posi- tioning before each down- and up-sampling block. These temporal layers also incorporate noise level information via mechanism analogous malization used in the 2D U-Net blocks. Figure 3 il- lustrates this interleaved modular design. This modular design could facilitate static pre- pre-training of the 2D back- bone, yet we observed that end-to-end training on the se- quential data was more effective for our tasks (see Ap- pendix E). While we found the proposed architecture to be key for generating good forecasts, it is slower and has higher memory needs than a 2D variant.
Noise Embadding (b, c, t)
Reshape
2D U-Net Block (bet, c,
Reshape (bst, c)
Reshape
Causal Temporal Attention Block
the adaptive layer nor-
(b+ c,
Figure 3: Sketch of one 2D U-Net block and one temporal attention block in our hy- brid U-Net topology with noise embedding to both spatial and temporal paths. Dimen- sions b, c, t, h, w refer to batch, chan- nel, window, height, and width. For simplic- ity, c, h, do not change in the sketch.
Experiments
5.1 Evaluation and Metrics
Due to the importance of probabilistic forecasts and uncertainty quantification, focus two key ensemble- based probabilistic metrics: Continuous Ranked Probability Score (CRPS) [37], and the spread-skil ratio (SSR) based on M-member ensembles, where M 50 for Navier-Stokes and M = 10 for ERAS. The CRPS is a proper scoring rule commonly used to evaluate probabilistic fore- casts [19]. The spread-skill ratio is defined as the ratio of the square root of the ensemble variance the corresponding ensemble-mean RMSE. It serves as a measure of the reliability of the ensemble, where values smaller than indicate underdispersion, and larger values overdispersion [16, 46).

---

Algorithm 2 Elucidated Rolling Diffusion Deterministic Sampler (Fuler-only)
1: Require: VL.w, N.Tooreasst
2: Al 1/N
Infer step size from desired number of steps per snapshot t- Initialize global diffusion time and emply generated sequence t> Initialize window with snapshot-dependenl rolling noise
bour 0: S Ø
4: sample Heur ~ NWiww (( (Iex))
5: while |S] Tioreaast do
t- Predict snapshot [S|+1 Global diftusion time after denoising
6: Inext teur At
our (teur)
Current noise levels
a next - (1next.
t> Noise levels at the end of this iteration
9:
- Do(Buur Teur)
p- Denoise sequence
10:
d (Tour D
>> Evaluate dæz/dt at fem- Euler step from lour Fnext
11:
Bnext Beur (aTnxxt our )
12:
Relean [lnext]
Infer finished snapshots, O, the following lines are no-op
13:
add first Ahlean snapshots of ý to S and discard first Velean snapshots of Tnext
14:
sample Znew (O, I Inttou DD)
15:
[Rnxxt Fnew
Concatenate fresh noisy snapshots to futuremost RR-adjust global diffusion 'time' be in [0, 1)
16:
four Enext 7ellea
17:
return S
ERDM
EDM W=1
EDM W=
4
DYffusion
PDE-Refirer
1.25
0.04
1.00
0.03
CRPS
SSR
0.75
0.02
0.50
0.01
20
40
20
40
Timestep
Timestep
Figure 4: Navier-Stokes test rollout over 64 time steps with 50 ensemble members. FRDM superior perfor- mance in both CRPS and calibration compared single- and multi-step EDM baselines, except for the initial timesteps. Beyond timestep 15, ERDM consistently delivers an approximate 50% improvement CRPS over the next best model (EDM W 4), demonstrating particular strength in long-range forecasting scenarios.
5.2 Navier-Stokes fluid dynamics
Dataset. We use the Navier-Stoke fluid dynamics benchmark from [41], defined on 221 x 42 grid. Each simulation features four randomly placed circular obstacles influencing the flow, with fluid viscosity set to 10-3 The dataset comprises æ and y velocities and pressure fields. All models receive boundary conditions and obstacle masks as auxiliary inputs. For testing, models predict a 64-timestep trajectory from a single initial snapshot.
Baselines. We benchmark ERDM against DYffusion [50], the current state-of-the-art method on this dataset, and PDE-Refiner [32]. To ensure a fair comparison, we retrained DYffusion using experimental setup, achieving improvement of over 3x its originally reported CRPS scores. Our primary focus. however, to evaluate ERDM's performance relative to common EDM-derived approaches for dynamics forecasting. Consequently, our key bascline is EDM denoiser parameterized by Do(x1.... zw; a, Yo), where all a. are corrupted according to the recover next-step forecasting conditional EDM in [45, 29]. We tried W E (1, 6). and found W = 4 to work best for EDM. All baselines are trained on three random seeds and share the same architecture as much as possible. For ERDM, we use W = 6, Jmin 0.002, max 200, p = 10, P Pmean = 0.5, Pad = 1.2, N = 1.25. Further de- tails provided in Appendix D.2.
same With W
Results. In Fig. 4, we benchmark ERDM against the baselines on forecasting the five Navier- Stokes test trajectories of length 64. While the EDM-based baselines and PDE-Refiner start well
7

---

EDM
ERDM
IFS ENS
NeuraGCM
Giop--ER EFM
1850
2t
t50
10u
15
CRPS ve EDM (%6
10
-10
-10
-15
2500
4700
91000
20
20
CRPS ve EDM [8]]
10
-5
6
-10
-10
-15
-20
-20
-30
-5
200
100
300
200 Lead time
200 300
Lead time
Load time
Lead time
Figure 5: Relative CRPS (lower is better) over single-step EDM baseline as a function of lead time, up to 15 days, for 8 selected variables. ERDM consistently outperforms EDM across most variables and lcad times by up to 10%. Furthermore, it performs competitively against the state-of-the-ar operational physics-based model IFS ENS and the hybrid model NeuralGCM ENS, especially for long lead times, while being more efficient.
in terms of CRPS, even beating ERDM for the very first few time steps, they exhibit a significantly higher error growth than ERDM. We attribute this to the explicit modeling of progressive uncertainty built into our model. At the end of the rollout, ERDM achieves a 30% better CRPS than the best baseline, EDM with window size of W 4. Similarly, DYffusion starts worse and catches up with the EDM W = 4 baseline at the end of the rollout, but not with ERDM. In terms of calibration, observe that ERDM consistently outperforms the EDM baselines, which are noticeably under- calibrated. We note that this is despite comprehensive tuning of these baselines, where we were only able to achieve calibration improvements the sigmificant expense of CRPS skill (not shown). Ex- amples of generated forecasts are visualized as videos at this URL:
5.3
ERA5 weather forecasting
Dataset. We benchmark our model on medium-range weather forecasting on the ERAS reanalysis dataset. We use the 1.5" resolution the data (a 240 121 grid) provided by [46], with 69 prognostic (input and output) variables: Temperature (t), geopotential (z), specific humidity (q), and the u and v components of wind (u, v) over 13 pressure levels (numbers after the abbreviation refer to the level in hPa) as well as the four surface variables 2m temperature (2t), mean sea level pressure (mslp), and u and v components of winds at 10m (10u, 10v). We train our models on 12- 12-hourly data, as in [45], from 1979 to 2020 and evaluate on 64 2021 initial conditions 00/12 UTC.
Baselines. Our primary baseline is conditional next-step forecasting EDM, trained using methodology consistent with ERDM's development. To gauge the absolute performance of ERDM, also benchmark it against two prominent external models: (1) IFS ENS: The European Cen- tre for Medium-Range Weather Forecasts' (ECMWF) operational, physics-based ensemble fore- casting system [15]; (2) NeuralGCM ENS: A hybrid ML-physies stochastic model [28]. Offi cial 1.5--resolution forecasts for both external models were sourced from Weatherbench-2 [46] for all variables where available. The IFS ENS evaluation uses the same 2021 initial condi- tion dates as our EDM and ERDM models, but is verified against operational analyses.2 Due to the unavailability of NeuralGCM forecasts for 2021, we use 2020 forecasts and evaluate them against the corresponding 2020 ERA5 data. Our conditional EDM baseline can be seen as a reproduction of GenCast [45], for which forecasts are only available higher spatial reso- lutions and a limited set of variables, but using our experimental setup and neural architecture. Notably, training ERDM is considerably less computationally expensive than NeuralGCM ENS 1.5" (GenCast 1º), which required 128 (32) v5 TPUs and 10 (3.5) days to train. In contrast,
As opposed to ERA5 targets. This choice is standard practice [46] and favorable to IFS ENS.

---

ERDM was trained on 4 H200 GPUs in 5 days only. Lastly, we also include scores for z500 and 2t from Graph-EFM [40]. taken from their paper. These scores are provided for reference, noting that Graph-EFM is trained on 20 more years and evaluated in 2020. For ERDM, we use
W min 0.002, mmxx 500, p 10, Preca 2, Paua
1.2, N 2. See Appendix D.2 for
more experimental details.
EDM ENS
NC
EROM
Results. An evaluation of ERDM's performance, il- lustrated in Fig. 5, reveals its consistent superiority in relative CRPS skill over our primary EDM baseline, by up to 10%. Similarly, ERDM demonstrates a clear advantage over related methods, significantly outper- forming Graph-EFN Fraph-EFMM on all four CRPS metrics reported in their study [40]. Against more computationally de- manding models such as IFS ENS and NeuralGCM, our approach is broadly competitive. Its primary lim- itation is a noted weakness in some short-range fore- casts compared to IFS ENS, which attribute our EDM-based initialization strategy and a backbone ar- chitecture not fully specialized for weather prediction. In terms of spread-skill ratio, ERDM, along with IFS ENS, delivers the most calibrated forecasts (see Fig. 12). In contrast, EDM and NeuralGCM often produce un- derdispersed short-range forecasts. crucial aspect of evaluating any ensemble forecast is its physical realism. In this regard, FRDM performs exceptionally well. As depicted in Fig. 6, its normalized power spectra are on par with those of the physics-based IFS ENS. This significant achievement, struggle to reproduce such physically consistent spec- tra-a challenge evident with NeuralGCM in Fig. 6 and documented by Rasp et al. [46]. See Appendix for extended results, including analyses of extra variables, spread-skil ratios, visualizations, and an EDM vs. ERDM score card.
10
10
Normalized spectral density
z500
8
mslp
110"
10
10'
10²
Zonal wavenumber
Figure 6: Normalized spectral density of 14- day forecasts, averaged [60', 90"]. Spectra are divided by the target ERAS reanalysis spectra. ERDM generates highly accurate spectra that match or slightly beat the physics-based model IFS ENS. Neu- raIGCM ENS undcrestimates energy the mid to high frequencies. See Appendix F3 for more variables and absolute spectra.
high latitudes,
many ML models typically
5.4 Ablations
We conducted a comprehensive ablation study on the Navier-Stokes task to validate ERDM's design. Our findings reveal that several architectural and training strategies fundamental its success; their removal causes performance to collapse, often falling below that of the single-step EDM base- line. In contrast, the model is robust to certain hyperparameters, showing little sensitivity to the window size, W, or the noise bounds, mii and max within a reasonable range. We focus on the most impactful ablations below and provide full analysis in Appendix
1. Appropriate progressive noise schedule & fixed training:
The design of the noise sched- ule is paramount. Default EDM hyperparameters (p = 7) prove suboptimal for ERDM, yielding 2x worse CRPS than our proposed default schedule (p 100). Performance is robust within a reasonable range of e [-30, -5]. see Fig. 2 for intuition on why. Further- more, training with fixed, optimized noise schedule is critical; randomizing the schedule during training degrades performance by nearly 2x. Despite this, such randomized train- ing can serve as heuristic to identify an effective fixed schedule for subsequent retraining. This finding contrasts with the randomized training procedure from Chen et al. [9].
2. Strategic loss weighting: Reweighing the losses based the lognormal probability den- sity function of noise levels, (a) as proposed, is crucial. Removing this weighting causes 2x performance drop. We generally found that larger mean of the lognormal distri- bution, Pmean > O, than EDM's default, - 1.2, was necessary for ERDM to achieve optimal results, confirming the importance of tuning here.
3. Dedicated spatiotemporal architecture: A crucial element is the use of a proper archi- tecture that explicitly models spatiotemporal dependencies. Naively stacking the time di-

---

mension into the channel dimension of 2D architecture results in severe 4x performance degradation, confirming the need for our bespoke temporal architecture.
5.5 Computational complexity
To analyze the computational demands of ERDM relative to a standard autoregressive EDM, we benchmarked key efficiency metrics single A100 GPU. The results, summarized in Table 1, correspond to generating a 30-step (15-day), 5-member ensemble weather forecast. The primary trade-off lies in the architectural design. ERDM's hybrid 3D denoiser is inherently more memory- intensive than the 2D architecture used by EDM, requiring more than twice as much GPU memory. However, the rolling window mechanism makes ERDM significantly more efficient in terms of Neural Function Evaluations (NFEs). ERDM requires 5x fewer NFEs to generate the full 15-day forecast than the step-by-step EDM. Thus, despite the higher cost per step, ERDM's total inference time (including the initialization cost) is competitive with, and even slightly faster than, EDM's.
Table I:
Efficiency metrics for ERDM vs. EDM. All measurements were performed on a single A100 GPU using mixed precision. Inference figures correspond to a 15-day (30 time steps), 5-memher ensemble weather forecast with second-order Heun solver. Training uses a batch size of I. NFEs are Neural Function Evaluations.
GPU Memory (GB)
Model EDM
Inference NFEs
Inference Time (s)
Inference
Training
600
237
21
19
ERDM
120
209
49
53
Conclusion
We introduced the Elucidated Rolling Diffusion Model (ERDM), diffusion framework for long- range probabilistic forecasting in complex scientific systems. ERDM adapts EDM diffusion to se- quential data by integrating a progressive temporal noise schedule and snapshot-dependent precondi- tioning, enabling it to explicitly model the increasing uncertainty of chaotic dynamics. Our ablation studies validate that these components, combined with a bespoke spatiotemporal architecture, synergistic and essential for strong performance. On challenging benchmarks like ERA5 weather data and Navier-Stoke simulations, ERDM consistently outperforms relevant baselines in proba- bilistic metrics such as CRPS, calibration, and physical cealism-particus horizons. Future work could improve computational efficiency through latent-space modeling or explore the framework's 's applicability to wider range of physical systems.
over extended forecast
Limitations. The primary limitation of ERDM is the computational cost of its 3D denoiser ar- chitecture. While ERDM requires fewer sampling steps than autoregressive baselines (e.g., 40% less ERAS), each step is more expensive, resulting in a comparable total inference time but sig- nificantly higher memory usage. This memory complexity poses a key barrier to scaling to higher resolutions, though techniques like gradient checkpointing or latent diffusion could offer path for- ward. Beyond computational hurdles, ERDM's short-range weather forecasts are not yet on par with leading operational models like IFS ENS. The framework is also constrained by its reliance external model for initialization and its use of an explicit noise-level loss weighting, which may be suboptimal compared to importance sampling [14], offering exciting directions for future work.
Acknowledgements
S.R.C. acknowledges generous support from the summer internship and collaboration at NVIDIA. This research used resources from the National Energy Research Scientific Computing Center (NERSC), a Department of Energy User Facility, using NERSC awards DDR-ERCAP003414 ESCR-ERCAPP003329 and Army Research Office under Army-ECASE award DEIINNF-----000E Of Energy, Office of Science, IARPA HAYSTAC Program, and NSF Grants #2205093, #2146343, #2134274, CDC--RFA-FT-2330066 DARPA ATE FoundSci and DARPA YFA. We are grateful the anonymous reviewers for their valuable feedback that helped strengthen this work.
This work was supported in part by the U.S.
the U.S. Department
10

---

References
[1] Martin Andrae, Tomas Landelius, Joel Oskarsson, and Fredrik Lindsten. Continuous ensemble weather forecasting with diffusion models. International Conference on Learning Representa- tions, 2025. 3,
[2] Peter Bauer, Alan Thorpe, and Gilbert Brunet. The quiet revolution numerical weather prediction. Nature, 525(7567):47-55, Sep 2015. ISSN 1476-4687. doi: 10t1100.08//aaure1499900
[3] Emanuele Bevacqua, Laura Suarez-Gutierree, Aglaé Jézéquel, Flavio Lehner, Mathieu Vrac, Pascal Yiou, and Jakob Zscheischler. Advancing research on compound weather and climate events via large ensemble model simulations. Nature Communications, 14(1):2145, April 2023.
[4] Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. Accurate medium-range global weather forecasting with 3D neural networks. Nature, 619(7970):533- 538, 2023. doi: 10. 10000000/11111--3-006155-33
[5] Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Do- minik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, Varun Jampani, and Robin Rombach. Stable video diffusion: Scaling latent video diffusion models large datasets. 2023.
URL https: org/abs/////eooooooooo...ooeee 2311 15127. 3,
[6] Cristian Bodnar, Wessel P. Bruinsma, Ana Lucic, Megan Stanley, Anna Allen, Johannes Brand- stetter, Patrick Garvan, Maik Riechert, Jonathan A. Weyn, Haiyu Dong, Jayesh K. Gupta, Kit Thambiratnam, Alexander T. Archibald, Chun-Chieh Wu, Elizabeth Heider, Max Welling, Richard E. Turner, and Paris Perdikaris. Aurora: A foundation model for the earth system, 2024. URL /attps://arxils wtt//or////////o// 13063.
[7] Zied Ben Bouallègue, Mariana C. A. Clare, Linus Magnusson, Estibaliz Gascón, Michael Maier-Gerher, Martin Janoušek, Mark Rodwell, Florian Pinault, Jesper S. Dramsch, Simon T. K. Lang, Baudouin Raoult, Florence Rabier, Matthieu Chevallier, Irina Sandu, Peter Duehen, Matthew Chantry, and Florian Pappenberger. The rise of data-driven weather forecasting: first statistical assessment of machine learning-basee weather forecasts in operational- like context. Bulletin of the American Meteorological Society, 105(6):E864 E883, 2024.
doi: T11.5///AMS---2--1166
[8] Pascal Chang, Jingwei Tang. Markus Gross, and Vinicius C. Azevedo. How i warped your noise: a temporally-coorelated noise prior for diffusion models. International Con- ference Learning Reprexentations, 2024. URL https: http://ppeneeeeeeeeeeetetoo PzEInMrgSD. 5
[9] Boyuan Chen, Diego Marti Monso, Yilun Du Max Simchowitz, Russ Tedrake, and Vincent Sitzmann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. Advances in Neural Information Processing Systems, 2024. 3,
[10] Lei Chen, Fei Du, Yuan Hu, Zhibin Wang, and Fan Wang. Swinrdm: integrate swin- mn with diffusion model towards high-resolution and high-quality weather forecasting. In Proceedings of the Thirty-Seventh AAAI Conference Artificial Intelligence, 2023. doi: 10.110...66 2.009//ai..37....5515. x3711.25105..
[11] James S. Clark, Steven R. Carpenter, Mary Barber, Scott Collins, Andy Dobson, Jonathan A. Foley, David M. Lodge, Mercedes Pascual, Roger Pielke, William Pizer, Cathy Pringle, Wal- ter V. Reid, Kenneth A. Rose, Osvaldo Sala, William H. Schlesinger, Diana H. Wall, and David Wear. Ecological forecasts: An emerging imperative. Science, 293(5530):657-660, 2001. wtw.11///science293.555555
[12] Guillaume Couairon, Renu Singh, Anastase Charantonis, Christian Lessig, and Claire Mon- teleoni. ArchesWeather Arches Weatherea:::::en: deterministic and generative model for effi- cient MI. weather forecasting. 2024. doi: 10..448550000rii22222.222999

---

[13] Prafulla Dhariwal and Alex Nichol. Diffusion models beat GANs on image synthesis, Ad- vances in Neural Information Processing Systems, 2021. dor110488500mmii22055.62233. 2, 6, 29,30
[14] Sander Dieleman. Noise schedules considered harmful, 2024. URL https://sandeer ai/ 2024/06/ 10///14/// 14//1///oo/ ht////sssscccchllleeeeee html.
[15] ECMWE IFS Documentation CY46R! Part V: Ensemble Prediction System. 2019.
doi: do1:10.21957738yyuceee 1,8
[16) V. Fortin, M. Abaza, F. Anetil, and R. Turcotte. Why should ensemble spread match the rmse of the ensemble mean? Journal of Hydrometeorology. 15(4):1708 - 1713, 2014.
doi.hutps://doi.org//... htw.://///...1S//--M----EELM 115/JHM-D-14-0000. 6, 31
[17] Zhihan Gao, Xingjian Shi, Boran Han, Hao Wang, Xiaoyong Jin, Danielle C Maddix, Yi Zhu, Mu Li, and Bernie Wang. PreDiff: Precipitation nowcasting with latent diffusion models. Advances in Neural Information Processing Systems, 2023.
[18] Songwei Ge. Seungjun Nah, Guilin Liu, Tyler Poon, Andrew Tao, Bryan Catanzaro, David Jacobs, Jia-Bin Huang. Ming-Yo Liu, and Yogesh Balaji. Preserve your own correlation: A noise prior for video diffusion models. ICCV, 2024. 5, 27, 31,
[19] Tilmann Gneiting and Matthias Katzfuss. Probabilistic forecasting, Annual Review of Statistics and Its Application, 1(1):125-151, January 2014. ISSN 2326-831X. doi:10.1 doi:10.1146/annuree 146/annurev- matisties-062713-08555
6
[20] William Harvey, Sacid Naderiparizi, Vaden Masrani, Christian Weilbach, and Frank Wood. Flexible diffusion modeling of long videos. Advances in Neural Information Processing Sys- tems, 2022. doi: 4tt.110055////xx/.22205 11405.
[21] Jonathan Ho. Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Ad- vances in Neural Information Processing Systems, 2020. doi: 10t:10048500aaxxx220066 1239.
[22] Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P. Kingma, Ben Poole, Mohammad Norouzi, David J. Fleet, and Tim Salimans. Imagen video: High definition video generation with diffusion models. arXiv, 2022. 11t:100485500axxx2221000000
[23] Langwen Huang, Lukas Gianinazzi, Yuejiang Yu, Peter Dominik Dueben, and Torsten Hoefler. DifiDA: a diffusion model for weather-scale data assimilation. International Conference on Machine Learning, 2024. URL https::/opeereviiiii wtwpeeewwwwwwwwweeeeeeeeeeee aet//foouumm.d-whh/ae33334 3
[24] Aapo Hyvärinen. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(24):695-709, 2005. URL http: /tatt:://jmiii //jrrr www.oooggeeeeeeeeeoeooeee papers/v6/ hyvarinen05a. html.
[25] Matthias Karlbauer, Danielle C. Maddix, Abdul Fatir Ansari, Boran Han, Gaurav Gupta, Yuyang Wang, Andrew Stuart, and Michael W. Mahoney. Comparing and contrasting deep learning weather prediction backbones on navier-stokes and atmospheric dynamics. 2024.
URL httts: //axxiv..//////////a org arg/abs/// ab//aabbs.....///////////////a// 2407.14129. 14129.
[26] Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems, 2022.
2, 4, 6, 24, 25, 26, 27, 30
[27] Jihwan Kim, Junoh Kang, Jinyoung Choi, and Bohyung Han. EIFO-diffusion: Generating infinite videos from text without training. Advances in Neural Information Processing Systems, 2024. URL https:://// /ttp:////ppeeeeeeeeee
tttt//oorummdd-iiiaaaaaaaaaa 3
[28] Dmitrii Kochkov, Janni Yuval, Ian Langmore, Peter Norgaard, Jamie Smith, Griffin Mooers, Milan Klöwer, James Lottes, Stephan Rasp, Peter Düben, Sam Hatfield, Peter Battaglia, Alvaro Sanchez-Gonzalez, Matthew Willson, Michael P. Brenner, and Stephan Hoyer. Neural general circulation models for weather and climate. Nature, (06000 July 2024. ISSN 1476-4687. 10. 1038/s41 4 586-024-07744--. 3,8
12

---

[29] Georg Kohl, Liwei Chen, and Nils Thuerey. Benchmarking autoregressive conditional diffu- sion models for turbulent flow simulation. 2023. 2, 3, 7, 30
[30] Remi Lam, Alvaro Sanchez-Gonzales Matthew Willson, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton-Rosen, Weihua Hu, Alexander Merose, Stephan Hoyer, George Holland, Oriol Vinyals, Jacklynn Stott, Alexander Pritzel, Shakir Mo- hamed, and Peter Battaglia. Learning skillful medium-range global weather forecasting, Sci- ence, 382(6677):1416) 1421, 2023. ISSN 1095-9203. doi: 126/science.adi25.... 3, 30
[31] Lizao Li, Robert Carver, Ignacio Lopez-Gomez Fei Sha, and John Anderson. Generative emulation of weather forecast ensembles with diffusion models. Science Advances, 10(13); cadk4489, 2024. doi: 10.
URL https: / http://wwwinnc.... science. htg//oo/abs//////
10. / adk4489.
[32] Phillip Lippe, Bastiaan S. Veeling, Paris Perdikaris, Richard E Turner, and Johannes Brandstet- ter. PDE-Refiner: Achieving accurate long rollouts with temporal neural pde solvers, Advances in Neural Information Processing Systems, 2023. doi: 00te:000485500/mm.2220...0..5..
3, 7, 29
[33] Chao Liu and Arash Vahdat. Equivdm: Equivariant video diffusion models with temporally consistent noise. 2025. 5
[34] Ignacio Lopez-Gomest Zhong Yi Wan, Leonardo Zepeda-Núñez, Tapio Schneider, John Anderson, and Fei Sha. Dynamical-gennrattn downscaling of climate model ensem- bles. Proceedings of the National Academy of Sciences, 122(17):e220028812, 2025.
100.10073/pnas.2220888.
[35] Peter Manshausen, Yair Cohen, Peter Harrington, Jaideep Pathak, Mike Pritchard, Piyush Garg, Morteza Mardani, Karthik Kashinath, Simon Bymee and Noah Brenowitz. Genera- tive data assimilation of sparse weather station observations kilometer scales. 2025. URL https://arxi:: org/ abs/2406. 16947. 3
[36] Morteza Mardani, Noah Brenowitz, Yair Cohen, Jaideep Pathak, Chieh-Yu Chen, Cheng- Chin Liu, Arash Vahdat, Mohammad Amin Nabian, Tao Ge, Akshay Subramaniam, Karthik Kashinath, Jan Kautz, and Mike Pritchard. Residual corrective diffusion modeling for km- scale atmospheric downscaling, Communications Earth & Enviroument, 6(1), February 2025. ISSN 2662-4435. 10.
[37] James E. Matheson and Robert L. Winkler. Scoring rules for continuous probability distribu- tions. Management Science, 10800166,, 1976. 6, 61
[38] Tung Nguyen, Johannes Brandstetter, Ashish Kapoor, Jayesh K Gupta, and Aditya Grover. ClimaX: A foundation model for weather and climate. International Conference on Machine Learning, 2023. 000110048500/amxx.2301....
3
[39] Tung Nguyen, Rohan Shah, Hritik Bansal, Troy Arcomano, Sandeep Madireddy, Romit Maulik, Veerabhadra Kotamarthi, Ian Foster, and Aditya Grover. Scaling transformer neu- ral networks for skillful and reliable medium-range weather forecasting. Advances in Neural Information Processing Systems, 2024. 10..1.855///aaxx2212..2....
[40] Joel Oskarsson, Tomas Landelius, Mare Peter Deisenroth, and Fredrik Lindsten. Probabilistic weather forecasting with hierarchical graph neural networks. Advances in Neural Information Processing Systems, 2024. URL /ttttss://pperrviiiii net/forumiid
3,
[41] Karl Otness, Arvi Gjoka, Joan Bruna, Daniele Panozzo, Benjamin Peherstorfer, Teseo Schnei- der, and Denis Zorin. An extensible benchmark suite for learning simulate physical systems. Advances in Neural Information Processing Systems Track on Datasets and Benchmarks, 2021. 7, 18, 28
[42] T. N. Palmer. The economic value of ensemble forecasts as tool for risk assessment: From days decades. Quarterly Journal of the Royal Meteorological Society, 128(581):477-774 2002. doi.http:://oii.rr//// httpp////.......//03000000013330
13

---

[43] Jaideep Pathak, Shashank Subramanian, Peter Harrington, Sanjeev Raja, Ashesh Chattopad- hyay, Morteza Mardani, Thorsten Kurth, David Hall, Zongyi Li, Kamyar Azizzadenesheli, Pe- dram Hassanzadeh, Karthik Kashinath, and Anima Anandkumar. FourCastNet: Accelerating global high-resolution weather forecasting using adaptive Fourier neural operators. Proceed- ings of the National Academy of Sciences (PNAS), 119448))22088655111 2022. 3
[44] Jaideep Pathak, Yair Cohen, Piyush Garg, Peter Harrington, Noah Brenowitz, Dale Durran, Morteza Mardani, Arash Vahdat, Shaoming Xu, Karthik Kashinath, and Michael Pritchard. Kilometer-scale convection allowing model emulation using generative diffusion modeling, 2024. URL hwthhttp///// http///axxvvv.o///e abs abs//a/s...... 10958. 3
[45] Ilan Price, Alvaro Sanchez-Gonzales Ferran Alet, Tom R. Andersson, Andrew El-Kadi, Do- minic Masters, Timo Ewalds, Jacklynn Stott, Shakir Mohamed, Peter Battaglia, Remi Lam, and Matthew Willson. Probabilistic weather forecasting with machine learning. Nature, 637 (8044):84-90, December 2024. ISSN 1476-4687. doi: 10.:10.100 1001100//41166-024204000 http://dx. doi org/oo..//.. 10. 10210003/444 s41586 02002--882299... 9. 2, 3, 7, 8, 28, 30
URL
[46] Stephan Rasp, Stephan Hoyer, Alexander Merose, Ian Langmore, Peter Battaglia, Tyler Rus- sell, Alvaro Sanchez-Gonzalez, Vivian Yang. Rob Carver, Shreya Agrawal, Matthew Chantry, Zied Ben Bouallegue, Peter Dueben, Carla Bromberg, Jared Sisk, Luke Barrington, Aaron Bell, and Fei Sha. WeatherBench 2: A benchmark for the next generation of data-driven global weather models. Journal of Advances in Modeling Earth Systems, 16(6), June 2024. ISSN 1942-2466. doi: 2001000002/00233000000 6, 8, 9, 18, 28, 30, 31
[47] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. pages 10684-10695, 2022. 3
[48] François Rozet and Gilles Louppe. Score-based data assimilation. Advances in Neu- ral Information Processing Systems, 2023. URL VUvLSnMZaX. 3
[49] David Ruhe, Jonathan Heek, Tim Salimans, and Emiel Hoogeboom. Rolling diffusion models. International Conference on Machine Leurning, 2024. URL https://openreviewt net/ forum?id=a9bzTvSS:
4,6
[50] Salva Rühling Cachay, Bo Zhao, Hailey Joren, and Rose Yu. DYffusion: A dynamics-informed diffusion model for spatiotemporal forecasting. Advances in Neural Information Processing Systems, 2023. doi: 10.448550arrww.233600099999
3,7
[51] Salva Rühling Cachay, Brian Henn, Oliver Watt-Meyer, Christopher S. Bretherton, and Rose Yu. Probabilistic emulation of a global climate model with Spherical DYffusson. Advances in Neural Information Processing Systems, 2024. 3
[52] Jascha Gohl-Dickstein Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. International Conference on Machine Learning, 2015. doi: 10,48550/arxiv, 15w.3i....11..... 1503.03585. 2
[53] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. Inter- national Conference on Learning Representations, 2021.
[54] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. International Conference on Learning Representations, 2021. 485:100.4550//axxx22001 1.14566.
2, 4, 24, 25
[55] Prakhar Srivastava, Ruihan Yang. Gavin Kerrigan, Gideon Dresdner, Jeremy J McGibbon, Christopher S. Bretherton, and Stephan Mandt. Precipitation downscaling with spatiotemporal video diffusion. Advances in Neural Information Processing Systems, 2024. 3
[56] Arash Vahdat, Karsten Kreis, and Jan Kautz. Score-based generative model- ing in latent space. Advances in Neurul Information Processing Systems, 2021. URI adcalcchee/4424430444440000.0000 Papeer .af.r.pdf.
https://proceedingss
wtwpeeeeddinnn.nu.iiieeee neurips cc/
14

---

[57] Tong Wu, Zhihao Fan, Xiao Liu, Yeyun Gong, Yelong Shen, Jian Jiao, Hai-Tao Zheng, Juntao Li, Zhongyu Wei. Jian Guo, Nan Duan, and Weizhu Chen. AR-Diffusion: auto-regressive diffusion model for text generation. Advances in Neural Information Processing Systems, 2023. 2,3
[58] Ruihan Yang, Prakhar Srivastava, and Stephan Mandt. Diffusion probabilistic modeling for video generation. Entropy, 25(10):1469, 2023.
[59] Michaël Zamo and Philippe Naveau. Estimation of the continuous ranked probability score with limited information and applications to ensemble weather forecasts. Mathematical Geo- sciences, 20(2):00--234, 2018. doi: 0oi:10.1007/: 10040077777--- 31
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We clearly mention our key contributions at the end of the introduction, which reflect our experimental results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
See the bespoke limitations paragraph the end of our conclusion section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specificattin, asymptotic approximations only holding locally). The au- thors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution is low images taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to ad- dress problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't i acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [es]]
16

---

Albeit not central to our work, we include complete derivation of the prob- ability flow ODE of our model in Appendix C.I.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the of the paper should be comple- mented by formal proof' provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: All experimental configurations, including dataset details and hyperparame- ters, are fully described in Appendix D. Algorithmic details for training and sampling are provided Algorithm and Algorithm 2.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., g., in the case of a large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or a way produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which thors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
17

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Code, including detailed README with instructions, will be open-sourced. Datasets that are publicly available. Refer to Otness et al. [41] and Rasp et al. [46] for instructions for the Navier Stokes and ERA5 data, respectively.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https: http://iipscccccocc
lttpe//ggiidee//oeeeuueeee sos.ssoooiiiiooooccoc.oooo for more details.
While we encourage the release code and data, we understand that this might not be possible, so "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //nis. /cc/pblic//ae seccc/guiesseee Cubmmmumiiii ssisn ssionnolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: Data splits are described the main text, where we introduce the respective datasets. All experimental configurations, including baseline details and hyperparameters, are fully described in Appendix D.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer: [Ys]]
For the Navier Stokes experiments, train every method with three different random seeds and report error bars based on these sets of 3 runs per method. For ERAS, we only train single model per method for computational efficiency reasons.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
18

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of eπrоrs is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Yes, the used compute resources are described in Appendix D.5 for each dataset.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't t make it into the paper).
9. Code ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: : /ttp///eurrpp..p/ee cc Setpblic/Ethissuuddeeeoo
Answer: [Yes]
Justification: The authors have reviewed the NeurIPS Code of Ethics and believe that the conducted research the paper conforms, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there is a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
19

---

Justification:
Potential positive and negative societal impacts are discussed in special section (Appendix A).
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper does not deal with data or models with a high risk of misuse or related to internet scraping.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards allow for controlled use of the model, for example by quiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We explicitly reference the creators of the datasets and baseline forecast data for ERA5 that we use. The licenses of the datasets, which we fully respect, are explicitly mentioned in Appendix D.1
Guidelines:
The answer NA means that the paper does not use existing assets.
20

---

The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use the pack- age should be provided. For popular datasets, paperswith thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the li- cense of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged reach out the asset's ' creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documenta- tion provided alongside the assets?
Answer: [Ys]]
Our method's training and sampling algorithms are fully described in Algo- rithm 1 and Algorithm 2. Our proposed method is fully reproducible in our source code, including clear instructions. Model weights for our ERA5 model will be open-sourced together with the code.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mmoel as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the pa- per include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Question:
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethies, workers involved in data collection, cura- tion, or other labor should be paid at least the minimum wage the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
21

---

Answer: [NA]
Justification:
Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equiva-
lent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: LLMs are only used for helping with editing and formatting of latex and figures. LLMs not related to the core methods in our work in any way.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy (https:// htt////eeersppeeeeee wtt.c//aeeeeeccce/22222222 200.cceL////22LLL for what should or should not be described.
22

---

Appendix
Contents
A Broader Impact
23
Background on Diffusion Models
23
C ERDM
25
C.I Probability flow ODE derivation
25
C,2 Sampling
26
C.3 Stochastic Heun Sampler
27
C.4 Temporal noise prior
27
Experimental Details
28
D.1 Dataset details
28
D.2
Implementation details and hyperparameters
29
D.3 Baseline details
29
D.4
Distinctions between GenCast and our EDM model for ERA5
30
D.5
Compute resources
30
D.6
Metrics
30
E
Ablations
31
E.1
ERDM initialization..
33
E,2
ERA5 evaluation year ablation
34
F
Additional Results
34
F.
Additional Navier-Stokes results
F2
Detailed ERA5 quantitative results
35
F3
Comprchensive ERA5 power spectra
35
F.4
ERA5 forecast visualizations
37
A
Broader Impact
Our research holds the potential for societal benefits, primarily through improved disaster prepared- ness, enhanced decision-making across sectors like agriculture or energy. However, these advance- ments also necessitate careful consideration of potential negative impacts, such over-reliance on ML-based forecasts or socioeconomic disparities in access and benefit. Mitigating these risks requires comprehensive evaluations, communication about forecast uncertainties, and efforts to sure equitable access to these new tools. Any deployment of our model, or derived versions, should comprehensive evaluations than in our paper, careful disclaimers
be accompanied by either to not blindly trust its forecasts, especially for data outside of the training data distribution.
B Background on Diffusion Models
Consider the data distribution represented by (a)) The forward diffusion process transforms this distribution by incorporating time-dependent Gaussian noise, yielding modified distributions
23

---

p(:: G(t)). This transformation is achieved by adding Gaussian noise with zero mean and covari- ance G(t) to the data. Gtt) d x d positive definite noise covariance matrix that evolves over diffusion time I. When the "magnitude" of G(t) (e.g.. its trace or smallest eigenvalue) is sufficiently large, the resulting distribution p(a; (t)) approximates a pure Gaussian distribution (1)).
Conversely, the backward diffusion process operates by initially sampling noise, represented as æo- from prior distribution, typically (O, GO)). where G(0) corresponds to the maximum noise level. The process then focuses on denoising this sample through a sequence, L; characterized by a sequence of decreasing noise covariance matrices: G(ty) G(t)) > .... > Gtv R 0 (in the Loewner order), where to = 0. Each intermediate sample I; is drawn from z; p(a;; G(t))). The terminal sample of the backward process, NN is expected to approximate the original data distribution Paata y). Note that for the sake of consistency with our ERDM formulation, we reversed the diffusion time order. That is, sometimes described in opposite order in the diffusion literature.
(1) corresponds to maximal (minimal) noise levels, which
SDE formulation. To present the forward and backward processes rigorously, they can be captured via stochastic differential equations (SDEs), Such SDEs ensure that the sample, 2t)), aligns with the designated data distribution, p(æ; G()).. as it evolves through diffusion time [54]. The Elucidated Diffusion Model (EDM) by Karras et al. [26] provides a principled design for diffusion models, typically based on scalar noise schedule (t) (leading to isotropic noise o² The formulation presented here extends these principles to accommodate a general, potentially anisotropic, time- varying noise covariance matrix (tt). Let G(t) = (t) /t be the time derivative of the noise covariance matrix. We assume Gtt) is positive cemindefinite for all t, and let K(t) be any matrix such that ((t) = Git) (e.g., via Cholesky decomposition if Gtt) strictly positive definite, or other matrix square roots). The forward SDE, describing the process of adding noise such that (t) x11 (O Gt)) (assuming (11 = 0, and x=1) ~ Putaa (3)), is: dx (t))
(5)
where (t) is a standard d-dimensional Wiener process. The corresponding backward (reverse-time) SDE is given by [54]:
dx = -())V) logp(æ;; G(t))dt + ((t)dū(t),
(6)
where (t)) is a standard Wiener process in reverse time. The term 'time' t here is a conceptual dimension for the denoising steps. The backward SDE comprises a deterministic drift term related to the score of the perturbed data distribution and a stochastic noise injection term.
Denoising score matching. An examination of the SDE in equation 6 indicates the necessity of the score function, v log p(x; G()),, for sampling. This score function can be estimated using the relationship derived from Tweedie's formula. Given that æ y + 6 where y Pdata (y) and E ~ NO, G(t)), the score is:
Va loggp(a; G()) = (G(I)) G(1) -x)- x))
A denoising neural network, Do(æ: G(t).. is trained to approximate the conditional expectation Blylæ, G(f)] x y. Thus, the score is approximated as (G(t)) (Do(( G(t)) x). The network
is trained by minimizing the following objective, typically by sampling & (and thus G()) according
to predefined schedule or distribution (t) over diffusion times:
min En--raun En (o, [1) + e; [I)
-G())
The denoiser Do is now conditioned on the full noise covariance matrix Gtt..
Sampling. To generate samples from the model, one typically discretizes the backward SDE equa- tion and simulates from = 0 to i s 1. A common discretization is the Euler Maruyama method. Given sequence of diffusion times 0- ITo ~ ~N G()),, the update rule for i = O...., N - 1s:
< TI- <... < TN x 1, and initial sample
Tra. Zr,- (G( - G(T.1)) Va log p(x,,; ; G(T,)) + v G(T)) G(T++))z
(9)
where AG(T) = G(T,) G(T+1) > O, we have used the approximation G(T;) x G(T,) G(T41) (assuming (G is small), and z, - I) is a standard Gaussian noise sample. The term Va jog,( G(T))) is the score estimated using the trained denoiser:
V n logp(=..; G(T.)) = (G(T.) - (Do(z,, G(T,)) x.,))
(10)
24

---

More sophisticated samplers, such as those proposed by Karras et al. [26] (e.g., second-orde Heun or DPM--olver++e can also be adapted for the general covariance case to improve sample quality or reduce the number of sampling steps N. These often involve more complex update rules, potentially incorporating predictor-corrector steps or higher-order approximations the SDE.
EDM preconditioning. Recall the denoiser network parameterization in EDM:
Dy(x;0) Cskip (0) (o(C(()))))) (a))su( (a)iso(
(11)
where a corresponds to data corrupted with Gaussian noise of standard deviation a and Fg the raw neural network trained to denoise æ. One of the key contributions in HDM is the careful choice of preconditioning (Cskip: Cour: Gn) The preconditioning ensures unit variance for the denoiser net- work's inputs and targets. Consistent scales, regardless of noise levels, simplify the task for the denoiser network, which does not need to adapt to changing magnitudes for different noise standard deviations. Mathematically, these functions are defined as:
data
d
Cin V02
Cakip(a)
Com(
(12)
r² + data
V 0² + data
data
The transformation of the noise level is chosen empirically (moise(0) = In(o)/4. In ERDM, we vectorize the preconditioning to function on each snapshot independently, thus ensuring that the same EDM design principles apply to the noisy sequence that ERDM's 's denoiser network ingests.
C ERDM
C.1
Probability flow ODE derivation
Recall that ERDM operated on noisy sequences &L.W of size W, with each C.. E RR corrupted according increasing noise level standard deviations 0 2 (t) < 02(() <... < @t). The noise covariance matrix for the entire window #1.W at diffusion time t is block diack-diagonal:
G window (t) (I ... ow (D)
(13)
Its time derivative is:
Gwindow (t) diag(26 25 (1)o
25w (f) (t)
(14)
Let K window (t) be matrix such that window (t) window (f)T = G Gwiniow (t). Specifically:
K wiiddww (1) diag( 2 (1)
(15)
assuming T. (t) O. The forward SDE for the window ax() is:
dæ
Kwintow (f) (t),
(16)
where (t) is standard W D-dimensional Wiener process. The corresponding backward (reverse- time) SDE, which the primary SDE used for generation, [54]:
dæ = winndow (t) log p( Gwindow (1)) dt Kwindow Kwindow(tt)dau (t).
(17)
The score for the w-th snapshot of the window, Bw. is approximated with the learned denoiser, Dg:
V log log (T:; Gwindow (t)) = (t) (Do - Zw),
(18)
where Do o (f)... e. is the w-th snapshot output of the denoiser Do. which is conditioned on the full noisy window & and the vector of current noise standard deviations (t). The probability flow ODE, which provides a deterministic path for generation. is obtained by removing the stochastic term from the backward SDE and adjusting the drift (see Song et al. [54], Eq. (14) and Karras et al. [26], Eq. (5)):
dæ = G Window Vt) V log (o;;; Gwindow (t) dt.
(19)
Using the specific EDM parameterization where the ODE drift is =D(IV= logp(a; with D(1) = di) ( o (w(()) the ODE becomes:
dæ (t) (()))..........() Va lo (t))dtt.
(20)
25

---

C.2 Sampling
ERDM generates sequences with a sliding window of size W. Generating each window requires solving the backward SDE in equation 17 (or ODE in equation 20). In each iteration & where the SDE is fully solved, ERDM operates on the noisy window Ti:w After solving the SDE backwards
k > 1 times, the first snapshot of the window, [ (with relative window index
1), corresponds
to a forecast of yk. where k is the absolute time step of the forecast time dimension. Solving the SDE requires discretizing it into N₂ diffusion "time" steps, each of size Ale- == -_ In general, NN No
may vary across iterations. To simplify our illustrations of the sampling process in the following, we assume that N₂ and A are identical for all k denoted as and At respectively. This can be achieved by setting N in our sampling algorithms
a positive integer. Note that in practice N need not be an integer (e.g., we use N = 1.25 for our Navier-Stokes experiments), which is accounted for in our sampling algorithms. With this simplification, the SDE evolves, for all k, following
ō (t1) - (t²2) -
(tx)), where f++1 = t/= At for > 1, t1 = O, and ÊN = 1. For example,
N 1 the SDE is solved from (0) to (1) in one sampling step. while N 2 would be solved following (0)) - (12) - (1) in two sampling steps (see illustration in Fig. 1). At the end of the N sampling steps. 01 (1) Omin and the first snapshot the window can be emitted as the forecast for lead time k. =
(k) yv- In the next iteration of N sampling steps, the next snapshot
will be predicted by operating on a new window, ((++1)) Because w++1(1) (.) for > 1, this new window can be constructed by shifting the noisy parts of the old window by one position. That is, Fhe [k++] : w+1 for w < W, and new pure-noise snapshot is appended as Tw (k-1) (0, max I). The new window is thus noised according standard deviations (0) and the process can be restarted and repeated indefinitely, as illustrated in Fig. 1.
(k)
Discretization. As mentioned above, the SDE probability flow ODE need to be solved through discretization over N diffusion steps. We denote the noisy window at diffusion step n E (1,..., ......, N) as d(n). We ignore the superscript & since the ODE discretization is independent of it. Assuming that (0) corresponds to a window corrupted according to a (0), an Euler-Maruyama step for the probability flow ODE and the whole window is
s(n + 1) = I(n)+ + ((n + 1) (nni) [((n) - Do( (nnAt)
(21)
The ODE is thus solved in parallel for all snapshots in the window. When #(N) is reached, its noise levels correspond to (N)t) (11. Thus, the first snapshot is emitted, and the window is shifted as describe above. Higher-order stochastic integrators- e.g. the second-order Heun scheme of EDM [26] can replace
as decribed in the next subsection C.3 below.
Continuous-time view of a tracked snapshot. Focus refined across windows. Let be the rollout step at which x is finalised. The snapshot first appears rollout step le- W (relative position w W) and moves
single physical snapshot
it is
1 at step k. Define Sw : ā (0) as the initial noise level when the snapshot is at position wo, and treat w as a continuous variable the normalised interval [0, 1] (with N O as refinement proceeds). Let zn denote the state of the tracked snapshot when its current noise level is sw. Its evolution is governed by
1 ds dæ,
ds
dw V log p(zwi Swi Cw) dw +
dew:
(22)
dw
where dw < 0 since decreases. The context C.p comprises all other snapshots in the window: The past [0, w) and the future (w, 1]. Note that
do? de > 0, and the score obtained as
V Vz log (Zz0;
Zio Ste, C.o)
(Do(zw5 (ollz551w. Cu Zw),
where Do(zw: 81. C. is the fully denoised estimate of the snapshot at position w. SDE in equa- tion 22 therefore captures how a snapshot is progressively refined as shifts forward through suc- cessive windows, with its effective noise level sw decreasing monotonically.
26

---

C.3 Stochastic Heun Sampler
In the main text, we introduced our proposed deterministic, first-order sampling algorithm (AI- gorithm 2). We now extend this into a more sophisticated sampler, detailed in Algorithm 3, by incorporating two key modifications: a second-order Heun correction step and optional sampling stochasticity via "churn" mechanism. Both components are inspired by EDM's stochastic sam- pling algorithm [26], but require specific adaptations for our ERDM to work.
Heun correction step. The Heun method enhances sampling accuracy by incorporating a correc- tor step over the basic Euler step during the denoising of the data. However, a complication arises because when mext > 1, the first snapshot of E =extt is already fully denoised (i.e., "clean"). Apply- ing the denoiser to such a clean snapshot as part of the Heun step would be inappropriate. To address this, we pad the noisy data by one extra pure-noise snapshot to the end of the sequence, resulting in window of size W + 1. The core denoiser, Do. is then selectively applied on the sub-window of size W, depending on whether the first snapshot is already clean or not. That is, if next >> 1, the denoiser would be applied on the snapshots with indices 2.... ., W + I. Otherwise, is applied, as usual, to the indices 1,..., W. To formalize this selective application. we define De (a. (t)) such that it applies the actual denoiser on all snapshots with indices I {|t] 1...., [t] + W) and leaves all other snapshots untouched. This ensures that the denoiser evaluations within the Heun step confined to snapshots that are indeed noisy.
((()),(())))1p))) if w Zw
D (x: (t))
(23)
otherwise
Churn mechanism adaptation. The second modification is the integration of a "churn" step, which introduces new noise at each iteration to inject stochasticity into the sampling process. Unlike the EDM algorithm, which applies churn at the beginning of each iteration, we position it at the end of each iteration. This change is motivated by the fact that the first snapshots of the very first initial window are noised based on low levels of noise, so that artificially injecting more noise before performing any denoising could be harmful. This contrasts with the EDM setup, where the noisy data at the very first iteration is typically maximally noisy.
Experimental configuration. For the experiments reported in this work, the stochastic Heun sam- pler (Algorithm 3) is always used with the Heun correction step. Our ablations in Appendix E demonstrate that our second-order sampler provides better results than first-order version (Euler- only; Algorithm 2). However, we disable sampling stochasticity by setting the churn rate Saham = O. In our experience, we found that small values of Setur & 0.1 can increase the ensemble spread, albeit it had minimal impact on error metrics such as the CRPS. We defer more comprehensive analysis of stochastic sampling to future work. Our algorithm can, in principle, be used to fully denoise multiple snapshots at once (e.g., output two snapshots per iteration when N 4). This could be useful in certain modalities or problems where subsequent snapshots are highly correlated, but is left to future work too.
C.4 Temporal noise prior
In our training and sampling algorithms (Alg. 1-3), we use i.i.d. noise to corrupt the first window as well as append new pure-noise snapshots when sliding the window. As mentioned in the main text, it has been shown to be useful to sample temporally correlated noise snapshots. In our experiments, we use the simple "progressive noise model" technique proposed by Ge et al. [18], which we found to improve results on the Navier-Stokes dataset compared to using i.i.d. noise (see Appendix E). This noise prior generates the noise for each snapshot autoregressively by combining a perturbed version of the previous noise with a new i.i.d. noise sample. Mathematically, the noise for the first snapshot random, (1)) NO I). while for the following snapshots k
(k-1)
ind
(R)
End
(k)
N(O,
1 1+ I),
1
(24)
where controls the correlation strength between snapshot noises and a 0 corresponds to i.i.d. noise. In our experiments, we use a = following the recommended practice in [18). These temporally correlated noise samples replace the i..dd draws in lines 6 and 27 of Algorithm 3. For
27

---

Algorithm 3 Elucidated Rolling Diffusion Stochastic Heun Sampler
1: Require: gintt N, TToorcasss Saham
0, Snaiee =
> Choose Sewm 0 for sampling stochasticity
2: # Initialization
3: Al <- 1/N
Infer step size from given number of steps per snapshot Size of larger. initial step, before backtracking (if Saimm 0) to A Initialize global diffusion time and empty generated sequenec to Initialize window with 7: concatenate Ppdd == |At'] fully noisy snapshots to the end of Tour
4: A/' A1/(1 Schmm
5: four
S Ø
6: sample Heur ~ N Nyiw o (feur) (IXD))
rolling noisc >- Necessary for Heun step p- Predict snapshot |S|+1 Global diffusion time after denoising, before chum >> Global diffusion time after churn (fnext & (eex)
8: while [S] Tirecaes do
9: fuext terr At'
10: fnext < leur
11:
Our - ō (teut)
t- Current noise levels t- Noise levels before churn t- Noise levels at the end of this iteration
12:
next - o (Inext)
13:
next o (Inext)
14:
Fuler step
15:
y Do (Feur, ceur)
Denoise sequence Evaluate /æz/d at feu- Euler step from teur to L
16:
d - (Nour - D)/
I'eett Deus (o c d
18:
Heun step (rdd- correction)
19:
d' (= DD (Tx desss ))/ / next
Denoise at Erext t> 2dd order Heun correction step
20:
F'eext < Beur { our (d d')
21:
# Churn step (stochastic backtrack)
22:
sample E ~ ~NO IwDD)
> Sample churn noise
23:
Bnextt F'exxt + Vo
next o
Backtrack neex noise levels
24:
Sliding-window shift and forecast snapshot extraction to Infer finished snapshots. 0, the following lines are a no-op
25:
Relean - [taext]
26: add first Pelean snapshots of ÿ to S and discard first Relean smapshots of Znext
27:
sample Znew NO0 0 mnxxnnt
28:
Beur [Tnext; #new]
Concatenate fresh noisy snapshots to futuremost t- Re-adjust global diffusion 'time" to be in [o. 1)
29:
feur fneett 7elean
30:
return
example, line 6 becomes Heur ginit + (feur)) where <1-W (e(")...., (W)). Note that the noise prior is applied during both training and sampling, enabling the model learn how to deal with temporally correlated noise samples. A more comprehensive study of alternative noise priors in the context of ERDM is left to future work.
D Experimental Details
D.1 Dataset details
Licenses. We use the Navier-Stokes dataset with four obstacles introduced by Otness et al. [41]- Its license is CC BY 4.0. We also use the ERA5 dataset provided by Weatherhench-2 [46]. Its license is CC BY 4.0.
ERA5 variables. We use the 1.5 resolution of the data (a 240 121 grid) provided by [46], with 69 prognostic (input and output) variables: Temperature (t), geopotential (z), specific hu- midity (q). and the u and components of wind (u, v) over 13 pressure levels as well as the four surface variables 2m temperature (2t), mean sea level pressure (malp), and u and v components of winds at 10m (10u, 10v). Following Weatherbench-2, the 13 levels are (50, 100, 150, 200, 250, 300, 400, 500, 600, 700, 850, 925, 1000). Numbers after any variable ab- breviation refer to the level in hPa. For example, z500 refers to geopotential 500 hPa. Besides these prognostic variables, we also use six static maps as additional inputs: A land-sea mask, soil type, orography, and sin(lat), cos(lat) (af) cos(lon), (as(lat))e cos(lat)si((on)) where lat, lon are the latitude and longitude of each location. We train our models on 12-hourly data, as in [45], from 1979 to 2020 and evaluate the models based on 64 initial conditions at 00/12 UTC, evenly spaced in 2021.
Sizes. The Navier-Stokes training set consists of around 6100 samples, and the corresponding validation set consists of 118 samples (slightly less for W > 1). We use the ERA5 training data
28

---

set at 12-hourly resolution with possible initial condition times in 00/06/12/18 UTC. The training data size is thus around 61000 samples. Following Weatherbench-2. we only use 00/12 UTC start times for evaluation. We use 64 samples for evaluation. During validation, we use ensemble sizes of M 10 and M = 5 for Navier-Stokes and ERA5 respectively, which we increase to M 50 and M = 10 during final testing.
D.2 Implementation details and hy hyperaarameters
Architecture. For all experiments and baselines that we train, we use the same basic ADM U- Net [13] architecture, only modified where necessary depending on the method. We always use 4 down- and up-sampling blocks, with channel multipliers 1, 2, 3, The base channel dimension is 64 for Navier-Stokes and 256 for ERA5 experiments. We use attention layers at the two lowest spatial resolutions. We use a dropout rate of 0.15 for Navier-Stokes, but disable it for ERA5. Other architectural hyperparameters are kept to the defaults from EDM. This results in a parameter count of 29.5 million for Kavier-Stokes, and 517 million for ERA5. For the models operating on sequence, ERDM and EDM with W > 1, we integrate causal temporal attention layers into the 2D U-Net before each down- and up-sampling block. This increases the respective parameter counts slightly to 31.5 and 537 million for Navier-Stokes and ERA5, respectively. For ERA5, we use circularly padded convolutions along the longitude dimension (and zero- vero-padding for the latitude dimension) to better respect the spherical structure of the Earth.
Navier-Stokes hyperparameters. We train each model for 300 epochs with a global effective batch size of 32, leading to almost 57, 000 gradient steps. The effective batch size is kept the same by balancing Per-GPU batch size and gradient accumulation steps appropriately over different training runs. We use a cosine learning rate schedule with a linear warmup period of 10 epochs and peak learning rate of x 10 We use the Adam optimizer with a weight decay of 10 During evaluation,
on an EMA decay rate of 0.995- version of the model. Because the input spatial grid, 221 x 42, after four halvings does not result in even number-a problem for the residual connections in the upsampling blocks-we bilinearly interpolate it to 256 x 64 grid at the input layer (and back before the final convolution layer).
an exponential moving average (EMA)-basee
ERAS hyperparameters. We use cosine learning rate schedule with línear warmup period of 5 epochs and a peak learning rate of x 10-1 ERDM and EDM are trained with a global effective batch size of 32 and 512, respectively. The difference in this choice was informed by how many data points we could fit into GPU memory (ERDM has higher memory needs due to operating on window of data) and the observation that EDM benefited more strongly from higher effective batch sizes. The maximum length of the schedule is 60 epochs for ERDM and 1000 epochs for EDM, resulting in a comparable number of total gradient steps. We use the AdamW optimizer without weight decay, clipping gradients above norm of 0.8. During evaluation, we use an EMA version of the model, with an EMA decay rate of 0.9999. Because the input spatial grid, 240 x 121, after four halvings does not result in an even number-a a problem for the residual connections in the upsampling blocks-we bilinearly interpolate it to a 240 x 128 grid at the input layer (and back before the final convolution layer).
D.3 Baseline details
DYffusion. We follow the default values used by DYffusion for its Navier-Stokes experiments, but replace the interpolator and forecaster architecture with the more performant U-Net and use our optimization hyperparameters, both described above. We tuned the interpolator dropout rates and found rate of 0.5 to produce the best results. The interpolator dropout rate is key hyperparameter in DYffusion, since it is responsible for the stochasticity in the model (if O, it reduces to a deter- ministic model). As for ERDM, we use a window size of W 6. Our improved architecture and training hyperparameters jointly improved the DYffusion CRPS scores by over 3x compared to the results in the original paper.
PDE-Refiner PDE- Refiner [32] is a next-step diffusion-derived forecaster. We tune the two key hyperparameters of PDE-Refiner, K and min- The best forecasts were obtained with K 6, a 0.002. The recommended values from [32], K = 4, amin 2e-7, did not work as well. We use the same denoiser architecture as for EDM. PDE-Refiner performs extremely well for the first few
29

---

timesteps of the Navier-Stokes rollout, but rapidly diverges after that. We believe this is because it cannot model temporal interactions.
Our primary focus is to evaluate ERDM's performance relative to common EDM-derived approaches for dynamics forecasting. Consequently, our key baseline is an EDM denoiser parame- terized by Do(x; .... æw; a, Mo), where all zw are corrupted according to the same noise standard deviation, a. With W = we recover next-step forecasting conditional EDM as in [45, 29]. On Navier-Stokes, we tried W {1.2 2, 4, 6], and found W 4 to work best for EDM. For compu- tational reasons, we only report an EDM run with = for ERA5. network the initial condition, yo- concatenate along the channel dimension to the input, noisy window, Ti.W. When W > 1. Yo is thus first duplicated along the window dimension. We found this to perform better than operating position of the window. Preconditioning is applied before concatenation, only to the noisy window.
EDM.
condition the denoiser
W + 1 window, where yo is inserted into the first
To be clear, the EDM "video" baseline with W > 1 does not use rolling diffusion. It is a standard sequence-to-sen tossuuenceee EDM, using the same 3D denoiser architecture as ERDM, that operates au- toregressively. Conditioned
a single clean frame (e.g., at time l), it jointly predicts a window of future frames (L. + to 1+ + W). Unlike ERDM, all frames this window are denoised from the same initial noise level. To generate long forecast, the last frame of the predicted window (at time + W) is then used as the new "clean" condition for the next prediction step.
D.4 Distinctions between GenCast and our EDM model for ERAS
Our primary baseline, conditional EDM, applied to ERA5 shares several similarities with Gen Cast [45]. A primary distinction lies in the spatial resolution: GenCast trained on spatial res- olution of 1 and 0.25%, whereas our model, EDM, and other baselines are exclusively trained at 1.5 resolution. The denoiser network architecture presents another significant divergence. GenCast uses graph neural network, similar to the approach in Lam et al. [30], while we use 2DD U-Net for our EDM baseline borrowed from [26, 13]. Furthermore, GenCast incorporates SST as both an input and an output variable. surface temperature as an input and output, which we do not. More minor differences between GenCast vs. our EDM are: ODE solver (DPMSolver++ vs. Heun), training noise level distribution (uniform vs. lognormal), noise distribution (spherical vs. i..dd), optimization hyperparameters, and residual vs. full-state prediction.
D.5 Compute resources
For Navier-Stokes experiments, we trained all models and baselines on 2 - 8 L40S or A100 GPUs. For ERA5, we trained some models (including our final ERDM version) on 4 H200 GPUs, but performed most of our development work on 8 - 16 A100 GPUs (per training run). For ERDM with W = 6 (W = 4), a full training run on L40S GPUs took around 16 (10.5) hours. For ERA5 with W 6, a full training run on 4 H200 GPUs took almost days.
D.6 Metries
Let y E IxJ indicate the targets for specific time step, and ý E M the corresponding predictions with an ensemble size of M. I and J are the size of the height dimension and width dimension, respectively. For ERAS, these are latitude and longitude, respectively. The definitions of the metrics below follow common practices (e.g., ef. Weatherbench-2 [46]).
Weighting. For ERA5 experiments, we follow the common practice of area- weighting all spatially aggregated metrics according to the size of the grid cell, ensuring that they are not biased towards the polar regions [46]. The unnormalized area weights are computed as w(() = sin oN sin df. where di and di represent upper and lower latitude bounds, respectively, for the grid cell with latitude index i E (1,2,... 73. The normalized area weights, used in all metrics below are thus (6) w(i)- S a(t)* where I is the number of latitude indices. For the Navier-Stoker experiments, we report unweighted spatially aggregated metrics (i.e., w(i) 1 for all i).
Ensemble-mean RMSE. In basemble-based forecasting, the RMSE typically computed based on the ensemble mean prediction as follows:
30

---

RMSEens
w (maan (9m.s.) yt.))²
(25)
where meall refers to the average over the ensemble dimension.
Spread-skill ratio (SSR). Following Fortin et al. [16], the spread-skill ratio is defined the ratio between the ensemble spread and the ensemble-mean RMSE. The ensemble spread is defined as the square root of the ensemble variance. Thus, the SSR is defined as:
1 Σ w(i) (mm..))
M + Spread
Spread
SSR V M RMSE
(26)
where varm refers to the variance over the ensemble dimension, and v (M + 11/M is a correction
factor which is especially important to include for small ensemble sizes. The SSR serves simple measure of the reliability of the ensemble, where values closer to are better. Values smaller (larger) than indicate underdispersion (overdispersion). In our ablations, we also report the average squared deviation, (1 SSR)² (lower better), where the overline indicates averaging over all rollout/lead times. This is more useful than reporting SSR, since such a metric would be sensitive to canceling out deviations from I on both sides (e.g., an SSR of 0.5 and 1.5 on the first and last half of the rollout, respectively, would average 1).
Continuous ranked probability score (CRPS). Following [59, 46), we use the unbiased version of the CRPS [37] (lower is better)
M Σ
CRPS TJ Σ w(i
1
Σ
9 Mm
Vis
Σ
Am... Doniol
(27)
M
2M 1)
CRPS skill score (CRPSS). The CRPSS is commonly used to compare the CRPS scores relative to baseline. It is computed
CRPS CCRPS where CRPS and CRPS refer to the CRPS scores for the evaluated model and baseline, respectively. Values larger than indicate that the model is better than the baseline model in underscore, while values smaller than indicate that the model is worse than the baseline model.
E
Ablations
Starting from our proposed ERDM configuration ("Base" in Table 2), which uses the second-order Heun sampler, loss weighting with Pmean 0.5, Pad 1.2, temporally correlated progressive noise prior following [18] with frog == a 1 (see Appendix C.4), noise schedule parameters Tmin 0.002, Tmax 200, p 10, and window size of W 6, we systematically varied individual components. For sampling, we use N = 1.25 (Ar 0.8) and Setar = by default. The first window initialized based on predictions from the EDM W 4 baseline. Our findings, summarized in Table 2, highlight several critical elements for achieving optimal performance.
Heun versus Euler sampler. Compared to the first-order Euler method (Algorithm 2), the second- order sampling algorithm detailed in Algorithm achieved superior performance in CRPS, MSE, and SSR metrics. This improvement comes at the cost of doubling the neural function evaluations due to the correction step, which significantly reduces sampling speed. Nevertheless, we deemed this increased computational overhead justifiable for the enhanced accuracy.
Number of sampling steps. Our base ERDM model uses N 1.25 sampling steps "per snap- shot", which results in step size of At 1/N = 0.8. That is, depending on the noise levels at sampling steps before outputting the
the start of an iteration for solving the ODE, may use first frame and sliding the window. Increasing N extends sampling time, but optimal performance found at "sweet spot": for example, N 1.25 outperforms N for Stoker.Stoke.. This was also confirmed for ERA5, where CRPS scores showed minimal improvement for N > 2 (with an optimum around = 2). The parameter is also crucial for tuning the model's smaller N increases spread, larger N reduces it itethereby providing control for adjusting the spread-skill ratio. Ultimately, the ideal N and step size depend on the dataset complexity and the window size, where larger window sizes generally tolerate larger step sizes (smaller N).
31

---

Table 2: Ablation study results on the Navier-Stokes dataset. The most important design choices, without which ERDM would degrade to worse performance than the EDM baseline (highlighted in rod), are 1) proper 3D architecture; 2) appropriate noise schedulc (in particular, weighting; 4) a fixed training schedule. CRPSS refers to the CRPS skill score, where > means that the model better than the baseline model in underscore. CRPSS values smaller than 0 indicate that the model is worse than the baseline model. For all other metrics, lower is better.
EDM's default p 7); 3) our proposed loss
Ablation
CRPS x 10a
MSE x 10a
CRPSSERDM
CRPSSEDM
SSR)
Base
0.904
0.588
0.000
0.432
0.052
Sampling Euler Sampler
1.160
0.752
-0.279
0.288
0,227
At
1,270
0.810
-0.399
0.224
0,343
At 0.9
1.067
0.823
-0.160
0.349
0.056
At 0.7
0,986
0.668
-0.084
0.389
0.080
At 0.5
1.042
0.648
-0.159
0.349
0.121
Loss Weighting
No f(o)
2.168
2.987
-1.296
-0.243
0.244
P Pmeaa
1.082
0.783
-0.181
0.338
0,057
P Pmeaa 0
1,431
1.517
-0.489
0.177
0.119
P Pmean 1.22
2.001
2.694
-1.086
-0.134
0.259
Noise Prior (c)
Gndep
1,176
0.977
-0.240
0.309
0.134
Grog-=
1,060
0.715
-0.151
0.357
0.065
6prog=10
1,404
1.353
-0.474
0.190
0.098
Noise schedule Random training sched.
1.904
1.998
-1.065
-0.130
0.347
Omin = 0.001
1.040
0.758
-0.102
0.383
0.062
Min 0.01
1.511
1.342
-0.692
0.052
0.219
O maax 80
1.094
0.878
-0.175
0.346
0.080
maxx 800, 20
1.249
1.086
-0.355
0,244
0,056
P = -5
1,529
1.569
-0.615
0.121
0,189
P -20
1.060
0.695
-0.180
0.339
0.045
P = 7
1.982
2.382
-1.273
-0.265
0.098
Window Size (W)
W =
1,411
1.308
-0.487
0.185
0.104
W 8
1,258
1.134
-0.313
0.276
0.196
W 12
1.709
1.841
-0.765
0.041
0.422
Architecture
Uncond. Pre-training 2D arch.
1,475
1.732
-0.528
0.157
0.074
3,931
2.979
-3,202
-1.209
3.476
Initialization
Init=EDM W =
1,176
0.977
-0.240
0.309
0.134
Intt-persistence
14.789
172.233
25.361
-20.412
0.765
Init=truth
0.899
0.593
0.053
0.492
0.286
32

---

Loss weighting. As detailed in the main text, well-tuned loss weighting can significantly im- prove results. In particular, using no loss reweighting based on the PDF of the lognorma distri- bution, f(a; P mem:: Pua) ("No [") degraded CRPS and MSE scores by more than 2x and 5x, respectively. It also resulted
worse calibration score. Similarly, choosing poor value for the center of the lognormal distribution, e.g.. maan = 1.2, resulted in almost suboptimal results. Interestingly. I' mean - 1.2 corresponds to the default value in EDM, but we found values > 0.5 to be necessary for ERDM. We found EDM's default Pad = 1.2 to work well for ERDM generally.
Noise prior. As hinted in Appendix C.4, we found it beneficial to use a temporally correlated noise prior as opposed to simply sampling i.i.d. noise for each snapshot ("Gindep") "). The noise prior that we use [18] introduces one hyperparameter, a, which balances the strength of the correlation. We generally found a [0.5, 1.5] to work best. Similarly to the findings of Ge al. [18], using overly
large (e.g., 10, which introduces large correlations between noise snapshots) is detrimental to performance and worse than simply using i.i.d. noise.
Other design choices. Surprisingly, unconditional static pre-training of the non-temporal com- ponents of the model before fine-tume tunitt on the forecasting task was found to degrade performance ("Uncond. Pre-training"), Preliminary ERA5 experiments seemed to confirm this finding. More- over, we did not find ERDM to be overly sensitive to the window size within reasonable range (W 6 as base). However, overly large window sizes (e.g,, W 12) significantly degrade per- formance, potentially due to fixed network capacity. Note that for each window size, we tuned the sampling step size from At (0.25, 0.5, 0.8, 1} and report the best results only.
E.1 ERDM initialization
Init=EDM W
Init=EDM W 11
Init=truth
0.015
3
CRPS
0.010
5SR
2
0.005
0.000
20
40
60
20
40
60
Timestep
Timestep
Figure 7: Impact of first-window initialization methods on FRDM's Navier-Stokes forecast performance. The chosen initialization strategy significantly affects short-range skill (up to timestep 10), but its influence di- minishes over longer forecast horizons. While initializing with ground truth data (an impractical approach) unsurprisingly yields the best initial results, it also produces an artificially inflated Spread-Skil Ratio (SSR) due to a near-zero initial RMSE. Notably, this early advantage dissipates throughout the rollout, leading to final CRPS and SSR scores that are largely indistinguishable regardless of the initialization method.
In Fig. 7 and the last part of Table 2, we ablate possible initialization schemes for ERDM. These in- clude neural initializations based on the EDM baselines with W 1 and W (our default for Navier-Stokes being EDM W = 4), alongside two simple non-neural approaches. One non-neural method, Init=truth, initializes ERDM using ground truth data for the first window. While clearly impractical, it serves as an informative upper bound on the achievable performance of ERDM. The other, Ini==perisstent sitencee, populates the first window by duplicating the initial condition yo: this ap- proach performed very poorly and is omitted from Fig. 7 for visual clarity. Expectedly. Init=truth yields optimal initial CRPS results but cannot be used in practice. More significantly, Fig. 7 demon- strates that the choice of initialization predominantly impacts ERDM's short-range forecast skill (up to approximately timestep 10), The initial performance advantages conferred by methods like Init=truth, or differences among other practical schemes, diminish substantially as the forecast horizon extends. By the end of the rollouts, both CRPS and SSR scores become largely indis- tinguishable across the various practical initialization methods, including our default EDM W
33

---

ERDM 2021
EROM 2022
1850
2t
t50
10u
0.0
12.5
0.6
-0.5
10.0
0.0
-1.
-0.5
CRPS ERDM 2021 [%]
2
-25
2.5
-1.5
0.0
2300
mslp
9700
91000
0.50
-5
-2
0.28
0.00
-5
-5
-0.75
1.0
100
100
200
300
Lead (h)
Lead (h)
Lead time
Lead time
Figure 8: Impact of ERA5 evaluation year on ERDM's CRPS performance. The model, trained on data up to 2020 (inclusive), generally shows improved CRPS scores when evaluated on initial conditions from 2022 compared to 2021 (the baseline year used throughout this paper), with t50 being an exception. While most variables exhibit CRPS changes of less than 3%, geopotential and more sensitive to evaluation year, particularly for times under days (e.g.. z500 scores improve by up to 15%). This variability underscores the need for careful interpretation when comparing model results from different evaluation years, especially for geopotential and stratospheric fields.
level pressure significantly
initialization. This convergence suggests that while the first-window initialization is crucial for early forecast accuracy, its specific choice becomes less critical for the long-range performance of ERDM (except when using a naive initialization such as persistence,
E.2 ERA5 evaluation year ablation
In our quantitative comparison, we evaluated ERDM, EDM, and IFS ENS on the same 64 initial conditions evenly spaced in 2021, ensuring fair comparison. Unfortunately, only have cess to the NeuralGCM ENS and Graph-EFM results for 2020. This discrepancy highlights the importance of understanding how model scores might vary across different evaluation years. We investigate this for our ERDM model (trained on data up to 2020 inclusive) in Fig. 8. As detailed in the figure and its caption, ERDM's CRPS performance indeed shows sensitivity the evaluation year. For example, when evaluated on 2022 initial conditions, scores often improved relative to our 2021 baseline, though for most variables this change notably geopotential (with 2500 improving by up to 15%) and mean sea level pressure, exhibited significantly greater sensitivity, particularly lead times under days. These findings suggest that
less than 3%. However, certain variables,
while our 2021-based evaluations provide a consistent benchmark for ERDM, EDM, and IFS ENS, direct comparisons with results from NeuralGCM ENS and Graph-EFM on 2020 data should be made with an awareness of this potential inter-annual performance variability.
tRUM
HDM W
EDM W
DYfiuusion
Additional Results
0.008
F1 Additional Navier-Stokes results
0.006
MSE scores. For completeness, we report the ensemble-mean MSE test rollout scores for all Navier- Stokes models in Fig.
MSE
0.004
analogously to our main CRPS and SSR results in Fig. 4. The MSE scores are strongly correlated to the CRPS scores, which why we deferred their discussion to the appendix. Similarly the CRPS scores, ERDM achieves significantly bet- ter MSE scores, especially for long-range lead times.
0.002
0.000
20
40
60
Timestep
Figure 9: Ensemble-me mean MSE scores for the Navier-Stoke test rollout.
34

---

EROM
IFS
SPM
[
[]
(m/s]
1.75
1.2
1.50
0.8
1.0
1.25
1.00
0.6
0.8
1.00
0.75
0.20
0.6
0.50
CRPS
0.25
0.2
0.4
0.25
[m]/s²]
mslp
q700 (kg/kg]
(kg/kg)
300
3.0007
900
0.0037
250
250
0.0006
3.0008
200
200
3,0008
160
0.0004
100
100
0.0003
0.0032
3,0002
100
200
300
100
200
300
100
200
300
100
20C
300
Lead Sime
Lead time (h)
Lead time (h)
Lead time
Figure 10: Absolute CRPS corresponding to the relative scores shown in Fig. 5. Lower is better.
slight difference to the CRPS results is that the relative improvement of the block-wise autore- gressive EDM W 4 baseline over the DYffusion baseline is larger in terms of MSE than CRPS.
F.2 Detailed ERA5 quantitative results
A comprehensive quantitative assessment of forecast skill is presented in Fig. 11, which displays CRPS for ERDM, IFS ENS, NeuralGCM, and Graph-EFM, relative to the EDM baseline. These relative CRPS values are reported as percentages (where lower indicates better performance) for numerous atmospheric variables (extending set shown in Fig. 5) over the full 15-day forecast horizon. These results underscore that ERDM consistently achieves significantly lower CRPS values than the EDM baseline. The corresponding absolute CRPS scores are visualized in Fig. 10, and selection of them summarized in Table 3. A full relative comparison between ERDM and EDM presented the form a scorecard in Fig. 13. A quantitative assessment of forecast calibration presented in Fig. 12, which displays the spread-skill ratio for the same of models and variables. ERDM generally generates well-calibrated forecasts on par or better than IFS ENS, and consistently more calibrated forecasts than the EDM baseline NeuralGCM.
Table 3: Tabular CRPS scores for selection of forecast horizons (three, seven, and ten days) and variables (10m u-component of wind and 500 hPa geopotential). Lower is better.
u10m
z500
Model
3d
7d
14d
3d
7d
14d
EDM 0.6923
1.324
1.769
76.04
204.9
327.4
ERDM
0.6748
1.312
1.737
69.29
197.5
316.7
IFS ENS
0.6830
1.318
1.785
58.41
183.4
317.3
NeuralGCM
N/A
54.59
173.1
311.1
F.3
Comprehensive ERA5 power spectra
To assess the physical realism of the generated forecasts, we analyze their spectral properties, as presented in Fig. 14. This figure shows the spectral density of 14-day forecasts for several key atmospheric variables, averaged over high latitudes ([60", ,90").. The absolute spectra (subfigure (a)) and spectra normalized by ERA5 reanalysis (subfigure (b)) consistently demonstrate the high fidelity of ERDM. Our model's spectra closely align with the ERA5 target across broad range of zonal wavenumbers and frequently match even slightly outperform those from the operational
35

---

ESM
BROM
NarabON
Oripf 6PM
2t
10u
10v
mslp
-10
-10
41000
u1000
20
800
12
10
10
CRPS vs EDM (SS
1850
z050
9850
u$50
10
-10
-10
-20
-20
1700
2700
4700
u700
-20
-20
9500
20
-10
-10
-20
-20
100
100
250
200
Figure 11: CRPS relative to next-step autoregressive EDM baseline (in %; lower better) for 15-day rollouts across a more comprehensive set variables than Fig.
EDM
ERDM
IFS ENS
...
Nc.rFOCM
2t
10v
mslp
1.0
0.8
t1000
z1000
g1000
u1000
1/2
10
1.0
0.5
0.8
0.8
t850
z850
q850
uB50
1.25
1.00
0.75
0.75
1700
z700
q700
u700
1.00
1.00
0.75
0.75
0.75
t500
z500
q500
u500
1.00
1.00
0.75
0.75
200
Lead (h)
Load (n)
Lead
Spread Skill Ratio (SSR) computed as the ensemble spread ensemble-mest mean RMSE based on 10- member ensembles for each model. Closer to 1 is better and represents well-calibrated ensemble. Among the ML-based methods, ERDM tends to produce the best calibration in terms of SSR, together with IFS ENS, except for the first few lead times of z500 and 2700 where is overcalibrated. NeuralGCM and EDM tend be severely underdispersed, while IFS ENS tends to slightly overdispersed, the short range.
Figure 12:
36

---

mslp
10.0
100
5.0
200
2.5
DOURARUUO %
Pressure (hPa)
0.00
300
-2.5
500
700
-7.5
925
-0.00
3
10
15
3
10
15
10
15
Lead time (days)
Lead time (days)
Lead time (days)
Lead time (days)
Figure 13: Scorccard of ERDM's CRPS relative next-step autorcgressive EDM bascline (in %; lower is better) for all predicted variables (except v. which is correlated with u), pressure levels, and lead times (up 15 days). ERDM consistently outperforms the EDM baseline, especially for geopotential high-altitude levels.
physics-based IFS ENS model. This accurate representation indicates that ERDM effectively cap- tures the energy distribution from large planetary scales down to smaller synoptic scales for diverse atmospheric fields. In contrast, NeuralGCM ENS exhibits marked underestimation of energy at mid to high frequencies for several variables, evident by its spectra decaying more rapidly in the absolute plots and falling below unity in the normalized plots these scales. The robust spectral characteristics of ERDM across multiple variables underscore its capability to produce physically consistent and realistic long-range weather forecasts.
F.4
ERA5 forecast visualizations
See Fig. 15 and Fig. 16 for visualizations of example forecasts by ERDM for specific initial condition (2022-01-01) and several lead times. As suggested by the good power spectra (see section above), ERDM produces realistic forecasts, including at long lead times (c.g., 15 days).
37

---

10
10
10"
10
10"
10
10"
10
10
10²
16
10
Spectral density
10*
10'
10
mslp
4700
g1000
10º
16a
16
10
10
10
16
10
10
10"
10" -
10
10
10"
10" 4
10
10
10
10
10
10
10'
10
16
10'
10"
10
10"
10
wavenumber
wavenumber
wavenumber
(a) Absolute spectral density.
1850
10u
10
1.0
1.5x 10
141
10
Normalized spectral density
105
10
10
10
10
16
90"10"
z500
mslp
q700
q1000
10"
10a
10"1
10'
10
10
10"
10
wavenumber
wevenumber
(b) Normalized spectral density (relative to ERAS reanalysis).
Figure 14: Spectral density of 14-day forecasts, averaged over high latitudes ([60", 90"D) ERDM generates highly accurate spectra that match or even slightly beat the physics-based model IFS ENS, while NeuralGCM ENS underestimates energy at mid to high frequencies. Subfigure (a) displays the absolute spectral density, and subfigure (b) presents the normalized spectral density (absolute spectra divided by the target ERA5 spectra).
38

---

2m Temperature 2022-01-01
Truth
EROM
Errer (Pood Thuth)
(a) 2 meter temperature (2t).
10m U-Wind Component Init. time: 2022-01-01 00:00 UTC
34
10d
15d
Truth
EALOM
Error (Pred Truth)
(b) 10m component of wind (10u).
Figure 15: Example visualizations of FRDM (second row), the corresponding ground truth (first row), and the hias (last row) for example variables and forecast lead times 3, 10, and 15 days.
39

---

Geopotential 500hPa Inlt. 2022-01-01 00:00
IRON
redd Trutth
(a) Geopotential at 500 hPa (z500).
Specific Humidity at Init. time: 2022-01-01 00:00 UTC
Trusth
ERDM
Eroor (Pred Trutth)
-30M
(b) Specific humidity at 700 hPa (g700).
Figure 16: Example visualizations of ERDM (second row). the corresponding ground truth (first row), and the bias (last row) for two example variables and forecast lead times 3, 10, and 15 days.
40


Reviews:
[{"review_id": "UtKv4CDCYd", "reviewer": "Reviewer_yaoq", "summary": "This paper introduces the Elucidated Rolling Diffusion Model (ERDM) that incorporates the general framework of Rolling Sequence Diffusion Models and design choices in EDM for spatial temporal probabilistic forecasts. In addition, the authors propose an uncertainty-aware weighting scheme, leverage pretrained next-step EDM models to initialize the first forecast window and design a hybrid 2D U-Net with causal temporal attention to capture spatial and temporal dependencies. ERDM outperforms baseline models on 2D Navier Stokes simulation and is competitive against state-of-the-art models on ERA5 data. With improved performance and more efficient training, ERDM promises accurate and skillful forecasts for other spatiotemporal forecasting problems.", "questions": "* Since the authors mentioned the inferior performance of the ERDM is due to the EDM prediction initialization, could the authors replace the current initialization with IFS ENS or  deterministic model (e.g. FourCastNet)  and repeat the experiment? to close the gap\n* The noise level for the first snapshot within the window is small, does this mean it takes fewer solves to fully denoise the first snapshot? Does this have any implications of the number of NFEs compared to other models? Also, have you profiled whether the 3 D attention layers or memory bandwidth dominate? An ablation with a lighter temporal backbone would clarify where to optimize.\n* How does the choice of window size and accordingly $\\sigma_{\\rm min}$ and $\\sigma_{\\rm max}$ affect the performance? \n* What is the energy cost per 15 day ensemble forecast relative to IFS ENS, measured on the same hardware? This would strengthen the efficiency claim and inform practitioners.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* The proposed ERDM improves CRPS on the Navier Stokes problem compared to the best EDM and DYffuison models. Also, the ERDM is competitive with IFS ENS and NeuralCGM model with improved training efficiency.\n* ERDM captures the temporal dependency and automatically account for the horizon uncertainty with the progressive noise schedule, leading to better long-term rollout.\n* During sampling, the progressive noise schedule potential enables fewer solve steps than GenCast like model. After each denoised first step, the reuse of the partially denoised tail naturally moves the forecasting forward.\n\nWeakness:\n* The short-range performance on ERA5 data is inferior to IFS ENS and NeuralCGM. The authors attribute this to the less optimal initialization of the sequence. However, this needs further experiments to validate.\n* It is unclear how the window size W affect the performance. Accordingly, the choice of $\\sigma_{\\rm min}$ and $\\sigma_{\\rm max}$ needs more discussion.\n* the work demonstrates that diffusion models can yield calibrated, medium range probabilistic weather forecasts at modest compute cost, suggesting relevance to climate down scaling, oceanography, and other chaotic systems; however, the iterative sampling still imposes latency comparable to the EDM baseline, limiting immediate operational adoption.\n* Uses existing building blocks (EDM, Heun, causal attention); conceptual novelty is incremental", "was_revised": true, "final_justification": "I will keep my score and leaning towards acceptance. However, the methods' performance, especially in short-range skill seems to be sensitive to the initial condition but that is not evaluation and solution is address this is not provided, which can limit the applicability to other problems/areas.", "rebuttal": "Thank you for your thoughtful and constructive review, and for recognizing our model's improved performance over strong baselines, as well as its ability to effectively capture long-range dependencies through an efficient sampling process.\n\n* **W1:** The short-range performance on ERA5 data is inferior to competitors, and the provided explanation \"needs further experiments to validate.\"\n\nThank you for this excellent point. We agree that our short-range performance on ERA5 data merits further discussion and that suboptimal initialization is likely only one contributing factor.\n\nWe believe a more significant factor is the denoiser architecture. For both the Navier-Stokes and the ERA5 experiments, we used the same general-purpose 3D U-Net. Our method performs extremely well on the simpler Navier-Stokes problem, where designing a bespoke architecture is not as critical. However, for a complex system like ERA5, state-of-the-art performance is increasingly tied to specialized architectures (e.g., graph networks or transformers) that carefully model the spherical structure of the Earth, and its various short- and long-range dependencies. The focus of our paper was on introducing the **ERDM framework** for sequential probabilistic forecasting, rather than designing a new SOTA *architecture* for weather.\n\nWe are confident that future work extending current SOTA architectures like GraphCast, Aurora, or Pangu-Weather to ingest temporal data, for use within our proposed ERDM framework, would significantly advance weather forecasting performance. We will clarify this distinction in the revised manuscript to better contextualize our results.\n* **W2:** It is \"unclear how the window size $W$ affect the performance,\" and the choices for $\\sigma_{min}$ and $\\sigma_{max}$ require more discussion.\n\nThank you for raising this important point. We have performed an ablation study on the window size $W$, $\\sigma_{min}$, and $\\sigma_{max}$, with the results presented in Table 1 of the Appendix. We agree that these findings should be highlighted more clearly.\n\nThe ablation shows no significant sensitivity to these hyperparameters within a reasonable range. We only observed a degradation in performance for overly large window sizes, which makes the current backbone architecture less efficient for feature extraction. In our revision, we will add a sentence to the main text to explicitly summarize this and direct the reader to the detailed ablation study in the Appendix.\n\n* **W3:** Latency that limits \"immediate operational adoption,\" despite the model's relevance to other scientific domains.\n\nThank you for this very relevant comment on the practical implications of sampling latency. We agree with your assessment. If minimal inference latency is the single most important criterion, our model—like other iterative diffusion models such as EDM—may not be the preferred choice.\n\nIn the context of operational weather forecasting, our model (like most other AI-based models) is orders of magnitude faster than incumbent physics-based systems like IFS ENS. Thus, for many applications in this domain, inference speed is not the primary limiting factor for adoption.\n\nFurthermore, this sampling speed issue could be alleviated via established techniques, such as diffusion in a compressed latent space (common in video generation), to significantly reduce the inference cost in future work. \n\n* **W4:** The conceptual novelty is \"incremental\" as the model uses existing building blocks.\n\nWhile we build upon established components, we respectfully argue that their successful integration into our ERDM framework is a non-trivial contribution tailored specifically for chaotic dynamical systems. The novelty lies in elucidating the specific adaptations required—which are not features of a standard EDM or RSDM—including a snapshot-dependent noise schedule, a bespoke temporal architecture, a specialized initialization strategy, and a tailored loss weighting. \n\nAs our ablation studies demonstrate (Appendix, Table 1), these carefully engineered components are crucial for the final performance, showing that a naive combination of the building blocks would not have been sufficient.\n\n* **Q1:** \"Since the authors mentioned the inferior performance of the ERDM is due to the EDM prediction initialization, could the authors replace the current initialization with IFS ENS or deterministic model (e.g. FourCastNet) and repeat the experiment? to close the gap\"\n\nThank you for this excellent suggestion for a follow-up experiment.\n\nIt would certainly be possible to initialize our ERDM framework with predictions from a different model. However, we intentionally chose to use our next-step EDM baseline for initialization to ensure a carefully controlled and fair comparison. Our goal was to isolate and benchmark the specific advantages of the proposed ERDM framework against a baseline trained with the exact same setup. Using a more powerful, external model for initialization would confound the analysis, making it difficult to determine whether performance gains came from our framework or the superior initialization.\n\nAs we discussed in our response to **W1**, we believe that another significant factor for the short-range performance gap is the general-purpose 3D U-Net architecture, which is not as specialized for complex weather data as SOTA architectures.\n\n* **Q2:** a) \"The noise level for the first snapshot within the window is small, does this mean it takes fewer solves to fully denoise the first snapshot?\" b) \"Does this have any implications of the number of NFEs compared to other models?\" c) \"have you profiled whether the 3 D attention layers or memory bandwidth dominate? An ablation with a lighter temporal backbone would clarify where to optimize.\"\n\na) Yes, the first snapshot of the first window is emitted after $N$ sampling steps (recall, $N=2$ for ERA5). \n\nb) Indeed, our ERDM framework is ***significantly more efficient in terms of the NFEs*** compared to the autoregressive EDM baseline. To generate a 30-step (15-day) forecast for our ERA5 experiments with a Euler solver, EDM requires 20 solver steps for *each* forecast, totaling $20 \\times 30 = 600$ NFEs. In contrast, ERDM has a cost of only $N=2$ solver steps per forecast, resulting in just $2 \\times 30 = 60$ NFEs (excluding a one-time initialization cost). These NFE counts double when using a second-order Heun solver, but the *10x* relative efficiency of ERDM is maintained.\n\nc) We experimented with a lighter 3D U-Net architecture by removing temporal attention from the shallower blocks. However, this modification resulted in negligible memory savings.\n\n\n\n* **Q3:** \"How does the choice of window size and accordingly σmin and σmax affect the performance?\"\n\nSee response to **W2** above.\n\n* **Q4:** \"What is the energy cost per 15 day ensemble forecast relative to IFS ENS, measured on the same hardware? This would strengthen the efficiency claim and inform practitioners.\"\n\nA direct energy comparison on the same hardware is impractical, as IFS ENS runs on specialized supercomputers, not GPUs. Nevertheless, the trend of AI models being vastly more energy-efficient is well-established; ECMWF's AI model, for example, is cited as being 1,000x more efficient than IFS ENS. Our model aligns with this trend, generating a 15-day, 5-member ensemble forecast in just 209 seconds on a single A100 GPU."}, {"review_id": "XG46OkImlJ", "reviewer": "Reviewer_5Mcc", "summary": "This paper proposes the Elucidated Rolling Diffusion Models (ERDM), a framework for probabilistic forecasting of high dimensional dynamical systems. It incorporates a loss weighting scheme, an efficient initialization strategy, and a bespoke hybrid sequence architecture to facilitate improved performance. The authors demonstrate the effectiveness of the proposed method through a series of comprehensive experiments.", "questions": "1. The authors should provide a clearer explanation of the rationale and applicability of diffusion models for this task.\n\n2. In the Rolling EDM noise schedule, the authors identify the setting of the $\\rho$ as particularly important. However, they only provide a simple empirical value without discussing its optimality or offering a thorough analysis.", "limitations": "The authors could provide a more thorough discussion of the societal limitations and potential impact of their work.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Paper Strengths: \n\n1. The application of elucidated rolling diffusion models to explicitly capture the sequential nature of data and the escalating physical uncertainty inherent in chaotic systems is compelling. The authors validate the effectiveness of their approach through a comprehensive series of experiments.\n\n2. The paper is relatively well-organized and clearly written. And it is also easy to be followed by later works.\n\nWeaknesses:\n\n1. The motivation of the paper appears to be unclear. It is not evident what limitations in existing methods the proposed techniques, such as the weighted loss and initialization, are intended to address. The contributions seem more like engineering refinements rather than principled innovations.\n\n2. The authors provided some visual results in the form of charts, but there is a lack of objective quantitative tables to evaluate the performance of the proposed method compared to baseline approaches.\n\n3. The paper lacks an analysis of its limitations. Moreover, the experiments should include measurements of inference time and computational overhead, especially considering that diffusion models often entail significant time costs during inference.", "was_revised": true, "final_justification": "The response has addressed most of my concerns.", "rebuttal": "Thank you for your review and constructive feedback. We are encouraged that you found our core approach to be \"compelling\". We also appreciate you recognizing that our method is validated by a comprehensive series of experiments and that the paper is well-organized, clearly written, and easy for future work to build upon.\n\n* **W1:** \"The motivation of the paper appears to be unclear. It is not evident what limitations in existing methods the proposed techniques...are intended to address. The contributions seem more like engineering refinements rather than principled innovations.\"\n\nThank you for this feedback, which helps us clarify the core motivation and principled nature of our work. Our approach is designed to address key limitations of existing generative methods when applied to chaotic dynamical systems, particularly in data-scarce (non Internet-scale) scenarios. Standard video diffusion models often struggle with limited data, while next-step autoregressive diffusion models may not adequately capture non-stationarity and non-Markovianity. Our ERDM framework is proposed as a *principled middle ground*, using a rolling window approach to better model the dynamics.\n\nRegarding our contributions, components like the weighted loss and initialization strategy are not mere engineering refinements but crucial, principled innovations required to make ERDM effective. For example, the weighted loss is *essential* for (1) correctly adapting the EDM training noise distribution to a sequential rolling setting and (2) is empirically vital for the model's performance, as validated by our ablation studies (Appendix, Table 1). The novelty lies in the synergistic integration of these carefully engineered components (including the noise schedule, initialization, and architecture), which are indispensable for adapting diffusion models to this challenging problem space.\n\n* **W2:** \"...there is a lack of objective quantitative tables to evaluate the performance of the proposed method compared to baseline approaches.\"\n\nThank you for this suggestion. While we believe our charts offer a more comprehensive view across all lead times—an approach common in the AI weather forecasting literature—we agree that a table serves as a valuable and objective reference. Therefore, for completeness, we provide the requested table below and will add it to the Appendix of our revised manuscript.\n\n**Tabular CRPS for ERA5**\n\n| Model | u10m (3d) | u10m (7d) | u10m (14d) | z500 (3d) | z500 (7d) | z500 (14d) |\n|:---|:---:|:---:|:---:|:---:|:---:|:---:|\n| EDM | 0.6923 | 1.324 | 1.769 | 76.04 | 204.9 | 327.4 |\n| **ERDM** | **0.6748** | **1.312** | **1.737** | 69.29 | 197.5 | 316.7 |\n| IFS ENS | 0.6830 | 1.318 | 1.785 | 58.41 | 183.4 | 317.3 |\n| NeuralGCM | N/A | N/A | N/A | **54.59** | **173.1** | **311.1** |\n\n*Legend: CRPS (lower is better) for 10m u-component wind (`u10m`) and 500 hPa geopotential (`z500`) at lead times of 3, 7, and 14 days.*\n\n* **W3:** *a)* \"The paper lacks an analysis of its limitations.\" *b)* \" Moreover, the experiments should include measurements of inference time and computational overhead...\"\n\n*a)* Thank you for this valuable feedback. We agree that a dedicated discussion of limitations and computational overhead will significantly strengthen the paper.\n\nIn our revision, we will add a new **Limitations** section to the conclusion. This section will consolidate and expand upon several points. We will reiterate the key limitation already noted in our conclusion—the higher memory footprint and cost per function evaluation of our 3D architecture compared to 2D alternatives. Furthermore, we will explicitly discuss other limitations of the current work, including:\n1.  The scope of our ERA5 experiments, which are not performed at the highest 0.25° resolution.\n2.  The framework's reliance on a pretrained forecaster for initialization.\n\n*b)* Excellent point. To address the need for concrete efficiency metrics, we'll include the following *efficiency benchmark* in our revised paper. \n\n| Model | Inference NFEs | Inference Time (s) | Inference GPU Memory (GB) | Training GPU Memory (GB) |\n|:---|:---:|:---:|:---:|:---:|\n| EDM | 600 | 237 | 21 | 19 |\n| ERDM | 120 | 209 | 49 | 53 |\n\n*Legend: Efficiency metrics for ERDM vs. EDM. All measurements were performed on a single A100 GPU using mixed precision. Inference figures correspond to a 15-day (30 time steps), 5-member ensemble forecast with second-order Heun solver. Training uses a batch size of 1.*\n\n\n* **Q1:** \"The authors should provide a clearer explanation of the rationale and applicability of diffusion models for this task.\"\n\nThe rationale for using diffusion models is that they often represent the state-of-the-art for probabilistic forecasting, which is essential for two primary reasons in the context of chaotic dynamical systems. First, their ability to generate *sharp, diverse ensembles* is crucial for robust *uncertainty quantification*. Second, they produce *high-fidelity samples* that are more *physically consistent* than the blurry, averaged-out predictions often generated by deterministic models trained with mean-squared error. We will add this clarification to the revised manuscript. See our *Diffusion models for spatiotemporal data* related work section for relevant references.\n\n\n\n* **Q2:** \"In the Rolling EDM noise schedule, the authors identify the setting of the $\\rho$ as particularly important. However, they only provide a simple empirical value without discussing its optimality or offering a thorough analysis.\"\n\nThe insight guiding our approach is that while progressive noise is necessary, an excessively large $\\rho$ destroys the temporal information that is crucial for the sequence model to learn dependencies between frames (see Fig. 2). As detailed in our ablation studies (Appendix, Table 1), the model's performance is robust across a wide range of small $\\rho$ values (e.g., -5 to -30). Note that we use the same $\\rho$ for Navier-Stokes and ERA5 experiments. We will add this detailed analysis to the revised manuscript.\n\n\n* **L1:** \"The authors could provide a more thorough discussion of the societal limitations and potential impact of their work.\"\n\nWe discuss the broader impact of our work in Appendix A. Is there anything specific that the reviewer is missing there?"}, {"review_id": "jEk5elNjMg", "reviewer": "Reviewer_yCRd", "summary": "This paper proposes an approach for modelling dynamical systems that combines the ideas from two different diffusion-based frameworks: rolling sequence diffusion models (RSDM) and elucidated diffusion models (EDM). The former models sequences, and introduces a timestep-dependent noise schedule, with increasing noise for more distant states in order to reflect the underlying uncertainty of the system. However, this approach has only been studied in the context of the DDPM formulation. Meanwhile, another popular approach emerged in the diffusion literature---EDM---which proposes carefully tuned choices for a more efficient design of the diffusion process, including noise schedules, loss weights, optimised network architectures, and faster sampling techniques. The approach proposed in this paper, ERDM, combines the strengths of the two methodologies by 1) designing a snapshot‐dependent noise schedule that assigns each future time point the appropriate noise magnitude, 2) applying a loss-weighting scheme across the sequence that emphasises the \"harder\" noise levels, 3) extending the backbone architecture to better capture the temporal axis, and 4) integrating efficient second-order solvers during sampling. Experiments are performed on two systems: a 2D Navier–Stokes flow simulation and the multivariate ERA5 atmospheric reanalysis dataset. The authors show that the approach generally outperforms a plain autoregressive EDM baseline (in terms of CRPS and SSR), highlighting the advantages of the rolling diffusion formulation for modelling dynamical systems.", "questions": "1. The authors state that it is important to tune the noise schedule parameters $\\sigma_{\\text{min}}$, $\\sigma_{\\text{max}}$, and $\\rho$ during training. For example, $\\rho$ is chosen such that the snapshots are under less noise than in the normal EDM formulation. How dataset-dependent are the most reasonable choices for these hyperparameters? For example, in the context of Navier-Stokes, would a different temporal resolution require re-tuning of these hyperparameters, given that the underlying dynamics change (e.g. if we use 10x a smaller timestep, the states would be much more \"similar\" to each other and perhaps we should add even less noise). \n2. Could the authors comment on why they haven't included an RDM-based baseline too (with the DDPM formulation)?\n3. Could you clarify how the sampling procedure looks like when $N$ is not an integer?\n4. How much of a limitation are the increased memory and computational complexity, with the vision that such a model could be used to model, for example, higher resolution weather data (0.25° or even 0.1° ERA5)?\n5. The SSR shows that ERDM goes from a region of underdispersion (up to about 10 timesteps) to overdispersion (10-20 timesteps) followed by underdispersion again. In contrast, the trend for plain EDM models is monotonically decreasing. Can the authors comment on this---what are the potential causes of overdispersion at medium timesteps, and whether the SSR could be stabilised through other hyperparameter choices?", "limitations": "The authors mention the increased memory and computational cost of their method, but do not necessarily discuss how practical this would become with higher resolution (in space and time) data of practical significance. This would be especially relevant in the weather modelling field, where the community is heading towards 0.25° or even 0.1° spatial resolution and up to 1 hour time resolution.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n1) **Comprehensive ablations** - One of the main strengths of the paper lies in the ablations it performs. They improve the level of significance, as they help clearly identify and isolate the components that contribute most to the performance of the model.\n2) **Good presentation** - The manuscript is well written, with a clear narrative flow and only minor typographical errors.\n3) **Novel Integration of previous methods** - Although it does not come up with new methods, combining the RDM and the EDM formulation is original.\n4) **Timely and relevant** - The topic it addresses---improving the modelling of dynamical systems with diffusion models---is of interest in the community, especially in the light of the latest advancements from communities such as weather modelling.\n\n**Weaknesses**\n1) **Variance in Loss Weighting vs. Noise Sampling** - The authors replace sampling $\\sigma$ from a log-normal distribution (an importance-sampling strategy that emphasises intermediate noise levels) with an explicit loss-weighting scheme. While these approaches are equivalent in the continuous-time limit—when the loss is written as an integral over time [1]—their gradient estimates under minibatch training would differ in practice. In general, incorporating weights into the noise sampling distribution often reduces variance, so the chosen loss-weighting strategy may be suboptimal from a training perspective.\n2) **First-window initialisation** - To avoid the extra hyperparameters introduced by Ruhe et al. [2], ERDM trains a separate “initial” model for the first rollout window. However, this still entails training and tuning a second network. In this work, the trained EDM model acted as a baseline anyway, but on large-scale datasets (e.g., native-resolution ERA5), maintaining two models would probably be more demanding than tuning some additional hyperparameters.\n3) **Missing Baseline: Original RDM formulation** - The paper positions ERDM as an improvement over rolling diffusion, yet it does not compare directly to the original DDPM formulation of RSDM from [2]. Demonstrating gains over a DDPM-based rolling model—as well as over plain EDM—would more convincingly attribute performance improvements to the EDM design choices within the rolling framework.\n4) **Experiments Setup and Benchmarks**\n  - **Navier-Stokes** - Only five test trajectories are evaluated, which may not capture the full variability of initial conditions of the system. Moreover, I think the authors could include one or two additional baselines in order to be able to correctly position the performance of the method. For example, PDE-Refiner [3] showed good performance on modelling the KS and Kolmogorov systems. Another related approach is DyDiff [4]. Dyffusion does not compare to any of these, and I am not sure it's clear that Dyffusion is currently SOTA.\n  - **ERA5 reanalysis** - As the authors acknowledge, comparing ERDM against NeuralGCM ENS and Graph-EFM—both evaluated on 2020—weakens the analysis due to potential distribution shifts in weather patterns during 2020/2021 (e.g., pandemic-related anomalies). Likewise, the IFS ENS comparison is flawed because IFS ENS is verified against operational analyses rather than the ERA5 reanalyses, further undermining its relevance. These mismatches leave the autoregressive EDM baseline as the only truly comparable benchmark. To strengthen their claims, the authors should: (1) include an RDM baseline following Ruhe et al. [2], and (2) consider adding a SOTA weather model—such as GenCast—to contextualise the performance gap, even if the comparison is inherently imperfect given ERDM’s coarser training data and limited computational budget. Without these additional benchmarks, it remains difficult to fully assess and contextualise the benefits of the proposed framework.\n  \n    Moreover, the results are fairly hard to interpret in the way they are currently presented. I personally think that a better way of visualising and summarising these results is through score cards.\n5) **Memory Requirements** - The paper does not quantify the additional computational and memory costs of ERDM versus baseline EDM. As weather and fluid-dynamics applications move toward higher spatiotemporal resolutions, understanding ERDM’s resource footprint is crucial for assessing its practical scalability.\n6) **Minor Issues**\n  - **Energy Spectra** - Energy spectra would be useful in the Navier-Stokes experiment too. I am expecting a similar behaviour as in the ERA5 experiment (shown in Figure 12), but it would be worth introducing them for completeness.\n- **Appendix References** - There are minor misreferences in the Appendix (e.g., the noise-prior discussion is in C.4, not C.6; the network-topology section is supposed to be in C.5., but there is no such section).\n- **Sampler Efficiency Comparison** - When comparing Heun versus Euler, it would be informative to compare performance under a fixed budget of neural function evaluations (e.g., 20 Euler steps vs. 10 Heun predictor + 10 corrector steps) to account for the doubled evaluations in Heun’s corrector step.\n\n**References**\n\n[1] Dieleman, S. (2024). Noise schedules considered harmful. Sander.ai.\n\n[2] Ruhe, D. et al. (2024). Rolling diffusion models. arXiv:2402.09470.\n\n[3] Lippe, P. et al. (2023). PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers. arXiv:2308.05732.\n\n[4] Guo, X. et al. (2025). Dynamical Diffusion: Learning Temporal Dynamics with Diffusion Models. arXiv:2503.00951.", "was_revised": true, "final_justification": "As noted in my initial review, I believe the paper presents a novel combination of two previously proposed methods—Rolling Diffusion (RSDM) and EDM—adapted effectively for probabilistic spatiotemporal modelling. The paper includes comprehensive ablation studies that clearly identify which modifications lead to the most significant performance gains. Finally, the results are encouraging.\nThat said, I continue to believe that the authors should include a DDPM-based RSDM baseline. While the superiority of the EDM approach has been shown in the image generation literature, I think including this baseline is important for completeness and would strengthen the story of the paper. However, since I expect the proposed method to outperform the DDPM-based baseline, I have chosen not to penalise the authors heavily for this omission.\nAn additional comment I have after the rebuttal and reading other reviewers' comments is that the authors might stress the applicability of their approach to general probabilistic spatiotemporal tasks for chaotic systems, rather than focusing on weather forecasting right from the title. This might improve the reach and impact of their approach.", "rebuttal": "Thank you for your thorough and constructive review. We are grateful for your positive feedback, particularly on our *comprehensive ablations*, which you noted help to clearly identify the most impactful components of our model. We also appreciate you recognizing the paper as *well-written with a clear narrative*, and the integration of RDM and EDM as a *novel, timely, and relevant* contribution to the field.\n\n* **W1:** \"Variance in Loss Weighting vs. Noise Sampling - The authors replace sampling σ from a log-normal distribution [...] with an explicit loss-weighting scheme. [...] incorporating weights into the noise sampling distribution often reduces variance, so the chosen loss-weighting strategy may be suboptimal from a training perspective.\"\n\nWe thank the reviewer for this insightful technical point. The potential difference in gradient variance between weighting the loss versus importance-sampling the noise distribution is a valid consideration. While we acknowledge the theoretical benefits of noise sampling, we found our chosen explicit **loss-weighting** scheme to be highly effective in practice, yielding the strong empirical results reported. We will add a discussion of this trade-off, along with the suggested reference, to the section on limitations and future work.\n\n\n* **W2:** \"First-window initialisation - [...] ERDM trains a separate “initial” model for the first rollout window. However, this still entails training and tuning a second network. [...] on large-scale datasets [...], maintaining two models would probably be more demanding than tuning some additional hyperparameters.\"\n\nThis is a valid point regarding the practicality of our initialization strategy. In the context of our study, this approach was practical as a strong pretrained model (our EDM baseline) was already available. However, we acknowledge this may not be generalizable. We will explicitly state in our new *limitations section* that the reliance on a readily available, strong pretrained forecaster is a limitation of our current approach, particularly for novel problem domains.\n\n* **W3:** \"Missing Baseline: Original RDM formulation - The paper [...] does not compare directly to the original DDPM formulation of RSDM from [2]. Demonstrating gains over a DDPM-based rolling model [...] would more convincingly attribute performance improvements to the EDM design choices...\"\n\nThis is a valid point. Our rationale for not including a direct comparison to the DDPM-based RSDM is that the *EDM framework is a direct generalization of DDPM*. As shown in the original EDM paper (c.f. Table 1), EDM recovers the DDPM formulation for specific choices of its noise schedule and preconditioning.\n\nGiven the demonstrated success of the specific design choices proposed in the EDM paper, we focused our efforts on adapting these superior settings to the rolling context. While our ERDM framework inherits this generality and could be configured to replicate a DDPM-style RDM, we concentrated on what we believe to be the most promising formulation. We agree, however, that a direct empirical comparison is valuable and will note this as an important direction.\n\n\n* **W4:** \"Navier-Stokes - Only five test trajectories are evaluated... Moreover, I think the authors could include one or two additional baselines... For example, PDE-Refiner [3]... Another related approach is DyDiff [4].\"\n\nWe thank the reviewer for these suggestions regarding the Navier-Stokes evaluation. Our use of five test trajectories follows the protocol established in the NeurIPS benchmark paper that introduced this dataset. While we agree that including additional baselines would be valuable, implementing and carefully tuning & evaluating new complex models is unfortunately hard to fulfill within the rebuttal period. We hope that the comprehensive ERA5 experiments serve as, complementary, compelling empirical evidence.\n\n* **W5:** \"..., the IFS ENS comparison is flawed because IFS ENS is verified against operational analyses rather than the ERA5 reanalyses, further undermining its relevance These mismatches leave the autoregressive EDM baseline as the only truly comparable benchmark.\"\n\nYou're right that we verify our model against *ERA5 reanalysis* while IFS ENS is verified against *operational analysis*. However, this isn't a flaw that favors our model—it's actually the opposite. This evaluation setup is the **recommended best practice** by the established WeatherBench-2 (WB2) benchmark, precisely because it's known to be more favorable to the IFS ENS baseline.\n\nAn external check of the official WB2 CRPS scores for \"IFS vs. Analysis\" versus \"IFS vs. ERA5\" will confirm that the former are consistently lower (better). Therefore, our comparison is intentionally conservative. Our model's competitive performance is achieved against a baseline that is being evaluated under its most advantageous conditions. We will make sure to clarify this in the paper.\n\n\n* **W6:** \"Moreover, the results are fairly hard to interpret in the way they are currently presented. I personally think that a better way of visualising and summarising these results is through score cards.\"\n\nThat's a great suggestion, thank you. We agree that score cards are an excellent way to visualize and summarize results. \n\nBased on your feedback, we have now created a score card comparing ERDM vs. our key EDM baseline, and we will add it to the paper. Since we're not allowed to share images here, this is a high-level summary of it: The score card is predominantly \"blue\", indicating that ERDM outperforms EDM on the vast majority of variables and lead times, with minor exceptions for the geopotential field at 12 hours and for the q50 variable.\n\n\n* **W7:** \"Memory Requirements - The paper does not quantify the additional computational and memory costs of ERDM versus baseline EDM.\"\n\nExcellent suggestion. We will add the following *efficiency benchmark* to our revised paper.\n\n| Model | Inference NFEs | Inference Time (s) | Inference GPU Memory (GB) | Training GPU Memory (GB) |\n|:---|:---:|:---:|:---:|:---:|\n| EDM | 600 | 237 | 21 | 19 |\n| ERDM | 120 | 209 | 49 | 53 |\n\n*Legend: Efficiency metrics for ERDM vs. EDM. All measurements were performed on a single A100 GPU using mixed precision. Inference figures correspond to a 15-day (30 time steps), 5-member ensemble forecast with second-order Heun solver. Training uses a batch size of 1.*\n\n\n* **Q1:** \"How dataset-dependent are the most reasonable choices for these hyperparameters? For example, in the context of Navier-Stokes, would a different temporal resolution require re-tuning of these hyperparameters...?\"\n\nIn our experiments, $\\rho$ was kept constant for both the Navier-Stokes and ERA5 datasets, suggesting a degree of robustness. The noise boundaries, $\\sigma_{min}$ and $\\sigma_{max}$, are more data-dependent, which is consistent with the standard EDM framework. We used the same $\\sigma_{min}$ (0.002) for both problems, but $\\sigma_{max}$ was set higher for ERA5 (500 vs. 200) to account for the larger dynamic range of its variables (e.g., humidity).\n\nBased on our results across these heterogeneous datasets, our current settings provide a robust baseline. While exploring the precise dependency of the noise schedule on data characteristics is a valuable direction for future work, and further tuning could yield marginal improvements, we found these settings to generalize well.\n\n\n* **Q2:** \"Could the authors comment on why they haven't included an RDM-based baseline too (with the DDPM formulation)?\"\n\nSee response to **W3**. Besides, the rolling diffusion model paper [2] doesn't provide open-source code.\n\n\n* **Q3:** \"Could you clarify how the sampling procedure looks like when N is not an integer?\"\n\n$N$ doesn't represent a literal number of discrete solver steps. Instead, it controls the amount of denoising per step, defined as $\\Delta t = 1/N$. Our sampler takes discrete steps until an internal 'global diffusion time' accumulator crosses 1, at which point the window is rolled forward.\n\nFor example, when $N=1.25$, then $\\Delta t = 0.8$. If the global time starts at 0.0, it takes two steps to reach 1.6, crossing the integer threshold and triggering a roll. The new global time becomes 0.6, etc. Thus, the number of actual sampling steps would always be 1 or 2 here.\n\n* **Q4:** \"How much of a limitation are the increased memory and computational complexity, with the vision that such a model could be used to model, for example, higher resolution weather data (0.25° or even 0.1° ERA5)?\"\n\nYou're right to point out that the increased memory complexity from our 3D architecture is the primary bottleneck for scaling to higher-resolution data.\n\nWe are hopeful: While standard techniques like gradient checkpointing can trade speed for memory, we believe a more powerful solution is to integrate ERDM with latent space diffusion. Performing the diffusion process on a compressed representation would drastically reduce the memory and compute costs of the 3D backbone, making the framework much more scalable without sacrificing the benefits of the rolling formulation. \n\n* **Q5:** \"The SSR shows that ERDM goes from a region of underdispersion [...] to overdispersion [...] followed by underdispersion again. [...] Can the authors comment on this---what are the potential causes of overdispersion at medium timesteps, and whether the SSR could be stabilised through other hyperparameter choices?\"\n\nGreat observation! Note that for ERA5 (Fig. 11 in the appendix) the SSR curves are qualitatively different and this behavior doesn't happen for EDM any more. Generally, we've found that the SSR can be \"improved\" (for ERDM, EDM and other models) simply by stopping the training earlier. This comes at a cost of accuracy, however. Because we consider the CRPS as the key metric, we avoid doing this in this paper."}, {"review_id": "H7CgMGhUGw", "reviewer": "Reviewer_jthi", "summary": "The paper proposes Elucidated Rolling Diffusion Models (ERDM), which combines Rolling Sequence Diffusion with the Elucidated Diffusion Model (EDM) framework. The paper investigates various design choices, including hyperparameters and (denoising) model architecture. For the former, some modifications of the suggestions in the EDM paper are proposed. For the latter, temporal attention is added to the standard UNet model to acknowledge the spatio-temporal properties of the forecasting model. The method is tested on both Navier-Stokes fluid dynamics and weather forecasting.", "questions": "1.\tThe proposed method has a higher computational cost (per NFE) than a standard EDM since it operates on windows of lengths W. This is briefly acknowledged in the conclusion section, where it is mentioned that ERDM and EDM still have roughly the same computational cost in practice. However, this is not obvious from the empirical evaluation and I wonder if it would be possible to use the extra compute for EDM in a better way to improve it’s performance? In other words, does the observed improvement for ERDM primarily come from the fact that you effectively use W times the number of diffusion steps in ERDM (since each forecast is progressively denoised W times)? \n2.\tHow are the forecasts produced with EDM, W=4? Does it also use rolling diffusion, or do you simply extract the first forecast in the window and throw away the rest?\n3.\tI found it a bit strange that you evaluate your method against NeuralGCM using different test sets. Even if you don’t have results for the different baselines for the same year (and don’t want to run them yourselves), could you not at least run the proposed method for both 2020 and 2021 and carry out two separate evaluations depending on the test sets for the different baselines?\n4.\tWhat’s the point of having two weighting functions $\\lambda(\\sigma)$ and $f(\\sigma)$? You write (line 337) that it’s important to tune the $f(\\sigma)$ function anyway, so I wonder how important the particular choice $\\lambda(\\sigma)*f(\\sigma)$ is. Could you simply replace this product with some function $g(\\sigma)$ that you tune based on empirical performance?\n5.\tIt’s written (line 36) that you _“model the progressively increasing uncertainty [… by …] escalate noise for distant future snapshots”_. Is this really a correct interpretation? It is true that the model forecasts using higher noise levels at future time points (end of the W-window), but these time points are not completely denoised, so the model does not actually learn to forecast multiple steps into the future – correct? A better interpretation, in my opinion, would be to say that you produce 1-step forecasts by progressive denoising in multiple steps, in a way which is made possible by the temporal structure of the problem. When I read the intro (sentence above) and abstract I got the feeling that your proposed model could accurately simulate from the multi-step forecasting distribution $p(y_{t+W} | y_{t})$ in a way which respects the increased uncertainty with increasing W (in a principled way), but I don’t believe that this is the case..?\n6.\tWhy do you have “weather forecasting” in the title when the method is more general and you evaluate it on one fluid dynamics problem and one weather forecasting problem?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "The proposed method shows appears to have strong empirical performance compared to the baselines. In particular, it is shown that the ERDM outperforms a standard autoregressive EDM model, suggesting that there is a value in the rolling diffusion formulation of the probabilistic forecasting problem.\n\nA weakness is that the contribution is somewhat limited, considering that is mainly consists of combining existing ideas for rolling diffusion with the EDM framework.\n\nI’m also a bit sceptical about the word “Elucidating” in the title. I understand that this is inherited from the EDM paper, but I’m not convinced that this paper is “elucidating” how one should design diffusion model for forecasting in the best way. To me, the paper reads more like a typical methodological paper where the authors propose a particular method, which is evaluated empirically against alternative methods. Most design choices are taken from the EDM framework without shedding much additional light on these choices. This is perfectly fine, but I feel that the word “Elucidating” in the title promises more than what the paper delivers.\n\nRegarding the resulting method, the proposed temporal UNet model used for denoising seems to be a key component, and perhaps this deserves to be emphasized more, at least if the focus is shifted from “elucidating the design space” to “proposing a method”.", "was_revised": true, "final_justification": "The authors addressed W1, W2, Q1 in my original review to a satisfactory degree.", "rebuttal": "Thanks for your detailed review and insightful feedback. We're encouraged that you found our method to have strong empirical performance compared to the baselines. \n\n* **W1:** \"the contribution is somewhat limited, considering that is mainly consists of combining existing ideas for rolling diffusion with the EDM framework.\"\n\nWhile we build upon established components, we respectfully argue that their successful integration into our ERDM framework is a non-trivial contribution tailored specifically for chaotic dynamical systems. The novelty lies in elucidating the specific adaptations required—which are not features of a standard EDM or RSDM—including a snapshot-dependent noise schedule, a bespoke temporal architecture, a specialized initialization strategy, and a tailored loss weighting.\n\nAs our ablation studies demonstrate (Appendix, Table 1), these carefully engineered components are crucial for the final performance, showing that a naive combination of existing ideas would not have been sufficient. This is complemented by strong empirical results.\n\n* **W2:** \"...sceptical about the word “Elucidating” in the title... I’m not convinced that this paper is “elucidating” how one should design diffusion model for forecasting in the best way... [it] promises more than what the paper delivers.\"\n\nWe thank the reviewer for this fair point regarding the paper's title and its perceived promise. The primary reason for the name is make the link to the *Elucidated Diffusion Models (EDM)* framework, from which our method is derived, explicit; an alternative such as \"EDM Rolling Diffusion Models\" would be redundant.\n\nWe agree that our paper is primarily methodological. Our use of \"Elucidating\" isn't meant to cover the entire design space, but to clarify the core principles—in architecture, initialization, and noise scheduling—needed to successfully adapt the EDM framework for dynamics forecasting.\n\n* **W3:** \"...the proposed temporal UNet model used for denoising seems to be a key component, and perhaps this deserves to be emphasized more...\"\n\nYou're right, the temporal U-Net is essential to our method's success. We chose not to over-emphasize this specific architecture because our *primary contribution is the more general ERDM framework, which is agnostic to the specific backbone*.\n\nOur temporal U-Net could be swapped with any other capable 3D or sequential architecture, which might even yield better performance (see also discussion with reviewer yaoq). The key architectural takeaway isn't our specific U-Net, but that a capable temporal backbone is necessary, as a naive, non-temporal model would fail. We don't want to tie the ERDM framework too closely to one architectural choice, but rather highlight the principles that make it work.\n\n* **Q1:** \"The proposed method has a higher computational cost (per NFE) than a standard EDM (...) does the observed improvement for ERDM primarily come from the fact that you effectively use W times the number of diffusion steps in ERDM...?\"\n\n***This is a misunderstanding***. ERDM actually needs *less* NFEs than EDM. However, each NFE is more costly for ERDM (due to the 3D vs. 2D denoiser architecture).\n\nThe key difference is:\n* The standard autoregressive **EDM** must perform a full, expensive denoising schedule (with $N_{EDM} \\approx 20$ steps) for *every single time step* it predicts.\n* In contrast, **ERDM's** denoises in parallel. The first snapshots of the window start in a state that is already \"almost denoised\" and only needs a very small number of refinement steps (`$N=2$` for our ERA5 experiments) to produce the next forecast before rolling forward.\n\nTo make this relationship perfectly clear and quantitative, we will add the following *efficiency benchmark* to our revised paper.\n\n| Model | Inference NFEs | Inference Time (s) | Inference GPU Memory (GB) | Training GPU Memory (GB) |\n|:---|:---:|:---:|:---:|:---:|\n| EDM | 600 | 237 | 21 | 19 |\n| ERDM | 120 | 209 | 49 | 53 |\n\n*Legend: Efficiency metrics for ERDM vs. EDM. All measurements were performed on a single A100 GPU using mixed precision. Inference figures correspond to a 15-day (30 time steps), 5-member ensemble forecast with second-order Heun solver. Training uses a batch size of 1.*\n\n* **Q2:** \"How are the forecasts produced with EDM, W=4? Does it also use rolling diffusion, or do you simply extract the first forecast in the window and throw away the rest?\"\n\nThis baseline does not use rolling diffusion. It is a standard *sequence-to-sequence EDM* (using the same 3D denoiser arch. as ERDM) that operates autoregressively. Conditioned on a single frame (e.g., at time $t$), it jointly predicts a window of future frames ($t+1$ to $t+W$). Unlike ERDM, all frames in this window are denoised from the same initial noise level. To generate a long forecast, the last frame of the predicted window (at time $t+W$) is then used as the new condition for the next prediction step. We will clarify this distinction in the revised manuscript.\n\n* **Q3:** \"...could you not at least run the proposed method for both 2020 and 2021 and carry out two separate evaluations depending on the test sets for the different baselines?\"\n\nWe couldn't evaluate our model on the 2020 test set as it would be an unfair comparison; our model's training data includes the year 2020. Re-training our model with a different data split just for this comparison wasn't feasible due to the high computational cost of a full training run. \n\n* **Q4:** \"What’s the point of having two weighting functions `$\\lambda(\\sigma)$` and `$f(\\sigma)$`? ... Could you simply replace this product with some function `$g(\\sigma)$` that you tune based on empirical performance?\"\n\nThank you for this insightful question. Our formulation using the product of $\\lambda(\\sigma)$ and $f(\\sigma)$ was deliberately chosen to maintain consistency with the EDM framework, from which our method is derived. $\\lambda(\\sigma)$ is adopted directly from EDM, while our novel weighting $f(\\sigma)$ is an adaptation of the EDM training noise distribution.\n\nWe agree with your assessment that these could be merged into a single tunable function, $g(\\sigma)$, and we believe exploring such alternative weighting schemes is a valuable direction for future work. We will add this clarification to the manuscript.\n\n\n* **Q5:** \"...Is this really a correct interpretation? ...I got the feeling that your proposed model could accurately simulate from the multi-step forecasting distribution `$p(y_{t+W}|y_t)$` ... but I don’t believe that this is the case..?\"\n\nThank you for this excellent question. We agree that a complete denoising of the window to $t+W$ is not performed at each step. Our statement about \"modeling progressive uncertainty\" pertains to the effect of the noise schedule on the *joint denoising process* itself.\n\nBy escalating noise for distant future snapshots, the framework compels the denoiser to treat information from these frames as less reliable. This process implicitly teaches the model that uncertainty grows over the forecast horizon. Therefore, the principle of increasing uncertainty is embedded within the model's learning mechanism. We will clarify this distinction in the revised manuscript.\n\n* **Q6:** \"Why do you have “weather forecasting” in the title when the method is more general...?\"\n\nWe agree that our method is more general. We chose to emphasize \"Weather Forecasting\" in the title because it's the primary application in our experiments and because it motivated our work. We view the Navier-Stokes experiment as a valuable but simpler proof-of-concept, as it's a smaller-scale problem that shares similar underlying chaotic dynamics with weather."}]

Author rebuttal:

Meta-review: The paper introduces *Elucidated Rolling Diffusion Models (ERDM)*, a framework for spatiotemporal probabilistic forecasting that integrates the structure of Rolling Sequence Diffusion Models with the design principles of Elucidated Diffusion Models (EDM). ERDM features a snapshot-dependent noise schedule, an uncertainty-aware loss weighting scheme, and a hybrid U-Net with causal temporal attention. It also leverages pretrained models to improve initialization for the first forecast window. The method is evaluated on two tasks: a 2D Navier–Stokes simulation and ERA5 atmospheric data, showing improvements over autoregressive EDM baselines and competitive performance compared to state-of-the-art systems like NeuralGCM and IFS ENS.

**Strengths:**
- Demonstrates superior performance over autoregressive EDM and DYffusion on Navier–Stokes, and is competitive with NeuralGCM and IFS ENS on ERA5.
- Incorporates a progressive noise schedule that better captures horizon uncertainty and enables fewer sampling steps during inference.
- Includes thorough ablation studies that isolate the contributions of each architectural and training component.
- Integrates prior methodologies (EDM and RSDM) in a novel and practical way for modelling chaotic systems.
- Paper is clearly written and well-structured, making it accessible and easy to follow.
- Shows practical training efficiency improvements and potential for broader applicability in climate and fluid dynamics modeling.

**Weaknesses:**
- Some key performance limitations remain, particularly short-range accuracy on ERA5 compared to other models, attributed to suboptimal initialization without full experimental validation.
- Lacks direct comparisons to the original DDPM-based RSDM baseline, weakening claims of methodological improvement.
- Benchmarks on ERA5 and Navier–Stokes have mismatches or limited test diversity, making it harder to fully assess generalization.
- Inference efficiency and computational overhead are insufficiently analyzed; practical deployment concerns like memory cost and inference latency are underexplored.
- Conceptual contributions may be seen as incremental engineering refinements rather than fundamentally novel ideas.
- Several empirical design choices (e.g., window size, loss weighting functions) lack principled justification or tuning analysis.

All concerns have been addressed by the authors during the rebuttal period, with all reviewers satisfied with the response. I am therefore recommending acceptance. 

I would also suggest that 

- The authors provide a more comprehensive literature review to include related work on diffusion models for weather forecasting like [1].
- The authors test generalization across various values of Reynolds numbers for the Navier Stokes experiments and highlight a lack of comprehensive testing across different Reynolds numbers as future work.


[1] PreDiff: Precipitation Nowcasting with Latent Diffusion Models. NeurIPS 2023.
Decision: Accept (poster)