Title: Elucidated Rolling Diffusion Models for Probabilistic Forecasting of Complex Dynamics
Venue: NeurIPS 2025
Accepted: yes
Keywords: diffusion models

AI for science

probabilistic forecasting

weather forecasting
Abstract: Diffusion models are a powerful tool for probabilistic forecasting, yet most applications in high-dimensional complex systems predict future states individually. This approach struggles to model complex temporal dependencies and fails to explicitly account for the progressive growth of uncertainty inherent to the systems. While rolling diffusion frameworks, which apply increasing noise to forecasts at longer lead times, have been proposed to address this, their integration with state-of-the-art, high-fidelity diffusion techniques remains a significant challenge. We tackle this problem by introducing Elucidated Rolling Diffusion Models (ERDM), the first framework to successfully unify a rolling forecast structure with the principled, performant design of Elucidated Diffusion Models (EDM). 
To do this, we adapt the core EDM components--its noise schedule, network preconditioning, and Heun sampler--to the rolling forecast setting. The success of this integration is driven by three key contributions: $(i)$ a novel loss weighting scheme that focuses model capacity on the mid-range forecast horizons where determinism gives way to stochasticity; $(ii)$ an efficient initialization strategy using a pre-trained EDM for the initial window; and $(iii)$ a bespoke hybrid sequence architecture for robust spatiotemporal feature extraction under progressive denoising. 
On 2D Navier–Stokes simulations and ERA5 global weather forecasting at $1.5^\circ$ resolution, ERDM consistently outperforms key diffusion-based baselines, including conditional autoregressive EDM.
ERDM offers a flexible and powerful general framework for tackling diffusion-based dynamics forecasting problems where modeling  uncertainty propagation is paramount.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes Elucidated Rolling Diffusion Models (ERDM), which combines Rolling Sequence Diffusion with the Elucidated Diffusion Model (EDM) framework. The paper investigates various design choices, including hyperparameters and (denoising) model architecture. For the former, some modifications of the suggestions in the EDM paper are proposed. For the latter, temporal attention is added to the standard UNet model to acknowledge the spatio-temporal properties of the forecasting model. The method is tested on both Navier-Stokes fluid dynamics and weather forecasting.\n\nSTRENGTHS AND WEAKNESSES:\nThe proposed method shows appears to have strong empirical performance compared to the baselines. In particular, it is shown that the ERDM outperforms a standard autoregressive EDM model, suggesting that there is a value in the rolling diffusion formulation of the probabilistic forecasting problem.\n\nA weakness is that the contribution is somewhat limited, considering that is mainly consists of combining existing ideas for rolling diffusion with the EDM framework.\n\nI’m also a bit sceptical about the word “Elucidating” in the title. I understand that this is inherited from the EDM paper, but I’m not convinced that this paper is “elucidating” how one should design diffusion model for forecasting in the best way. To me, the paper reads more like a typical methodological paper where the authors propose a particular method, which is evaluated empirically against alternative methods. Most design choices are taken from the EDM framework without shedding much additional light on these choices. This is perfectly fine, but I feel that the word “Elucidating” in the title promises more than what the paper delivers.\n\nRegarding the resulting method, the proposed temporal UNet model used for denoising seems to be a key component, and perhaps this deserves to be emphasized more, at least if the focus is shifted from “elucidating the design space” to “proposing a method”.\n\nQUESTIONS:\n1.\tThe proposed method has a higher computational cost (per NFE) than a standard EDM since it operates on windows of lengths W. This is briefly acknowledged in the conclusion section, where it is mentioned that ERDM and EDM still have roughly the same computational cost in practice. However, this is not obvious from the empirical evaluation and I wonder if it would be possible to use the extra compute for EDM in a better way to improve it’s performance? In other words, does the observed improvement for ERDM primarily come from the fact that you effectively use W times the number of diffusion steps in ERDM (since each forecast is progressively denoised W times)? \n2.\tHow are the forecasts produced with EDM, W=4? Does it also use rolling diffusion, or do you simply extract the first forecast in the window and throw away the rest?\n3.\tI found it a bit strange that you evaluate your method against NeuralGCM using different test sets. Even if you don’t have results for the different baselines for the same year (and don’t want to run them yourselves), could you not at least run the proposed method for both 2020 and 2021 and carry out two separate evaluations depending on the test sets for the different baselines?\n4.\tWhat’s the point of having two weighting functions $\\lambda(\\sigma)$ and $f(\\sigma)$? You write (line 337) that it’s important to tune the $f(\\sigma)$ function anyway, so I wonder how important the particular choice $\\lambda(\\sigma)*f(\\sigma)$ is. Could you simply replace this product with some function $g(\\sigma)$ that you tune based on empirical performance?\n5.\tIt’s written (line 36) that you _“model the progressively increasing uncertainty [… by …] escalate noise for distant future snapshots”_. Is this really a correct interpretation? It is true that the model forecasts using higher noise levels at future time points (end of the W-window), but these time points are not completely denoised, so the model does not actually learn to forecast multiple steps into the future – correct? A better interpretation, in my opinion, would be to say that you produce 1-step forecasts by progressive denoising in multiple steps, in a way which is made possible by the temporal structure of the problem. When I read the intro (sentence above) and abstract I got the feeling that your proposed model could accurately simulate from the multi-step forecasting distribution $p(y_{t+W} | y_{t})$ in a way which respects the increased uncertainty with increasing W (in a principled way), but I don’t believe that this is the case..?\n6.\tWhy do you have “weather forecasting” in the title when the method is more general and you evaluate it on one fluid dynamics problem and one weather forecasting problem?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an approach for modelling dynamical systems that combines the ideas from two different diffusion-based frameworks: rolling sequence diffusion models (RSDM) and elucidated diffusion models (EDM). The former models sequences, and introduces a timestep-dependent noise schedule, with increasing noise for more distant states in order to reflect the underlying uncertainty of the system. However, this approach has only been studied in the context of the DDPM formulation. Meanwhile, another popular approach emerged in the diffusion literature---EDM---which proposes carefully tuned choices for a more efficient design of the diffusion process, including noise schedules, loss weights, optimised network architectures, and faster sampling techniques. The approach proposed in this paper, ERDM, combines the strengths of the two methodologies by 1) designing a snapshot‐dependent noise schedule that assigns each future time point the appropriate noise magnitude, 2) applying a loss-weighting scheme across the sequence that emphasises the \"harder\" noise levels, 3) extending the backbone architecture to better capture the temporal axis, and 4) integrating efficient second-order solvers during sampling. Experiments are performed on two systems: a 2D Navier–Stokes flow simulation and the multivariate ERA5 atmospheric reanalysis dataset. The authors show that the approach generally outperforms a plain autoregressive EDM baseline (in terms of CRPS and SSR), highlighting the advantages of the rolling diffusion formulation for modelling dynamical systems.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1) **Comprehensive ablations** - One of the main strengths of the paper lies in the ablations it performs. They improve the level of significance, as they help clearly identify and isolate the components that contribute most to the performance of the model.\n2) **Good presentation** - The manuscript is well written, with a clear narrative flow and only minor typographical errors.\n3) **Novel Integration of previous methods** - Although it does not come up with new methods, combining the RDM and the EDM formulation is original.\n4) **Timely and relevant** - The topic it addresses---improving the modelling of dynamical systems with diffusion models---is of interest in the community, especially in the light of the latest advancements from communities such as weather modelling.\n\n**Weaknesses**\n1) **Variance in Loss Weighting vs. Noise Sampling** - The authors replace sampling $\\sigma$ from a log-normal distribution (an importance-sampling strategy that emphasises intermediate noise levels) with an explicit loss-weighting scheme. While these approaches are equivalent in the continuous-time limit—when the loss is written as an integral over time [1]—their gradient estimates under minibatch training would differ in practice. In general, incorporating weights into the noise sampling distribution often reduces variance, so the chosen loss-weighting strategy may be suboptimal from a training perspective.\n2) **First-window initialisation** - To avoid the extra hyperparameters introduced by Ruhe et al. [2], ERDM trains a separate “initial” model for the first rollout window. However, this still entails training and tuning a second network. In this work, the trained EDM model acted as a baseline anyway, but on large-scale datasets (e.g., native-resolution ERA5), maintaining two models would probably be more demanding than tuning some additional hyperparameters.\n3) **Missing Baseline: Original RDM formulation** - The paper positions ERDM as an improvement over rolling diffusion, yet it does not compare directly to the original DDPM formulation of RSDM from [2]. Demonstrating gains over a DDPM-based rolling model—as well as over plain EDM—would more convincingly attribute performance improvements to the EDM design choices within the rolling framework.\n4) **Experiments Setup and Benchmarks**\n  - **Navier-Stokes** - Only five test trajectories are evaluated, which may not capture the full variability of initial conditions of the system. Moreover, I think the authors could include one or two additional baselines in order to be able to correctly position the performance of the method. For example, PDE-Refiner [3] showed good performance on modelling the KS and Kolmogorov systems. Another related approach is DyDiff [4]. Dyffusion does not compare to any of these, and I am not sure it's clear that Dyffusion is currently SOTA.\n  - **ERA5 reanalysis** - As the authors acknowledge, comparing ERDM against NeuralGCM ENS and Graph-EFM—both evaluated on 2020—weakens the analysis due to potential distribution shifts in weather patterns during 2020/2021 (e.g., pandemic-related anomalies). Likewise, the IFS ENS comparison is flawed because IFS ENS is verified against operational analyses rather than the ERA5 reanalyses, further undermining its relevance. These mismatches leave the autoregressive EDM baseline as the only truly comparable benchmark. To strengthen their claims, the authors should: (1) include an RDM baseline following Ruhe et al. [2], and (2) consider adding a SOTA weather model—such as GenCast—to contextualise the performance gap, even if the comparison is inherently imperfect given ERDM’s coarser training data and limited computational budget. Without these additional benchmarks, it remains difficult to fully assess and contextualise the benefits of the proposed framework.\n  \n    Moreover, the results are fairly hard to interpret in the way they are currently presented. I personally think that a better way of visualising and summarising these results is through score cards.\n5) **Memory Requirements** - The paper does not quantify the additional computational and memory costs of ERDM versus baseline EDM. As weather and fluid-dynamics applications move toward higher spatiotemporal resolutions, understanding ERDM’s resource footprint is crucial for assessing its practical scalability.\n6) **Minor Issues**\n  - **Energy Spectra** - Energy spectra would be useful in the Navier-Stokes experiment too. I am expecting a similar behaviour as in the ERA5 experiment (shown in Figure 12), but it would be worth introducing them for completeness.\n- **Appendix References** - There are minor misreferences in the Appendix (e.g., the noise-prior discussion is in C.4, not C.6; the network-topology section is supposed to be in C.5., but there is no such section).\n- **Sampler Efficiency Comparison** - When comparing Heun versus Euler, it would be informative to compare performance under a fixed budget of neural function evaluations (e.g., 20 Euler steps vs. 10 Heun predictor + 10 corrector steps) to account for the doubled evaluations in Heun’s corrector step.\n\n**References**\n\n[1] Dieleman, S. (2024). Noise schedules considered harmful. Sander.ai.\n\n[2] Ruhe, D. et al. (2024). Rolling diffusion models. arXiv:2402.09470.\n\n[3] Lippe, P. et al. (2023). PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers. arXiv:2308.05732.\n\n[4] Guo, X. et al. (2025). Dynamical Diffusion: Learning Temporal Dynamics with Diffusion Models. arXiv:2503.00951.\n\nQUESTIONS:\n1. The authors state that it is important to tune the noise schedule parameters $\\sigma_{\\text{min}}$, $\\sigma_{\\text{max}}$, and $\\rho$ during training. For example, $\\rho$ is chosen such that the snapshots are under less noise than in the normal EDM formulation. How dataset-dependent are the most reasonable choices for these hyperparameters? For example, in the context of Navier-Stokes, would a different temporal resolution require re-tuning of these hyperparameters, given that the underlying dynamics change (e.g. if we use 10x a smaller timestep, the states would be much more \"similar\" to each other and perhaps we should add even less noise). \n2. Could the authors comment on why they haven't included an RDM-based baseline too (with the DDPM formulation)?\n3. Could you clarify how the sampling procedure looks like when $N$ is not an integer?\n4. How much of a limitation are the increased memory and computational complexity, with the vision that such a model could be used to model, for example, higher resolution weather data (0.25° or even 0.1° ERA5)?\n5. The SSR shows that ERDM goes from a region of underdispersion (up to about 10 timesteps) to overdispersion (10-20 timesteps) followed by underdispersion again. In contrast, the trend for plain EDM models is monotonically decreasing. Can the authors comment on this---what are the potential causes of overdispersion at medium timesteps, and whether the SSR could be stabilised through other hyperparameter choices?\n\nLIMITATIONS:\nThe authors mention the increased memory and computational cost of their method, but do not necessarily discuss how practical this would become with higher resolution (in space and time) data of practical significance. This would be especially relevant in the weather modelling field, where the community is heading towards 0.25° or even 0.1° spatial resolution and up to 1 hour time resolution.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes the Elucidated Rolling Diffusion Models (ERDM), a framework for probabilistic forecasting of high dimensional dynamical systems. It incorporates a loss weighting scheme, an efficient initialization strategy, and a bespoke hybrid sequence architecture to facilitate improved performance. The authors demonstrate the effectiveness of the proposed method through a series of comprehensive experiments.\n\nSTRENGTHS AND WEAKNESSES:\nPaper Strengths: \n\n1. The application of elucidated rolling diffusion models to explicitly capture the sequential nature of data and the escalating physical uncertainty inherent in chaotic systems is compelling. The authors validate the effectiveness of their approach through a comprehensive series of experiments.\n\n2. The paper is relatively well-organized and clearly written. And it is also easy to be followed by later works.\n\nWeaknesses:\n\n1. The motivation of the paper appears to be unclear. It is not evident what limitations in existing methods the proposed techniques, such as the weighted loss and initialization, are intended to address. The contributions seem more like engineering refinements rather than principled innovations.\n\n2. The authors provided some visual results in the form of charts, but there is a lack of objective quantitative tables to evaluate the performance of the proposed method compared to baseline approaches.\n\n3. The paper lacks an analysis of its limitations. Moreover, the experiments should include measurements of inference time and computational overhead, especially considering that diffusion models often entail significant time costs during inference.\n\nQUESTIONS:\n1. The authors should provide a clearer explanation of the rationale and applicability of diffusion models for this task.\n\n2. In the Rolling EDM noise schedule, the authors identify the setting of the $\\rho$ as particularly important. However, they only provide a simple empirical value without discussing its optimality or offering a thorough analysis.\n\nLIMITATIONS:\nThe authors could provide a more thorough discussion of the societal limitations and potential impact of their work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces the Elucidated Rolling Diffusion Model (ERDM) that incorporates the general framework of Rolling Sequence Diffusion Models and design choices in EDM for spatial temporal probabilistic forecasts. In addition, the authors propose an uncertainty-aware weighting scheme, leverage pretrained next-step EDM models to initialize the first forecast window and design a hybrid 2D U-Net with causal temporal attention to capture spatial and temporal dependencies. ERDM outperforms baseline models on 2D Navier Stokes simulation and is competitive against state-of-the-art models on ERA5 data. With improved performance and more efficient training, ERDM promises accurate and skillful forecasts for other spatiotemporal forecasting problems.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* The proposed ERDM improves CRPS on the Navier Stokes problem compared to the best EDM and DYffuison models. Also, the ERDM is competitive with IFS ENS and NeuralCGM model with improved training efficiency.\n* ERDM captures the temporal dependency and automatically account for the horizon uncertainty with the progressive noise schedule, leading to better long-term rollout.\n* During sampling, the progressive noise schedule potential enables fewer solve steps than GenCast like model. After each denoised first step, the reuse of the partially denoised tail naturally moves the forecasting forward.\n\nWeakness:\n* The short-range performance on ERA5 data is inferior to IFS ENS and NeuralCGM. The authors attribute this to the less optimal initialization of the sequence. However, this needs further experiments to validate.\n* It is unclear how the window size W affect the performance. Accordingly, the choice of $\\sigma_{\\rm min}$ and $\\sigma_{\\rm max}$ needs more discussion.\n* the work demonstrates that diffusion models can yield calibrated, medium range probabilistic weather forecasts at modest compute cost, suggesting relevance to climate down scaling, oceanography, and other chaotic systems; however, the iterative sampling still imposes latency comparable to the EDM baseline, limiting immediate operational adoption.\n* Uses existing building blocks (EDM, Heun, causal attention); conceptual novelty is incremental\n\nQUESTIONS:\n* Since the authors mentioned the inferior performance of the ERDM is due to the EDM prediction initialization, could the authors replace the current initialization with IFS ENS or  deterministic model (e.g. FourCastNet)  and repeat the experiment? to close the gap\n* The noise level for the first snapshot within the window is small, does this mean it takes fewer solves to fully denoise the first snapshot? Does this have any implications of the number of NFEs compared to other models? Also, have you profiled whether the 3 D attention layers or memory bandwidth dominate? An ablation with a lighter temporal backbone would clarify where to optimize.\n* How does the choice of window size and accordingly $\\sigma_{\\rm min}$ and $\\sigma_{\\rm max}$ affect the performance? \n* What is the energy cost per 15 day ensemble forecast relative to IFS ENS, measured on the same hardware? This would strengthen the efficiency claim and inform practitioners.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}