Probablistic Emulation of a Global Climate Model with Spherical DYffusion
Authors: Salva Rühling Cachay
Brian Henn
Oliver Watt-Meyer
Christopher S. Bretherton
Rose Yu
Venue: NeurIPS 2024 spotlight (2024)
Keywords: deep learning
climate modeling
climate science
ai for science
climate change
Primary area: machine_learning_for_physical_sciences
TL;DR: We present the first conditional generative model for efficient, probabilistic emulation of a realistic global climate model, which beats relevant baselines and nearly reaches a gold standard for successful climate model emulation.

Abstract: Data-driven deep learning models are transforming global weather forecasting. It is an open question if this success can extend to climate modeling, where the complexity of the data and long inference rollouts pose significant challenges. Here, we present the first conditional generative model that produces accurate and physically consistent global climate ensemble simulations by emulating a coarse version of the United States' primary operational global forecast model, FV3GFS.
Our model integrates the dynamics-informed diffusion framework (DYffusion) with the Spherical Fourier Neural Operator (SFNO) architecture, enabling stable 100-year simulations at 6-hourly timesteps while maintaining low computational overhead compared to single-step deterministic baselines.
The model achieves near gold-standard performance for climate model emulation, outperforming existing approaches and demonstrating promising ensemble skill.
This work represents a significant advance towards efficient, data-driven climate simulations that can enhance our understanding of the climate system and inform adaptation strategies. Code is available at [https://github.com/Rose-STL-Lab/spherical-dyffusion](https://github.com/Rose-STL-Lab/spherical-dyffusion).

Probabilistic Emulation of a Global Climate Model with Spherical DYffusion
Salva Rühling Cachay UC San Diego
Brian Henn Allen Institute for Al
Oliver Watt-Meyer Allen Institute for Al
Christopher S. Bretherton Allen Institute for Al
Rose Yu UC San Diego
Abstract
Data-drived deep learning models are transforming global weather forecasting. It is open question if this success can extend to climate modeling, where the complexity of the data and long inference rollouts pose significant challenges. Here, we present the first conditional generative model that produces accurate and physically consistent global climate ensemble simulations by emulating a coarse version of the United States' primary operational global forecast model, FV3GFS. Our model integrates the dynamics-informed diffusion framework (DYffusion) with the Spherical Fourier Neural Operator (SFNO) architecture, enabling stable 100-year simulations at 6-hourly timesteps while maintaining low computational overhead compared to single-step deterministic baselines. The model achieves near gold-standare performance for climate model emulation, outperforming existing approaches and demonstrating promising ensemble skill. This work represents a significant advance towards efficient, data-driven climate simulations that can enhance our understanding of the climate system and inform adaptation strategies.
Introduction
Climate models are foundational tools used understand how the Earth system evolves over long time periods and how it may change as a response to possible greenhouse gas emission scenarios. Such climate simulations are currently very expensive to generate due to the computational complexity of the underlying physics-based climate models, which must be run supercomputers. As a result, scientists and policymakers are limited exploring only a small subset of possibilities for different mitigation and adaptation strategies [48].
Training relatively cheap-to-run data-driven surrogates emulate global climate models could provide compelling alternative [15]. Although recent deep learning models are on the verge of transforming the conceptually similar field of medium-range rrange weather forecasting [5, 38, 11, 51], these advances do not directly transfer to long-term climate projec-
Tiraeste meer
tions [37]. Indeed, most such models only report forecasts up to two weeks into the future and may diverge become physically inconsistent over longer simulations. In contrast, climate projections demand accurate and stable simulations of the global Earth system spanning decades or centuries, requiring reliable reproduction of long-term statistics.
Figure 1: Weather performance (x- axis) is not strong indicator of cli- mate performance (y-axis). Each dot corresponds to distinct sample or checkpoint epoch.
Code available http::/hhw....oooeeeee./../aaaaa
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

In Figure I we quantitatively show this divergence between the medium-range range weather forecasting skill of ML models (measured as the average RMSE on 5-day forecasts) and their performance on longer climate time scales (measured as the RMSF of the 10-year time-mean). We have verified that this finding holds regardless of the analyzed variable and the proxy used for weather performance, which we discuss in more detail in Appendix E.3. Heuristically, optimizing weather skill ensures that a climate model takes a locally accurate path around the climate "attractor', but it does not guarantee that small but systematic errors may not build up to distort that simulated attractor to have biased long-term climate statistics. While this is a little-discussed observation in the ML community, the climate modeling community has documented it for physics-based based models [1].
recent breakthrough is a deterministic surrogate called ACE (Ai2 Climate Emulator) [67], which remains remarkably stable and physically consistent over 10-year simulations at 6-hourly time steps, forced by time-varying specified sea-surface temperature and sea-ice. Its success can be attributed to careful data processing, problem design, and the Spherical Fourier Neural Operator (SFNO) [8] architecture. ACE is trained to emulate the United States' primary operational global forecast model, the physics-based FV3GFS [73], which is operationally used at the US National Weather Service and US National Centers for Environmental Prediction. ACE produces encouragingly small ten-year mean climate biases (i.e. biased long-term averages), but they are still significantly larger than the theoretical minimum imposed by internal variability of the reference physics- mhasics-based model.
ACE's s deterministic nature restricts its ability to model the full distribution of climate states or to facilitate ensemble simulations, which involve drawing multiple samples from the same model. These capabilities are crucial for climate modeling, as they enable better uncertainty quantification, more robust and physically consistent predictions, and a deeper understanding of potential future climate scenarios and associated risks [32]. While it is possible to ensemble a deterministic model by perturbing its inputs, this approach often leads to under-dispersed (i.e. overly confident) ensembles compared to generative or physics-based approaches [57]. Even then, the problem remains that due to optimizing them on MSE-based loss functions, the deterministic predictions may degrade to a mean prediction for longer forecast time scales and underestimate unlikely events [9].
generative modeling approach, particularly the use of diffusion models [59, 25]. appears to be a promising solution to these challenges. However, standard diffusion models are computationally intensive to train and sample from. This complexity poses significant problems for climate modeling because: 1) atmospheric data is extremely high-dimensional, making the use of video diffusion models [63, 27, 69, 58, 26, 23] prohibitive, even more so as this class of models still struggle with videos longer than a few seconds; and 2) the sampling speed of standard diffusion models is particularly problematic for long, sequential inference rollouts. For instance, generating a single 10- year-long simulation, as in our experiments, with a standard autoregressive diffusion model [35, 51] that uses N diffusion steps would require 14600 x N neural network forward passes. a second-order solver for sampling is used [31, 51], this number doubles. Even with N small as 30, this results in half a million forward passes to generate a single sample trajectory, severely limiting the potential of data-driven models to serve as fast surrogates for expensive physics-basee models.
As solution to this computational problem, we build upon the dynamics-informed diffusion model framework, DYffusion, from Rühling Cachay et al. [57], which caps the computational overhead at inference time (as measured by the number of neural net forward passes) to less than 3x as much as for a deterministic next-step forecasting model such SFNO ACE. Unfortunately, the original DYffusion method relies on an UNet-based architecture designed for Euclidean data rather than physical fields on a sphere. As we show in Figure I, this mismatch of inductive biases becomes more problematic at the long climate time scales that we focus on in this paper.
We address these limitations by carefully integrating the DYffusion framework with the SFNO architecture from Bonev et al. [8], and the data and evaluation procedure from Watt-Meyer et al. [67]. To achieve this integration, we extend SFNO with time conditioning and inference stochasticity modules. Our proposed framework, Spherical DYffusion, achieves strong results: On average, across all 34 predicted fields, our model reduces climate biases to within 50% of the reference model, which is more than 2x and 4x lower than the best baselines. For critical fields, such as the derived total water path quantity, our method achieves results within 20% of the reference model, representing 5x improvement over the next best baseline (see Fig. 2). Additionally, our method proves effective for ensemble climate simulations, reproducing climate variability consistent with the reference model and further reducing climate biases towards the theoretical minimum through ensemble- ensemble-averaging
2

---

Surface pressure (Pu)
Total water path (egion")
RMSE rel. Reference (in
200
Retwencc
Dart
5T
Releeecce
Dur
Dur
ACC
Figure 2: RMSE of 10-year time-means for subset of important fields. The leftmost bar in the first two subplots shows the reference noise floor, determined by comparing ten independent 10-year reference FV3GFS simulations with the validation simulation. The scores computed using the mean over these ten simulations (a proxy for an "ensemble prediction") are shown in light shade. The subsequent bars show the corresponding scores for our method and the deep-learning baselines, using 25-member ensemble for the probabilistic methods (all except ACE, which only reports scores for its single deterministic prediction). Scores computed using the ensemble- mean prediction are shown in light shade. The dark shaded bar on top indicates the performance drop when using single member's prediction only, with error bars representing the standard deviation over the 25 different member choices. The rightmost subplot displays the average time-mean RMSE of the ML--based emulators relative to the reference across all 34 variables. On average, our method's time-mean RMSEs 50% higher than the noise floor, which is less than half the average RMSE of the next best method, ACE. When using the 25-member ensemble mean prediction, this reduces to 29.28%.
Our generative model is a leap forward toward purely ML-based large ensemble climate projections that are both efficient and accurate. Our main contributions are:
1. We present the first conditional generative model for probabilistic emulation of realistic climate model, with minimal computational overhead over deterministic baselines.
2. We carefully integrate two distinet frameworks, ACE and DYffusion, including additional modifications to the SFNO architecture such as conditindi modules.
3. We show that our integrated method performs considerably better than relevant baselines in terms of reduced climate biases, ensemble-based climate modeling, and consistent variability of the climate predictions.
4. We show that short-term weather performance does not necessarily translate to accurate reproduction of long-term climate statistics.
2 Related Work
ML for weather and climate modeling. There are fundamental differences in weather and climate modeling. Climate refers to the average weather over long periods of time². While weather forecasting focuses on short time scales in the order of days or weeks, climate modeling simulates longer periods of decades to centuries. Weather forecasting is primarily an initial-value problem, for which it is important to analyze short-term time-specific predictions, Climate modeling is primarily a boundary- condition (or foreing-driven) problem [65], characterized by long-term averages and distributions.
Deep learning-base models have emerged as a much more computationally efficient alternative to traditional physics- based numerical weather prediction (NWP) models, showing impressive skill for deterministic medium-range weather forecasting [49, 33. 5, 8. 10, 47, 38]. This success has been more recently extended ensemble-based probabilistic weather forecasting [34, 51]. An alternative approach is hybrid modeling, where a physics- based component is complemented by ML-based parameterizations or corrections [52, 71, 1, 36, 70, 34). At longer lead times, when weather becomes chaotic and less predictable, the ensemble mean prediction of a physics- based or probabilistic ML-based ensemble improves deterministic metries such as root mean squared error (RMSE) over non-ensembled methods [51, 34, 53].
However, advances in weather forecasting hardly transfer to long- long-term climate projections. Fully data- driven models fail to maintain stability beyond two-week-ahee forecasts, as errors accumulate over their autoregressive rollouts. Weyn et al. [68] and Bonev et al. [8] showed stable forecasts for horizons of up to six weeks and one year, respectively. Only recently, Watt-Meyer et al. [67] notably achieved stable and accurate 10-year simulations, followed by another deterministic SFNO-based climate emulator showing promising results using four prognostic variables [22]. Easier, but less flexible and
For example, http:///ccaaaeeevveecccc../aattt
3

---

informative, alternatives to full-scale temporal modeling of atmospheric dynamics, include emulation of annual means given an emission scenario [66, 30, 46, 42], temporal super-resolution monthly means [4], or debiasing climate model output [3. 7, 45].
Diffusion models. Diffusion models [25, 59-61] have demonstrated significant success in generat- ing data such as natural images and videos. While traditionally formulated for finite-dimensional spaces, these models have been extended to function spaces [40]. Their direct applications to autore- gressive forecasting [35, 51] and downscaling [64, 43, 20] of physical data have shown promising results. However, these approaches inherit the computational complexity associated with training and sampling from standard diffusion models. This is particularly prohibitive for autoregressive predictions climate time scales, the total number of neural network forward passes increases proportionally with the number of sampling steps, typically ranging from 20 to 1000. Consequently, recent research that leverages insights from diffusion models balance predictive performance and sampling speed appears more promising for assessing their viability in climate simulations [57, 41J. While diffusion models traditionally rely on U-Net architectures [55, 13], vision transformers have shown promising results in image synthesis [50, 29, 24]. Our work explores a different, neural operator- based, architecture for Earth data.
3 Background
We first define the problem and then introduce the key components in our framework, namely DYffusion and SFNO. We abbreviate a time series of tensors yo..... Me with yo...
3.1 Problem Setting
Our goal is to learn the probability distribution P( No. fo.H) over a horizon of H time steps, conditional on initial conditions 2n and scenario of forcing variables f Jo. (i.e. time-varying boundary conditions). In our paper, these forcings correspond to prescribed sea surface temperatures and incoming solar radiation (see Section 5.1), leaving it to future work to force based on greenhouse gas emission scenarios explicitly. Each H; E IR" represents the state of the atmosphere at a given timestep, t, consisting of two- and threc-dimensional surface and atmospheric variables across latitude-longitude grid. These variables, which serve as both input and output, are referred to as prognostic variables. We assume constant time interval between successive time steps t and t ++1. To make training feasible, it is necessary to train on a much shorter horizon h, i.e. learn the distribution P(æ++11:44 æ.,, f f+++),, ), and apply the model autoregressively. This process begins with P(ær æn, f f and continues until reaching time step H at inference time.
3.2 Diffusion Models and DYffusion
Diffusion models can be seen a general paradigm to learn the target distribution p(s( by iterating over N diffusion steps of a forward or reverse process. We denote the states of each diffusion step with (*)), using a superscript n to clearly distinguish them from the physical time steps of the data Tt. Standard diffusion models [59, 25, 31], initialize the reverse process from simple isotropic Gaussian distribution (N) ~ NN(O, I) so that as - 0 the intermediate states s(n)) are gradually denoised towards real data sample (0))
In Cold Diffusion [2]. this paradigm is extended to more general data corruption processes such as blurring. Rühling Cachay et al. [57] propose DYffusion, by adapting cold diffusion models to forecasting problems. The key idea is make the forward and reverse processes dynamics-informed by directly coupling them to the physical time steps of the data. That is, the reverse process is initialized with s (N)) = Zn and iteratively evolves jointly with the dynamics of the data æI...., T/-1 to reach the data at some target time step, s0)) 3h.-
In DYffusion, the forward and reverse processes are informed by temporal dynamics in the data and do not rely on data corruption. Their only source of stochasticity comes from using a stochastic neural network as operator for the forward process and is implemented by using Monte Carlo (MC) dropout [19]. This forward process essentially corresponds to a temporal interpolator network, while the reverse process is represented by a multi-step forecasting network. Thus, compared to standard diffusion models, DYffusion requires training one more neural network, which they propose doing in separate stages, beginning with the interpolator model. Due to its informe

---

AIII
Insene SET
3t
1++1
SIFT
SENO
Figure 3: The diagram shows how proposed approach functions at inference time. Given an ini- tial condition 2; and forcings f fut+h- method uses the DYffusion framework, integrated with two SFNO backbone networks, generate predictions for the next time steps based on an alternation of direct multi-step forecasts and temporal interpo- lations. To simplify the visualization, we exclude the facts that the interpolator network, SFNO,, is conditioned on æ, and f, in addition to an estimate of ætth- We also exclude the timeccondiioniig of both networks. To forecast more time steps beyond t + h. our method applied autoregressively.
Impars
Time Enicolding
Figure 4: Diagram of one of the blocks of the modified SFNO architecture for our proposed method. The full architecture con- sists of sequence of such blocks. Our newly introduced time-conditioning mod- ules correspond to the Time Embedding, followed by the MLP on the right, and the scale-shift operation. Our method relies on dropout, which is part of the two-layer MLP on the top. SFNO-based baselines use the same architecture and hyperparameters without the time embedding module.
nature, DYffusion was shown to be faster at sampling time and more memory-efficiet than standard diffusion models, while matching or outperforming their accuracy.
3.3 Spherical Fourier Neural Operator (SFNO)
The SFNO architecture [8] extends the FNO framework from Li et al. [39] to spherical data and symmetries such as the Earth. FNOs efficiently model long-range interactions the Fourier space, but because the underlying Fast Fourier Transform is defined on Euclidean domain, this can lead to modeling artifacts. SFNOs overcome this issue by using the spherical harmonic transform (SHT) [14], generalization of the Fourier transform, instead. The SFNO model achieves higher long-term stability of autoregressive rollouts than the FNO model, showing stable forecasts of Earth's atmospheric dynamics for up to 1-year-long rollouts at six-hourly time steps. The ACE model from Watt-Meyer et al. [67] is based on the SFNO architecture, modifying some of the hyperparameters and the grid used for the first and last SHT of the SFNO. We the SFNO configuration from ACE in our experiments.
4 Spherical DYffusion
SFNO and ACE are deterministic models that cannot be readily used for uncertainty quantification ensemble-based climate modeling. DYffusion introduces efficient diffusion-based approach specifically for forecasting problems but only for Euclidean data. Thus, we propose Spherical DYffusion, deep generative model for data-driven probabilistic climate simulations that carefully integrates SFNO and DYffusion into an unified framework.
DYfffusion requires two neural networks that used for temporal interpolation and direct multi-step forecasts. In the original framework, these are UNet-like networks. For our approach, we propose to replace them with modified versions of the SFNO architecture, which denote by SFNO, and SENO,, respectively.
Training. We follow the original training procedure from DYffusion, complementing it with the of the input-only forcing variables. That is, for specified training horizon h, these networks
5

---

trained in two stages such that for sequences of prognostic data Tea+h and forcings festh
S (xt, Leth: f (E) 2 a+ti
sFNO, (2,, Lith f 15) (t5..)) Rs with.
where E (1,...,h- h 1} and we use E (0, 1, ..... h 1}, defining SFNO, (# [c) 2..- In
our experiments, we use h = 6. Here, & refers to the random variable representing the interpolator network's inference stochasticity. We discuss its implementation further below. The forecaster network, SFNO,, is deterministic. The full training scheme is defined in Algorithm 1.
Inference. At inference time, we follow the DYffusion sampling scheme based on cold sampling [2]. Essentially, we start with the initial conditions zo generate a first forecast of time step h through a forward pass of the forecaster network, i.c. 3 = SFNO, æo. fo. 0). Given this prediction, we can now use the interpolator network to interpolate 1 SFNO (Zog Ry. fo. KK). In practice, cold sampling applies correction term to this estimate. The prior forecast of 2% can now be refined with 2b SFNO, (1. f1. 1). The alternation between forecasting and interpolation continues until SFNO,, predicts - and the forecaster network performs a last refinement forecast of time step h, conditioned on the time j = h and interpolated sample - After this final forecast of at, the process is repeated autoregressively. starting with 2 the new initial condition. This slightly simplified sampling process is illustrated in Figure and fully described in Algorithm 2. Repeating this sampling process multiple times using the same initial conditions will lead to an ensemble of samples, thanks to the interpolator network being stochastic.
SFNO time-conditioning. To use SENO as described above, it is necessary to implement time- conditioning modules that allow the interpolator and forecaster networks to be conditioned on the time and j. respectively, given that the original SFNO architecture does not support this. We follow the same approach taken by standard diffusion models [13]. which consists of transforming the time condition into vector of sine/cosine Fourier features at 32 frequencies with base period 16, then pass them through 2-layer MLP obtain 128-dimensional time encodings that are mapped by linear layer into the learnable scale and offset parameters. We scale and shift the neural representations of every SFNO block directly following the normalization layer and preceding the application of the SFNO spectral filter, as shown in Figure 4.
SFNO inference stochasticity. A stochastic interpolator network, made explicitly through the random variable & above, was shown to be a key design choice the original DYffusion framework. However, to the best of our knowledge, the SFNO model has been only used for deterministic modeling. We overcome this issue through MC dropout [19], i.c. enabling dropout modules [62] at inference time. Following the original SFNO implementation (of training- -imee-onm dropout), we propose to use a dropout module inside the MLP of cach SENO block. In addition, we enable stochastic depth [28]-also known drop path-at inference time at rate of 0.1. Stochastic depth randomly skips a whole SFNO block. When this happens the whole block reduces to the identity function, since only the residual connection is enabled. To the best of our knowledge, this has not been explored before as a source of inference stochasticity.
5 Experiments
5.1 Dataset and Experimental Setup
To compare our proposed method against ACE [67], we use the same dataset, training and evaluation setup. The dataset consists of 11 distinct 10-year- simulations from the state-of-the-arn global atmospheric model FV3GFS [73], saved every 6 hours. The forcings consist of annually repeating climatological sea surface temperature (1982-2012 average) and incoming solar radiation. Greenhouse gas and aerosol concentrations are kept fixed. The data was regridded conservatively from the cubed- sphere geometry of FV3GFS a 1 Gaussian grid, and filtered with a spherical harmonic transform round-trip to remove artifacts in the high latitudes. We train on 100 years of simulated data from FV3GFS, and evaluate the models on how well they can emulate a distinet 10-year-long validation simulation (i.e. H = 14600- = 10 x 365 x 4). The 11 simulations form an initial-condition ensemble, where each simulation independent of the other-after some discarded spinup time-due to the chaoticity of the atmosphere [32]. For more details, see Appendix B.

---

5.2 Baselines
We compare with the following baselines for climate projection.
ACE [67] applied the SFNO architecture to the FV3GFS dataset described above.
ACE-STO: We re-train ACE but use MC dropout, in the same way how it is applied in s for our method, to generate stochastic predictions.
DYffusion [57]: We train DYffusion using the original UNet-based architecture as its interpolator and forecaster neural networks.
Reference [73]: physics- based FV3GFS climate model simulations. We use the ten training simulations to create 10-member reference ensemble that we use to more robustly estimate the 'noise floor' introduced in [67] and to compare the variability of the reference ensemble with sample simulations from our method. Note that this reference is not appropriate for weather forecasts given that it is initialized from different initial conditions.
is worth noting that ACE also compared their results against a physics-based baseline called C48, which corresponds to running FV3GFS at half the original spatial resolution. This makes C48 around 8x less computationally costly to run compared to the reference simulations but was shown to underperform ACE, which our method is shown outperform in the experiments below.
For ACE, we directly use the pre-trained model from the original paper. ACE was trained on a next-step forecasting objective based on MSE loss. For ACE-STO, we re-train ACE from scratch with the only difference being that we use a dropout rate of 10% for the MLP in the SFNO architecture. We use the same dropout rate for the interpolator model, SFNO,, in method. For both DYffusion and our approach, we choose h = 6. That is, these models are trained to forecast up to 36 hours into the future. We use the same training and sampling procedures for both, the only difference being the underlying neural architectures.
Runtime analysis. In Table we report the computational complexity in terms of the number of neural function evaluations (NFEs) needed to predict f time steps, and the wall clock runtime for simulating one complete val- idation trajectory of 10 years. For our method, NFEs is not 3h because in the first and last iter- ation we do not need to actually run line 8 and lines
Table I: Computational complexity of the different deep learning methods in terms of: 1) the number of neural function evaluations (NFEs) needed to predict h time steps. and 2) Total inference runtime (simulating 10 years), including the time needed to compute metrics (in hours:minutee). N refers to the number of diffusion steps which usually ranges
between 20 to 1000.
in Algorithm 2, respectively. Our runtime analysis confirms that the computational overhead at inference time for using our method, is less than 3x much for deterministic next-step forecasting model such as SFNO or ACE. This enables our method to provide sig- nificant 25x x speed-ups and associated energy savings over using the emulated physics-based model, FV3GFS.
Method
NFE
Runtime
ACE/ / SFNO Standard diffusion Ours
01:08
Nh
N/A
3(h 1)
02:56
Physics-based FV3GFS
N/A
78:04
FV3GFS (2x coarser)
N/A
45:38
All models were trained A6000 GPUs using distributed training on 2 up to 8 GPUs, ensuring that the effective batch size remains the same (see Figure 8). For a fair inference runtime comparison measuring the wall clock time needed to simulate 10 years (i.e. one full validation rollout), we run all deep-lcarning basclines on one 100 GPU. We also include the runtime for the physics-base based FV3GFS climate model which was on 96 cores (24 cores for the 2x coarser version) of AMD EPYC 7H12 processors. The deep learning methods are not only much faster, but also much more energy-efficient than FV3GFS.
For illustrative purposes, we also report the complexity of a standard autoregressive diffusion model [25, 35, 51] approach in terms of the number of neural function evaluations (NFEs) needed to predict time steps, totaling to Nh where N' is the number of sampling steps required to reverse the diffusion process. N usually ranges from between 20 to 1000. This makes the use of such an approach less attractive for climate emulation since the resulting inference runtime would not offer as significant speed-ups over the physics-based reference model.
7

---

Reference RMSF 0.36, bas 0.03
Ours RMSE C 39, hiss 0.03
ACE RMSF - 0.65, bas 0.02
Figure 5: Global maps of the y0-y time-mean biases of a single sample from the reference noise floor simulation, our model, and the ACE baseline for the total water path field. Each subplot reports the global mean RMSE and bias of the respective bias map. Our model reproduces biases similar location and magnitude to the reference noise floor, suggesting they are mainly due to internal climate variability rather than model bias, while the baseline exhibits larger climate biases.
5.3 Climate Biases
The most crucial quality of an ML-based climate model its ability to reproduce the climatology of the emulated reference system, i.e. the long-term average ("time-mean") of weather states. The time-mean of the validation simulation is defined 1 " zt.- The time-mean for each model is defined as if Σt= Le, where à; is the model's prediction for time step t. These two quantities are then compared against each other using the bias, i.e. prediction target, and root mean squared error (RMSE) as key metries of interest for analyzing climate biases. For the probabilistic methods, i.e. ours, DYffusion, and ACF-STO, we generate simulation ensembles by sampling from the model multiple times using the same initial conditions. Unless specified otherwise, all ensemble 25 ensemble members. We evaluate the ensemble performance using two metrics: the RMSE of the ensemble-mean prediction (
Metrics.
results are based on
EH Σe= Σ kt,e) and the RMSE of
1
member-wist time-meam means ( Σ/ et,e), where indexes individual ensemble members. For the latter, standard deviations are computed over the member-wise errors. The corresponding "optimal noise floor" for the ML-based emulators is estimated by comparing the validation simulation with the 10-member reference ensemble. All metrics, which are fully defined in Appendix D, are weighted by grid cell area. It is important acknowledge the potential for improving the estimate of the "noise floor" based on statistical significance testing and improved metrics [21].
Quantitative analysis. Our method and all baselines consistently produce stable long- term climate simulations without diverging. In Figure 2, we compare the RMSE of the time-means of the reference, our method, and all baselines.
Our method significantly reduces climate biases compared to baseline methods across most fields, with errors often closer to the reference simulation's 's noise floor than to the next best bascline. The performance of ACE notably degraded when made stochastic through MC dropout. Similarly, direct application of DYffusion fails to accurately reproduce long-term climate statistics. Both these baselines are unable to outperform our proposed careful integration of these two paradigms leads to skillful climate model emulator: On average, our method's 's time-mean mean RMSEs are only 49.8 49.36% higher than the noise floor, which is less than half the average RMSE (110.47%) of the next best method, ACE.
even match the scores of the deterministic ACE baseline. Only
Ensemble averaging significantly enhances our method's performance, reducing climate biases by 29.28% on average across all variables. As shown by the light shading in Fig. 2, the ensemble-meat predictions consistently achieve lower time-mean RMSEs compared single-member predictions (dark shading). This ensemble-base DYffusion, where ensemble averaging proves less effective, and from ACE, where initial-condition perturbations would be required for ensembling. Additional results for more fields are available in Figure of the Appendix. Our comprehensive evaluation Table includes ensemble metrics such as the Continuous Ranked Probability Score (CRPS) and spread-skill ratio. The results demonstrate that our method outperforms alternatives in emulating the 10-year time-mer menn climatology of the reference model for most variables and metrics. However, some challenges remain, particularly in matching the reference ensemble's performance for stratospheric (level 0) variables and in achieving better ensemble scores.
improvement distinguishes our approach from ACE-STO and
8

---

Qualitative analysis. In Figure 5 we show the corresponding global maps of the time menn biases for the total water path (TWP) field. Our model reproduces small biases of remarkably similar location and magnitude to the "perfect-model"" reference simulation, with spatial pattern RMSEs of approximately 1% of the global-time-mean TWP. The perfect-model bias is due to unforced random decadal variability in the mean climate of the reference model each 10-year period has randomly different weather, leading to a slight difference in 10-year time-mean averages across this weather. The reference bias is due to comparing one such decade simulated with the reference model with other simulated decades; its spatial pattern depends strongly on which decade is used for computing the reference model climatology. That our model (trained on 100 years of output) reproduces this pattern suggests that it emulates the long-term (e.g. century-long time-mean statistics of the reference model even more accurately than a 10- 10-year-mean RMSE can reliably resolve. On the other hand, the baseline ACE model exhibits somewhat larger climate biases, indicative of an actual, albeit small model deficiency that is already evident with single 10-year estimate of climatology.
In Appendix E.4, we visualize two sample 10-year trajectories simulated by Spherical DYffusion as well as the corresponding validation simulation from FV3GFS. Supplementary videos demonstrate the full temporal evolution of key derived variables: near-surface wind speed and total water path'. The emulated fields demonstrate high realism, closely mimicking the patterns and variability observed in actual climate model outputs. This showcases Spherical DYffusion's 's capability to generate plausible and physically consistent climate scenarios over decadal timescales.
Climate variability. Above, we have verified that sampling 10-year- long trajectories from our model pro- duces encouragingly low ensemble mean and member-wise time-mean biases. An important feature of cli- mate is its natural variability on time scales of years, decades, or even cen- turies even when external forcings (e.g. sunlight or greenhouse gas con- centrations) remain unchanged. For instance, multi-decadal periods of rel-
Table 2:
Global wrighted mean of the spread of an en- semble of 10-yr mime'sean's for surface pressure, total water path, air temperature, zonal wind, and meridional wind (the last three the near-surface level). The climate variability of our method is consistent with the reference model.
Model
Ps
TWP
T7
ut
07
Reference Ours
19.96
0.199
0.090
0.142
0.110
23.52
0.214
0.094
0.167
0.121
DYffusion
24.75
0.223
0.082
0.169
0.127
ACE-STO
30.32
0.256
0.135
0.192
0.131
ative drought have stressed many past human civilizations. The present simulations are more constrained than natural climate variability because they employ repeating cycle of sea-surface temperature and thus do not allow for feedbacks between the atmosphere, ocean, vegetation, and cryosphere. Nevertheless, an important quality of an ML emulator of the global atmosphere suitable for climate studies is that it simulates similar level of wwwffreuency climate variability as the reference model.
Here, we verify that our time-mean ensemble passes this challenging test, measured using the intra-ensemble variability of time-mean averages of a few important climate statistics simulated by 25-member ensembles of the emulators vs. the ten reference simulations. We measure this variability by computing the area-weighted average of the standard deviation of time-means across the ensemble dimension. In Table 2 we show that the resulting global mean variability of the ensemble of time-means of our method is within 10-20% of those of the reference simulations for all tabulated variables (and other predicted fields). DYffusion achieves similarly accurate ensemble variability. while ACE-STO In Appendix 1.2 we show that the corresponding global maps of the time-mean variability reveal similar spatial patterns. That is, our method generates ensemble climate simulations with decadal variability consistent with the underlying climate model.
100-year-long simulation. We evaluate the long-term stability of Spherical DYffusion through a 100-year simulation, a critical timescale for many climate modeling applications. Figure demon- strates the model's robustness through time series of key global mean variables from a single (random) simulation, which completed in approximately 26 hours of wall-clock time. The model generates physically consistent temporal patterns in response to annually repeating forcings. Notably, Spherical
http://youuuu.e///HHaagBgBBB httpp:///youuubbHHae..ooo

---

Model
ACE
Spherical DVffusion
Global mean
286
Tx
282
TWP
24
98650
98600
P, [Pa]
98550
98500
20
40
60
80
100
Years since initialization
Figure 6: Comparison of 100-year global mean simulations between Spherical DYffusion and ACE. From top bottom: near-surface air temperature (T7), total water path (TWP), and surface pressure (pz). Both models are driven by identical annually repeating foreings. Spherical DY ffusion demonstrates more stable trajectories, particularly evident in the surface pressure predictions, while maintaining physically realistic variability patterns. The consistent behavior across all variables indicates the model's s robustness for long-term climate simulations.
DYffusion exhibits improved variability patterns compared to the baseline ACE model, which suffers from unrealistic annual fluctuations (e.g. see surface pressure).
Conclusion
introduce Spherical DYffusson, novel approach that combines efficient diffusion modeling with spherical-aware neural architecture to probabilistically emulate complex global climate dynamics across decadal to centennial timescales. Our model achieves lower climate biases than relevant deterministic and probabilistic baselines, getting significantly closer the optimal performance provided by the emulated climate model. For climate model emulation problems, our approach presents unique solution for balancing generative modeling. computational efficiency, and low climate biases. This opens up the ability to perform fully data-driven ensemble climate simulations.
Limitations. To achieve real-world impact, the dataset will need to be expanded that ML emulators can be evaluated (and trained) on climate change scenarios/simulations. This will require using time-varying climate change forcings such greenhouse gas and aerosol concentrations. Although our use of the state-of-the-art FV3GFS atmospheric model enables generation of such training data, any emulator will inherently reflect biases present in the base model. Additionally, we only considered emulating the atmosphere, but to achieve a full Earth System Model (ESM) we also need emulate (or couple to physics-based model of) other components such as ocean, land, sea-ice, etc. It is important to stress that while our method is more than 25x faster than the reference physics- based climate model. is still slower than deterministic emulators such as ACE. Though our method characterizes model uncertainty through its generative design, extending it to incorporate initial condition uncertainty- a key component of traditional ensemble physics-basee based models could further enhance its capabilities. The method also needs extension to handle output-only variables like precipitation, either through dedicated prediction heads or modifications to the DY framework.
10

---

Acknowledgements
This work supported in part by the U.S. Army Research Office under Army-CCASE award 1911NF-23-1-0231 the U.S. Department Of Energy, Office of Science IARPA HAYSTAC Pro- gram, CDC-RFA--T-23-0069, DARPA AIE FoundSei, DARPA YFA, NSF Grants #2205093, #2100237, (200027,,214440 (244H1143444 146343, and #2134274. S.R.C. acknowledges generous support from a summer internship and subsequent collaboration with the Allen Institute for Al (Ai2), which pri- marily funded by the estate of Paul G. Allen. We are grateful to Zihao Zhou, Gideon Dresdner, and Peter Eckmann for their insightful feedback, and the anonymous review- ers for their constructive comments and valuable suggestions that helped strengthen this work.
References
[1] Troy Arcomano, Istvan Szunyogh, Alexander Wikner, Brian R Hunt, and Edward Ott. A hybrid atmospheric model incorporating machine learning can capture dynamical processes not captured by its physics-based component. Geophysical Research Letters, 50(8), April 2023.
10. 1029/2022|1102649. 3
[2] Arpit Bansal, Eitan Borgnia, Hong-Min Chu, Jie Li, Hamid Kazemi, Furong Huang, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Cold diffusion: Inverting arbitrary image transforms without noise. Advances in Neural Information Processing Systems, 2023.
4,
[3] B. Barthel Sorensen, A. Charalampopoulos, S. Zhang, B. E. Harrop, L. R. Leung, and T. P. Sapsis. A non-intrusive machine learning framework for debiasing long-time coarse resolution climate simulations and quantifying rare events statistics. Journal of Advances in Modeling Earth Systems, 16(3), 2024. doichttss:/doi..or//00 httptp/s//ooooo///01111///////10
4
[4] Seth Bassetti, Brian Hutchinson, Claudia Tebaldi, and Ben Kravitz. DiffESM: Conditional
emulation of earth system models with diffusion models. ICLR Workshop on Tackling Climate Change with Machine Learning, 2023. doi: 10.488850//xxi.22300000 116999990.11.9.... 4
[5] Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. Accurate medium-range global weather forecasting with 3D neural networks. Nature, 619(7970):533- 538, 2023. doi:
1,3
[6] Lukas Biewald. Experiment tracking with weights and biases, 2020. URL https://www. wandb. com/. Software available from wandb.com. 18
[7] Antoine Blanchard, Nishant Parashar, Boyko Dodov, Christian Lessig, and Themis Sapsis. A multi-scale deep learning framework for projecting weather extremes. In NeurlPS 2022 Work- shop on Tackling Climate Change with Machine Learning, 2022. doi: 10.48550/rxxx.2220.111
4
[8] Boris Bonev, Thorsten Kurth, Christian Hundt, Jaideep Pathak, Maximilian Baust, Karthik Kashinath, and Anima Anandkumar. Spherical fourier neural operators: Learning sta- ble dynamics the sphere. International Conference Machine Learning, 2023.
doi:10. 100..48500/axx.22330000 03838. 2, 3,333
[9] Noah D. Brenowitz, Yair Cohen, Jaideep Pathak, Ankur Mahesh, Boris Bonev, Thorsten Kurth, Dale R. Durran, Peter Harrington, and Michael S. Pritchard. A practical probabilistic benchmark for Al weather models. arXiv, 2024. doi: 100.14555//aaai2220.......
[10] Kang Chen, Tao Han, Junchao Gong, Lei Bai, Fenghua Ling, Jing-Jia Luo, Xi Chen, Leiming Ma, Tianning Zhang, Rui Su, Yuanzheng Ci, Bin Li, Xiaokang Yang. and Wanli Ouyang. Wu: Pushing the skillful global medium-range weather forecast beyond 10 days lead. (0.48550/axxxv22300 0004000028... arXiv, 402.
[11] Lei Chen, Xiaohui Zhong, Feng Zhang, Yuan Cheng, Yinghui Xu, Yuan Qi, and Hao Li. FuXi: a cascade machine learning forecasting system for 15-day global weather forecast. upj Climate and Atmospheric Science, 6(1), November 2023. ISSN 2397-3722. doi:10. 10. 1038/s41612-023-
00512-1.

---

[12] Kai- Yuan Cheng, Lucas Harris, Christopher Bretherton, Timothy M. Merlis, Maximilien Bolot, Linjiong Zhou, Alex Kaltenbaugh, Spencer Clark, and Stephan Fueglistaler. Impact of warmer sea surface temperature on the global pattern of intense convection: Insights from a global storm resolving model. Geophysical Research Letters, 496),), 2022. doi: 1029/2222g(0999999 18
[13] Prafulla Dhariwal and Alex Nichol. Diffusion models beat GANs on image synthesis. Advances Neural Information Processing Systems, 2021. 48w550/axiw.2105.0000 05.05533....... 4,
[14] J.R. Driscoll and D.M. Healy. Computing fourier transforms and convolutions on the 2-sphere. Advances in Applied Mathematics, 15:202-250, 1994. ISSN 01968858.
doi: 10. 1006/00/a.aaa.........mma..aaaa. wwa.ma.194..... 1008. 5
[15] Veronika Eyring, William D. Collins, Pierre Gentine, Elizabeth A. Barnes, Marcelo Barreiro, Tom Beucler, Mare Bocquet, Christopher S. Bretherton, Hannah M. Christensen, Katherine Dagon, David John Gagne, David Hall, Dorit Hammerling, Stephan Hoyer, Fernando Iglesias- Suarez, Ignacio Lopez-Gomez, Marie C. McGraw, Gerald A. Meehl, Maria J. Molina, Claire Monteleoni, Juliane Mueller, Michael Pritchard, David Rolnick, Jakob Runge, Philip Stier, Katja Weigel, Rose Yu, and Laure Zanna. Pushing the frontiers in climate modelling and analysis with machine learning. Nature Climate Change, pages 1-13, 2024. doi:10. 1038//4.1///11111 -58-022--0.
Oliver
[16] William Falcon and The PyTorch Lightnin team. PyTorch Lightning. March 2019. URL https://githin htt./////ttttu..//// won.LLightainggAA/IBaa AIIiightnig.. 18
[17] K. Fletcher, C. S. Bretherton, H. Xiao, R. Sun, and Han. Improving subtropical boundary layer cloudiness in the 201 I NCEP GFS. Geoscientific Model Development, 7(5):2107-2100)
2014. 1ee9110559//gm-222 2210072004 2.
[18] V. Fortin, M. Abaza, F. Anctil, and R. Turcotte. Why should ensemble spread match the rmse of the ensemble mean? Journal of Hydrometeorology, 15(4):1708 1713, 2014. doi.https://ooi.rr//100 10.1
1ttpg//.....11/MMM11 1755/HMMDD-144 1-100008.1- 21
[19] Yarin Gal and Zoubin Ghahramani. Dropout bayesian approximation: Representing model uncertainty in deep learning.
International Conference on Machine Learning, 2016.
4,6
[20] Han Gao, Sebastian Kaltenbach, and Petros Koumoutsakos. Generative learning for forecasting the dynamics of high dimensional complex systems. Nature Communications 15, 8904, 2024. doi: 10. 46760/4444--024311666
[21] Robert C. Garrett, Trevor Harris, Bo Li, and Zhuo Wang. Validating climate models with spherical convolutional wasserstein distance. Advances in Neural Information Processing Systems, 2024. doi: 4b0::0044550/arx..220.....4
[22] Haiwen Guan, Troy Arcomano, Ashesh Chattopadhyay, and Romit Maulik. Lucie: A lightweight uncoupled climate emulator with long-term stability and physical consistency for 0(1000)- member ensembles. arXiv, 2024. doi: (0.48550//rxi..2455..u 200.24665.....
[23] William Harvey, Saeid Naderiparizi. Vaden Masrani, Christian Weilbach, and Frank Wood. Flexible diffusion modeling of long videos. Advances in Neural Information Processing Systems, 2022. (0.485550/axxx.2200... 1495. 2
[24] Ali Hatamizadeh, Jiaming Song, Guilin Liu, Jan Kautz, and Arash Vahdat. DiffIT: Diffusion vision transformers for image generation. arXiv, 2023. doi: 000:10048550/amxi22112200011 4
[25] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 2020. (0.445550/mrvv.2000..1323 2, 4,
[26] Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P. Kingma, Ben Poole, Mohammad Norouzi, David J. Fleet, and Tim Salimans. Imagen video: High definition video generation with diffusion models. arXiv, 2022.
2
12

---

[27] Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 2022. 0001100048500/arx.222444.4
2
[28] Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kilian Q. Weinberger. Deep networks with stochastic depth. In Computer Vision ECCV 2016, page 646 166-661. Springer International Publishing, 2016. ISBN 9783319464930,
6, 20
[29] Allan Jabri, David Fleet, and Ting Chen. Scalable adaptive computation for iterative generation. International Conference Machine Learning, 2023. 200:100045550/amxx22221111
4
[30] Julia Kaltenborn, Charlotte E. Lange, Venkatesh Ramesh, Philippe Brouillard, Yaniv Gurwicz, Chandni Nagda, Jakob Runge, Peer Nowack, and David Rolnick. ClimateSet: A large-scale climate model dataset for machine learning. In Advances in Neural Information Processing 4
Systems Track on Datasets and Benchmarks, 2023. doi: 100.44550a2//.2111..0000
[31] Tero Karras, Mika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Advances in Neural Information Processing Systems, 2022. doi: 10..1100//raiii.......5 22..22222000022 2,
[32] J. E. Kay, C. Deser, A. Phillips, A. Mai, C. Hannay, G. Strand, J. M. Arblaster, S. C. Bates, G. Danabasoglu, J. Edwards, M. Holland, P. Kushner, J.-F. Lamarque, D. Lawrence, K. Lindsay, A. Middleton, E. Munoz, R. Neale, K. Oleson, L. Polvani, and M. Vertenstein. The community earth system model (cesm) large ensemble project: A community resource for studying climate change in the presence of internal climate variability. Bulletin of the American Meteorological Society, 96(8):1333 1349, 2015. doi: 111.1117////AM-------0000
2, 6, 18
[33] Ryan Keisler. Forecasting global weather with graph neural networks. doi: 000110048550//am1122000.53333350
arXiv, 2022.
[34] Dmitrii Kochkov, Janni Yuval, Ian Langmore, Peter Norgaard. Jamie Smith, Griffin Mooers, Milan Klöwer, James Lottes, Stephan Rasp, Peter Düben, Sam Hatfield, Peter Battaglia, Alvaro Sanchez-Gonzalee Matthew Willson, Michael P. Brenner, and Stephan Hoyer. Neural general circulation models for weather and climate. Nature, 632(8027):1000-1000 July 2024. ISSN 1476-4687.
[35] Georg Kohl, Li-Wei Chen, and Nils Thuerey. Benchmarking wwwoorrgeeesivvecooiiiioiooo diffusion models for turbulent flow simulation. arXiv, 2023. doi: 460::100.45550ammm223300.....44. 2, 4,
[36] Anna Kwa, Spencer K. Clark, Brian Henn, Noah D. Brenowitz, Jeremy McGibbon, Oliver Watt- Meyer, W. Andre Perkins, Lucas Harris, and Christopher S. Bretherton. Machine-learned climate model corrections from global storm-resolving model: Performance across the annual cycle. Journal of Advances in Modeling Earth Systems, 15(5), 2023. 1 (0.110990022mm0034400 3
[37] Ching- Yao Lai, Pedram Hassanzadeh, Aditi Sheshadri, Maike Sonnewald, Raffaele Ferrari, and Venkatramani Balaji. Machine learning for climate physics and simulations. arXiv, 2024.
[38] Remi Lam, Alvaro Sanchez-Gonzaie Matthew Willson, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton-Rosen, Weihua Hu, Alexander Merose, Stephan Hoyer, George Holland, Oriol Vinyals, Jacklynn Stott, Alexander Pritzel, Shakir Mohamed, and Peter Battaglia. Learning skillful medium-range range global weather forecasting. Science, 382(6677):
2023. ISSN 1095-9203. doi: 112///cience.di23333.. 1,
[39] Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadeneshelli Burigede liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar, Fourier neural operator for parametric partial differential equations. International Conference Learning Representations, 2021. 5
[40] Jae Hyun Lim, Nikola Kovachki, Ricardo Baptista, Christopher Beckham, Kamyar Azizzade- nesheli, Jean Kossaifi, Vikram Voleti, Jiaming Song, Karsten Kreis, Jan Kautz, Christopher Pal, Arash Vahdat, and Anima Anandkumar. Score-based diffusion models in function space. arXiv, 4
2023. doi:
13

---

[41] Phillip Lippe, Bastiaan S. Veeling, Paris Perdikaris, Richard E Turner, and Johannes Brandstetter. PDE-Refiner Achieving accurate long rollouts with temporal neural pde solvers. Advances in Neural Information Processing Systems, 2023. 100:10..40555550 235.50//axxi.208.0577.777 4
[42] Björn Lütjens, Raffaele Ferrari, Duncan Watson-Parris, and Noelle Selin. of internal variability on benchmarking deep learning climate emulators. arXiv, 2024. doi: 44.
The impact
[43] Morteza Mardani, Noah Brenowitz, Yair Cohen, Jaideep Pathak, Chieh-Yu Chen, Cheng- Chin Liu, Arash Vahdat, Mohammad Amin Nabian, Tao Ge, Akshay Subramaniam, Karthik Kashinath, Jan Kautz, and Mike Pritchard. Residual corrective diffusion modeling for km-scale atmospheric downscaling. arXiv, 2023. doi: 1000:00044550/aaxx22300.... 152144 4
[44] James E. Matheson and Robert L. Winkler. Scoring rules for continuous probability distributions. Management Science, 22 10810)00007-1696 1976. 21
[45] McGibbon, K. Clark, B. Henn, A. Kwa, O. Watt-Meyer, W. Perkins, and C. Bretherton. Global precipitation correction across a range of climates using CycleGAN. Geophysical Research Letters, (1), 2024.
4
[46] Tung Nguyen, Johannes Brandstetter, Ashish Kapoor, Jayesh K Gupta, and Aditya Grover. ClimaX: A foundation model for weather and climate. International Conference on Machine Learning, 2023. doi: 10.48550/arxxi.2333333 2301........11.33....... 4
[47] Tung Nguyen, Rohan Shah, Hritik Bansal, Troy Arcomano, Sandeep Madireddy, Romit Maulik, Veerabhadra Kotamarthi, lan Foster, and Aditya Grover. Scaling transformer neural networks for skillful and reliable medium-range weather forecasting. Advances in Neural Information Processing Systems, 2024. 105
[48] Brian C. O'Neill, Claudia Tebaldi, Detlef P. van Vuuren, Veronika Eyring, Pierre Friedlingstein, George Hurtt, Reto Knutti, Elmar Kriegler, Yean-Francori Lamarque, Jason Lowe, Gerald A. Meehl, Richard Moss, Keywan Riahi, and Benjamin M. Sanderson. The scenario model intercomparison project (ScenarioMIP) for CMIP6. Geoscientific Model Development, 9()): 3461-3482, September 2016. ISSN 1991-9603. doi: 100.119//mmdm-334611111
[49] Jaideep Pathak, Shashank Subramanian, Peter Harrington, Sanjeev Raja, Ashesh Chattopadhyay, Morteza Mardani, Thorsten Kurth, David Hall, Zongyi Li, Kamyar Azizzadenesheli, Pedram Hassanzadeh, Karthik Kashinath, and Animashree Anandkumar. FourCastNet: global data- driven high-resolution weather model using adaptive fourier neural operators. arXiv, 2022.
doi: 1tt:10448550//mxii222222. 1214.
[50] William Peebles and Saining Xie. Scalable diffusion models with transformers. In IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, October 2023. 1o::10.1 100.1100//icc55550000000000.0000
4
[51] Iln Price, Alvaro Sanchez-Gonzaler Ferran Alet, Timo Ewalds, Andrew El-Kadi, Jack- lynn Stott, Shakir Mohamed, Peter Battaglia, Remi Lam, and Matthew Willson. Gen- Cast: Diffusion-based ensemble forecasting for medium-range weather. arXiv, 2023.
4o0::0048550/arri..232..555 1579. 1, 3,
[52] Stephan Rasp, Michael S. Pritchard, and Pierre Gentine. Deep learning to represent subgrid processes in climate models. Proceedings of the National Academy of Sciences, 115(39): 9684-9689, September 2018. ISSN 1091-6490. 1oi:10..173/paas..... 1810086111 15.
[53] Stephan Rasp, Stephan Hoyer, Alexander Merose, Ian Langmore, Peter Battaglia, Tyler Russell, Alvaro Sanchez-Gonzalez, Vivian Yang, Rob Carver, Shreya Agrawal, Matthew Chantry, Zied Ben Bouallegue, Peter Dueben, Carla Bromberg, Jared Sisk, Luke Barrington, Aaron Bell, and Fei Sha. WcatherBench 2: A benchmark for the next generation of data-driven global weather models. Journal of Advances in Modeling Earth Systems, 16(6), June 2024. ISSN 1942-2466. doi: 1tt110.122//2233m00000999 3, 20, 21
[54] M. J. Rodwell and T. N. Palmer. Using numerical weather prediction to assess climate models. Quarterly Journal of the Royal Meteorological Society, (33()) 126-62))::221466 129-146, 2007. doi.https://boioor//// 100200/0...2....... 27
14

---

[55] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional networks for biomedical image segmentation. International Conference on Medical image computing and computer-ussisted intervention, pages 234-241, 2015. doi: 001100779988--3319-25555
4
[56] Salva Rühling Cachay, Venkatesh Ramesh, Jason N. S. Cole, Howard Barker, and David Rolnick. ClimART: A benchmark dataset for emulating atmospheric radiative transfer in weather and climate models. Advances in Neural Information Processing Systems Track on Datasets and Benchmarks, 2021. doi: 10.48550/arxiv221111
[57] Salva Rühling Cachay, Bo Zhao, Hailey Joren, and Rose Yu. DYffusion: A dynamics-informed diffusion model for spatiotemporal forecasting. Advances in Neural Information Processing Systems, 2023. doi: 200.500/axxi.2230000199000 2, 4,
[58] Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman. Make-a-video: Text-to-video generation without text-video data. 2022. doi: 20044455/xxxi.222.....
2
[59] Jascha Sohl-DDcksttein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. International Conference on Machine Learning, 2015. doi: 10.448500//arii 150.0035550003385... 2,
[60] Yang Song and Stefano Ermon. the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
Generative modeling by estimating gradients of
1tt:10448550/arxv.199077 05660..
[61] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar. Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. International Conference on Learning Representations, 2020. doi: 1tt.:00.8550//ax.2011....
4
[62] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, llya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 1929-191 2014. 6
[63] Vikram Voleti, Alexia Jolicocur-Martineau and Christopher Pal. MCVD: Masked conditional video diffusion for prediction, generation, and interpolation. Advances in Neural Information Processing Systems, 2022.
2
[64] Zhong Yi Wan, Ricardo Baptista, Anudhyan Boral, Yi-Fan Chen, John Anderson, Fei Sha, and Leonardo Zepeda-Nunee Debias coarsely, sample conditionally: Statistical downscaling through optimal transport and probabilistic diffusion models. Advances in Neural Information Processing Systems, 2023. 0ete10.88/0//xii..2222 1565. 4
[65] D. Watson-Parris. Machine learning for weather and climate are worlds apart. Philosophical Trunsactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 379 (2194):20200098, 2021. doi: 10. 1098/1sta. 2020.0098. 3
[66] D. Watson-Parris Y. Rao, D. Olivié, Ø. Seland, P. Nowack, G. Camps- P. Stier. S. Bouabid, M. Dewey, E. Fons, J. Gonzalez, P. Harder, K. Jeggle. J. Lenhardt, P. Manshausen, M. Novi- tasari, L. Ricard, and C. Roesch. ClimateBench v1.0: A benchmark for data-driven cli- mate projections. Journal of Advances in Modeling Earth Systems, 14(10), October 2022. doi: 1029/2021mm00229999
[67] Oliver Watt-Meyer, Gideon Dresdner, Jeremy McGibbon, Spencer K Clark, James Duncan, Brian Henn, Matthew Peters, Noah D Brenowitz, Karthik Kashinath, Mike Pritchard, Boris Bonev, and Christopher Bretherton. ACE: A fast, skillful learned global atmospheric model for climate prediction. NeurIPS 2023 Workshop on Tackling Climate Change with Machine Learning, 2023. doi:
3, 5, 6, 17, 18, 19, 20,
[68] Jonathan A. Weyn, Dale R. Durran, Rich Caruana, and Nathaniel Cresswell-Clay. Sub- seasonal forecasting with large ensemble of deep-learning weather prediction models. Journal of Advances Modeling Earth Systems, 13(7), July 2021. ISSN 1942-2466. (0002//22lss02500.22 lms002502.
15

---

[69] Ruihan Yang, Prakhar Srivastava, and Stephan Mandt. Diffusion probabilistic modeling for video generation. Entropy, 25(10):1469, October 2023. ISSN 1099-4300. 06::10.3390/22511144.
[70]
Sungduk Yu, Walter Hannah, Liran Peng, Jerry Lin, Mohamed Aziz Bhouri, Ritwik Gupta, Bjöm Lütjens, Justus Christopher Will, Gunnar Behrens, Julius Busecke, Nora Loose, Charles I Stern, Tom Beucler, Bryce Harrop, Benjamin R Hillman, Andrea Jenney, Savannah Ferretti, Nana Liu, Anima Anandkumar, Noah D Brenowitz, Veronika Eyring, Nicholas Geneva, Pierre Gentine, Stephan Mandt, Jaideep Pathak, Akshay Subramaniam, Carl Vondrick, Rose Yu, Laure Zanna, Tian Zheng, Ryan Abernathey. Fiaz Ahmed, David C Bader, Pierre Baldi, Elizabeth Barnes, Christopher Bretherton, Peter Caldwell, Wayne Chuang, Yilun Han, YU HUANG, Fernando Sulare, Sanket Jantre, Karthik Kashinath, Marat Khairoutdinov, Thorsten Kurth, Nicholas Lutsko, Po-Lun Ma, Griffin Mooers, J. David Neelin, David Randall, Sara Shamekh, Mark A Taylor, Nathan Urban, Janni Yuval, Guang Zhang, and Michael Pritchard. ClimSim: A large multi-scale dataset for hybrid physics-ML climate emulation. Advances in Neural Information Processing Systems Track on Dulasets and Benchmarks, 2023.
[71] Janni Yuval and Paul O'Gorman. Stable machine-learning parameterization of subgrid processes for climate modeling at range of resolutions. Nature Communications, 11(1), July 2020. ISSN 2041-1723. doi:10. 001111///44477022------
[72] Michaël Zamo and Philippe Naveau. Estimation of the continuous ranked probability score with limited information and applications ensemble weather forecasts. Mathematical Geosciences, 50(2):209-23,, 2018. doi: 10070
21
[73] Linjiong Zhou, Shian-Jiann Lin, Jan-Huey Chen, Lucas M. Harris, Xi Chen, and Shan- non I.. Rees. Toward convective-scale prediction within the next generation global predic- tion system. Bulletin of the American Meteorological Society, 100(7):1225 1243, 2019. bothtngs/bbooor//11117551444/770
6, 7, 18
16

---

Appendix
A
Broader Impact
The goal of this work is to advance the application of machine learning to climate modeling, specifi- cally for generating fast and cheap ML-based climate simulations. This could significantly democra- tize climate modeling, improve scientific understanding of the earth system, and enhance decision- and policy-making in a changing climate. However, to realize this goal, the reliability and limitations of such ML models will need to be much better understood.
B Dataset
In the subsections below, we elaborate on the dataset and variables that we use, including background information FV3GFS and how it was configured in order generate the training and validation data. Any listed data preprocessing steps below are also deseribed in appendix A from [67]. The final training and validation data can be downloaded from Google Cloud Storage following the instructions of the ACE paper at http:///nooooorg/eecor////////// Commons Attribution 4.0 International.
The data are licensed under Creative
B.1 Input, output and forcing variables
Table 3: Input and output variables used in this work. The table was adapted based on Table 1 of [67]. The & subscript refers to vertical layer index and ranges from 0 to starting at the top of the atmosphere and increasing towards the surface. The two prognostic surface variables, T, and ps. do not have this additional vertical dimension. Each of their snapshots is 2D latitude-longitude matrix. The Time column indicates whether variable represents the value particular time step ("Snapshot"), the average across the 6-hour time step ("Mean")), or quantity that does not depend on time (* ("Invariant"). "TOA" denotes "Top Of Atmosphere", the climate model's upper boundary.
Prognostic variables (input and output)
Symbol
Description
Units
Time
Is 3D?
Tk
Air temperature
K
Snapshot
Yes
qi
Specific total water (vapor condensates)
kg/kg
Snapshot
Yes
Uk
Wind speed in eastward direction
m/s
Snapshot
Yes
Wind speed in northward direction
m/s
Snapshot
Yes
T,
Skin temperature of land or sea-ice Atmospheric pressure at surface
K
Snapshot
No
Ps
Pa
Snapshot No
Forcing variables (input-only)
Symbol
Description
Units
Time
DSWRFTOA
Downward shortwave radiative flux at TOA
W/m²
Mean
T,
Skin temperature of open ocean
K
Snapshot
Additional input-only variables
Symbol
Description
Units
Time
2s
Surface height of topography
m
Invariant
ft fo fsi
Land grid cell fraction
Invariant
Ocean grid cell fraction
Snapshot
Sea-ice grid cell fraction
Snapshot
Derived, evaluation-only, variables
Symbol
Description
Units
Time
3D?
WSk
Wind speed
m/s
Snapshot Snapshot
Yes
TWP
Total water path
No
17

---

The complete list of input, output, and forcing variables used in this work is given in Table 3. The only difference to the work from [67] that we do not consider diagnostic (ouiput only) variables. The foreings consist of annually repeating climatological sea surface temperature (1982-2012 average), Ts. and incoming solar radiation, DSWRFTOA- Prescribed sea surface temperatures are simply "overwritten" on the skin temperature predictions of the ML models over all open ocean locations (when rolling out the ML-based simulation). The other forcing or input-only variables are added as an additional channel dimension. Derived variables are computed from the (predicted) prognostic variables as described below.
Derived variables. For evaluation, we also consider the derived variable called total water path which is computed as TWP = Ex 4% dp i.e. as a function of surface pressure and the profile of specific total water. Its units are mm (or sg/m², assuming that water has density of 1000 kg/ 2g/m²). The derived wind speed variable for level k is computed based on the simulated meridional and zonal wind variables as WSg V +
2zz Its units are m/s.
B.2 Background on FV3GFS
Our dataset and physics- based baselines (including our "noise-floor" reference baseline) are based on simulations from comprehensive global atmospheric model called Finite- Volume on Cubed- Sphere Global Forecasting System (FV3GFS) [73]. was developed by the National Oceanic and Atmospheric Administration (NOAA) Geophysical Fluid Dynamics Laboratory (GFDL)). A very similar model version is operationally used by the US National Centers for Environmental Prediction (NCEP) and the US weather forecasting service Its scalability to horizontal grid spacings as fine as 3 km [12] makes it an excellent candidate for generating training data for future ML-based climate model emulators, including out-of-distributin climate change simulations that may be necessary to train ML emulators on so that they can generalize.
B.3 FV3GFS configuration for data generation
In the following we summarize the reference data for this study, as also discussed in Section 2.1 of [67]. The training and validation data is generated by running an ensemble of 11 10-year (after discarding a 3-month spinup period) FV3GFS simulations 100 km horizontal grid spacing) with 63 vertical levels. The simulations are an imitial-condition ensemble. That is, they are identical except for using different initial atmospheric states. Initial- condition ensembles are popular tool in climate modeling [32]. Discarding a 3-month spinup period ensures that each simulation is independent of each other due to the chaoticity of the atmosphere' Each simulation is forced by repeating annual cycles of sea-surface temperature and insolation. The temperature, humidity, two wind components at each grid point, and selected vertical fluxes the surface and top of the atmosphere in each grid column are saved every six hours. For ML training, the temperature, humidity, and two wind components are averaged along FV3GFS's 63 levels to vertical layers, and the data are interpolated to latitude-longitude grid of 180 x 360 dimensions.
C96 cubed-sphere grid (approximately
C Implementation details
All methods and baselines are conditioned on the forcings, fr, by simple concatenation of the forcings with the remaining input variables across the channel dimension. We use PyTorch Lightning [16] and Weights & Biases [6] as part of our software stack.
C.1 Training and inference pseudocode
In Algorithms and we provide the procedures used to train and sample from our proposed method, respectively.
'htpss//www.gtt/ "https://www. wwtp::/wwwwwwwww..ww..owooo www.wwherr.oo///ws/s/ssss
'E.g. see Kay et al. [32], who note that: "After initial condition memory is lost, which occurs within weeks in the atmosphere, each ensemble member evolves chaotically, affected by atmospheric circulation fluctuations characteristic of random, stochastic process (e.g.. Lorenz 1963; Deser et al. 2012b)".
18

---

Algorithm Spherical DYffusion, Training
Input:
networks sF SFNO,, norm ||-11, horizon / 6
Stage 1: Train interpolator network, SFNO,
1. Sample ~ Uniform (11,...... --1)--1})) 1})
2. Sample æt, 2c+4, Ut++h ~ IR, and corresponding forcing fc
3. Sample network stochasticity (dropout),
4. Optimize minş
|SFNO (Ig, f:. (5) xe+ill?
Stage 2: Train forecaster network, SFNO,
1. Freeze SFNO, and enable its inference stochasticity g
2. Sample ~ moiifrmm((O,... .,. 11) and Tt, Tith ~ RRD
3. Retrieve corresponding forcings frti
4. æt+j f SENO, (a,, f..
# with Et4j 2, for 0 5. Optimize mine ||SNO,, (Ze+s. (4)) 2eth
Algorithm 2 Spherical DYffusion, Inference
Input: Initial conditions tp == zu, training and inference horizon and 14600, forcings
fa.11
2: Autoregressive loop:
3: for = 0, h,, 2 h, ................................ ([H/h] - 1) h do
4: # &Sampling loop for time steps 1,..., L h:
5: for j 0.1..... h do
6: Ct+h + SFNO, fe:4:.ftjjin
(Refine) forecast
7: Uz+j+1 f , (i,, Be+h: J: + 1 (E) # #Itterpolate
8: Ør+j+1 Tz+j+1 + Li-j SENO, & (āt,, Creh, Su.j (()
# Cold sampling
9: end for
10:
end for
11: Return: $1.H
C.2 Discussion on the training horizon
The training horizon, h, a critical hyperparameter for both DYffusion and our proposed method. Throughout this study, we use h = 6 (corresponding to 36 hours) for both approaches. While we initially explored other horizons, we chose = as strikes an optimal balance: A smaller horizon (e.g., h 3) reduces the number of sampling steps since the reverse sampling process directly corresponds to physical time steps, potentially degrading performance. Conversely, a larger horizon makes the forecasting task more challenging, predicting 24+h from æ; becomes increasingly difficult for the forecasting model.
Our choice is further supported by the DYffusion paper, which successfully used h 7 for sea surface temperature forecasting. While we believe that values close to would likely perform similarly well, comprehensive ablation studies would require re-training two neural networks sequentially, making such experiments computationally expensive to run.
C.3 Hyperparameters
Architectural hyperparameters. To fairly compare against the deterministic SFNO model from [67]. we use exactly the same hyperparameters for training the interpolator and forecast- ing networks for our method, described in Table 78. For the stochastic version of ACE, ACE-STO, we re-train ACE from scratch with the only difference being that we use a dropout rate of 10% for the MI.P in the SFNO architecture. We train the stochastic interpolator model, SFNO g. in our method using the same dropout rate. Both of these stochastic models are run using MC dropout
Names correspond to the definition of the SpperrcaFFooiieeeuuuaaeleerteeee class found at: com wwwm///aac////////cccoe/eo htt/a//aacm/mmmuuuaaaaaaaa
https://githhu: ctmpal...ooo.//////////eaae modiiuss/nooeess//ttt//fssoooo L222299. Unless specified otherwise, defaults are used.
19

---

Figure 8: Optimization hyperparameters. The effective batch size is calculated as data loader batch size x number of GPUs X number of gradient aceumulation steps, and is ensured to be the same for all our trained models regardless of the number of GPUs used.
Figure 7: Table is directly taken from [67], and reports the SFNO hy- perparameters used for ACE as well as the interpolator and forecasting net- works of our method.
Name
Value
Name
Value
Optimizer
AdamW
embed_dit
256 linear
Initial learning rate
4 x 10-1
filter_tyy type
Weight decay
5 x 10-3
num num_layers
Learning rate schedule
Cosine annealing
aperator_type scale_factor spectral_layers
dhconv
Number of epochs
60
Effective batch size Exponential moving average decay rate
72
0.9999
Gradient clipping
0.5
(i.e. enabling the dropout layers at inference time). For our interpolator network, we also use a 10% rate for stochastic depth [28], which is also enabled at inference time. This choice was informed by preliminary experiments focused on training good interpolator network. There, we found the addition stochastic depth to slightly improve interpolator's validation CRPS scores (for interpolated timesteps to 5) and significantly improve the calibration of the interpolation ensemble based on the spread-skill ratio (averaged across variables from around 0.26 0.35). We found worse results when using stochastic depth for ACE-STO inference time.
Optimization hyperparameters. We train the interpolator networks for DYffusion and our method on the same relative L2 loss function used for the baseline from [67], and the corresponding forecaster networks on the L1 loss. The models that we train on our own, i.e. the interpolation and forecasting networks of DYffusion and our method are trained with mixed precision. Inference is always run full precision. For the non-interpolation networks, we perform early stopping based on the best CRPS averaged over 500-step (125 days) rollout. More optimization--elated hyperparameters are discussed in Table 8.
D Metrics
Unless specified otherwise, all ensemble results are based on E 25 ensemble members. All metrics are area- area-weighted according to the size of the grid cell, as described below.
D.1 Preliminaries
Let X E ExXxx denote an ensemble of predictions, and Y E [xx the corresponding targets, where E' is the number of ensemble members, is the number of latitudes, and J the number of longitudes in the grid. In the context of this paper, Y usually corresponds to the validation, reference 10-year time-mean and X corresponds reference climate model (excluding the validation time-mean,, our proposed method, or any of the baselines.
ensemble of 10-year time-means simulated by the
Let (i) denote the normalized latitude dependent area weights at latitude such that Σ w(() which ensure that spatial means are not biased towards the polar regions (see e.g. Rasp et al. [53]).
20

---

D.2
Member-wise Metrics
We report the average, member-wise area- area-welge wrighwe bias, Mean Absolute Error (MAE) and Root Mean Square Error (RMSE), which are defined as follows
Bias
Y(J)
(1)
EIJ
E
MAE
W Xeis YYul
(2)
EIJ
E (X))(X Yzj)²
RMSE
(3)
For the bias, closer to zero better, for the MAE and RMSE lower better.
D.3 Ensemble Metrics
Ensemble-mean RMSE. For skillful ensemble, the magnitude of the average, member-wiss RMSE (see above) can be reduced by computing the RMSE on the ensemble-mest prediction, defined as Xij = E Eemi Leji.j instead.
RMSE
Σ w -X.,) TJ
Spread-Skill Ratio (SSR). Following Fortin et al. [18], the spread-skill ratio is defined as the ratio between the ensemble spread and the ensemble-mean RMSE. The ensemble spread is defined the the square root of the ensemble variance
Spread
w(1) w(r)var (Xeig))
TJJ
where var, computes the variance of the ensemble. Then, we can compute the spread-skill ratio simply
SSR V E +1 Spread
(6)
E RMSE
is correction factor which especially important to include for small ensemble sizes. where V E
Note that this factor is omitted in e.g. WeatherBench-2 [53]. The SSR serves simple measure of the reliability of the ensemble, where values smaller than indicate underdispersion (i.e. the model is overconfident in its predictions), and larger values overdispersion. That is, closer to is better.
Continuous Ranked Probability Score (CRPS). Following Zamo and Naveau [72], we use the unbiased version of the CRPS [44], which a proper scoring rule:
E
E X
1 Σ w(i CRPS IJ
Xess Y 2E(E 1) f=1==
X
(7)
E
where the first term represents the skill, and the second term represents the spread. The biased CRPS averages over the spread with the factor 1
2g2. which is biased-especins for small ensemble
sizes-come compprcd to using the unbiased version with the factor 25(E-1) Note that common Python packages such as xski llscore and properscoring use the biased version. Lower is better.
Note on deterministic models. For deterministic models like ACE without initial condition ensem- bling, the ensemble size trivially E 1, causing X;, to identical X. This results RMSF reducing to standard RMSE, MAE equaling CRPS, and a spread-skill ratio. To accurately
21

---

Zonei
ond
Zanal reed- mrnnd
level
Zon0 pareet
20
Figure 9: Same as Figure 2 but showing more fields for the RMSE of 10-year time-mean's ss Bars (left to right) show 1) the noise floor calculated from the pairwise differences of ten independent 10-year reference model simulations with respect to the validation simulation. In light shade we report the score computed using the mean over the ten reference simulations as "prediction". 2) the member- wise scores of 25-member ensemble of our method in dark shade, and the corresponding ensemble mean score in light shade, 3) the score of the deterministic ACE baseline, 4) the member-wise scores of 25-member ensemble of the DY baseline in dark shade, and the corresponding ensemble mean score in light shade. The standard deviation error bar is computed over the set of pairwise (member-wise) time-mean RMSEs for the reference (Ours and DYffusion). ACE does not have standard deviation since it is deterministic model. Turning ACE stochastic through MC dropout (ACE-STO) degrades its performance. Our method significantly reduces climate biases over the baseline methods and can be effectively ensembled to reduce its climate biases further, approaching the theoretical lower limit imposed by the noise floor of the reference simulation.
reflect that ensemble metrics are not meaningful for denote these metrics as for ACE in Table 4. While incorporating initial condition ensembling would enhance ACE's performance on these metrics beyond the naive deterministic baseline, such techniques are orthogonal to the model-based ensembling approaches explored this work. We leave this extension future research, noting that initial condition ensembling could potentially improve results for all models in our comparison, including the inherently stochastic ones.
deterministic predictions,
E
Additional results and figures
E.1
Climate Biases
We quantitatively analyze the 10-year time mean biases of our model and the baselines in terms of the global mean RMSE in Figure The time-mean prediction is the average over the 14,600 predicted snapshots during the 10 years. Our method significantly reduces climate biases over the baseline methods across most fields. Notably, the errors of our method are often closer to the noise floor the reference simulation than to the next best baseline. We also show that our method can be effectively ensembled to further reduce climate biases, its ensemble- mean reliably improving time-mean scores across all fields. Interestingly, the stochastic version of ACE, ACE-STO, significantly underperforms the deterministic version. Similarly, the direct application DYffusion fails to match the deterministic ACE baseline, even after ensemble averaging. This shows that MC dropout and DYffusion alone are not the reason for the encouraging performance of our method, but rather the holistic integration of all components, including MC dropout in our SFNO-bast based interpolator network.
In Table 4, we report a comprehensive evaluation of the (ensemble) of 10-year time-means of each method for subset of ten representative variables. We report the mean bias error, mean absolute error (MAE), root mean square error (RMSE), ensemble-mean RMSE, spread-skill ratio, and Continuous Ranked Probability Score (CRPS), which are rigorously defined in Appendix D.
22

---

Table 4: Comprehensive evaluation of simulated 10-year time-means Bias, RMSE, and MAE
represent average member-wise scores. For Bias (Spread-skil ratio; SSR) closer to (1) is better. For the other metrics, lower is better, with relative changes from the reference shown in parentheses. See Appendix D for mathematical formulations and Table 3 for variable descriptions and units.
Variable
Metric
Reference
Ours
ACE
ACE-STO
DYffusion
Bias
0.004
-0.043
0.021
0.017
0.686
RMSE
0.336
0.404 (+20%)
0.653 (+94%)
1.372 (+308%)
0,965 (+187%)
RMSE
0.249
0.327 (+31 %)
1.206 (+385%)
0,934 (+276%)
TWP
SSR
1.017
0.760
0.574
0.273
MAE
0.245
0.303 (+24%)
0.459 (+88%)
0,957 (+291%)
0.768 (+214%)
CRPS
0.125
0.178 (+43%)
0.639 (+413%)
0.644 (+417%)
Bias
0.036
4.820
45.47
34.82
-126.4
RMSE
39.37
48.79 (+24%)
103.5 (+163%)
131.1 (+233%)
151.5 (4285%)
RMSF
31.50
39,59 (+26%)
120.1 (+281%)
149.0 (4373%)
Ps
SSR
0.847
0,766
0,470
0.190
MAE
26.26
35.60 (+36%)
71.69 (+173%)
93.14 (+255%)
134.8 (+413%)
CRPS
14.44
21.91 (+52%)
66.48 (+360%)
121.6 (+742%)
Bias
0.011
-0.049
0.121
0.369
0.311
RMSE
0.172
0.290 (+699%
0.349 (+103%)
0.831 (+383%)
0.692 (+302%)
RMSECsS
0.124
0.267 (+114%
0.734 (+490%)
0.684 (+450%)
T7
SSR
1.065
0.474
0.634
0.158
MAE
0.108
0.187 (+73%)
0.224 (+108%)
0,510 (+373%)
0.408 (+278%)
CRPS
0.054
0.132 (+147%)
0.343 (+540%)
0.360 (+573%)
Bias
0.005
-0.068
0.079 0.333 (+94%)
0.173
0.377
RMSE
0.171
0.244 (+42%)
0,610 (+256%)
0,540 (+215%)
RMSE
0.132
0.211 (+60%)
0.525 (+299%)
0,527 (4301%)
T3
0.933
0.619
0.657
0,228
MAE
0.117
0.171 (+46%)
0.243 (+108%)
0,451 (+286%)
0,388 (+232%)
CRPS
0.060
0.110 (+84%)
0.299 (+402%)
0.330 (+455%)
Bias
0.000
0.034
0.162
-0.127
0.517
RMSE
0.124
0.220 (+78%)
0.444 (+259%)
0.767 (+520%)
0,592 (4379%)
RMSF
0.074
0.202 (+174%)
0.674 (+815%)
0,585 (+695%)
To
SSR
1.533
0,517
0.599
0.161
MAE
0.084
0.150 (+78%)
0.316 (+277%)
0,550 (+555%)
0.526 (+527%)
CRPS
0.034
0.102 (+200
0.348 (+921%)
0.481 (+1312%)
Bias
0.012
0.038
-0.170
-0.023
0.077
RMSE
0,240
0.307 (+28%)
0.456 (490%)
0,935 (+2899)
0,462 (492%)
RMSE
0.178
0.239 (+355%
0.874 (+391%)
0,427 (+140%)
U1
SSR
1.012
0.846
0,412
0.438
MAE
0.173
0,226 (+31%%
0.343 (+98%)
0,693 (+300%)
0,339 (+96%)
CRPS
0.087
0.129 (+48%)
0.519 (+494%)
0.249 (815))
Bias
0.005
0.015
0.009
0.044
-0.067
RMSE
0.196
0.224 (+14.3%)
0.299 (+53%)
0,592 (+202%)
0,320 (+64%)
RMSF
0.152
0.178 (+17.0%)
0.548 (+260%)
0,292 (492%)
17
SSR
0.910
0.802
0.439
0.471
MAE
0.138
0.164 (+18.6%)
0.224 (+62%)
0,440 (+218%)
0.247 (+79%)
CRPS
0.072
0.094 (+30%)
0.325 (+351%)
0.179 (+ (+148%)
Bias
0.003
-0.053
-0.017
-0.000
-0.001
RMSE
0.243
0.303 (+24%)
0.437 (+79%)
0,886 (+264%)
0,450 (485%)
RMSF
0.183
0.238 (+30%)
0.823 (+349%)
0.415 (+126%)
WS7
SSR
0.976
0.830
0,430
0.445
MAE
0.175
0,224 (+28%)
0.331 (+89%)
0.659 (+277%)
0,334 (+91%)
CRPS
0.089
0.128 (+44%)
0.488 (+449%)
0.244 (+175%)
Bias
0.022
0.058
-0.104
-0.036
-0.081
RMSE
0.324
0,398 (+23%%
0.626 (+93%)
1.128 (+248%)
0,591 (+82%)
RMSE
0.240
0 (+30%)
1.030 (+329%)
0,543 (+126%)
WS:
SSR
1.013
0.837
0.475
0.452
MAE
0.248
0,311 (+25%)
0.492 (+98%)
0.878 (+254%)
0,456 (+84%)
CRPS
0.124
0.176 (+422%
0.636 (+412%)
0.329 (61%)
Bias
0.151
-0.022
-0.167
0.854
1.642
RMSE
0.450
0,944 (+110 %
2.163 (+381%)
2,661 (+491%)
3.158 (+602%)
RMSF
0.307
0.887 (+189% %
2.349 (4664%)
3.142 (4922%)
WS
1.203
0.397
0,573
0.110
MAE
0.336
0.752 (+124%)
1.626 (+384%)
2,035 (+506%)
2.044 (+509%)
CRPS
0.152
0.589 (+287%) %
1.354 (+789%)
1.874 (+1131%)
23

---

E.1.1 Zonal time-means
In this section, we analyze the absolute magnitudes the simulated time-means by examining their zonal averages (aggregated over the longitude dimension). We also visualize the standard deviation of the respective ensembles of time- and conal-means for the reference and stochastic methods. We visualize these in Figures 10 and 11. For several fields, including surface pressure, total water path (not shown), and near-surface temperature (top left subplot in Fig. 10), differences between the simulations are not visually noticeable, except for polar biases in baseline methods. However, discrepancies become pronounced higher-altitude and wind fields, where our method generally achieves the closest agreement with the reference model. Although near-surface fields are the most relevant for society and decision-making levels might contribute to long-term term biases, especially in longer simulations, due the interactions of atmospheric dynamics across all levels. This observation may partly explain why our method achieves the lowest time-mean biases and RMSEs, as discussed in Appendix E.
the clear biases of the baseline method high-altitude
Temporature level
200
Temperature level
212
254
221
DV
DTtsion
230
LaGoade
Lotkudk
Figure 10: Zonal means of the simulated 10-year time-mean climatologies for a representative subset of four temperature fields. Level 7 represents near-surface conditions, while Level 0 corresponds to the highest altitude. Our method generally provides the closest emulation to the reference data. The most notable biases in the emulations occur Levels 2 and O, indicating greater discrepancies higher altitudes. Emulation challenges are also significant near the poles, including at near-surface levels, particularly for DY ffusion.
Meridional wind level-7
Maridional wind level- -
15
-02
ACE
-75
-30
-23
X
>5
-04
+73
-30
-13
25
Meridional wind level 2
Meridional wind level -
05
155
Latiuade
tudi
Figure 11: Zonal-means of the simulated 10-year time-mean climatologies for representative subset of four northward (meridional) wind fields. Our method generally provides the closest emulation to the reference data, except for the level-0 polar latitudes.
24

---

E.1.2 Climate variability
In Fig. 12 we show the global maps corresponding to the global means of Table 2. Our method shows a consistent ensemble variability terms of the simulated climate that also largely reflects the spatial patterns and magnitudes of the reference ensemble.
Reference
Qurs
100
100
Surfece preveare
75
75
g
50
50
25
25
Reference
Ours
0.6
0.8
Totas watee patth
0.4
0.4
Reference
Ours
0.4
0.4
Temperature lovel
03
03
Reference
Ours
0.4
0.4
Zonal wilnd Incel
Reference
Ours
Haridional wind lavel-
0.4
0.4
022
D2.2
Figure 12: Global maps of the standard deviation of the 10-year time-mean of the reference ensemble and 25-member ensemble of our method. The climate variability of our method is consistent with the reference model, and largely follows similar spatial patterns with adequate magnitudes. The global mean standard deviation is reported in Table 2.
E.2 Weather forecasting
While we focus on climate time scales in this work, climate is formed by the statistics of weather, so is important verify that our method also generates reasonable forecasts of the weather simulated by the reference model. In Figure 13, we analyze the medium-range forecasting skill of our method and the baselines for lead times up to two weeks. Interestingly. ACE and DYffusion show persistent biases for the surface pressure field that are clearly visible from the first few days of forecasts already do not seem reflect on the RMSEs at weather time scales. Such persistent biases, however, may be magnified over longer simulations and could explain why the baselines have problems reproducing accurate long-term climate statisties. terms of RMSE, the deterministic model ACE generally has slight edge over method and DYffusion, especially on lead times of less than week. After that,
25

---

04% Frc
Curr.
on Eefludorr Frs
Surface pressuee X
wind leral Insba]
Hardioral whs leat (vler
Figure 13: Comparison of medium-range weather forecasting skill between Spherical DYffusson (25-member ensemble and single forecast), DYffusion (25-member ensemble and single forecast), and ACE (single, deterministic forecast). Our method generates competitive probabilistic ensemble weather forecasts, a necessary but sufficient prerequisite for achieving good climate simulations.
Time nich RISE Tempera sturettue leedl N 82
140
120
100
3.15
3.10 1.15
1.20
1.00
3.05 1.15
1.20
Figure 14: We visualize the performances DYffusion and Spherical DYffusion (in different marker colors) at multiple checkpoint epochs and for multiple generated samples. We plot the 10-year time-mean RMSE ("climate skill") of three example fields versus the time-step-wiss near-surface temperature RMSE averaged out over the first 20 forecasts (5 days; "weather skill"). The 5-day weather forecast performance shows no correlation with the long-term climate biases (indeed, there seems exist an inverse correlation). This has important implications for practitioners, implying that optimizing for short-term forecasts alone - as is current practice for most ML-based weather forecasting models may be suboptimal for attaining accurate climate simulations. We have verified that the behavior shown above holds for fields other than near rear-surface temperature too (not shown).
the ensembles of our method and DYffusion perform best in terms ensemble-mean RMSE. As expected, the ensemble mean significantly reduces the RMSE compared to using single sample from our method or DYffusion, especially at longer lead times. The ensemble metrics, CRPS, and spread/ RMSE ratio show that our method's and DYffusion's ensemble perform quite similarly, even though they are based on completely different MI architectures. Both ensembles tend to be underdispersed (Spread / RMSE 1) short time scales but quickly converge to well-dispersed ensemble at longer lead times which persists for the whole 10-year climate simulations (not shown).
E.3 Weather vs. climate performance
In Figure 14, we illustrate that weather performance does not correlate with the climate biases of same model. We plot the average RMSE over the first days simulation (here, using the near- surface temperature field) against the 10-year time-mean RMSE of various fields, and do not observe any correlation between the two metrics. We have verified that this observation holds independently of the analyzed field. This is ditllediscussed observation that has important implications for ML
26

---

practitioners since it implies that optimizing for short-term forecasts alone as is current practice for most ML-based weather forecasting models may be suboptimal for attaining accurate climate simulations. Heuristically, optimizing weather skill ensures that climate model takes locally accurate path around the climate "attractor", but does not guarantee that small but systematic errors may not build to distort that simulated attractor to have biased time-mean statistics. This observation has been documented for the case of physics-based climate models [17, 54].
E,4 Qualitative samples
Figures 15, 16, and 17 compare near-surface air temperature, near-surface wind speed. and total water path between the FV3GFS validation simulation, two randomly selected 10-year trajectories generated by Spherical DY DYffusion, and the trajectory predicted by ACE, For both variables, we show the final ten snapshots of each simulation. The complete temporal evolutions for near-surface wind speed and total water path can be viewed at https://youtu. htt.yyouuuuuuuuuoooeeeeeee be/7lHra7gBiile and https://youtu.be//// be//ac_xGsJJae Y respectively. The emulated fields demonstrate high realism, closely mimicking the patterns and variability observed in actual climate model outputs. This showcases Spherical DYffusion's capability to generate plausible and physically consistent climate scenarios over decadal timescales.
these simulations
Srraada--O0i-1--1 Cimmat
Tiraidap-O00--U 12-23
12-29
Tirastag www.:/oo00--... 32-29
5029-3 Pode
Duin pargle
Duss Etangee 13
Dur Hample
Dur tample
Derscoan
Distonatt 11
ACC ACE ACE ACT AEE ACT
We visualize the final 10 predictions from two random 10-year trajectory samples (i.e, end of the ninth year) generated by Spherical DYffusion (middle rows) and ACE (bottom row). Here, we show the near-surface air temperature variable, Tę for level & = It is important to note that at these extended time scales, simulated trajectories expected to diverge significantly from one another for any given time step.
Figure 15:
27

---

Traasap-0000 13-2 LOUTE
TTrulap-OM0- 12-23
12-29
Trrasta-rOok 12-29
Tirastg-dookk 32
Darscong
Da 11
ACC ACE XCZ ACt i0t ACE
Figure 16: We visualize the final 10 predictions from two random 10-year trajectory samples generated by Spherical DYffusion (middle rows) and ACE (bottom row). Here, we show the derived near- surface wind speed variable, WS, for level k It important to note that at these extended time scales, simulated trajectories are expected to diverge significantly from one another for any given time step. video visualizing the full 10-year simulations accessible
http:///youuubb///HHaa
1ineSipp mude Mide
Elimaae Madel
Eiren Model
Climate Node
C Chaeee Model
Dus Hergle 21
0.65 (imple
Dur 1ampl
Our jample
Oersssanp +1
Og 0y515ongf 21
Figure 17: Same Figure 16 but for the derived total water path variable, TWP. A video visualizing the full 10-year simulations is accessible at https://youtu, be/Hac xGsJ1qY.
28

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The paper's s main contributions are enumerated at the end of the introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
See the limitations paragraph at the end of the main text (at the end of Section 6).
Justification:
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]

---

Justification:
This work does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof's provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Data-for both training and evaluation-are publicly available (see Appendix B). Open-source code will be made available
dyffusion. All important hyperparameters are discussed in Appendix C.3. The training and sampling algorithms used by our method are fully described in Appendix C.I. In Figure 4, we include diagram of the modified SFNO architecture used by our proposed method, which is discussed in the text the end of Section 4 (see SFNO time-conditioning paragraph).
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibiliity In the case of closed-source models, it may be that access to the model is limited in
30

---

some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Data (for both training and evaluation) are publicly available (see Appendix B). Open-source code will be made available at http:/giwwb...oooooosSSS-aaaaaaa dyffusion.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm cc/
www.iiissssniiiiiioooooomoo olicy) for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ //nips. wcwppclpiiicccccccccccccccccccbe geuuide//oodeSiie wwweeeuumiissiiooo asionnolicyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The experimental setup is discussed in Section 5.1. All important hyper- parameters are further discussed in Appendix C.3, and data details are further discussed in Appendix B. Our method's training and sampling algorithms are fully described in Appendix C.I.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Error bars are reported for the reference noise-floor baseline and all probabilis- tic/stochastic methods, including our proposed method, by sampling multiple predictions and computing the standard deviation of the metric (c.g. RMSE) over them.
31

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Section 5.2 on information compute resources used for method and baselines as well as a fair inference runtime benchmark across all methods, including the emulated physics-based climate model.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: //terri//../..././/// cc cc/publicc/thicccuiieeee
Answer: [Yes]
Justification: The authors have reviewed the NeurIPS Code of Ethics and believe that the conducted research in the paper conforms, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
32

---

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See Appendix A for section discussing potential positive societal impacts and negative societal impacts of our work.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
This paper does not deal with data or models with high risk of misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: In our paper, we always cite the works where models (e.g. SFNO [8]) or data (e.g. ACE [67], including URL and license in Appendix B) originated from.
Guidelines:
33

---

The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Our method's training and sampling algorithms fully described in Ap- pendix C.I and fully reproducible in our source-code, including clear instructions.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
34

---

Justification:
Our paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
35


Reviews:
[{"review_id": "tQiBokkiy3", "reviewer": "Reviewer_mrTD", "summary": "This paper presents a new method for probabilistic emulator of climate models, based on a type of diffusion model with spherical geometry. The results for climate model emulation are generally quite impressive.", "questions": "1. The authors do not include any images of generate fields. This makes it hard to qualitatively understand how well the model works, e.g., if it looks realistic. It would be informative to show images of fields at the end of a 10 year simulation and compare them to the training dataset. I suspect such comparisons could be very informative, e.g., I would expect that ACE may show more blurry predictions than this new approach. I would encourage the authors to try to find qualitative examples of how their model obtains better results.\n\n2. The broader point about a lack of correlation between weather and climate model accuracy is well-taken, but I would omit the calculation of a correlation coeffient combining both types of dyffusion models. Within each type of dyffusion model, there is a not a clear trend in performance, as is clear from looking at Figure 10.\n\n3. Do you use the \"fair\" CRPS (which is bias corrected for ensemble size)? This is not a major concern for sufficiently large ensembles, but should be clarified. On a similar note, the spread-skill ratio should not be expected to be 1 for finite ensembles. There is a [correction factor](https://journals.ametsoc.org/view/journals/hydr/15/4/jhm-d-14-0008_1.xml) of sqrt(1+1/m), which I believe would explains away at least the apparent under-dispersion of total water path for the reference simulation.\n\n4. Why do you set h=6? This is worthy of some discussion.\n\n5. Nit: Line 110 references FuXi and GenCast for ensemble weather prediction. I would omit FuXi, given its extremely underdispersed ensemble and poor results, and consider referencing NeuralGCM instead.", "limitations": "Yes", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. The paper establishes what appears to be a new state of the art for climate model emulation, significantly improving upon a previous effort (ACE) by a very experienced team of AI/climate scientists.\n2. Very clear results, showing large improvements both in reduced bias, and reaosnably variability\n3. The method is quite computationally efficient compared to standard diffusion models.\n\nOverall, this problem makes a very significant contribution to the field of AI-based climate modeling, and I strongly support publishing it at NeurIPS.", "weaknesses": "1. ACE is stable for up to 100 year time integration. Is this model similarly stable? My default assumption is that this model is not, which indicates it is not necessarily an improvement in practice. (If I'm mistaken, please correct me and the manuscript!)\n\n2. This is not necessarily a weakness of this paper, per se, but I think the \"noise floor\" approach of ACE for estimating the significance of improvements in terms of reduced bias could be significantly improved upon. You are attempting to estimate a difference in means, so errors and statistical signifiance can be estimated using the central limit theorem, e.g., by taking the average over each year as a (mostly) statistically independent sample.\n\n3. Using CRPS to measure distance between probability distributions, as is done in Figure 8, is a weird choice. It would be better to use a distance measure appropriate to distributions, like the Earth mover's distance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive reception of our work and “strongly” supporting its publication at NeurIPS.\n\n**W1:** That is an excellent point. We did not try running our method for 100 years. While our current simulations cover 10-year segments due to reference data limitations, we recognize the importance of longer-term stability. We are actively working on 100-year inference runs and aim to provide preliminary results during the rebuttal period, with a full analysis to follow in the revised manuscript.\n\n**W2:** Thank you for the suggestion. We are eager to improve the estimate of the “noise floor”. For consistency with ACE, we follow their approach and complement it with the estimation of the “ensemble noise floor”. We will include a discussion of alternative methods for estimating the 'noise floor' in our revised manuscript, acknowledging the potential for improving the estimate using approaches based on the central limit theorem.\n\n**W3:** We appreciate your suggestion. Our choice for the CRPS to measure the quality of the ensembles of time-means was mostly due to its popularity in the time series and, especially,  weather forecasting literature. We agree that better approaches exist (as also discussed with reviewer u1ux) and will note so in our revised Fig. 8 and corresponding text in the Appendix.\n\n**Q1:** Thank you for the great suggestion. We have created videos of two random sample 10-year trajectories by Spherical DYffusion and shared it with the AC (since we are not allowed to directly share links here)*. For the analyzed near-surface wind speed variable (a derived variable from the meridional and zonal wind predictions), we see promisingly realistic outputs compared to the validation climate model simulation.We would be happy to include these visualizations in the final paper.\n\n*We have also included the corresponding snapshots as a set of images in our rebuttal PDF (in case the AC is not able to share the videos with you for any reason). If possible, we would encourage you to look at the videos.\n\n**Q2:** We appreciate this feedback. We will remove the correlation computation from the figure.\n\n**Q3:** Great points. We use the basic formulation of the CRPS as implemented in the python packages properscoring and xskillscore. We will make sure to clearly state this in our updated draft. Similarly, we did not include the correction factor when computing the spread-skill ratio. Note that for the 25-member ensemble, this correction factor is just 1.0198. We will fix this in our revised draft.\n\n**Q4:** This is a great point! The training horizon, $h$, is indeed an important hyperparameter. In our work, we briefly experimented with other horizons (3 and 9) at the initial stage of our project, but then decided to stick with $h=6$ for the following reasons: We believe that it sets a sweet spot between being too small and too large. When it is too small (e.g. 3), it immediately reduces the number of sampling steps for DYffusion and our method, since the reverse sampling process directly corresponds to the time steps $t=0, 1, …, h-1, h$ which can lead to subpar performance. If it is too large, we run the risk that predicting $\\mathbf{x}_h$ from early time steps (e.g. based on $\\mathbf{x}_0$) is too challenging for the forecasting model. Additionally, the DYffusion paper used a similar horizon of $h=7$ for their sea surface temperature forecasting experiments, which is the data that is most similar to ours. We believe that using $h=9$ or similar values close to $6$ would probably work fine too, but we did not have the compute to run ablations to support this (every new choice of $h$ would require re-training two neural networks sequentially). We will add a discussion of this choice to our revised paper, acknowledging its importance and the rationale behind our selection.\n\n**Q5:** Thank you for the reasonable suggestion, we will adapt our draft to reflect it.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "v05gxfSis3", "reviewer": "Reviewer_JrWU", "summary": "In this manuscript, the authors demonstrate a domain-specific, generative approach for a climate model emulator (ACE) trained to reproduce a climate model (FV3GFS) by training on FV3GFS simulation. Their approach is stable and reproduces the climate of the reference climate model with minimal biases. The addition of diffusion-based methodology significantly reduces the biases compared to the original ACE model. The spherical DYffusion approach addresses several important issues: 1.) the long inference and training time for diffusion-based models, 2.) helps introduce correct inductive biases as the atmosphere has spherical geometry, and 3.) allows for climate ensembles.", "questions": "Does the original ACE model run stability for 10 years without the inclusion of diagnostic variables? In the original ACE paper, there are several claims these variables help with the conservation of energy and moisture, however, the results in this paper suggest that at least in terms of stability, these variables aren't required. \n\nDoes the FV3GFS have a good representation of stratospheric dynamics (e.g. QBO and sudden stratospheric warming)? If so how are the stratospheric dynamics in the emulator?", "limitations": "The authors are very upfront with the limitations of the work. My only suggestion in the limitation section is to mention this is climate simulation and trained on observation-based products (ERA5) and thus has some biases with respect to the real atmosphere.", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The spherical DYffusion is an important contribution to both the weather/climate domain as it incorporates the spherical nature of the atmosphere and demonstrates the success of the DYffusion approach on a complex, large system. One of the biggest problems with diffusion-based modeling is the long training and inference times (see GenCast where a single forecast takes 8 minutes compared to GraphCast which takes a few seconds). The spherical DYffusion is a good compromise between the benefits of a generative modeling approach and computational time during inference (something important for climate modeling). This approach only increases inference time by a factor of 3 and opens up the possibility for scalable, fast diffusion-based models for weather and climate.\n\nThe DYffusion methodology presented in this paper has a large number of applications well outside the weather and climate field. \n \nFor climate research, the ability to rapidly run an ensemble of climate simulations is a large selling point for data-driven emulators compared to traditional physics-based, numerical methods.", "weaknesses": "Some of the claims in the conclusion are misleading and I suggest toning down the climate change-related conclusions (e.g. \"Our method’s climate biases nearly reach a gold standard for successful climate model emulation and thus represent a significant leap towards efficient, data-driven climate simulations that can help society to better understand the Earth and adapt to a changing climate\"). Right now this model makes a stationary climate assumption (e.g. boundary conditions and forcings are consistent in both the training data set and during model inference). As mentioned in the limitation section, there is still significant research and evaluation needed before these emulators can start tackling climate change. \n\nThe emulator is also trained on climate simulation not reanalysis (e.g. ERA5) which is typically done for data-driven weather models. The FV3GFS itself has biases with respect to the past and present climate which should be noted.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive reception of our work and viewing it as an “important contribution” that has “a large number of applications well outside the weather and climate field”.\n\n**W1:** Sincere thanks for bringing this up. Firstly, we plan to tone it down by rewriting to *“(...) a gold standard for successful climate model emulation under our simplified setting (see limitations)”*. Secondly, we will make clear in our limitations section that a trained emulator should be expected to inherit any biases of the underlying training dataset (here, FV3GFS) with respect to the real atmosphere. We appreciate your insight, which has helped us refine our presentation of results.\n\n**Q1:** That is an interesting question. Based on training our own method as well as DYffusion baseline, our intuition is that yes, stability would not be hurt by removing the diagnostic variables from the training set, but we haven’t actually re-trained ACE this way. Also, from reading the paper it seems to us that the main motivation for including these variables was to be able to evaluate how well ACE conserves energy and moisture rather than helping stability.\n\n**Q2:** This is an excellent question that touches on important aspects of stratospheric dynamics. The top model layer in the FV3GFS-derived dataset encompasses 0-50 hPa, lying entirely in the stratosphere. We note that FV3GFS itself does not have a good representation of the QBO. For sudden stratospheric warming, we are unsure ourselves and would have to dig deeply to evaluate this properly. Generally, stratospheric variables seem particularly challenging to learn due to their slower time scales and partial decoupling from tropospheric weather. Indeed, meridional wind and specific total water at level 0 are amongst the few variables where Spherical DYffusion underperforms ACE in terms of RMSE of the time-means. A more detailed analysis of our model's performance on and potential improvements for stratospheric variables could be an interesting direction for future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vBWU18ehth", "reviewer": "Reviewer_u1ux", "summary": "The paper presents a method for approximating a physics-based climate model with a faster data-driven model. To make the model probabilistic the method uses diffusion models (specifically DYffusion). Predictions over longer timescales are generated using autoregressive rollouts. Spherical Fourier Neural Operators (SFNOs) are used to handle the non-euclidean geometry of the earth. Training is performed on a large climate simulation (FV3GFS). \n\nIn experiments, the method is compared to a set of alternative approaches. Each method is used to generate an ensemble of climate simulations over 10 years, and the statistics of these runs can be compared to a similar ensemble from the physics-based model. Comparison metrics include the bias and variability of time-averages of atmospheric variables.  In addition to the standard physics-based climate models, experimental comparisons are made to (1) the deterministic ACE model (which uses the SFNO), (2) a probabilistic version of ACE (using MC-dropout), and (3) a DYffusion model with a U-Net architecture instead of SFNO.", "questions": "None.", "limitations": "The paper clearly addresses the limitations.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The paper is very well written. It is well-organized and the methods used are clearly explained. \n- The relation to previous work is clearly explained.\n- The experiments compare against strong baselines and support the hypotheses.\n- The topic is of significant interest. Physics-based climate models are key to our understanding of climate change, but they are fundamentally limited by the computational costs. There is keen interest in using machine learning to address these challenges by building fast emulators that can be used to generate larger ensembles and help quantify variability and uncertainty. The authors address the key problem by proposing a method that attempts to balance performance with computational complexity.", "weaknesses": "- The proposed model is never evaluated using probabilistic performance metrics (such as data likelihood). Diffusion models are attractive for probabilistic modeling because they have been shown to be good at maximizing the training data likelihood (e.g. Song, et al. Maximum Likelihood Training of Score-Based Diffusion Models).  The authors use DYffusion for its speed and say it matches or outperforms the accuracy of standard diffusion models, but is accuracy the right metric for emulating a physical system like this? The experimental results suggest that the model is working well for estimating climate variables, but it would be nice to have a stronger argument that the DYffusion model is a good probabilistic model of this data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for their positive reception of our work, particularly noting its organization, clear relation to previous work, and the practical significance of our contribution to climate modeling and climate change.\n\n**W1:** This is an excellent point. Our paper focuses on standard evaluation assessments for climate modeling. We cannot compute the data likelihood exactly because the diffusion model can be understood as optimizing a variational lower bound. This is complicated further by the DYffusion framework only falling under the umbrella of generalized diffusion models (not a type of Gaussian Diffusion). It is certainly interesting to evaluate the ensemble simulations with probabilistic metrics. In Figure 7, we take a simple approach by evaluating the spread across ensembles of 10-year time-means. Moreover, in Figure 8, we include a comparison of the ensembles of time-means via the Continuous Ranked Probability Score (CRPS), which shows agreement with the ensemble-mean RMSE of the same. However, the CRPS is a pixel-and timestep-wise metric which may not tell the whole story. To our knowledge, more advanced metrics still need to be carefully  developed (for example, https://arxiv.org/abs/2401.14657 presents an interesting approach).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qlkCIZxvpg", "reviewer": "Reviewer_2sHD", "summary": "In the paper the authors develop a generative model for global climate simulations. By combining the DYffusion framework for generative modeling with Spherical FNOs that respect the spherical earth geometry the resulting model has stable rollouts for 10-year climate simulations. Experiments show that the resulting model achieves more accurate climate simulations than existing machine learning baselines and the sampled ensemble reproduces meaningful climate variability. Due to the DYffusion framework the generative modeling is achieved without a massive increase in the number of forward passes required to sample a state trajectory.", "questions": "1. The DYffusion baseline is described as using the original U-net architecture. This is, as accurately represented in the paper, a poor choice that does not take the spherical geometry or any longitudinal periodicity into account. I would expect that this creates issues with artefacts or instability on the \"edges\" of the flattened earth, and close to the poles. Do you see such problems in practice? How much does these issues explain the poor performance of the DYfussion baseline? I think more discussion about how DYffusion performs empirically could be valuable, as its shortcomings are the main motivation for the use of SFNO in the proposed model.", "limitations": "The authors extensively and accurately discuss the limitations of the work.", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "1. ML has a high potential for impact in the development of climate models. As outlines in the paper, ML has seen great success in medium-range weather forecasting and we are starting to see this also translate to climate modeling. This paper significantly pushes this development forward in bringing important advances in terms of both model accuracy and ensemble modeling.\n2. The paper is well written and relatively easy to follow, despite containing combinations of multiple complex components (DYfussion, SFNOs, climate modeling). The authors do a good job at making climate modeling concepts accessible to a wider ML audience.\n3. The proposed method of combining SFNOs and DYfussion is well-motivated by requirements on stable rollouts + sampling speed + ensemble modeling.\n4. The evaluation includes relevant analysis, baselines and metrics.", "weaknesses": "1. The stochasticity in the ensemble comes from MC dropout + Stochastic depth in the inference network. There is no clear motivation for this choice. \n    1. While MC dropout is an easy way to introduce stochasticity, it is generally known to not represent the true uncertainty very accurately. It is unclear to me if this is a suitable choice for representing the full distribution of possible interpolated states. On the other hand, the SFNO network with MC dropout here sits inside the DYfussion framework. It is now obvious to me how crucial it is for interpolation network to accurately represent the distribution of the atmospheric state for the final ensemble to match the distribution. Perhaps the authors can shed some light on this.\n    2. While MC dropout is used in the original DYffusion framework, it is unclear why Stochastic depth was added here. It would be valuable with an ablation to see the importance of MC dropout vs Stochastic depth.\n2. The ability of the ensemble to accurately capture climate variability is mainly evaluated in Table 2. In this table I am missing the same spread computed for the (generative) baselines. It is somewhat hard to put the spread of the proposed method into context without comparing to the baselines.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive reception of our work and its \"important advances\". We are especially glad to hear that you think we have done \"a good job at making climate modeling concepts accessible to a wider ML audience.\" We sincerely hope that, similarly to ML-based weather forecasting, we will see a flurry of research on this in the coming years at top ML conferences.\n\n**W1.1:** That is a great point. The main reason for using MC-dropout is its simplicity and the fact that the original DYffusion framework also uses it with promising results. You are right that MC-dropout alone does not lead to a skillful ensemble, but the inclusion of the DYffusion framework is key.  ACE-STO, made stochastic with MC-dropout in the same way as our method's interpolator network, did not show improvement over deterministic ACE (we also verified that stochastic depth does not improve the MC-dropout-only version of ACE-STO).\nA possible explanation is that while MC-dropout is crucial for stochastic sampling, the forecaster's predictions remain deterministic and are used to initialize the new autoregressive rollout iteration.\n\nWe will further clarify these in the updated version. \n\n**W1.2:** Thank you for bringing this up. This choice was informed by preliminary experiments focused on training a good interpolator network. There, we found the addition of stochastic depth to slightly improve the interpolator’s validation CRPS scores (for the interpolated timesteps 1 to 5) and significantly improve the calibration of the interpolation ensemble based on the spread-skill ratio (averaged across variables from around 0.26 to 0.35). We will discuss this motivation in our revised draft.\n\n**W.2:** Thank you for the great suggestion. Below, we have added the corresponding spread’s from the baselines. DYffusion reproduces the ensemble spread similarly well as our method, while ACE-STO clearly has troubles reproducing a variability of similar magnitude as the reference.\n\n| Model      | $p_s$ | $TWP$ | $t_7$ | $u_7$ | $v_7$ |\n|------------|---------|---------|---------|---------|---------|\n| Reference  | 19.961  | 0.199   | 0.090   | 0.142   | 0.110   |\n| Ours       | 23.517  | 0.214   | 0.094   | 0.167   | 0.121   |\n| DYffusion  | 24.753  | 0.223   | 0.082   | 0.169   | 0.127   |\n| ACE-STO    | 46.720  | 0.554   | 0.295   | 0.306   | 0.205   |\n\n\n**Q1:** We did not see any obvious visual artefacts at the boundaries of the “image”. However, we did observe biases arise in the DYffusion baseline. For example, Figure 9 shows that DYffusion develops clear biases (left column) for the surface pressure, total water path and meridional wind level-7 variables during the first 90 timesteps that are not (less) present in our method (SFNO-based ACE). Interestingly, these biases are not captured by other weather metrics CRPS, RMSE, spread-skill (right columns). We believe that minimizing such biases is crucial for the long rollouts used to evaluate our climate model emulators, which explains DYffusion's poor performance in terms of climate biases.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank all reviewers for their thoughtful, valuable, and encouraging feedback. 

We are very encouraged by the consistently positive reception of our work by all reviewers, who value its organization, clear relation to previous work, and the practical significance of our contribution to climate modeling and climate change (u1ux). We appreciate that reviewer mrTD *"strongly"* supports its publication at NeurIPS, while reviewers JrWu and 2HsD see it as an *"important contribution"* and bringing *"important advances"* respectively. We are particularly pleased that reviewer JrWu recognizes its potential for *"a large number of applications well outside the weather and climate field."*

We are especially glad to hear that reviewer 2HsD thinks that we make *"a good job at making climate modeling concepts accessible to a wider ML audience."* We hope this work will catalyze a surge of research in this area of ML-based climate modeling at leading ML conferences, mirroring the recent expansion of ML-based weather forecasting studies.

We have addressed the questions and weaknesses noted by the reviewers in the respective rebuttals.

In response to reviewer mrTD, we attach a PDF that shows qualitative samples from our method as well as the climate model validation simulation. The emulated field demonstrates a high degree of realism, closely mimicking the patterns and variability observed in actual climate model outputs. We have also generated corresponding videos of the full 10-year-long trajectory for the same two random samples of Spherical DYffusion. We shared the anonymous link with the AC (due to the discouraging wording of the rebuttal instructions regarding sharing links) and hope that they can share it with the reviewers.

Thank you again,

The authors of submission 11626

Meta-review: The paper proposes an interesting probabilistic emulator for climate models, integrating diffusion models with spherical geometry. The approach, leveraging Spherical Fourier Neural Operators (SFNOs) and DYffusion, offers computational efficiency and accuracy for long-term climate simulations. The method was validated against various baselines, showing superior performance in terms of bias reduction and climate variability. 

All reviewers are positive for this paper from the beginning. The authors’ response and additional discussion/results further convinced the reviewers that this work is a solid contribution to weather forecasting as well as climate models. I therefore recommend acceptance of the paper. 

The reviewers did mention that some of the claims in the conclusion are misleading, e.g. "Our method’s climate biases nearly reach a gold standard for successful climate model emulation and thus represent a significant leap towards efficient, data-driven climate simulations that can help society to better understand the Earth and adapt to a changing climate". Therefore I encourage the authors to tone down these claims as they promise in the rebuttal.
Decision: Accept (spotlight)