Predicting partially observable dynamical systems via diffusion models with a multiscale inference scheme
Authors: Rudy Morel
Francesco Pio Ramunno
Jeff Shen
Alberto Bietti
Kyunghyun Cho
Miles Cranmer
Siavash Golkar
OLEXANDR GUGNIN
Geraud Krawezik
Tanya Marwah
Michael McCabe
Lucas Thibaut Meyer
Payel Mukhopadhyay
Ruben Ohana
Liam Holden Parker
Helen Qu
François Rozet
K.D. Leka
Francois Lanusse
David Fouhey
Shirley Ho
Venue: NeurIPS 2025 poster ()
Keywords: Probabilistic forecasting
diffusion models
dynamical systems
partial observability
solar prediction
Primary area: machine_learning_for_sciences
TL;DR: We tackle prediction of partially observable systems with a multiscale inference scheme for diffusion models, improving accuracy and stability on solar dynamics without increasing computational cost.

Abstract: Conditional diffusion models provide a natural framework for probabilistic prediction of dynamical systems and have been successfully applied to fluid dynamics and weather prediction. However, in many settings, the available information at a given time represents only a small fraction of what is needed to predict future states, either due to measurement uncertainty or because only a small fraction of the state can be observed. This is true for example in solar physics, where we can observe the Sun’s surface and atmosphere, but its evolution is driven by internal processes for which we lack direct measurements. In this paper, we tackle the probabilistic prediction of partially observable, long-memory dynamical systems, with applications to solar dynamics and the evolution of active regions. We show that standard inference schemes, such as autoregressive rollouts, fail to capture long-range dependencies in the data, largely because they do not integrate past information effectively. To overcome this, we propose a multiscale inference scheme for diffusion models, tailored to physical processes. Our method generates trajectories that are temporally fine-grained near the present and coarser as we move farther away, which enables capturing long-range temporal dependencies without increasing computational cost. When integrated into a diffusion model, we show that our inference scheme significantly reduces the bias of the predicted distributions and improves rollout stability.

Predicting partially observable dynamical systems via diffusion models with a multiscale inference scheme
Rudy Morel Francesco Pio Ramunno Jeff Shen'
Alberto Bietti', Kyunghyun Choo, Miles Cranmer", Siavash Golkar Olexandr Gugnin", Geraud Krawezik", Tanya Marwah', Michael McCabe Lucas Meyer Payel Mukhopadhyay Ruben Ohana', Liam Parker 188, Helen Qu', François Rozet", K.D. Leka 10,11 François Lanusse David Fouhey², Shirley Ho
The Polymathic Al Collaboration
Flatiron Institute, University of Geneva, RFHNW, "Princeton University, "New York University, Univerrityy of Cambridge. University of Kyiv, University of California, Berkeley, Uniiversity of Liège, 10 North West Research Associates, Nagooa University, Univerrsité Paris-Saclay, Université Paris Cité, CEA, CNRS, AIM.
Abstract
Conditional diffusion models provide a natural framework for probabilistic predic- tion of dynamical systems and have been successfully applied to fluid dynamics and weather prediction. However, in many settings, the available information given time represents only small fraction of what is needed to predict future states, either due to measurement uncertainty or because only a small fraction the state be observed. This is true for example in solar physies, where observe the Sun's surface and atmosphere, but its evolution driven by internal processes for which lack direct measurements. In this paper, we tackle the probabilistic prediction of partially observable, long-memory dynamical systems, with applications solar dynamics and the evolution of active regions. We show that standard inference schemes, such as autoregressive rollouts, fail to capture long-range dependencies in the data, largely because they do not integrate past information effectively. To overcome this, we propose a multiscale inference scheme for diffusion models, tailored physical processes. Our method generates trajectories that are temporally fine- fineggraieed near the present and coarser as we move farther away, which enables capturing long-range temporal dependencies without increasing computational cost. When integrated into a diffusion model, we show that our inference scheme significantly reduces the bias of the predicted distributions and improves rollout stability.
Introduction
Probabilistic prediction of dynamical systems is at the heart of many challenging tasks science and engineering. Diffusion models have recently shown success in probabilistic prediction for physical systems, especially when they are applied to simulated environments (39 or to settings such as terrestrial weather prediction [62], where laboratory settings or advanced data assimilation can recover
much of the current system state [27].
Many real systems are partially observable, meaning that data is missing, unobtainable, or sufficiently noisy such that at any given time there is inadequate information accurately infer the underlying state of the system. It follows, then, that there is inadequate information to predict its exact evolution. In these settings, the correct incorporation of past information can help predict future trajectories.
"Contact: rwwoee@fffeit aflreellllairronnssttttee.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

A prime example of such partially observable system is our nearest star. Key components governing the dynamics of the Sun are not directly observable (e.g, the driving forces beneath the visible "surface"), ) and what is observable is only available via remote sensing. Nonetheless, predicting this particular system's 's evolution is important due to the potential impact on technology-basee sectors of society arising from solar energetic events (56). While domain experts have identified physical descriptors associated with energetic phenomena such as solar flares [40 I0 42), and relevant ML-ready datasets have been curated and published [22 4 [6]. there does not yet exist model
(physics-based or ML-based) that can predict future states of solar active regions and their magnetic
fields across the spatial and temporal scales relevant to significantly improve prediction for these events
In this paper we study the problem of predicting partially observable dynamical systems with diffusion models [29], motivated by the challenging problem of learning solar dynamics from data. As benchmark to encourage community progress on this problem, we assemble an 8.5TB dataset of 512 x 512 videos of solar regions containing 12 fields with measurements of the magnetic vector field and the Sun's atmosphere. Diffusson models developed for well-observed fluid simulations [39] or reanalyzed terrestrial weather data [62] typically use an autoregressive inference scheme to generate future predictions, conditioning on only a few past frames (lypically two). For solar dynamics, however, we find that such models struggle to accurately predict the evolution, showing significant deviation from observations over time.
To address these limitations,, we introduce new multiscale inference scheme based on "multiscale templates", which provide an efficient way to condition on distant past information without increas- ing computational cost. These templates enable the generation of distant future time steps while conditioning on fine-grained present information and coarse- coarse-grained past times. A model trained on generating such videos can then be used to generate arbitrarily long trajectories in the future, by combining different multiscale templates. Compared to inference schemes such as standard autore- gressive rollouts used in the literature 33 62
our method predicts a distant future time step from past observations in a single call to the diffusion model, avoiding the accumulation of distribution errors. Furthermore, we condition more frequently, and on a larger portion of past observed data.
Contributions. Our key contributions are: (a) We introduce a new multiscale inference scheme tailored to partially observable dynamical systems encountered in Physics. (b) On the challenging task of solar prediction, our multiscale inference scheme outperforms standard schemes from the literature on diffusion models for physics and natural videos, reducing prediction bias and instability. (c) To the best of our knowledge, our model is the first multi-modal diffusion model trained to predict high-resolution solar videos: prior work focuses on single modality, low-resolution data (both in time and space). (d) To encourage competition on the challenging problem of solar prediction, we provide a new multi-modal 8.5TB dataset of 512 x 512 videos capturing solar regions. Upon publication, our dataset and model will be made publicly available.
2 Related works
Diffusion models for predicting dynamical systems. Unlike [47 60). which employ a diffusion model to learn the distribution of individual states in order refine predictions from a predictor network, our work falls within the scope of modeling the dynamic of the observations. Along these lines, [39 69 address highly observable dynamical systems, like fluids governed by the Ntvier-Stokes equations, where all relevant variables (e.g., velocity, pressure) are accessible. Other works (62) train on data from complex reanalysis of sparse observations (c.g.. the ERAS dataset (27). Full observation or re-analysis is not always feasible. For instance, in solar dynamics, it is challenging to accurately recover surface observations at even moderate scales [see, e.g.
14 and becomes
especially difficult when attempting to infer the state of the Sun's 's interior [4 49) energy transfer [82]
or forces acting on the plasma [I] 88 yet this information is key to predicting solar dynamies. Thus, while [39 62] see no benefit from using more than two past observations, incorporating additional past steps substantially improves results in our setting. In that sense, our findings align with those of (70) even though they focused on deterministic models. Diffusion models can perform data assimilation and prediction from incomplete observations simultaneously [66] 74 33,, but this requires a dataset of underlying system states to train the model an assumption we do not make in this paper.

---

Inference schemes for diffusion models. The standard autoregressive inference scheme for video diffusion [1
68] consists in generating progressively an entire video by sliding short window. Beyond this, Flexible Diffusion Models (FDM) [25] FDM) and Masked Conditional Video Diffusion X4 both adopt flexible conditioning strategies and train a single model with a randomized masking. In particular, [25] introduces two types of inference schemes. The first, called "long-range," generates progressively more distant future frames while conditioning only on recent ones, thereby discarding distant past information. The second, called "hierarchy-2," uses sliding-window with an initial long-range prediction, but it conditions on past information only at the first iteration. In contrast, our multiscale inference scheme generates videos multiple scales and conditions on past information across multiple iterations, which is crucial for recovering information in partially observable dynamical systems.
8 26
Machine learning for solar physics. Machine learning increasingly used across heliophysies
13
5) in particular for predicting solar energetic events [9] 57 [59 58
However, these approaches typically perform classification based on selected features rather than modeling the temporal evolution of the solar atmosphere. Other works apply ML to enhance data quality [12 35
20 44
34 or build large-scale pretrained models [85], but these also do not predict future physical states. When it comes to predicting future solar trajectories, many works either focus on a single quantity of interest [6 65 21] or operate on limited spatiotemporal resolutions. For example, [65 I use at least 4x spatial downsampling factor and temporal resolution no finer than 12h. In contrast, our dataset uses multiple modalities (associated to different instruments); is downsampled only 2x spatially, matching the optical resolution of the instrument; and is captured at Ih sampling rate.
86
24
3 Background: Conditional Diffusion models
This section presents the aspects of conditional diffusion models
75
29 most relevant to our work.
Score-based diffusion model. Score- base- generative models 77 78 are a class of generative
models that learn to sample from complex data distributions by reversing a gradual noising process. These models define a forward diffusion process in which the input data x E RR is progressively corrupted by adding Gaussian noise at various noise levels 0s
x, = x + O.C,
C c~ <~ (N,
(1)
The resulting distribution over the noisy data is denoted by p. (x.) and captures how the original data distribution evolves under increasing noise. The generative model learns reverse denoising process
which maps Gaussian distribution to the distribution of the data [77 3 81] This can be described as stochastic differential equation
=x - log + a.dW.,,
and involves the score function V Vlogp. This score can be obtained by solving a denoising task 229 77
78
46
76
37). Indeed, if we write D(x, 8) a function that minimizes the L loss
Ex-rasut (0,,(,
[ 8)
x]] , with x, = = +oC.
(3)
then we can show
18
52 that the score is given by V Vlog (xx) (D(x,, x,) /0z.
Therefore, a diffusion model is trained by learning a neural network Dg with parameters on the denoising loss 33, and sampled by discretizing the reverse process 2
Conditional diffusion model. the paper, beyond modeling the distribution p(x) of the data, we focus on modeling conditional distributions p(xly) where x is a trajectory and y is a part of the trajectory itself [84 67.. To that end, let m E {0, 11.² denote a vector (or mask) indicating which parts of the signal are used as conditioning. The conditioning data is written m & x, where @ is the element-wo wise product. As above, the distribution p(x @ x) can be modeled by learning a denoiser to reconstruct the "clean" data from its noised version x, with in addition the information of the conditioning:
E Ex-peeta: (o. [ m) o xx + m 3 x, m) x||] with x, = x o.e. (4)
where the mask m is fed to the denoiser Dy to help differentiate between noised data and conditioning data. This way, the denoiser is trained to retrieve the global noise from the noised data x, just like E3. but with additional conditioning clean information @
3

---

098869 090860
1 Autoregressive scheme
Multiscale templates
Multiscale inference scheme
Figure 1: Multiscale templates and inference scheme. (Left): Our multiscale templates in purple. (Right); Comparing a standard autoregressive scheme (on top) with our multiscale inference scheme. We use the visualization style of [25]. in which dark boxes indicate available steps (either observed or generated at previous iterations) and red and blue boxes indicate steps that are used as conditioning generated, respectively. Each row is
call to the conditional diffusion model with the used template indicated by the number next to the row. Our inference scheme enables capturing longer- range dependencies, conditions more often in the past, and mitigate rollout instability by generating distant future (9 on the figure) in one call to the conditional diffusion model.
Multiscale inference scheme for physical processes
In this paper, interested in predicting dynamical system from its observations x, e.g. the magnetic field at the surface of the Sun. At each time t, we denote x, the observation of the system, which provides only a partial view of the underlying true state.
At present time O, the goal is to generate future realization x1:7 horizon conditionally on the past K;50- In doing aim to approximate the following conditional distribution
p(xIT(X(5o)
(5)
Due computational constraints, modeling the full distribution over long horizons T is infeasible. A common approach is to compress the data to extend the effective context length, as done in latent diffusion models g 26 233, but the question remains, how generate arbitrarily long trajectories using a generative model with fixed trajectory length?
We assume that our conditional diffusion model can generate only subset of 2K + time steps at once. We seek to use the fixed-size model to produce samples over a far larger set of T >> 2K + 1 steps by repeatedly applying the fixed length model. For convenience, assume that the model always generates K future steps from white noise, and the remaining K + 1 are conditioning (from the past or present). Generating trajectory of length T thus requires at least [T/K] steps. If we define 7, as the set of K new time indices generated and Cn the set of K + frames used as conditioning, the iterated process amounts the following approximation:
N
p(x1:T (x5o) = II p(xı, x.
(6)
1==
A collection of pairs of index sets (In, Cx), 1 N, is called an inference scheme. Given the above fixed budget constrain, these sets must satisfy |C,| == + +1, In| = K. We write Pa the set of indices available step n, which is defined recursively as Pi (t < 0} (observed past) and Pa P In, (available time steps). To properly formalize the problem, we consider inference schemes that satisfy the following properties:
(completeness) UN (1.....
(admissibility) C2 C P.. the conditioning is done on already generated (or observed) steps (efficiency) Tg It @ for k * no future step is generated twice
For example, an autoregressive inference scheme consists sliding a fixed-size fine-grained window progressively forward in time, Cn {(n 1).....,77 and In {nK 1,..., (n + 1)K}

---

Autoregressive
Multiscale
Observed
Predicted
t= 0
20
40
60 80
Timestep
Figure 2: Performance of our multiscale inference scheme on a synthetic example. The observed data (blue) consists of Gaussian fluctuations around sinusoidal trend. Predictions (red) are from diffusion model with access only to past data & S 0. (Top): The global trend is barely observable at fine scale. Thus, model that generates small trajectory segments autoregressively tends accumulate errors, leading to biased and overly broad predicted distributions. (Bottom): Our multiscale inference scheme (see Fig. [ efficiently recovers the target distribution with Wasserstein distance of 0.021 vs. 0.23 for the autoregressive model. When restricted to the same 3-step past horizon, the multiscale inference still performs better, with a Wasserstein distance of 0.08.
as shown on Fig. I This autoregressive inference scheme has several downsides, evidenced in Tab. and illustrated in Fig. [[] The main one being that after the second iteration, there is explicit conditioning on observed data, which contributes to rollout instability.
4.1 Multiscale templates for physical processes
Finding an appropriate inference scheme for partially observable dynamical systems is challenging due to the large space of possibilities: many candidates exist for pairs of conditioned times C, and generated times 7, each step that satisfy the above properties.
To guide our design, we highlight two key challenges encountered in predicting physical systems:
(a) Partially observable. The state of the system any given time cannot be fully determined from the observations. Consequently, the distribution of future scenarios conditioned on past observed data may not be restricted state cannot be fully observed due to missing measurements of key physical variables (e.g.. velocity fields, or unresolved structures), insufficient observational resolution, or corruption arising from instrumental noise.
Dirac measure. In many cases, the system
(b) Long-memory. Many physical processes exhibit long memory, or long-range dependency, in time. This can be quantified by smooth decay of the autocorrelation (sometimes characterized in the frequency domain by a power-law decay of the power spectrum [51 77
4|48 53 55)). Intuitively, observations closer the present have a stronger impact on the future and the influence of distant past observations gradually diminishes while remaining significant.
Diffusion models have been applied to predicting dynamical systems without fully addressing challenge (a) or relying on additional information to overcome For example, (39 apply diffusion model to fully resolved fluids which are effectively Markovian. In weather prediction, although the observed data is sparse, data assimilation known as canalysis-enables the reconstruction of missing information, resulting in large datasets of highly informative states [27], on which diffusion
5

---

HMI Bp
HMI Br
HMI Bi
AIA 131A
AIA - 171A
AJA 1600A
time
Figure 3: (Above): Example full-disk solar images from 2015-12-12 (see 5.22 for details). The left three panels are photospheric vector magnetic-field field components; the right three panels are images of the solar corona and chromosphere. "Active regions" (intense magnetic fields connected bright coronal structures) are present in both modalities. (Below): A sequence of frames of cropped active region, corresponding to the red box in the row above.
models have been successfully trained (62]. Other models handle missing states, but require clean sates for training [4 33), which is always available.
In this paper, we tackle the challenging problem of predicting the observations a dynamical system presenting the two challenges (a) and (b) simultancously, as is common across many disciplines. For example, in oceanography and climatology, shallow ocean layers are observed while few observations exist for the deep ocean 45 and in seismology, subsurface stress is not directly measured (36). In solar physics, the goal of predicting a future trajectories of active solar regions from available observations (of the magnetic solar surface and hot coronal atmosphere) is challenged by: (a) missing key components of the sate in this case, observations of the interior of the Sun, with instrumental noise present in the data 33 72), which is sometimes not fully understood or mitigated [73]. And (b), the targets that are of predictive interest, e.g., sunspots, have long-range dependencies described by plasma diffusion and flow patterns on local, moderate, and global spatial scales [14
In principle, if the system state was knowable and described by well-constraieed partial differential equations (e.g., g. songneto-hddrdyyaamie framework (63]) one could solve the dynamics forward in time from a single time step (Markov process). Now, under assumption (a), even the underlying system is Markovian, its observations may not be predicted deterministically because of the lack of information; such systems are often called hidden Markov [19]). The combination of properties (a) and (b) it is often the case in real cases, encourages diffusion model consider not only information near the present but further back in time to access what is needed to predict the future.
Inspired by works long-range temporal processes 48 55 and wavelets [5]] 79 54
introduce a framework to do this.
A multiscale template T is set of 2K + 1 indices centered at the present to 0 and becoming progressively coarser farther from it, defined using time increments as powers of a 2
To [t= 18
1x) with t = th + a and thx = 1 k+1
This set of indices is symmetrical in 18 = 0. For a = 1, we retrieve a standard uniform window used in an autoregressive scheme. When a 1, the time indices are progressively more spaced as we move away from present. We allow to be real, in that case, the template is mapped to integers through I = (sign(()) [I -K < k < K} where [t] is the integer part of
For a fixed budget of times, a multiscale template allows to consider a horizon in the past (and in the future), that is exponential in K, while uniform template As we will see the next section, this is crucial for capturing long-range dependencies, and helps stabilize long predictions.
has horizon that is linear in K.
The term template reflects the flexibility to later separate into conditioning Cn and newly generated time indices In as needed, that is, to apply arbitrary conditioning mask m in Eq. 4).

---

t +
t ++ 4
t ++8 8
++ 16
t + 32
Observed
Antoreg.
Multiscale
Figure 4: Example of predictions, for different inference schemes: autoregressive and multiscale
(ours). Colorbar: -3000
3000 Gauss (magnetic field).
4.2 Multiscale inference scheme
We now design an inference scheme to produce arbitrarily long future trajectories, using the multiscale templates introduced above and motivated by the key properties of observed physical systems. As described above, this involves defining pairs (Cn, In) of conditioning indices and newly generated indices at each iteration n, that is, each call to the diffusion model, which progressively cover 3 new time steps
a future trajectory (see Eq. (6). In the experiments we choose to generate diffusion models generate small videos of length 2K + = 7.
at each iteration, which means and we choose to use templates T with a maximum Qmmx = 2.5 (see Fig. D in the following we K and write T directly. This means that the most extended video
drop the dependence generate at once goes up to 9 + 2.5 + 2.5² steps in the past and future (see Eq. 77 ). We refer the reader the Appendix for multiscale inference schemes with different choices of K and O'max-
will
Our inference scheme, illustrated in Fig. I begins by using the largest template To {-9, 3, 1, 0, 3, 9) to generate K steps in the future: 11 {1, 9} and conditioning on the + 1 = 4 observed steps Ci = (-9, -3, -1,0). This enables the model to generate the g step into the future while incorporating observed data that extends equally far into the past. Without completing an entire trajectory, this first step gives us predictions the physical system at multiple horizons in the future. Once this multiple-horizon prediction is performed, the goal is to "fill the gaps" in the future using the other, shorter-range templates.
Then, we iterate over all possible templates T with < S Omax in decreasing order, along with all their possible shifts into the future. For each candidate, we check whether the shifted template overlaps with at least K + 1 = 4 available time steps. This ensures sufficient conditioning data to generate K new steps. Among the valid options, we select the first template and shift whose final index aligns with the current maximum horizon, which is 9 in our experiments. This ensures that the generation proceeds a consistent way, gradually filling in missing future steps while maintaining coherence with earlier generated data. In the experiments, must be shifted by steps in the future. The overlap with the previously generated time steps defines C2 (-3, 1, 1,3, 9} and the newly generated indices at this second iteration /2 (2,4,5).
get T {-6, -2, -1, 0. 1, 2, 6} which
We repeat this procedure until all the gaps from the first applied largest template are filled. For the values chosen in the experiments, this requires applying last multiscale template T

---

Table 1: Predictions performance. We compare different inference schemes (Autoregressive, Hierarchy-2 [25]. Ours Multiscale) and models (AViT [50],AR [39) Ours). For each, we evaluate at three different time windows (1-4 hours, 4-16 hours, 16-32 hours) using multiple metries: the Wasserstein distance between the distributions; mean absolute error in the power spectrum; and normalized mean absolute error of representative solar physics quantities from 10 the Mean Horizontal Gradient of the Total Field (MeanGBT) and of the Vertical Field (MeanGBZ).
Wasserstein
MAE Power Spec.
NMAE MeanGBT
NMAE MeanGBZ
Model
Scheme
1:4
4:16
16:32
1:4
4:16
16:32
1:4
4:16
16:32
1:4
4:16
16:32
DIT
Autoreg.
3.9
5.6
7.9
0.25
0.36
0.53
0.18
0.30
0.37
0.15
0.25
0.31
DiT
Hiera. 25
3.0
4.6
6.0
0.12
0.27
0.38
0.12
0.28
0.38
0.09
0.22
0.31
DiT
Ours
3.0
4.3
5.5
0.12
0.22
0.33
0.14
0.27
0.33
0.10
0.21
0.27
50
Autoreg.
12
13
15
0.11
0.35
0.81
0.40
0.44
0.45
0.40
0.43
0.44
39
Autoreg.
7.3
12
16
0.20
0,47
0.71
0.29
0.52
0.67
0.27
0.49
0.64
DiT
Ours
3.0
4.3
5.5
0.12
0.22
0.33
0.14
0.27
0.33
0.10
0.21
0.27
(-3.-2. 2,, which actually a uniform template, shifted by 6 in the future, and condi- tioned on the time steps C3 = [3, 5, and generating new time steps I3 = {6, 7.8}.
Once the first template span has been entirely generated, shift the current present the last generated step, 9 in the experiments, and can now repeat the above scheme to predict a complete video until 18 and so (see Fig.
This inference scheme offers key advantages. Compared to standard autoregressive or "hierarchy- 2" schemes [25]. it conditions more often on distant past and future information, better capturing long-range dependencies around the present. It predicts up to 9 steps ahead in a single diffusion call, whereas autoregressive methods require 3 calls for the same horizon. This improves error accumulation, though errors can still grow beyond the largest template's time scale.
The horizon of the largest template is chosen to be 9 in experiments but it can be adjusted (see Appendix for general algorithm). If the physical process exhibits finite decorrelation timescale, natural to choose largest template that spans this timescale to fully capture long-range dependencies and mitigate rollout instabilities. We refer the reader to the Appendix for multiscale inference schemes based on larger templates.
5
Numerical experiments
5.1 Synthetic example
We present a synthetic example time-series of observations x; = th where b4 is a deterministic sinusoidal trend, and is made partially observable by the addition of Gaussian noise n)- In the absence of noise, single time step suffices to determine the future trajectory completely. In the presence of noise, however, consider the times around negative peak (approximately Depending on the noise realization, the local trend may be upward or downward, making the state difficult recover locally. That is, partial observability induced by noise prevents accurate estimation of the underlying slowly varying component. It is thus necessary to look further into the past, which precisely what our multiscale inference scheme achieves.
30; see Fig. (2)
Fig. [] shows predictions with a small diffusion model, with either an autoregressive scheme or our multiscale inference scheme. Our scheme better captures the trajectories than the autoregressive one, as confirmed visually and by Wasserstein distance (0.021 vs 0.23). Because of the partial observability of the trend mentioned above, the autoregressive scheme produces errors that accumulate.
Our multiscale scheme efficiently captures long-range dependencies through its multiscale templates (see Section 4.1 When predicting the future at = 0, it also conditions on earlier steps (up to -9) compared to only -3 for an autoregressive scheme (see Fig T). To isolate the effect of the multiscale template from that of conditioning further in the past, restrict scheme in Fig 2 to the same past

---

horizon (-3), Performance slightly degrades (from 0.021 to 0.08), but still surpasses autoregressive baselines.
We refer the reader to the Appendix for another synthetic example of a partially observable fluid dynamical system.
5.2 Solar dynamics prediction
Solar dataset. To encourage competition on predicting partially observable long-memory dynamical systems, we introduce new ML-ready dataset (see Fig. 3) of reasonably high-resolution solar dynam- ies prediction based on real observations from the NASA Solar Dynamics Observatory mission [6]] in continuous operation since 2010. The data contains two modalities from two instruments, surface magnetic fields [7]]. and images of the solar atmosphere [3]. Each produces 4096 x 4096-pixel images of the full disk of the Sun (see Fig. 3 high cadence, making the data-handling very de- manding. As discussed in [16. because active regions occupy only a small fraction of the visible disk, we propose dataset of square-image videos of 512 x 512-pixel windows that track an active-region. This data is curated to carefully account for the rotation of the Sun, the limb of the Sun (its "edge"), co-alignment between the two modalities, potential overlap between targets, and uncertainty, artifacts, and missing data. Each day, we randomly sample 8 regions of the Sun to follow for 48h, sampled hourly. The regions are selected to avoid bias towards rare events. Our dataset consists of 8.5TB composed of == 15K multi-channe videos of shape 48 x 12 512 512. Each video contains 3 magnetic fields channels and 9 channels for the solar atmosphere at different wavelengths. In the following, all models are trained on images downsampled by a factor of 2 (to the instrument's 's optical resolution) and considering only 3 of the atmosphere channels, in order to reduce the computational cost of training multiple diffusion models.
We adopt Vision Transformer 17 ViT] architecture as our denoiser backbone, following the approach in (37, but extended to handle spatio-temporal data and inspired by the implementation in [67]. The denoiser takes as input 3D patches of size 8 x (no patchification in time), and consists of 16 attention-based layers with hidden dimension of 512 and 4 attention heads per layer. The resulting denoiser has 62 million Icarnable parameters. Time and spatial information the patches are added input and
Diffusion model hyperparameters.
RoPE positional encoding [80]. Like in [37], input, output, and noise levels are preconditioned to improve the training dynamics. For sampling, generate small trajectories of length with 100 diffusion steps with Adams-Bart Bashforth multi-step sampler [87 88
Evaluation metrics. We use several metrics that can be computed between a sample and an observation. In evaluating magnetogram predictions, per-pixel averages are not informative since they are dominated by quiet Sun pixels even in patches [86 28 We therefore use multiple other metrics (see Tab. IT. First, the Wasserstein distance assesses the fit between the predicted distribution of pixels and the observed one. Second, we compute the mean absolute error in the isotropic power spectrum, which provides information on the spatial frequency content of an image. This metric is less sensitive to noise in the data. Finally, consider physics-based summary statistics that characterize spatial gradients of the magnetic field. All metrics are averaged on all fields, on several realizations of the model, at several prediction dates, and averaged over several different time horizons.
Baselines. We compare our model to baselines. Two fix the denoising architecture and compare the multiscale inference scheme with: autoregressive inference scheme (a default choice in the literature) and the hierarchy-2 inference scheme from 255 (which sparsely completes missing frames, then autoregressively samples the remainder by conditioning on both past and future frames). The other two compare our model to existing spatiotemporal models for physical systems: [39] is a diffusion model tested on fluid dynamics data; and (50) is deterministic transformer based on axial attention (30] All models are trained with 40 epochs. We refer the reader to the Appendix for additional details.
Solar predictions. Tab. [ confirms that, in this more challenging case, our multiscale inference scheme better predicts the pixel distributions than an autoregressive scheme at all future horizons (1:4, 4:16 and 16:32) by achieving the lowest Wasserstein distance. The spatial content is better preserved, shown by the error in the power spectrum, and illustrated in the predictions in Fig. 4 Our multiscale inference scheme also outperforms the "Hierarchy-2" model introduced for natural videos 225, which was not designed for slow-decaying, autocorrelated
processes. Tab.

---

also shows that our diffusion model, equipped with our multiscale inference scheme, significantly outperforms existing models [50 39 A deterministic baseline such as AVIT [50] can predict future trajectory that is close to observed data but loses high frequency content, which gives rise to errors that accumulate with the rollout. Our model also compares favorably to the diffusion model of [39], which was developed for fluid dynamics data. These results showcase the limits of current models in probabilistic prediction of partially observable dynamical systems.
Conclusion and discussion
This work introduces and analyzes a multiscale inference scheme for predicting partially observable dynamical systems. Our approach efficiently incorporates past information-while -wille being refined around the present-to predict future time steps. We show superior performance in both synthetic settings and the challenging task of predicting solar dynamics, outperforming existing schemes (25) and models [50] [9] for video and spatiotemporal physical systems prediction. Our results suggest that multiscale temporal conditioning helps mitigate partial observability, especially when long-range precursors influence future evolution, as in solar dynamics. To support further work, we contribute a dataset of righrresolution multi-modal solar regions trajectories.
While our method well suited for long-memory systems with smoothly decaying temporal de- pendencies, it may not remain competitive when observations dominated by short-term patterns. Future work could explore adaptive or learned conditioning strategies.
Acknowledgments and Disclosure of Funding
The authors thank the Scientific Computing Core at the Flatiron Institute, division of the Simons Foundation, for providing computational resources and support. They also thank Mark Cheung, Patrick Gallinari, Florentin Guth, and Ruoyu Wang for insightful discussions.
Polymathic Al acknowledges funding from the Simons Foundation and Schmidt Sciences.
References
[1] Harris Abdul Majid, Pietro Sittoni, and Francesco Tudisco. Solaris: A Foundation Model of the Sun. arXiv e-prints, page arXiv:2411.1663 163399, November 2024.
[2] Patrice Abry, Patrick Flandrin, Murad S Tagqu, and Darryl Veitch. Wavelets for the analysis, es- timation, and synthesis of scaling data. Self-similar network traffic and performance evaluation, pages 39-88, 2000.
[3] Brian D.O. Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313-326, 1982.
[4] Rafal A. Angryk, Petrus C. Martens, Berkay Aydin, Dustin Kempton, Sushant S. Mahajan, Sunitha Basodi, Azim Ahmadzadeh, Xumin Cai, Soukaina Filali Boubrahimi, Shah Muhammad Hamdi, Michael A. Schuh, and Manolis K. Georgoulis. Multivariate time series dataset for space weather data analyties. Nature / Scientific Data, 7(1):227, January 2020.
[5] Andrés Asensio Ramos, Mark C. M. Cheung, Tulia Chifu, and Ricardo Gafeira. Machine learning in solar physics. Living Reviews in Solar Physics, 20(1):4, December 2023.
[6] Liang Bai, Yi Bi, Bo Yang, Jun-Chao Hong. Zhe Xu, Zhen-Hong Shang. Hui Liu, Hai-Sheng Ji, and Kai-Fan Ji. Predicting the evolution of photospheric magnetic field in solar active regions using deep learning. Research in Astronomy and Astrophysics, 21(5):1 113, jun 2021.
[7] Graham Barnes, Marc L. DeRosa, Shacla I. Jones, Charles N. Arge, Carl J. Henney, and Mark
C. M. Cheung. Implications of Different Solar Photospheric Flux-transport Models for Global Coronal and Heliospheric Modeling. The Astrophysical Journal, 105, April 2023.
[8] Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages
June 2023.
10

---

[9] M. G. Bobra and S. Couvidat. Solar Flare Prediction Using SDO/HMI Vector Magnetic Field Data with a Machine-learning Algorithm. The Astrophysical Journal, 798(2):135, January 2015.
[10] M. G. Bobra, X. Sun, J. T. Hoeksema, M. Turmon, Y. Liu, K. Hayashi, G. Barnes, and K. D. Leka. The Helioseismic and Magnetic Imager (HMI) Vector Magnetic Field Pipeline: SHARPs Space-Weather HMI Active Region Patches. Solar Physics, 289:3549-333 September 2014.
[11] J. M. Borrero, A. Pastor Yabar, M. Rempel, and B. Ruiz Cobo. Combining magnetohydrostttic constraints with Stokes profiles inversions. I. Role of boundary conditions. Astronomy and Astrophssics, 632:A111, December 2019.
[12] E. G. Broock, A. Asensio Ramos. and T. Felipe. FarNet-Il: An improved solar far-side active region detection method. Astronomy and Astrophysics, 667:A 132, November 2022.
[13] E. Camporeale. The Challenge of Machine Learning in Space Weather: Nowcasting and Forecasting. Space Weather, 17(8):1166-123 12-1 August 2019.
[14] Ronald M. Caplan, Miko M. Stulajter, Jon A. Linker, Cooper Downs, Lisa A. Upton, Bibhuti Ku- mar Jha, Raphael Attie, Charles N. Arge, and Carl Henney. Open-source Flux Transport (OFT), I. HipFF--High-performaan pertomaccee Flux Transport. The Astrophysical Journal Supplement Series, 278(1):24, May 2025.
[15] Ee-Chien Chang, Stéphane Mallat, and Chee Yap. Wavelet foveation. Applied and Computa- tional Harmonic Analysis, 933::122-335, 2000.
[16] Karin Dissauer, KD Leka, and Eric L. Wagner. Properties of Active Regions from the Chromosphere rough the Corona I: Introduction of the AIA Active Region Patches (AARPs). Astrophys. J., 942:83, January 2023.
versus Flare-Quies
[17] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. image is worth 16x16 words: Transformers for image recognition at scale. In Proceedings of the 9th International Conference on Learning Representations, 2021. Accessed: 2025-10-23.
[18] Bradley Efron. Tweedie's formula and selection bias. Journal of the American Statistical Association, 106(496):600-16611 2011.
[19] Yariv Ephraim and Neri Merhav. Hidden markov processes. IEEE Transactions on information theory, 4816):1518-11660 2002.
[20] Grégoire Francisco, Sabrina Guastavino, Teresa Barata, João Fernandes, and Dario Del Moro. Multimodal Flare Forecasting with Deep Learning. arXiv e-prints, page arXiv:2410.161 October 2024.
[21] Grégoire Francisco, Francesco Pio Ramunno, Manolis K. Georgoulis, João Fernandes, Teresa Barata, and Dario Del Moro. Generative Simulations of The Solar Corona Evolution With Denoising Diffusion Proof of Concept. arXiv e-prints, page arXiv:2410.20843, October 2024.
[22] Richard Galvez, David F Fouhey, Meng Jin, Alexandre Szenicer, Andrés Muñoz-Jaramilly Mark CM Cheung, Paul Wright, Monica G Bobra, Yang Liu, James Mason, et al. A machine- learning data set prepared from the nasa solar dynamics observatory mission. The Astrophysical Journal Supplement Series, 242(1):7, 2019.
[23] songwei Ge. Seungjun Nah, Guilin Liu, Tyler Poon, Andrew Tao, Bryan Catanzaro, David Jacobs, Jia-Bin Huang, Ming-Yu Yu Liu, and Yogesh Balaji. Preserve your own correlation: A noise prior for video diffusion models. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 2930-22941, 2023.
[24] Olexandr Gugnin, Brian C. K. Wan, Charmaine S. M. Wong, and Shirley Ho. Spatial and temporal super-resolution methods for high-fidelity solar imaging. Astronomy and Astrophysics, A95: March 2025.
11

---

[25] William Harvey, Saeid Naderiparizi. Vaden Masrani, Christian Weilbach, and Frank Wood. Flexible diffusion modeling of long videos. Advances Neural Information Processing Systems, 35:27953-27965, 2022.
[26] Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent Video Diffusion Models for High-Fidelity Long Video Generation. arXiv e-prints, page #xXiv:2211.13221, November 2022.
[27] Hans Hersbach, Bill Bell, Paul Berrisford, Shoji Hirahara, András Horányi, Joaquín Muñoz- Sabater, Julien Nicolas, Carole Peubey, Raluca Radu, Dinand Schepers, et al. The era5 global reanalysis. Quarterly journal of the royal meteorological sociery, 146(730):1999-2064 2020.
[28] Richard E. L. Higgins, David F. Fouhey, Dichang Zhang, Spiro K. Antiochos, Graham Barnes, J. Todd Hoeksema, K. D. Leka, Yang Liu, Peter W. Schuck, and Tamas I. Gombosi. Fast and accurate emulation of the sdo/hmi stokes inversion with uncertainty quantification. The Astrophysical Journal (ApJ), 911(2), 2021.
[29] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 53:6840-6851 2020.
[30] Jonathan Ho, Nal Kalchbrenner, Dirk Weissenborn, and Tim Salimans. Axial attention in multidimensional transformers. arXiv preprint 17X111191..12180, 2019.
[31] Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in neural information processing systems, 35:8633-8646, 2022.
[32] J. T. Hoeksema, Y. Liu, K. Hayashi, X. Sun, Schou, S. Couvidat, A. Norton, M. Bobra, R. Centeno, K. D. Leka, G. Barnes, and M. Turmon. The Helioseismic and Magnetic Imager (HMI) Vector Magnetic Field Pipeline: Overview and Performance. Solar Physics, 289:3483- 3530, September 2014.
[33] Jiahe Huang, Guandao Yang, Zichen Wang, and Jeong Joon Park. Diffusionpde: Generative pde-solving under partial observation. Advances Neural Information Processing Systems, 37:130291 1080091--3000 130323, 2024.
[34] R. Jarolim, A. M. Veronig, W. Pötzi, and T. Podladchikova. A deep learning framework for instrument-to in-intuummetttt tenttruenttt-nistuuunntt translation of solar observation data. Nature Communications, 16(1):3157, 2025.
[35] Hyun-Jin Jeong, Yong-Jae Moon, Eunsu Park, Harim Lee, and Ji-Hye Back. Improved AI- generated Solar Farside Magnetograms by STEREO and SDO Data Sets and Their Release. The Astrophysical Journal Supplement Series, 262(2):50, October 2022.
[36] Ole Jørgensen and Dan Burns. Subsurface stress assessment from cross-coupled borehole acoustic eigenmodes. Geophysical Journal International, 235(1):565-77,, 08 2024.
[37] Tero Karras, Miia Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffussio-bbsed generative models. Advances in neural information processing systems, 35:26565-26577, 2022.
[38] Kwanyoung Kim and Jong Chul Ye. Noise2score: tweedie's approach to self-supervised image denoising without clean images. Advances in Neural Information Processing Systems, 14:864-874, 2021.
[39] Georg Kohl, Li-Wei Chen, and Nils Thuerey. Benchmarking autoregressive conditional diffusion models for turbulent flow simulation. arXiv preprint arXiv:2309.07745, 2023.
[40] K. D. Leka and G. Barnes. Photospheric Magnetic Field Properties of Flaring vs. Flare-Quiet Active Regions. IV: A Statistically Significant Sample. The Astrophysical Journal, 656: 173- 1 1186, 2007.
12

---

[41] K. D. Leka, S. H. Park, K. Kusano, J. Andries, C. Balch, G. Barnes, S. Bingham, S. Bloom- field, A. McCloskey, V. Delouille, D. Falconer, P. Gallagher, M. Georgoulis, T.A.M. Hamad Nageem, Y. Kubo, K. Lee, S. Lee, V. Lobzin, J.-C. Mun, S. Murray, R. Qahwaji, M. Sharpe, R. Steenburgh, Steward, and M. Terkildsen. A Comparison Benchmarks, Metrics and Performance Results for Operational Solar Flare Forecasting Systems. The Astrophysical Journal Supplement Series, 243(2):36, Aug 2019.
Flare Forecasting Methods. II.
[42] KD Leka, Karin Dissauer, Graham Barnes, and Eric L. Wagner. Properties of versus Flare-Quiee Active Regions from the Chromosphere through the Corona II: NonPara- metric Discriminant Analysis Results from the NWRA Classification Infrastructure (NCI). The Astrophysical Jourmal, 942:84, January 2023.
[43] James R. Lemen, Alan M. Title, David J. Akin, Paul F. Boemner, Catherine Chou, Jerry F. Drake, Dexter W. Duncan, Christopher G. Edwards, Frank M. Friedlaender. Gary F. Heyman, Neal E. Hurlburt, Noah L. Katz. Gary D. Kushner, Michael Levay, Russell W. Lindgren, Dnyanesh P. Mathur, Edward L. McFeaters, Sarah Mitchell, Roger A. Rehse, Carolus J. Schrijver, Larry A. Springer, Robert Stern, Theodore D. Tarbell, Jean-Pierre Wuelser, C. Jacob Wolfson, Carl Yanari, Jay A. Bookbinder, Peter N. Cheimets, David Caldwell, Edward E. Deluca, Richard Gates, Leon Golub, Sang Park, William Podgorski, Rock I. Bush, Philip H. Scherrer, Mark A. Gummin, Peter Smith, Gary Auker, Paul Jerram, Peter Pool, Regina Soufli, David L. Windt, Sarah Beardsley, Matthew Clapp, James Lang, and Nicholas Waltham. The Atmospheric Imaging Assembly (ATA) on the Solar Dynamics Observatory (SDO). Solar Physics, 275(1- 2):17-40, January 2012.
[44] Xuebao Li, Xuefeng Li, Yanfang Zheng, Ting Li, Pengchao Yan, Hongwei Ye, Shunhuang Zhang, Xiaotian Wang. Yongshang Lv, and Xusheng Huang. Prediction of Large Solar Flares Based on SHARP and sethh-enrryydensitt Magnetic Field Parameters. The Astrophysical Journal Supplement Series, 276(1): ):7, January 2025.
[45] Mingwei Lin and Canjun Yang. Ocean Observation Technologies: A Review. Chinese Journal of Mechanical Engineering, 33(1):32, December 2020.
[46] Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow Matching for Generative Modeling. arXiv e-prints, page arXiv:2210.02747, October 2022.
[47] Phillip Lippe, Bas Veeling, Paris Perdikaris, Richard Turner, and Johannes Brandstetter. Pde- refiner: Achieving accurate long rollouts with neural pde solvers. Advances in Neural Informa- tion Processing Systems, 36:67398 14467398-67433, 2023.
[48] Benoit B Mandelbrot. Multifractalss and 1/f noise: Wild self-affinity in physics (1963-1976).
Springer, 2013.
[49] Hiroyuki Masaki and Hideyuki Hotta. Detection of solar internal flows with numerical simula- tion and machine learning. Publications of the Astronomical Society of Japan, 76(6))L33-L33 December 2024.
[50] Michael McCabe, Bruno Régaldo-Sain Blancard, Liam Parker, Ruben Ohana, Miles Cranmer, Alberto Bietti, Michael Eickenberg. Siavash Golkar, Geraud Krawezik, Francois Lanusse, al. Multiple physies pretraining for spatiotemporal surrogate models. Advances in Neural Information Processing Systems, 37:1 19:119100-119333 2024.
[51] EJ McCoy and AT Walden. Wavelet analysis and synthesis of stationary bongmeeorry processes. Journal of computational and Graphical statistics, 5(1):26 -6, 1996.
[52] Chenlin Meng, Yang Song, Wenzhe Li, and Stefano Ermon. Estimating high order gradients of the data distribution by denoising. Advances in Neural Information Processing Systems, 34:25359-25369, 2021.
[53] Rudy Morel. Compact models of multi-scale processes. PhD thesis, École Normale Supérieure, 2023.
[54] Rudy Morel, Stéphane Mallat, and Jean-Philippe Bouchaud. Path shadowing monte carlo. Quantitative Finance, 1199 2024.
13

---

[55] Rudy Morel, Gaspar Rochette, Roberto Leonarduzzi, Jean-Phillppe Bouchaud, and Stéphane Mallat. Scale dependencies and self-similar models with wavelet scattering spectra. Applied and Computational Harmonic Analysis, 75:101724, 2025.
[56] National Science and Technology Council. weather strategy and action plan.
Implementation plan
of the national space
https
//biennhitthhusee. ar archiies.oo. gov http://peoootttettppaaaaaa/a///p /2/2t///2
Implem Implementation- Plan for-foorionnn Natiinal Space-
-pt.eaaaa-apce---eeeeeeeeeee Weather Strategy 122100222222... pdf
December 2023.
[57] N. Nishizuka, K. Sugiura, Y. Kubo, M. Den, and M. Ishii. Deep Flare Net (DeFN) Model for Solar Flare Prediction. The Astrophysical Journal, 858:113, May 2018.
[58] Chetraj Pandey, Rafal A. Angryk, Manolis K. Georgoulis, and Berkay Aydin. Explainable Deep Learning-Based Solar Flare Prediction with Post Hoe Attention for Operational Forecasting. Lecture Notes in Computer Science, 14276:567, October 2023.
[59] Brandon Panos and Lucia Kleint. Real-time Flare Prediction Based on Distinctions between Flaring and Non-flaring Active Region Spectra. The Astrophysical Journal, 891(1):17, March 2020.
[60] Chris Pedersen, Laure Zanna, and Joan Bruna. Thermalizer: Stable autoregressive neural emulation of spatiotemporal chaos. arXiv preprint arXiv:2503. 18731, 2025.
[61] W. Dean Pesnell, B. J. Thompson, and C. Chamberlin. The Solar Dynamics Observatory (SDO). Solar Physics, 275(1-2)33-15, January 2012.
[62] Ilan Price, Alvaro Sanzhez-Gonzalez, Ferran Alet, Tom R Andersson, Andrew El-Kadi, Dominic Masters, Timo Ewalds, Jacklynn Stott, Shakir Mohamed, Peter Battaglia, et al. Gencast: ceffusiin-baseee ensemble forecasting for medium-range weather. arXiv preprint arXiv:2312.15796, 2023.
[63] E. R. Priest. Solar magneto-nydrodyyamicc Springer Dordrecht, 1987.
[64] M. Cristina Rabello Soares, Sarbani Basu, and Richard S. Bogart. Exploring the Substructure of the Near-surface Shear Layer of the Sun. The Astrophysical Journal, 967(2):143, June 2024.
[65] Francesco Pio Ramunno, Hyun-Jin Jeong, Stefan Hackstein, André Csillaghy, Svyatoslav Voloshynovskiy, and Manolis K. Georgoulis. Magnetogram-to-maan magnetttt:ram Generative forecasting of solar evolution, October 2024.
[66] François Rozet and Gilles Louppe. Score-based data assimilation. Advances in Neural Informa- lion Processing Systems, 36:40521 2023.
[67] François Rozet, Ruben Ohana, Michael McCabe, Gilles Louppe, François Lanusse, and Shirley Ho. Lost in latent space: An empirical study of latent diffusion models for physics emulation. arXiv preprint arXiv:2507.02608 2025.
[68] David Ruhe, Jonathan Heek, Tim Salimans, and Emiel Hoogeboom. Rolling diffusion models. In Proceedings of the 41st International Conference on Machine Learning (ICML 2024), volume 235 of Proceedings of Machine Learning Research, pages 42818 42835. PMLR, 2024. Accessed: 10-2-1
[69] Salva Rühling Cachay, Bo Zhao, Hailey Joren, and Rose Yu. Dyffusion: A dynamics-informe diffusion model for spatiotemporal forecasting. Advances in neural information processing systems, 36:45259-45287, 2023.
[70] Ricardo Buitrago Ruiz, Tanya Marwah, Albert Gu, and Andrej Risteski, On the benefits of memory for modeling time-dependent pdes. arXiv preprint arXiv:2409.02313, 2024.
[71] H. Scherrer, J. Schou, R. I. Bush, A. G. Kosovichev, R. S. Bogart, J. T. Hoeksema, Y. Liu, T. Duvall, J. Zhao, A. M. Title, J. Schrijver, T. D. Tarbell, and S. Tomezyk. The Helioseismic and Magnetic Imager (HMI) Investigation for the Solar Dynamics Observatory (SDO). Solar Physics, 275(1-2):207-22 January 2012.
14

---

[72] J. Schou, P. H. Scherrer, R. I. Bush, R. Wachter, S. Couvidat, M. C. Rabello-Soodee Soares, R. S. Bogart, J. T. H oeksema, Y. Liu, T. L. Duvall, D. J. Akin, A. Allard, W. Miles, Rairden, A. Shine, T. D. Tarbell, A. M. Title, C. J. Wolfson, D. F. Elmore, A. A. Norton, and S. Tomczyk. Design and Ground Calibration of the Helioseismic and Magnetic Imager (HMJ) Instrument on the Solar Dynamics Observatory (SDO). Solar Physics, 275:229-221 259,, January 2012.
[73] P. W. Schuck, S. K. Antiochos, K. D. Leka, and G. Barnes. Achieving Consistent Doppler Measurements from SDO/HMI Vector Field Inversions. The Astrophysical Journal, 823:101, June 2016.
[74] Aliaksandra Shysheya, Cristiana Diaconu, Federico Bergamin, Paris Perdikaris, José Miguel Hemández-Lobatoo, Richard Turner, and Emile Mathieu. On conditional diffusion models for pde simulations. Advances in Neural Information Processing Systems, 17:23246-233000 2024.
[75] Jascha Dii--Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. In International conference on machine learning, pages 2256-2265 pmlr, 2015.
[76] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising Diffusion Implicit Models. arXiv e-prints, page arXiv:2010.02502, October 2020.
[77] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
[78] Yang Song, Jascha Sohl-Dickstein Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-Based Generative Modeling through Stochastic Differential Equations. arXiv e-prints, page arXiv:201 13456, November 2020.
[79] Mallat Stephane. wavelet tour of signal processing, 1999.
[80] Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocompatiigg 568:127063 2024.
[81] Simo Särkkä and Amo Solin. Applied Stochastic Differential Equations. Institute of Mathemat- ical Statistics Textbooks. Cambridge University Press, 2019.
[82] Dennis Tilipman, Maria Kazachenko, Benoit Tremblay, Ivan Milić, Valentin Martínez Pillet, and Matthias Rempel. Quantifying Poynting Flux in the Quiet Sun Photosphere. The Astrophysical Journal, 956(2):83, October 2023.
[83] MCK Tweedie. Functions of a statistical variate with given means, with special reference to laplacian distributions. In Mathematical proceedings of the cambridge philosophical society, volume 43, pages 41-49. Cambridge University Press, 1947.
[84] Vikram Voleti, Alexia Jolicoeur- Martineau, and Chris Pal. Mevd-masked conditional video dif- fusion for prediction, generation, and interpolation. Advances in neural information processing systems, 35:23371-23385, 2022.
[85] James Walsh, Daniel G. Gass, Raul Ramos Pollan, Paul J. Wright, Richard Galvez, Noah Kasmanoff, Jason Naradowsky, Anne Spalding, James Parr, and Atilm Güneş Baydin. A Foundation Model for the Solar Dynamics Observatory. arXiv e-prints, page arXiv:2410.02530, October 2024.
[86] Ruoyu Wang, David F. Fouhey, Richard E. L. Higgins, Spiro K. Antiochos, Graham Barnes, J. Todd Hoeksema, K. D. Leka, Yang Liu, Peter W. Schuck, and Tamas I. Gombosi. SuperSyn- thIA: Physics-ready Full-disk Vector Magnetograms from HMI, Hinode, and Machine Learning. The Astrophysical Journal, 970(2):168, August 2024.
[87] Gerhard Wanner and Ernst Hairer. Solving ordinary differential equations II, volume 375.
Springer Berlin Heidelberg New York, 1996.
15

---

[88] Kai E. Yang, Lucas A. Tarr, Matthias Rempel, S. Curt Dodds, Sarah A. Jaeggli, Peter Sadowski, Thomas A. Schad, lan Cunnyngham, Jiayi Liu, Yannik Glaser, and Xudong Sun. Spectropolari- metric Inversion in Four Dimensions with Deep Learning (SPIn4D). I. Overview, Magnetohy- drodynamic Modeling, and Stokes Profile Synthesis. The Astrophysical Journal, 976(2):204, December 2024.
[89] Qinsheng Zhang and Yongxin Chen. Fast sampling of diffusion models with exponential integrator. arXiv preprint arXiv:2204.1390,, 2022.
16

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes. The abstract and introduction correctly summarize the paper's 's core contributions: multiscale inference scheme for diffusion models to forecasting partially observable physical systems and a high resolution new sun region dataset well suited for spatio-temporal prediction of partially observable dynamical systems.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Yes, they are discussed in the "Discussion and conclusion" section.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: We do not have theoretical results.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Yes. The paper provides all necessary implementation details in Section (Diffusion model hyperparameters and evaluation setup), including architecture, training procedure, sampling parameters, and dataset construction (in the Appendix), enabling reproducibility of the main results even without access to code. In addition, Figure I the inference strategy is clearly explained.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: Yes the code will be published on GitHub unpon publication and if possible with dedicated scripts reproduce all the main experimental results. The data and scripts to generate from existing datasets will be published too upon publication.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification; Yes, the paper specify all the test details, including data splits, hyperparamters in Section and in the Appendix.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: due to compute budget limits.
17

---

8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Yes, estimates of the computational resources used contained the Ap- pendix.
Justification:
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http:: httpp//////erpppp..cc//bbb////ba httppppl.......i.ccccc..eeeeeee.
Answer: [Yes]
The research uses publicly available scientific data, respects data own- ership and attribution, and does not involve human subjects or sensitive content. All contributions intended for scientific and follow the NeurIPS Code of Ethics.
Justification: Yes.
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Yes, in the introduction explain why using solar data and studying the Sun can have important societal impacts.
Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The dataset and models designed for scientific study of solar dynamics and do not pose high risk for misuse; no safeguards are necessary beyond standard open-science practices.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
Yes. All external assets used in the article, including the SDO HMI and SDO
AIA datasets, are publicly available and properly cited ((22), 43), (71). We used the
curated dataset from [22], which includes appropriate preprocessing and suitable for ML applications. No external code models requiring attribution beyond standard libraries were used.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Yes. We introduce a new 8.5 TE multimodal dataset for solar forecasting, described in Section (0 and the Appendix, including the inference scheme. The dataset and documentation will be released publicly upon publication. It is not possible to include this data in the submission due to file size.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
18

---

Answer: [NA]
Justification: We did not do any crowdsourcing
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: NA. The paper does not involve human participants or any data derived from human subjects; solely uses observational solar data from publicly available sources.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research doesn't 't involve the usage of LLMs.
19


Reviews:
[{"review_id": "8MqhQ2Y1Eu", "reviewer": "Reviewer_1zXs", "summary": "The paper proposes a method to train a conditional diffusion model to predict future states in a dynamical system, while being able to learn and exploit long-range dependencies in the data remaining computationally efficient. The method is motivated by a problem in solar physics, for which an associated dataset will be released also.", "questions": "I suggest the authors to write down in what sense their generated samples can be seen as approximating the distribution of interest, and how making certain Markovian of order k assumptions about the data would relate to the adaptive choice of $\\alpha$ in the methodology. \nFinally, I express a good level uncertainty as I am not familiar with the baselines involved here.", "limitations": "The authors do discuss limitations.", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strenghts: the paper has a strong application; it does not involve \"mathiness\", which is to be appreciated these days; presents interesting and relatively well explained results. \n\nWeaknesses: \n\n**[Connections with related prediction approaches in dynamical systems are not very clear.]** Firstly, the field with which I am most familiar with, data assimilation (DA), is cited/mentioned in lines 182-183, but the connection remains unclear. The authors also cite DA works in lines 79-81, stating: \" Diffusion models can also be used for data assimilation and prediction from incomplete observations [67, 73, 34], but this is an orthogonal direction of work, and a dataset of fully observed states from the system must exist to train the model.\" It is not clear to me whether the last part of the sentence refers to the authors' work or to the cited DA work. In DA or Bayesian filtering, the \"states\" are not observed - they are latent - typically called x_t and only y_t is observed; partial observability in DA/filtering is the whole point. I guess it is not clear what \"partial observability\" means for the authors here. In DA, it is very mathematically defined by the framework of probabilistic state space models (see e.g., Kevin P. Murphy's book on probabilistic machine learning, section on state-space models, or Sarkka & Svensson 2023 \"Bayesian filtering and smoothing\"). \n\n**[Clarifications needed regarding assumption (b) in Sec 4.1]** : continuing really from the point above; authors claim that \" Diffusion models have generally been applied to dynamical systems without fully addressing challenge (b) \" but cite some DA works including eg [67] that do exactly this, in the meaning of partial observability. This point also adds confusion by mixing PO with Markovianity. Although the authors never state this fully explicitly (or clearly), they must be assuming non-first order Markovianity of their observations here, otherwise it would clearly be pointless to condition on anything beyond the previous state. The authors at some point mention \"noise in the data\" in the experiments, although this seems not explicitly modelled, which again leaves me confused. \n\nOverall, Eq (6) leaves a \"handwavy\" feel with respect to the statistical modelling done here, and it is not clearly explicited in mathematics what is assumed about the underlying data generating process. Again it is ok to use diffusion models and neural nets and this has been done in related works in data assimilation, but it has to be clearly explicited which parts of the dynamical system is the score modelling/related to, for example.  Section 3 feels isolated and disconnected from the rest, mathematics wise.", "was_revised": true, "final_justification": "See below comment for more details. I cautiously increase my score due to the potential improvements that can be made to the paper from the rebutal(s), yet I remain with low confidence due to my lack of experience with these type of approaches.", "rebuttal": "We thank the reviewer for their positive feedback, in particular for acknowledging the strength of the application, the readability of our paper, and for the valuable insights on data assimilation (DA).\n\n----\n\n##  Connections with related prediction approaches in dynamical systems\n\nAlthough we follow developments in data assimilation (DA) with interest, our paper does not directly address this topic. Instead, it focuses on probabilistic prediction of the observations themselves, without attempting to reconstruct the system's underlying states. We updated the “Related Works” section based on your comment about data assimilation (DA).\n- Line 68 now reads: “our work falls within the scope of modeling the dynamics of the observations”. \n- Line 76 now reads: “Thus, while [40, 63] see no benefit from using more than two past observations, incorporating additional past steps substantially improves results in our setting”. \n- Line 79 now reads: “Diffusion models can perform data assimilation and prediction from incomplete observations simultaneously [67, 73, 34], but this requires a dataset of underlying system states to train the model -- an assumption we do not make in this paper”, where the last part of the sentence refers to [67, 73, 34].\n\nIn our paper, partial observability is defined by the fact that, at any time, the conditional distribution of the state given its observation is not restricted to a Dirac measure, characterizing the impossibility of reconstructing the system’s state from the observation. This will be stated more clearly in the introduction and in section 4.1. In that sense, we align with the standard mathematical framework used in Bayesian filtering and DA, without explicitly writing out the equations, since, as you mention, we aimed to avoid any “mathiness” not directly useful for developing our approach. In the solar prediction application tackled in our paper, there are substantial unknown latent physical variables (e.g., all state variables of the interior, the magnetic field in the Sun's atmosphere) as well as open questions about dynamics (e.g., how the atmosphere of the Sun is actually heated) and entirely missing subgrid physics that cannot be observed in our data.\n\n----\n\n## Clarifications needed regarding assumption (b) in Sec 4.1\n\nWe thank the reviewer for pointing out this sentence. It has been replaced by: \\\n\"\nDiffusion models have been applied to prediction on dynamical systems by using additional methods to remove assumption (b). For example, [40] apply a diffusion model to fully resolved fluids which are effectively Markovian. In weather prediction, although the observed data are sparse, data assimilation -- also known as reanalysis -- enables the reconstruction of missing information, resulting in large datasets of highly informative states [28], on which diffusion models have been successfully trained [63]. Other models handle missing states, but require clean sates for training [73,34], which is not always available.\n\nIn this paper, we tackle the challenging problem of predicting a dynamical system that presents the three difficulties (a), (b), and (c) simultaneously, without assuming that (b) can be circumvented, as is often the case in many disciplines.\n\"\n\n----\n\n## Other clarifications\n\nAs you point out, the system has to be non-first-order Markov for our method to show an advantage over autoregressive systems. We did not mention this assumption as we allow ourselves to test it empirically. In practice, if a standard autoregressive scheme outperforms our multiscale approach, this may suggest that the process is Markovian of some order.\n\nIndeed, it is important to clarify that we learn the score of each of the distribution $p(x_{I_n}| x_{C_n})$ in Eq.6 in the same way as [Harvey et al., 2022] did, by training a denoiser which takes as input the relative positions of the time steps in the context $C_n \\cup I_n$ (see line 289-290 and Appendix, line 99-104). \n\nSection 3 will be renamed \"Background: Conditional Diffusion Models\".\n\n--- \n\n## Questions\n\nAs stated above, we learn the score of the distributions $p(x_{I_n}| x_{C_n})$, which can be used to approximate the distribution $p(x_{1:T} | x_{t\\leq0})$ (see Eq. 6). Let us recall that we assume the set of indices $I_n$ and $C_n$ to have $K$ and $K+1$ elements respectively. \n\nIn certain restricted cases, one can prove that Eq. 6 is actually an equality, for example, under a Markovian assumption of order $K+1$, and employing a standard autoregressive inference scheme (see line 155 in our paper). In that case, it makes sense to model the target distribution $p(x_{1:T} | x_{t\\leq0})$ by learning the score of $p(x_{I_n}| x_{C_n})$ for all $n$.\n\nStudying this approximation in the case of generic $I_n,C_n$ is hard, and it is not clear that a Markovianity assumption of order $K+1$ is well suited for justifying the choice of the maximum $\\alpha$ in our inference scheme. At this stage, a proper explanation would require a separate LaTeX note, which is not permitted this year.\n\nIntuitively, if we assume that the process has a maximum timescale of dependence, that is, there exists a time $T$ such that for any time $\\tau\\geq T$, the variables $x_t$ and $x_{t+\\tau}$ are independent (often referred to as the integral scale in signal processing), then it is natural to restrict the extent of the largest multiscale template $\\mathbb{T}^{\\alpha_{max}}_K$ (defined in Eq. 7) to be smaller than $T$, since there is no need to model dependencies beyond $T$. However, this intuition would need to be made more rigorous."}, {"review_id": "4wiW8TdmwE", "reviewer": "Reviewer_Ac1G", "summary": "In this paper, authors propose a multiscale inference scheme for diffusion models to solve the problem of insufficient information for prediction. The model can capture long-range temporal dependencies by generating trajectories that are temporally fine-grained near the given timestamp while coarser when moving farther away. The effectiveness of the proposed model has been evaluated on the solar prediction task, and authors also build a corresponding large multi-modal dataset.", "questions": "Please refer to the Weaknesses part.", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. Authors incorporate multiscale temporal templates to allow the model to focus on both short-term and long-term dynamics without increasing computational cost, breaking the limitations of traditional autoregressive rolling prediction that gradually ignores the ground-true historical observation and accumulates errors. \n2. The design of multiscale templates is flexible and controllable, which provides the possibility of easy deployment for real applications.\n3. The effectiveness of the proposed method has been sufficiently evaluated on both synthetic and real datasets, along with ablation studies and analyses. Besides, authors prepare a large-scale solar dataset, which should be helpful for the corresponding community.\n\nWeaknesses:\n1. The temporal template chosen on Sec.4.2 is a little confusing. In the first round, {-9, -3, -1, 0, 1, 3, 9} is chosen and history (<=0) is assumed to be fully available, i.e., history from t=-9 to t=0 should be available, then in the second round, why not directly choosing {-5, -4, -2, 0, 2, 4, 5} directly but using shift? Besides, after shifting the given {-6, -2, -1, 0, 1, 2, 6} 2 steps, it should be {-4, 0, 1, 2, 4, 8}, the overlap with previously generated steps is not {-3, 1, 3}.  \n2. Since the temporal template is not evenly sampled, can DiT well capture such flexible temporal relationships?\n3. As authors claimed, the paper aims at solving the solar prediction, besides diffusion-based methods, other temporal models designed for the same task should be involved as baselines.", "was_revised": true, "final_justification": "The multiscale temporal templates proposed in this paper are interesting and reasonable. Although initially confusing due to some unclear descriptions and typos, the authors' explanations have made things much clearer, and with the appropriate revisions, it should be easier to understand the underlying process. This multiscale template takes into account both short-term and long-term dynamics, and uses the generated long-term future step as a condition to complete the intermediate steps, thus ensuring temporal continuity. The overall design is sound and supported by experiments. Taking all factors into consideration, I think this paper exceeds the acceptance bar of NeurIPS, and I have increased my rating from 4 to 5.", "rebuttal": "We thank the reviewer for their positive feedback on our work, particularly for recognizing that our method is \"breaking the limitations of traditional autoregressive rolling prediction\" and for highlighting its \"possibility of easy deployment for real applications\".\n\n---\n\n## Question 1. Part 1.\n\nWe are not directly choosing the template {-5, -4, -2, 0, 2, 4, 5} because it is not, strictly speaking, a multiscale template in the sense of Definition Eq. 7 in our paper. This template is refined near $t = -5$ and $t = 5$, but not around $t = 0$, whereas the motivation behind our multiscale templates is to provide finer resolution around $t = 0$ and become progressively coarser farther from it (see line 199).\nMoreover, if we were to first use the template {-9, -3, -1, 0, 1, 3, 9} and then switch to {-5, -4, -2, 0, 2, 4, 5} as suggested, this would mean that future steps {1, 3, 9} are generated before steps {2, 4, 5}, potentially introducing temporal discontinuities. For instance, the model could generate a scenario where {1, 3, 9} become more active, while {2, 4, 5} become less active — even though both sets are conditioned on the same past steps {-5, -4, -2, 0}. This is precisely why our multiscale scheme (see Fig. 1, right) is designed to always condition on already generated future steps, thereby preserving temporal coherence.\n\n---\n\n## Question 1. Part 2.\n\nIndeed, the second multiscale template {-6,-2,-1,0,1,2,6} should be shifted by 3 (not by 2) to obtain {-3,1,2,3,4,5,9}, which overlaps with the steps {-3,1,3}. These correspond to either past steps {-3} or already generated future steps {1,3}. This has been updated in the manuscript (lines 234–235).\n\n---\n\n## Question 2\n\nHandling unevenly sampled trajectories is primarily achieved by concatenating time information to the input of our denoiser, as described on line 289 of the paper. More precisely, we add an additional channel to the input tensor that indicates, for each pixel in the input trajectory, its relative time index (e.g., -9, -3, -1, 0, 1, 3, 9). Further details are provided in Appendix (lines 99–104)\n\n---\n\n## Question 3\n\nTable 1 in our paper reports the results of a non-diffusion-based architecture, specifically a recent transformer architecture designed for next token prediction [52], that has been consistently used for the prediction of physical dynamical systems.\nBelow are the results for such baseline, as reported in our paper. Note that published models on solar prediction [1,66] operate in substantially worse spatiotemporal resolutions (e.g., a time resolution of 12h while we predict hourly, or 2x spatially downsampled compared to our data) than this baseline.\n\n| Model | Model type | Wasserstein 1:4 | Wasserstein 4:16 | Wasserstein 16:32 | PS MAE 1:4 | PS MAE 4:16 | PS MAE 16:32 |\n|---|---|---|---|---|---|---|---|\n| AViT [52]    | Transformer | 12 | 13 | 15 | **0.11** | 0.35 | 0.81 |\n| DiT (ours)   | Diffusion   | **3.0** | **4.3** | **5.5** | 0.12 | **0.22** | **0.33** |\n\nNote that a transformer, as any deterministic baseline, will suffer from the well-known smoothing effect, as shown below. The transformer model produces a prediction with significantly less power in the high-frequencies than the observed spectral content (measured as the sum of the power-spectrum (PS) over the top 75% high frequencies). \n\n| Model | Model type | PS relative energy (low frequencies) | PS relative energy (high frequencies) |\n|---|---|---|---|\n| AViT [52]    | Transformer | -0.89\\% | -82\\% |\n| DiT (ours)   | Diffusion   |  -3.8\\% | +17\\% |\n\nThis smoothing behavior would occur with any deterministic baseline, and we preferred to focus our paper on comparing different schemes within the same diffusion model."}, {"review_id": "rejP4J4BBk", "reviewer": "Reviewer_xJBu", "summary": "The paper is concerned with the problem of estimating the state of a dynamical system when the measurements only contain a subset of the states, referred to as partially observable dynamical systems. The motivation comes from physics and more specifically a problem of solar prediction. The main development of the paper is an inference scheme that is capable of working on several scales and in that way introduce a mechanism for including long-range dependencies. The approach is tested on a large dataset of videos on solar activity. The data used for the experiments will be  made available to the community to ensure reproducibility of the research.", "questions": "1. In many real-world situations the measurements are available as functions of the state. Would you be able to extent to that case or is that something that requires significant effort? \n2. In an effort to reduce the connection to the solar application and shift the focus more to the generality of the development, would it be possible to compare your developments on the test cases provided here \nhttps://arxiv.org/abs/2011.04006\nreferred to as the long range arena? The reason for this question is that these problems have become quite popular when it comes to profiling algorithms focusing on long range dependencies. \n3. More of a comment really, but by using a state-space construction you should be able to access the smoothing distribution (5) without risking the computational complexity to blow up.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths \n* The paper is well-written and easy to read. \n* The development is intuitive and comes across as natural.\n* Very good to see that the authors are making the data available. This is necessary for the results to be reproducible and it can also be a resource for other researchers, in particular those with an interest in this particular application. \n\nWeaknesses\n* Miss relationship to the state space model and in particular the so-called structured state space model, see e.g. \nhttps://arxiv.org/abs/2111.00396\n* Would have been good to see your performance on some of the existing long-range tasks that are available in order to gauge the performance compared to existing approaches. Note that this does not require you to implement an existing approaches, it is enough to run your method on these tasks.\n* Perhaps too much focus on the solar application.", "was_revised": true, "final_justification": "The authors responded very well to my questions and resolved the concerns I communicated in my questions in a good and convincing manner. Very nice that they also offered a new example on such short notice.", "rebuttal": "We thank the reviewer for their insightful comments and suggestions, and for highlighting both the clarity of the manuscript and the value of releasing the solar dataset to the community.\n\nBelow are our answers to your questions.\n\n---\n\n## Question 1\n\nYes, we can extend to the case where the observations are available as functions of the state without a lot of effort, so long as the function in question is differentiable. Indeed, in that case, one can learn a diffusion model on the states themselves through an expectation-maximization algorithm for example [Rozet et al., 2024]. Then, one can apply our multiscale inference to the diffusion model in the same way as in our paper. The expectation-maximization algorithm and the inference scheme are essentially orthogonal.\n\n---\n\n## Question 2 \n\nWhile interesting, the tasks introduced in the Long Range Arena benchmark [Tay, Dehghani et al., 2020] focus on compositionality, document similarity, and classification, and thus fall outside the scope of our work, which addresses probabilistic forecasting of partially observable dynamical systems.\nSince we agree that validating our method beyond the solar problem is important, we applied it to the PDEArena dataset [Gupta et al., 2022], specifically to the Navier-Stokes trajectories (see their Fig. 1). To make this system partially observable, we downsampled it from 128×128 to 32×32, retained only the density field (discarding velocity), and added small Gaussian noise to the observations.\nResults for this new partially observable system are reported in the table below, using the same metric as in our paper (MAE on the power spectrum, see line 296).\n\n| Inference scheme | 1–4 (MAE Power Spectrum) | 4–16 (MAE Power Spectrum) | 16–36 (MAE Power Spectrum) |\n|---|---|---|---|\n| Autoregressive | 0.39 | 0.43 | 0.35 |\n| Multiscale (ours) | **0.38** | **0.36** | **0.31** |\n\nWe observe that our multiscale inference scheme improves predictions over the autoregressive baseline, particularly in the long-term interval (16–36), where it achieves a significantly lower MAE on the power spectrum. This shows that our approach is not specific to solar prediction. Rather, we considered solar forecasting as a challenging scientific application where our model also performs well.\n\n---\n\n## Question 3\n\nThank you for the insightful comment. We now include a discussion of state-space models in the \"Related Works\" section. We would like to emphasize that the focus of our paper is not on the choice of architecture, but rather on how to perform prediction within a conditional diffusion model. Diffusion models are increasingly being applied to dynamical systems [63, 67, 73, 34], and have demonstrated strong performance in generative modeling, which aligns well with our task: probabilistic forecasting.\n\n---\n\nOur paper does focus on the solar prediction problem (although we also include a synthetic example and will add the results on PDEArena), because it is currently a very challenging task in probabilistic prediction, with the potential to significantly advance solar flare prediction in the future [41, 10, 43]."}, {"review_id": "UPfebN2Jjs", "reviewer": "Reviewer_EKRN", "summary": "This paper presents an “inference scheme” for using diffusion models to model dynamical systems and applies it to a dataset from solar physics. The key challenge being addressed is that a diffusion model only predicts a fixed number of frames into the future, which could be much less than the desired prediction horizon. Naively, one could solve this problem by applying the diffusion model autoregressively. The authors proposed an alternative scheme where non-contiguous frames from the past are used to predict non-contiguous frames into the future. They argue that this incorporates long-term information better without increasing the compute cost. The demonstrate the advantage of this scheme on the solar dataset, which they will provide to the community.", "questions": "1. Can you be more explicit about why the scheme is suitable for stochastic, partially observable systems? One would imagine that the probabilistic nature of diffusion models can handle the former. For the latter, it’s also not clear why having access to more distant pasts necessarily helps. Suppose you have a dynamical system with a fast component coupled to a slow component and you only observe the slow component. It’s not clear why having longer-range interactions will help you deal with the unobserved fast component.\n\n2. What is the design principle for the scheme? It may help to present a general algorithm for designing the scheme for various desired timescales and window sizes.", "limitations": "The apparent specificity of the scheme for the analyzed dataset is a significant limitation. The authors should perhaps consider how the scheme is to be adapted for data with a wide range of timescales.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 1, "originality": 2, "strengths_and_weaknesses": "The writing is generally clear, and the proposed scheme appears to achieve substantial performance gains.\nThe weaknesses are mainly two fold. First, the motivations are not unclear. The authors stated (line 164) that some physical systems are challenging to model because they are stochastic and partially observable. These are well-known problems in dynamical system modeling in general and have well-honed solutions, such as probabilistic models with latent variables (e.g. hidden Markov model). The authors claim that they are addressing both issues, but it’s not clear how. Simply incorporating information from the more distant past does not solve these problems unless one makes additional assumptions (for example, that the latent variables have very slow dynamics). However, making these assumptions essentially collapses the three challenges (line 164-179) into the third one, “long-memory”, alone. In other words, it’s not clear why the proposed scheme is particularly suited to stochastic or partially observable systems.\n\nThe applicability of the scheme also appears limited. As the authors noted, the temporal templates need to be “hand-designed” and seem to strongly depend the nature of correlations in the data. This reviewer is not familiar with solar physics and thus cannot verify what is the typical timescale in the dataset, but in general for significance it is desirable if the proposed method is more general and can accommodate a wide range of timescales.", "was_revised": true, "final_justification": "This work presents a novel scheme for using diffusion models to model temporal dynamics of physical systems. After back and forth with the authors, the assumptions about problems suitable for this approach are well stated (using autocorrelation instead of \"long memory\" is appreciated). The performance gain relative to the vanilla inference scheme is well supported by demonstrations on a solar physics dataset and a NS equation dataset. Nevertheless I am leaving the score at 3 (slightly below acceptance) to reflect the fact that I think for a NeurIPS paper one expects a broader scope.", "rebuttal": "We thank the reviewer for their feedback, especially regarding the motivation behind our approach and the types of physical systems we address, which has greatly helped improve the paper. Below, we respond to the main points concerning the motivation and applicability of our method.\n\n----\n\n## Question 1. Motivation of our multiscale templates\n\nThanks to the reviewer’s point, we would like to clarify the following. The paper tackles probabilistic forecasting of a dynamical system, which has two main properties.\n\n- Partial observability. The state of the system at any given time cannot be fully determined by the observations. Consequently, the distribution of future scenarios conditioned on past observed data may not be restricted to a Dirac measure. It is this point that justifies the task of probabilistic forecasting, rather than deterministic forecasting.\n\n- Long memory. The system has slowly varying components that are important to model (e.g., the motion of plasma across several millions of kilometers on the Sun).\n\nLong memory alone is not sufficient to justify our method. For example, consider the synthetic case of Fig. 2 in our paper. It exhibits long memory, since it has a slowly varying sinusoidal component. If the process had zero stochasticity (making it fully observable), then all trajectories would exactly follow this smooth component, and a single state would be sufficient to determine the position on the sinusoid and what comes next. In that case, there would be no need for our multiscale templates.\nNow, if the system exhibits stochasticity, imagine we are at the negative peak (around $t=30$). Depending on the noise realization, the local trend may be positive or negative, making the state difficult to recover locally. In other words, the system’s partial observability, due to noise in the data, prevents accurate estimation of the slowly varying component. In this case, it is thus necessary to look further into the past, which is precisely what our multiscale inference scheme achieves, at no additional computational cost.\n\nThis is supported by the experimental results reported below: as the noise level increases, the performance gain (measured as the ratio of Wasserstein distances; see Fig. 2) of our multiscale template becomes more pronounced. A ratio greater than 1 indicates improved performance with our model.\n\n| Noise level | 0.1 | 0.2 | 0.5 |\n|---|---|---|---|\n| Performance gain ratio | 2.71 | 3.95 | 4.28 |\n\nFollowing your comments, we would like to clarify that the “stochasticity” mentioned in the paper (line 164), such as measurement noise, can be considered part of the partial observability of the system. This has now been reflected in the abstract (line 9), which reads: “In this paper, we tackle the probabilistic prediction of partially observable dynamical systems with long-memory.”\n\n----\n\n## Question 2. Design principle\n\nThe method is not specific to the Sun and accommodates a wide range of timescales, as shown in Fig. 4 (Appendix). The general algorithm is described in lines 227–242 in the main text. To improve clarity and highlight the generality of our approach, we will include an algorithm box in the paper. \n\nThe multiscale templates are derived from first principles and are not hand-designed. The only dataset-specific adaptation lies in two hyperparameters: the time horizon that is generated at once (see Fig. 4, Appendix) and the number of future steps predicted per diffusion sampling process ($K$ in the paper). The later is driven by the computational budget allocated to the model (the more steps to generate at once, the larger the denoiser). The former depends on the time range of the dependencies. For example, if the system exhibits a maximum timescale $T$, after which any two observations $x_{t},x_{t+T}$ are independent, then this is a natural candidate for the horizon parameter. Otherwise, this can be tuned, and also provides a way to reveal the horizon of long-range dependencies in a dynamical system.\n\nTo showcase the genereticity of our multiscale inference scheme, we applied it to the Navier-Stokes data from PDEArena [Gupta and Brandstetter 2022]. \nTo make the dynamical system partially observable, we downsampled it to 32x32 (from 128x128), considered only the density field (discarding velocity) and added a small Gaussian noise on the observations.\nThe results on this new example of partially observable process are presented in the Table below with the same metric than in the paper (MAE on the power-spectrum, see line 296). \n\n| Inference scheme | 1–4 (MAE Power Spectrum) | 4–16 (MAE Power Spectrum) | 16–36 (MAE Power Spectrum) |\n|---|---|---|---|\n| Autoregressive | 0.39 | 0.43 | 0.35 |\n| Multiscale (ours) | **0.38** | **0.36** | **0.31** |\n\nWe observe that our multiscale inference scheme improves predictions over the autoregressive baseline, particularly in the long-term interval (16–36), where it achieves a significantly lower MAE on the power spectrum. This shows that our approach is not specific to solar prediction. Rather, we considered solar forecasting as a challenging scientific application where our model also performs well."}]

Author rebuttal:

Meta-review: The paper addresses the problem of prediction in partially observable dynamical systems. It proposes an inference scheme based on diffusion models that is able to predict future frames in a multiscale fashion by conditioning also on multiscale past frames. The idea here is that, compared to naive autoregressive prediction approach, multiscale inference can better capture long-range dependencies without sacrificing computation cost. The approach is tested on a large dataset of videos on solar activity. This dataset is very interesting in its own right, and the authors promise that they will make it available to the community.

This is a well written paper that deals with an interesting and challenging application in solar prediction. It develops a sensible multiscale diffusion inference method. The reviewers gave many useful comments that the authors should address when updating their paper.
Decision: Accept (poster)