Hybrid Latent Representations for PDE Emulation
Authors: Ali Can Bekar
Siddhant Agarwal
Christian Hüttig
Nicola Tosi
David S. Greenberg
Venue: NeurIPS 2025 poster ()
Keywords: PDE Integration
Physics Informed Learning
Neural PDE Solvers
Primary area: machine_learning_for_sciences

Abstract: For classical PDE solvers, adjusting the spatial resolution and time step offers a trade-off between speed and accuracy. Neural emulators often achieve better speed-accuracy trade-offs by operating on a compact representation of the PDE system. Coarsened PDE fields are a simple and effective representation, but cannot exploit fine spatial scales in the high-fidelity numerical solutions. Alternatively, unstructured latent representations provide efficient autoregressive rollouts, but cannot enforce local interactions or physical laws as inductive biases. To overcome these limitations, we introduce hybrid representations that augment coarsened PDE fields with spatially structured latent variables extracted from high-resolution inputs. Hybrid representations provide efficient rollouts, can be trained on a simple loss defined on coarsened PDE fields, and support hard physical constraints. When predicting fine- and coarse-scale features across multiple PDE emulation tasks, they outperform or match the speed-accuracy trade-offs of the best convolutional, attentional, Fourier operator-based and autoencoding baselines.

Hybrid Latent Representations for PDE Emulation
Ali Can Bekar Helmholtz Centre Hereon Geesthacht, Germany ali. www.beeereerrncccoomm
Siddhant Agarwal
Helmholtz Centre Hereon
Geesthacht, Germany si siddant.... Beahhant.agaraal@hheeee. wgarwal@hereon.ccmoomm
Christian Hüttig
Nicola Tosi
Institute of Space Research
Institute of Space Research
German Aerospace Center (DLR)
German Aerospace Center (DLR)
Berlin, Germany
Berlin, Germany nicola. tosi sehidddr.dd
christian parissiann.uettig@ddrr. huettigddlr.de
David S. Greenberg Helmholtz Centre Hereon
Geesthacht, Germany
david. www.dd.grebberr@hheeeo...c
Abstract
For classical PDE solvers, adjusting the spatial resolution and time step offers trade-off between speed and accuracy. Neural emulators often achieve better speed-accuracy trade-offs by operating a compact representation of the PDE system. Coarsened PDE fields are simple and effective representation, but cannot exploit fine spatial scales in the high-fidelit numerical solutions. Alternatively, unstructured latent representations provide efficient autoregressive rollouts, but cannot enforce local interactions physical laws inductive biases. To overcome these limitations, we introduce hybrid representations that augment coarsened PDE fields with spatially structured latent variables extracted from high-resolution inputs. Hybrid representations provide efficient rollouts, can trained on a simple loss defined coarsened PDE fields, and support hard physical constraints. When predicting fine- and coarse-scale features across multiple PDE emulation tasks, they outperform match the speed-accuracy trade-offs of the best convolutional, attentional, Fourier neural operator-based and autoencoding baselines.
Introduction
Integrating partial differential equations (PDEs) be computationally expensive, particularly for chaotic and turbulent systems with fine-scale structures requiring high spatiotemporal resolution. For traditional solvers, discarding fine- details reduces both cost and accuracy, but data-driven emulators can push the cost-accurace frontier beyond conventional limits. State-of-the-art emulators [Kochkov al. 202 Stachenfeld al. 2021 Li et 202T] employ supervised training on PDE fields coarsened by local averaging, straightlorward approach emphasizing speed and stability. But local averages an optimal representation for emulation? Fine-scale information crucial for convolutional network accuracy [Wang et al.] 2020a]. and emulators tend to neglect low-amplitude spatial frequencies [Rahaman et al.) [209 Lippe et al., 2023). Fine-scale features could potentially reduce emulation errors, which tend accumulate and amplify over autoregressive rollouts. The main alternative to coarsening unstructured latent representations [Wiewel et al., 2020), but problems with scalability, generalization and physical consistency hinder widespread adoption.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

We introduce compact hybrid representations of fine-scale PDE fields, combining local averages with spatially structured latent variables. Hybrid representations exploit fine-scale information, allow efficient coarse-scale emulation and admit hard physical constraints [McGreivy and Hakim 2023]. Hybrid representations provide more favorable cost-accuracy trade-offs than strong baselines on challenging emulation tasks, maintain accuracy at higher coarsening factors and benefit more from unrolled training. After time stepping, they can be used to reconstruct high-resolution PDE fields more accurately than super-resolution or high-resolution rollouts with baseline emulators.
2
PDE Emulation
For 2D PDE with m variable fields, the space- and time-discretized system state time step t is X, E RmxHxW reference solver S updates X,, starting from prior-drawn initial conditions:
Xo p(Xo)
(I)
X, = S(X-i).
IstST
(2)
We assume a fixed time step length between solver outputs. The reference solver runs slowly but accurately at the reference resolution H W generate training and testing data for emulation tasks. For some PDEs and solvers, S can be slow due to adaptive time steps that require many internal steps produce outputs every 8 time units, or because system of equations must be solved iteratively at each time step.
Coarsening Let Ar X, Jt. E RRmxx/rxW//r denote the coarsening operator reducing spatial resolution by a coarsening factor on each spatial axis. The nature of A, depends on the PDE and discretization (appendix [A] Fig. 6 we choose A, to respect conservation laws where possible. Most emulation studies use coarsening in some way, but consider it detail of data preparation. Since we compare different representations and resolutions deriving from the same reference simulation, we denote coarsening explicitly and consider it part of the emulator.
Problem statement We aim to predict coarsened future PDE fields quickly and accurately. We assume high resolution reference simulations {Xp, .I...... Xr are available as training data, and that the coarsening operator A, is known. PDE emulator is method that predicts future PDE fields from known initial conditions, with accuracy to be evaluated at larget resolution h² x w= H//r* x XW/r* that may be coarser than the simulations (r* 2 1). That is, given X, we wish to predict A,- A,-(X++ (X++)). The training data, reference resolution H x W, target resolution h² w² and rollout length k together define an emulation task.
Accuracy Metries We measure correlation and mean-squared-error (MSE) for the PDE's m fields. In some cases, we also compare the emulator's 's velocity and energy spectra those of the reference solver, or compare the time evolution of the LI norm of emulators' one-step updates to those of PDE fields the target resolution [Kochkov al., 202 Kohl al. 2024].
Inference speed Since comparison to reference solutions defines accuracy, only emulators with inference faster than S are useful. An emulator
accurate than alternatives with equal faster inference speed the best method for the given speed and task. We measure inference speed as the slope of wall-clock time as a function of k. We report additional details on timing overheads relevant for low k in appendix B
2.1 Fixed-resolution Neural Emulation
Most emulation studies trained fixed-resolution neural emulators (FRNEs) to operate autoregressively on coarsened PDE fields x, = (X,,). A loss defined on output fields is optimized, such as MSE:
LIRNE (M))) m2+1||2- Σ JM Ml|M (x)) - (x
(M is emulator autoregressively applied to the input fields k times. K= = 1 trains single-step predictions, while K > uses autoregressive rollouts. The rollout resolution h x w = H/rr x W/r used by the emulator advance the coarsened PDE fields in time.
common choice is matching the rollout and target resolutions (r = r*), but general r is an adjustable hyperparameter, and together with M's neural architecture can be chosen emphasize

---

accuracy or speed. r = 1 trains at the reference resolution, but for most tasks and architectures, this results in inference slower than [McGreivy and Hakim [024]. Instead, FRNEs for chaotic and
turbulent PDEs have been applied most effectively with 4 S < 64 [Zhou et al. 2024a Stachenfeld
et al. 2021, Gupta and Brandstetter, 2023] Kohl et al. 2024 Zhang et al. [024 Wang and Wang,
2024 Wang et a 2020b]. While our focus here is on PDEs defined on regular grids, this approach
has also been successful on irregular meshes (Brandstetter et al. 2022a] or for weather forecasting
Lam et al. 2023].
FRNEs exploit neural networks' better tolerance for coarsening than numerical solvers, and have employed wide range of architectures incorporating convolutional, Fourier, attention or graph-base
layers [Gupta and Brandstetter 2023 Li et al. [021, Hao et al., 2024] Brandstetter et al.) [023]
Horee and Mitsume, [2024] 2002 Sanchez-Gozzale
et a 2020]. Because the learned time stepping operator M operates on coarsened PDE fields, can be constructed impose physical constraints such as incompressibility [Wandel et al., [021] Wiewel et al. 2020 or conservation laws (Watt-Meyer
jet al.] 2025| Verma et al.)
2024|| Horie and Mitsume| 2024]. Ideally, MM should faster than but only slightly less accurate, and more accurate than decreasing the reference solver's resolution to obtain the same inference speed. A disadvantage of FRNES is that A, destroys information, so X may contain fine-scale features useful for predicting Tz+4 that are unavailable in
2.2 Unstructured Latent Space Emulation
Unstructured Latent Space Emulators (ULSEs) learn time stepping for a compact, unstructured latent representation of PDE fields. They are commonly trained in an encoder-processor processor-decodee framework: PDE fields coarsened to an encoding resolution H/T x W/r are encoded a vector 24 autoregressively rolled out, and decoded back to PDE fields at the encoding resolution. This results in the loss function:
LULSE = D M(K) o E(xx)) - = Do MC oE -x,
(4)
Where E is the encoder, D is the decoder and M_ is the propagator operating on latent representations. LULSE is efficient since it is only calculated once after an efficient K K-step rollout latent space. For
r = 1, the encoder operates directly on reference PDE fields, and can learn to exploit any features
of X,, but most studies r > [Hao et al. 2024 Li et al. 2023 Serrano et al. 2024 Alkin et al.
2024 Wu et al. 2022 Knigge et al. 2024 Wang and Wang 2024
Alternatively, and D can first be trained as an autoencoder, and then time stepping can be learned using fixed latent vectors 21:
K C. Σ
K Σ Axx -
(5)
L. efficiently computes a loss at each time step without involving PDE fields, but appropriately weighting the dimensions of = is challenging. Fluid dynamics have been emulated using L. WWeewel
ctt al., 2019 НAN et al. [2022] Yin et al. 2023 Li et al. 2025). albeit with few competitive results on challenging benchmark tasks.
ULSES provide fast training and inference with long rollouts, as their compact latent spaces lack spatial structure. However, ULSES lack the spatial inductive biases that improve performance and robustness in many computer vision tasks, and x, and Ø441 might not be close in the latent space. Furthermore, constructing Mg to obey physical constraints defined for æ, is usually not possible.
3 Emulation with Hybrid Representations
We introduce hybrid representations (Fig. [h), which extend coarsened PDE fields with additional channels of learned latent variables to combine the advantages of FRNEs and ULSES. Like the coarsened PDE fields used by FRNEs, they are spatially structured and support physical constraints on the learned time stepping operator. Like ULSE latent variables, they extract information with a trained encoder instead of relying solely on A,, before rolling out cost-effectively. Emulation with hybrid representations jointly trains an encoder and processor as in eq. 4 but does so using an efficient loss that has more in common with eq. and We first describe this strategy in full generality, reserving architectural details for sec.
3

---

Encoder Hybrid representations consist of coarsened PDE fields 2, E R= together with latent variables fields C, E Ruxhxm 2t is computed by A, and e by an encoder & applied to A, (X,),, with S r.
()) - (X,()), (X,) = (Xi,
(6)
&, is trained to extract latent variables from A, (X₂) while maintaining spatial structure. Its encoding resolution H/r' x W/r' is higher than its rollout resolution H/r x W/r.
Processor A second network MHR carries out time stepping on the hybrid representation:
(i1+1, E++1) MAR (zz, ct) = (Ma(z). Me(x).
(7)
MAR computes autoregressive rollouts entirely within the hybrid representation at the rollout resolu- tion H/r x W/r. As the representation advances in time, coarsened PDE fields and latent variables mutually influence each other. The notation 3y, é3 marks these variables as having been time-stepped by the processor, as opposed to 2₁ and C. which are computed by A, and & A. respectively.
Loss We wish to avoid costly loss calculations the encoding resolution (eq- 4 or over the numerous, non-physical channels of ca (eq. 5). We therefore jointly train all encoder and processor parameters by minimizing prediction errors only over coarsened PDE fields 24, the lower rollout resolution:
Ma o M o (X.) - Tz+h
LHR
While ChR does not include error terms on Ct, minimizing it still encourages & and MHR to generate latent variables that improve prediction of future states Tt+k. especially for long training rollout length K. We reasoned that while e ought to improve prediction of Te+ks it would be unrealistic to expect the processor to accurately predict Ct+A without access to Xe+k- We found this simple and efficient loss more effective in practice than alternatives involving error terms on C++-k or X1+k- Since LuR only includes error terms for variables with the same units the original PDE fields, it can data normalizations independent of the training process and architecture.
Physical Constraints An advantage of explicitly including 2t in hybrid representations is that Ma can be constructed to respect physical constraints. For example, for a conserved PDE field such as energy or density, can learn fluxes between neighboring grid cells [Le 2002]. Similarly, for a pair of channels (ue, vt) C. ze representing the velocity of an incompressible fluid, we can learn a vector potential d+++1 whose curl yields the desired divergence-fiee field (ū++1. Uz41) V ac+1 [Wandel et al. 2021]. Hybrid representations allow these physical principles to be enforced as hard constraints, while nonetheless improving the time stepping of x4 with additional information from ct- Together with a physically consistent local averaging operator & (Fig. 6), this provides end-to-end physical consistency from X, to Tz+k-
Decoder When the rollout resolution exceeds the target resolution, we can predict A,- (X++k) by coarsening Tz+R- Otherwise, we train decoder DIR to reconstruct X; xs A,- (X-),, using fixed interpolation operator DD, (Fig. 7 and learned correction B.:
X; DHR (., 2.) D2 + (.,,
DR and the output layer of B. are chosen in a PDE-specific way to enforce conservation laws and/or incompressibility as hard constraints (appendix A.3)
To train DHR we minimize:
K
An DHR M EHR (Xi) 1 >
Co CD=
(10)
For efficiency, and e emphasizes accurate rollouts of lo44 over decoding of X train DHR with the encoder and processor frozen after training. Thus, DHR is trained triplets (Ø,, E. X.) without backpropagation through time, and is not used to train the encoder and processor (eq.
While standard super-resolution maps 7, - X; and a ULSE decoder maps - X;, our decoder DuR takes both 7e and 7 as inputs. & provides information unavailable in ãt, while the presence of T allows physical constraints to be imposed.

---

a
Refrenee Sandalan
Xay
Revend Tribs
Oore
Boz Booeline
Vieletion
BRRB (onns)
Musli Bol-cwt Presanc
Leomed
X-
Corpan
Upentopli
xF
1012
MM
0.3-
0.87
Carelabon
Corsluban
9.4
0.4
U1S88
Recoder
Decodded
0.0
107
0.0
101
107
Inference Speed (oamply))
Inference Speed (oamlentt
Figure 1: (a) Top: hybrid representation emulators combine locally averaged PDE fields with latent variables. A processor network evolves both coarsened and latent fields in time, and an optional decoder reconstructs high-resolution fields. Teal arrows and boxes respect conservation laws and incompressibilityy Middle: fixed-resolution neural emulators. Bottom: unstructured compact latent space emulators. (b) Reference solver vs. HREs and mnet baseline (with super-resolution) with rollouts 64 for KF and 32 for IID and encoding from 5122. KF is shown after 8.5 and IDD after 7.75 seconds. (c) Violation of incompressbbility for HREs and mUnet. (d) Correlation with reference solver vs. inference speed for HREs and baselines. Multiple points on each cost-accuracy frontiers were obtained by adjusting the resolution used to advance rollouts (HREs/FRNEs) or to provide encoder input (ULSES).
4
Emulation Architectures
Here describe architectural details for baseline emulation approaches operating on coarsened and high-resolution PDE fields, alongside our hybrid representation--ased emulators. FRNES and HREs learn additive updates estimating 2; -1-1- We did not seek match parameter counts (Table 3 across emulators, instead choosing one or more hyperparameter settings for each emulator to obtain overlapping ranges of inference speeds and facilitate comparison of cost-accuracy trade-offs across architectures. Full architectural details appear in appendix A
Modern U-Net with Attention (mUnet) the U-Net architecture [Ronneberger al. 2015 has been optimized for use as an FRNE, and performed well in many emulation tasks and benchmark studies [uupta and Brandstetter] 2023 Kohl al. 2024). We also examined the SineNet variant (Zhang ct al. 2024], but found it to have a less favorable cost-accuracy trade-off (Fig. 16).
FactFormer [L et al. 2023 is Transformer Vaswani et al. 2017] designed for operator learning. uses axial factorized attention to learn integral operators while maintaining model scalability. We train FactFormer a ULSE at encoding resolutions 32 642, 256 and 512
DPOT (Hao al., 2024 is an attention-based FRNE, trained on diverse datasets and tasks as a foundation model using 1-step prediction loss and input noise. It aggregates coarsened fields from multiple input time steps, and its Fourier attention layers use an MLP for efffccieee approximation of frequency space computations. DPOT-S (31M parameters, matching our HREs) was fine-tuned in few-shot mode at resolutions 322 and 642 on each task before evaluation.
DINo Yin et al. 2023 is a ULSE combining implicit neural representations [Sitzmann et al. 2020
Tancik et al. 20200 with neural ODEs [Chen et al., 2018). Input fields are used compute an unstructured latent vector through an auto-decoding framework, propagated through time by a neural ODE and decoded predict PDE fields. DINo can produce outputs anywhere along continuous time axis, predict PDE fields from incomplete initial conditions and generalize across resolutions.
Dilated ResNet (DtlResNet) [Stachenfeld et al. 2021 is FRNE designed for turbulent flows using dilated convolutions. We train it using either input noise (Di)ResNNt-NTT) or unrolled training NeitResNee-UT).. In the main text, we use Net and refer to it simply DilResNet.
Fourier Neural Operator (FNO) Li et al. 2021 an FRNE that combines filtering in Fourier space with local linear operations, and was originally introduced for PDE emulation.
5

---

Hybrid Representation Emulators (HREs, ours) employ encoder, processor and decoder modules (Fig. [I sec. (3). Their design closely follows mUnet, our strongest baseline, evaluate specific benefits of hybrid representations instead of architectural differences. The encoder & uses (rgr(r/r') mUnet downsampling blocks to reach the desired coarsening factor. The processor is an mUnet with extra input/ouiput channels. Except where otherwise stated, we use n. = 16 learned latent variable fields. The decoder corrects interpolation D, using physics-constraine additive update (eq. [4] I6) from a network By constructed from mUnet upsampling blocks. The encoder, processor and decoder use flux/vector potential outputs to impose physical constraints.
LatRoll We implemented an HRE-like baseline with only learned variables in its latent representation. This model used the same encoder to calculate c₁. the same mUnet processor to compute latent space rollouts, the same decoder to compute X; from Cr+& and the same physical constraints but did not include coarsened PDE fields 24 as input to the processor or decoder. We trained LatRoll using a loss on Xx, obtaining best results when first training and freezing an autoencoder (LatRoll-FAA), We also report results for end-to-end training (LatRoll-E2F).
Super-resolution To evaluate baselines at higher target resolutions than their outputs, we trained super-resolution networks [Harder et al. 2023 Stengel al. 2020 Esmaeilzadeh et al. 2020). These took only T as input and their outputs directly estimated X,, but their architectures were otherwise identical the By of HREs, and were not trained jointly with emulators (Table Bp.
5 Training
Loss and Optimization We train baselines as described in the respective studies. We standardize reference fields X,. We normalize both components of each velocity field jointly, so the momen- tum conservation and incompressibility enforced by flux- and vector potential-based output layers are not violated. We use the A optimizer (Loshchilov and Hutter] 2019] except on DINo which uses Adam Kingma and Ba 2015]. A cosine scheduler decays learning rate from 10 to 10 [Lohhchlovv and Hutter 2017 for our HREs, mUnet, FNO and DilResNet, while DINo and FactFormer used their original schedulers. Further training details appear in appendix A
Unrolled Training and Input Noise We train mUnet, FNO, and hybrid representations on rollout curriculum of lengths K [1, 2, 4, 8, 16), each for 100 epochs with early stopping and patience of 75 epochs. We did not observe benefits from unrolled training with DilResNet, but instead added Gaussian noise (μ O, a = 10 10-4) to coarse inputs Ia as previously described (Stachenfeld et al. 2021]. For DINo and FactFormer, we follow the original papers" training schedules.
6 Datasets
We generated 5 datasets with numerical solvers chosen respect physical laws of the integrated PDEs. Each dataset employs a fixed time step 5 longer than the solver's internal time step; we choose 8 based previous studies where possible (details in appendix C.. Table 10 lists all model-dataset combinations we employed.
2D Kolmogorov Flow (KF) consists of solutions to the 2D incompressible Navier-Stokes equations:
d.u V (u 2 u)
Au
+ Vp
=00
V -
(11)
Re
P
for velocity u. constant density p. pressure p. forcing f and Reynolds number Re. We set (p, Re, v)
=
(1, 103, 10 10-3) and f = sin( é1 0.1u, resulting in statistically stationary turbulent flow.
2D Kolmogorov Flow with Re- 4000 (KFHR) As the smallest eddies scale with 1/VRe, we quadruple the Re to halve eddy size. This produces finer-scale structures in high-resolution fields.
2D Incompressible Decaying Turbulence (ID) has the same dynamics as KF but no forcing, resulting in transient rather than stationary dynamics. Larger structures develop as small eddies dissipate.
2D Compressible Decaying Turbulence (CD) solves the compressible Navier-Stokes equations:
O.p + V (pr) = 0
(12a)
(pv) + (pv @ v) = - Vp V E + p) v 8 7
(12b)
0 E + V
(12c)

---

for density P. velocity v, pressure p. viscous stress tensor T and total energy E = c (pv2)/2. (10 10100,,0000 0.4), with M the initial Mach number, resulting in decaying turbulence as small eddies coalesce through an energy cascade. This dataset is challenging due to CFL. Suurant-Friedrichs--le and the presence of shockwaves.
We use 7,
condition of 60, and nonlocal behavior of density and energy fields
ID Kuramoto-Sivashinsky Equation (KS) models chaotic stationary dynamics of a scalar field u: 8,u + (u² + 0g2 =00zauu)
(13)
Nonlinear advection transfers energy between low and high energy modes. KS is common bench- mark task for neural emulators [Stachenfeld et al. 2021 Lippe el al. 2023 Schiff et al. 2024
7 Experiments
Cost Accuracy of Neural Emulators We first evaluated HREs and baselines emulation tasks at target resolution 32 (r* =4). Where possible, we trained emulators at multiple rollout and encoding resolutions to obtain multiple points on their cost-accuracy curves (Fig. [d]. For all tasks, at 64 rollout resolution HRE rollouts were equally or more correlated to reference simulations than all baselines. Similarly, HRES with 32 encoding resolution were more accurate than all baselines except mUnet at 64 x 64 rollout resolution, which was slower. Only mUnet matched or approached the accuracy of HREs on any task, though DilResNet was sometimes comparable for shorter rollouts. When comparing methods at common rollout or encoding resolution of 32 the gap between HRE and baselines widened: HREs had 30-50% lower RMSE after 64 time steps (Table Since hybrid representations use a processor architecture nearly identical to mUnet, they achieve nearly identical inference speed. All emulators were faster than numerical simulations, and FactFormer and FNO were the fastest but considerably less accurate than HREs and slightly less accurate than mUnet on almost all tasks. A comprehensive quantitative comparison is provided in Tables 5
HRES showed the greatest benefit over baselines on the KF task (Fig. left 9 where they exhibited higher correlation and lower MSE than mUnet by wide margin across 64-step rollouts both resolutions. HREs also tracked the average l norm velocity tendencies more closely than other methods (Fig. bottom left). For the ID and KFHR tasks, HRE was more accurate at 322 rollout resolution but comparable to mUnet at 642. For the challenging CD task we analyzed only up to 32-step rollouts at inference time, revealing that HRES were more accurate than other baselines at both resolutions (Fig. 12.. However, the improvement from using hybria representations was less prominent compared to ID and KF cases. The weaker performance of HREs on CD may arise from discontinuities in the density and energy fields, making the extracted fine-scale information from high-resolution inputs harmful for longer rollouts. Moreover, the dataset's 's high CFL number and the limited receptive field of the propagators could exacerbate instabilities. Notably, DilResNet and FNO's non-local nature might the reason for their ability to match the performance of mUnet on this task.
We further compared the frequency spec- tra of TKE fields for both reference sim-
Table I: Rollout performance across emulators for KF and ID experiments at rollout resolution 322 (encoding resolution for DINo and FactFormer). We report MSE and correlation (Corr.) of field variable u after 64 rollout steps. Best model performance in bold and second underlined.
ulations and emulators (Fig. B). follow- ing previous work [Kochkov et al. [021] by normalizing with the fifth power of the wavenumber. After 64-step rollouts, hybrid representations produced spectra closer to the reference simulations than basclines relying solely on coarsened PDE fields. This improvement suggests that in- formation from high spatial frequencies en- coded in the c, fields (Figs. 13 44 enables the www-resolution emulator to capture the higher-frequent components of the TKE fields more accurately (additional results in Figs. 18 19).
KF
ID
RMSE 1
CORR. 1
RMSE
CORR.
OURS
0.71+.28
0.68
0.29+.08
0.964.08
MUNET
1.07-.20
0.35+.21
0.58+.34
0.78.425
FNO
1.29
0.124.19
0.71
0.744.20 20
DIERESNET
1.164.20
0.26+.20
1.77+.72
0.50++17
FACTFORMER
1.12..21
0.281.20
0.58
0.791.51
DIND
1.002.10
0.204.117
1.104.27
0.30
DROT-S
1,854.24
0.204-17
1.65+.64
0.184.24
DE RESNET-UT
1.19-.22
0.26
0.84
0.72-4.18
LATROLL-32 32- FAB
1.152.17
0.144.116
0.9641.22
0.39
LATROLI-52-E3E
1.234.17
0.204-10
1.204.38
0.274.48
To test generalization to data outside the training distributions, we compared HREs and mUnet on the IC task with the initial peak wave
7

---

number changed from 4.2 to 3, or 6. As expected, performance on uut-of-distribution (OOD) imputs worsened as peak wave number increased, but HREs continued outperform mUnet in every case (Table Thus, the representation learned by the HRE encoder does not become liability when encountering OOD data.
The Role of the Coarsening Ratio HREs retain information in or that FRNES at the same rollout resolution lack, so does their performance gap widen as r increases? KF and ID experiments at rollout resolutions 32 and 64 confirm this (r 16 and 8, Fig. 8
12 and in general HREs were better suited to large r than baselines (Fig. i.. FNOs showed the smallest accuracy decreases when doubling r, but were among the least accurate FRNEs overall. For KF, HREs outperformed mUnets at both resolutions, but for ID did so only at 32 while for 64 their performance was similar. To summarize, we observe that: (i) latent fields allow HREs cope with higher r increases, (ii) HREs outperform FRNES only for sufficiently large (emphasizing speed over accuracy), and (iii) the minimum r for which HREs beat FRNES varies across tasks.
Since HREs benefit from latent features ex- tracted from high-resolution PDE fields, why
TRR 230400
PNO
Diflielle
Rathfrre
neoos Tatidi PAR
not simply train FRNES on high resolution data? To test this, we trained mUnet with rollout res- olution 512?, resulting in inference >> 30 times slower than for rollouts at 642. Somewhat coun- terintuitively, mUnet rollouts at 512² were less accurate than at 322 or 642 rollouts, despite ac- cess to additional information from X, (Fig. It). One possible reason is that the physical size of receptive fields for a fixed mUnet architec- ture shrinks as r decreases. Training on high resolution data also emphasizes fine scales not reflected in accuracy measures at the target reso- lution. In contrast, HREs work efficiently across resolutions by encoding 512² PDE fields to pre- dict future fields 32
(sconds)
(swoads))
0.00
2.13
4.26
0.88
8G11
0:08
297
3.4 530 7.57 9.54 11.1
1.5
1.5
0.5
0.5
0.0
1.0
1.0
Comdlaion
0.5
0.6
0.4
0.4
0.3
150
190
100
110
18
302
18
522
-25
15
10
Tine Stepe
TIas: Stege
To test whether the trend would continue still higher coarsening ratio, we trained HREs and mUnet on data from the KS task with a reference resolution of 2048 and rollout resolution 32 (r 64), In contrast to other tasks for which HREs used an encoding resolution of 5122, here we used no coarsening before the encoder (r' = 1). HREs outperformed mUnet by a wider margin than in previous experiments, achieving near- pear-perfect predictions after 16 time steps in this challenging chaotic system, while mUnct's correlation dropped to 0.5 over the same rollout length (Fig. [5]
Figure 2: Emulator accuracy on KF and ID tasks (testing data, 16 ICs) at 32 rollout/encoding res- olution. Mean + s.d. over ICs of RMSE (top) and correlation (middle), and tendency & norm (bottom) vs. rollout length.
Benefits of Unrolled Training Consistent with previous studies [List et al. 2024 Lam et al. 2023 Kohl et al. 2024], we observed improved rollout performance for emulators trained autoregressively. However, while accuracy for 64-step rollouts plateaued
105-
function of training rollout length for mUnet, hybrid representations continued to benefit from training on curricula of longer sequences, up to at least 16 steps (Fig.
Grornd Truth
TKE
HRE (ous)
mlCnct
4... When training on shorter rollout lengths, mnne exhibits superior performance, suggesting that the encoder & requires lolg-rollout training to learn features in e, useful for longer prediction horizons.
FNO
DilRenNes
FactFormer
10
2
Latent Space Dimensionality We investigated how the dimensionality of affects HRE accuracy on KF and ID tasks. too small could lose useful information from (X,). while n too large could overfit or produce unstable rollouts; we tested = 4 and = 16 7). For KF at 64 rollout resolution, reducing n from 16 to roughly doubled RMSE and decreased correlation by 2/3. However, KF at 32 rollout resolution was similar for 4 and 16. For ID 16 gave
Figure 3: Scaled TKE values for high spatial fre- quencies on KF (32?).
8

---

RMSE 3 times lower, but had smaller and less consistent effect on correlation (Fig.) 8). 71 negligibly
impacted inference speed.
Prediction at High Resolution To evaluate PDE emulation of high-resolution input fields,
we computed accuracy for KF and ID tasks at target resolution 512 (r* = 4, Fig. 21 p2 Our HRE used encoding resolution 512 (r" 4),
rollout resolution 642 for KF (r 32) and 32 de-
Prediccoon nollod koggh
for ID (r = 32), and a physics-constranne coder. As baselines, we considered mUnet at r == 32 and r = 64 followed by learned super- resolution (essentially the same decoder with only 2e an input), mUnet with
12
21-
and
FactFormer with r = 4 and r = 8. HRE out- performed all baselines all rollout lengths both tasks (Fig. 20). Consistent with previous results r 64, mnnet with super-resolution consistently outperformed mUnet trained on 5122 for all but the shortest rollouts, and mUnet with r = 4 completely decorrelated with the reference solution before 64 steps. Both Fact- Former models performed slightly better than mUnet with 4.
Treining collout Jength
Thoining
length
(Commbetioa)
Figure 4: Correlation of predicted and reference PDE fields vs. rollout length during training and prediction on KF (322). Our HREs benefit from longer training rollouts, and achieve higher accu- racy, while mUnet plateaus after 8- 8-step training.
Ablations To identify benefits of hard phys- ical constraints and learned latent fields for HREs, we trained physics-constrained mUnet with no cts and an HRE without physical con- straints (Fig. 17, Table (8). For 64-step rollouts, both models surpassed mUnet accuracy but fell short of HREs. Unconstrained HREs yielded RMSE and correlation values only z 20% worse and 12% lower respectively, but the physics-constrained mUnet had RMSE nearly Rs 40% higher and correlation x 30% lower. Despite its relatively strong quantitative performance, the unconstrained hybrid model produced oscillatory tendency norms and led oscillatory behavior in the predicted fields during propagation (Fig. 17.. This suggests that physical constraints can regularize latent representations.
8 Related Work
Learning for Long Rollouts A key challenge for neural PDE emulation is eтror accumulation over autoregressive rollouts, causing distribution shift for network inputs, instability or high gradi- ent variance. To mitigate this, unrolled training and moise-injection
(otza)
mllan-3122
PartFomer-255
1elloa-31-55
KF w//0
Thur (scomds)
IP o/s.d)
Thtse (xnomade)
0,58
8.1
1.0
1.0
strategies List et al. 2024
0.6
0.5
Metz et al. [202 Mikhaeil et al. 2022 Stachen
feld et al. 2021 Sanchez-Gonzalez et al. 2020
1.0
Kochkov et al.) 2021 Lam al. al. 2023
Hao et al.
0.5
2024] expose networks to their own outputs or
corrupted inputs to enhance robustness. The pushforward trick [Brandstetter et al. 2022a
0.6
List al. 2025) trains with rollouts but no back- propagation over time.
34000
35500
33000
32000
Multiple Coarse Timesteps we use e to augment 2t, but other studies provide multi- ple coarsened states -i as additional inputs
54500
31000
32000
30000
Stepp
[Wang 2021 Li et al.) 2023) or via specialized
Figure 5: Accuracy 5122 target resolution.
aggregation layers [Buitrago et al. (2025] Hao
et al. 2024 predict It+1. This can improve performance, but incurs computational cost and

---

cannot learn which features to extract from X₁ Temporal bundling Buitrago et al. 2025] instead predicts several output steps at once.
Brandstetter et al. 2022a
Physics-derived Losses minimizing a known PDE's residuals [Wandel et al. 2021 Tompson et al. 2017 Raissi al. 20T9] removes reduces the need for reference simulations, but does not allow spatiotemporal coarsening for efficiency. Hybrid Solvers use machine learning to accelerate classical solvers or partial-physice models, to correct discretization errors [Kockkov al. 2021 Um et al.
2020
Pathak et al. 2020. Tompson et al. 2017 Paliard et al.
2022 Bar-Sinai et al.
2019
Neural Operators emulate PDEs with autoregressive mappings between function spaces, allowing
adaptation to different resolutions and geometries (Li et al. 2021 Alkin et al., 2024 Bonev et al.
2023
2021]. [Hagnberger et al., [024] take similar approach to the time axis, encoding initial conditions and decoding at desired output times non-autoregressively Likewise, foundation models pretraining across diverse data and tasks enable few- or zero-shot inference (Herde
Lu et al.
et al.
2024 Hao et al. 2024).
Climate and Weather Prediction drive PDE emulation research, large-scale turbulence and chaos
demand high spatiotemporal resolution, large domains and physical coherence [Kochkov et al. 2024
Keisler 2022 Verma et al. 2024 Lam et al. 2023 Watt-Meyer et al. 2023 2025 Price et al. 2024
Nguyen et al.
2023]. Some emu atoos enforce conservation laws and symmetries [McGreivy and
Hakim| 2023 Verma al. 2024 Wandel et al. [2021] Beucler et al. [202] Lee and Carlberg] 2021
Yuval and O' G'Goman 2020 Lee and Carlberg 2021 Brandstetter et al. 2022b 2023
Discussion and Outlook
By storing and propagating both coarsened and learned latent fields, HREs outperform diverse baselines with remarkable consistency across tasks and resolutions. We hope that this strategy will prove useful for tasks requiring long rollouts and physical consistency, such as predicting atmospheric or ocean dynamics.
Architectures and Hyperparameters HREs proved effective with mUnet processors rectangular grids, but our approach orthogonal to MIR 's architecture so long as physics-respecting coarsening and interpolation operators available. Which architectures benefit most from the HRE framework remains an open question, but a first step could be to consider our baselines and other successful emulators. Tuning of processor hyperparameters could prove beneficial, and the optimal values might not be the same as for the corresponding FRNEs. HRES could also be combined with hybrid solvers or numerical advection of ct.
Spatial Structure Weather forecasting models operating on spherical mesh data Lam et al. 2023 Oskarsson et al. 2024 Price et al. 2024] compute coarsened internal representations using graph- based message passing across resolutions, an approach compatible with physical and symmetry constraints (Horie and Mitsume 2024). However, mesh-based weather models have not yet included coarsened input fields, hard physical constraints, or multistep rollouts within a latent space. Similarly, cooss-attention-basee perceiver layers can drastically decrease token counts relative to the input dimensionality [Alkin et al. 2024]. These 'coarsening-like operators for meshes and point clouds could be used to extend HRRS beyond regular grids.
Domains and Boundaries Our PDEs were complex, but used simple spatial domains with periodic boundaries. Boundary conditions and geometric information could be provided as additional network inputs [Wandel et al. 202T Lam et al. 2023 Watt-Meyer et al. 2025 Horie and Mitsume 2024) to an HRE encoder. Such general-purpose -uurpss encoder could allow for HREE foundation model.
Generative Modeling Our HREs and baselines use MSE losses. For long rollouts of chaotic dynamics, this can produce blurry outputs, distribution shift and instability. Generative models remedy this somewhat (Lippe et al. 2023] Kohl et al. 2024 Price et al. 2024 Zhou et al., 2024b)) but reduce inference speed. Generative modeling with loss like eq. 8 remains an open challenge.
Further Constraints and Applications HREs could also support hard symmetry constraints (Huang loss terms [Schiff et al.) [024].. Beyond our 2D fluid dynamics tasks, HRES could also be applied to 3D systems, higher field counts, solid mechanics, mixeed-pass. flows or spatially extended chemical and biological systems.
and Greenberg] 2025) or stability-induee
10

---

Acknowledgments and Disclosure of Funding
We thank Nishant Kumar, Anthony Frion and Marcel Nonnenmacher for insightful comments the manuseript. ACB, SA and CH were supported by the BMFTR grant PLAGES (16DKWN1175 ACB, SA and DSG were supported by Helmholtz Al and by the Helmholtz Foundation Model Initiative project HC
References
Dmitrii Kochkov, Jamie A. Smith, Ayya Alieva, Qing Wang, Michael P. Brenner, and Stephan Hoyer. Machine learning- computational fluid dynamics. Proceedings of the Na- URL https://www pnas or ottooooo///ooo///o//// 1073 100. 1073/pnas. pnas. 2101784118 536 citations (Semantic Scholar/DOI) [2024-04-14] 14] 354 citations (Crossref) [2024-04-09] Publisher: Proceedings of the
fional Academy of Sciences, 118(21))e10018841 18, 2021. doi: 10.
National Academy of Sciences.
Kimberly Stachenfeld, Drummond Fielding, Dmitrii Kochkov, Miles Cranmer, Tobias Pfaff, Jonathan Godwin, Can Cui, Shirley Ho, Peter Battaglia, and Alvaro Sanchez-GGonzaeez. Learned Coarse Models for Efficient Turbulence Simulation. ArXiv preprint, abs/2112. 2021. URL
abs/ 2112. 15275
Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Burigede liu, Kaushik Bhat- tacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations, 2021. URL
https://openrrevie: net taaee/forrie htte//ooommmdddee odppmmiid-ssS9900emm
Haohan Wang, Xindi Wu, Zeyi Huang, and Eric P. Xing. High-frreueett component helps explain the generalization of convolutional neural networks. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 868 1- 8691. IEEE, 2020a. doi: 10.11000CY2422000220000000 CVPR42600 2020. 2020.00871
URL phttps:///ot org 10. 1109/
Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron C. Courville. On the spectral bias of neural networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pages 5301-5310. PMLR, 2019. URL http: 7/proceedings. mlr Prress/v997/rhaaman
html
Phillip Lippe, Bas Veeling, Paris Perdikaris, Richard Turner, and Johannes Brandstetter. Pde- refiner: Achieving accurate long rollouts with neural pde solvers. In A. Oh, T. Nau- mann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neu- ral Information Processing Systems, volume 36, pages 67398-67433. Curran Associates, Inc., 2023. URL https://proceeddngg newrriinnnnneu.ui........... 0tg.64444..3abba//ggaa///44efee9 Pawpeeeeeeeeeeee--aa-----oeeeeee
neurips. wwwpurips.cppppppppe cc/paper (files/paper/2023/3
pdf
S. Wiewel, B. Kim, V. C. Azevedo, B. Solenthaler, and N. Thuerey. Latent Space Subdivision: Stable and Controllable Time Predictions for Fluid Flow. Computer Graphics Forum, 35-5) 2020. ISSN 0167777555 com/doi/10 1111/cgf. 14097
doi: 10.1 00.1111/cgf.14097
Nick McGreivy and Ammar Hakim. Invariant preservation in machine learned PDE solvers via error correction. ArXiv preprint, abs/2303.161 2023. URL https://axxiv.org/bbbb abg//30
16110
Georg Kohl, Liwei Chen, and Nils Thuerey. Benchmarking autoregressive conditional diffusion models for turbulent flow simulation. ICML 2024 Al for Science Workshop, 2024. URL
httpp::/opennreview
Nick McGreivy and Ammar Hakim. Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations. Nature Machine Intelligence, URL htt.:htts/////t Publisher: Nature Publishing Group.
pages 1-14, 2024. ISSN 2522-5839,
doi: 10. 00000///2226-00-000099999
nature. com/ http/articclsseee /422///2222222 - 024
000222266024-000000 00897-5

---

Hang Zhou, Yuezhou Ma. Haixu Wu, Haowen Wang, and Mingsheng Long. Unisolver: PDE- Conditional Transformers Are Universal PDE Solvers. ArXiv preprint, abs/2405.17527, 2024a.
URL httpt:t//txxiiii wwwpexiv..orgaeeoo hwtgg/ab/2240555 17527
Jayesh K Gupta and Johannes Brandstetter. Towards muliisppatioeemporaassaalll generalized PDE
modeling. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https
//openreview. wtwp://wwwee//foummeeeeee F
Xuan Zhang, Jacob Helwig, Yuchao Lin, Yaochen Xie, Cong Fu, Stephan Wojtowytsch, and Shui- wang Ji. Sinenet: Learning temporal dynamics in time-dependent partial differential equa- tions. In The Twelfth International Conference on Learning Representations, 2024. URL
ftttps://openreviet
Tian Wang and Chuang Wang. Latent neural operator for solving forward and inverse pde problems. Advances in Neural Information Processing Systems, 37:33085-331 -3107,, 2024.
Rui Wang, Karthik Kashinath, Mustafa Mustafa, Adrian Albert, and Rose Yu. Towards physics- informed deep learning for turbulent flow prediction. In Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash, editors, KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, pages 1457-1466. ACM, 2020b. URL https: //al. acmm acm. ptrg/doi/110/ 1145/33944666 3403198.
Johannes Brandstetter, Daniel E. Worrall, and Max Welling. Message passing neural PDE solvers. In International Conference on Learning Representations, 2022a. URL /ttpssss////poereeiie
net/forum?id-v$ixxHte
Remi Lam, Alvaro Sanchez-Cronzllez, Matthew Willson, Peter Wirnsherger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton-Rosen, Weihua Hu, Alexander Merose, Stephan Hoyer, George Holland, Oriol Vinyals, Jacklynn Stott, Alexander Pritzel, Shakir Mohamed, and Peter Battaglia. Learning skillful medium-range global weather forecasting. Science, 382(6677); 1416-1421, 2023. doi: 10.1 1116//cincce.
URL http:::t////// htstpps:/www// science. . org /doi/10 1126/science. adi2336 Publisher: American Association for the Advancement of Science
Zhongkai Hao, Chang Su, Songming Liu, Julius Berner, Chengyang Ying, Hang Su, Anima Anandku- mar, Jian Song, and Jun Zhu. Dpot: auto-regressive denoising operator transformer for large-scale pde pre-training. In Proceedings of the 41st International Conference on Machine Learning, pages 17616-17635, 2024.
Johannes Brandstetter, Rianne van den Berg, Max Welling, and Jayesh K Gupta. Clifford neural layers for PDE modeling. In The Eleventh International Conference on Learning Representations, 2023.
Masanobu Horie and Naoto Mitsume. Graph neural pde solvers with conservation and similarity- equivariance, Proceedings of the 41 International Conference on Machine Leurning, pages 18785-18814, 2024.
Masanobu Horie and Naoto Mitsume, Physics-embedded neural networks: Graph neural pde solvers with mixed boundary conditions. Advances in Neural Information Processing Systems, 35:23218 23229, 2022.
A vvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter W. Battaglia. Learning simulate complex physics with graph networks. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 8459-8468. PMLR, 2020. URL gow.oezzgnnzaazeaa....
http::/proceedinn mlr. press//119//aacchez http11/////cccceeooooo//eeaaaaaa html
Nils Wandel, Michael Weinmann, and Reinhard Klein. Learning incompressible fluid dynamics from scratch towards fast, differentiable fluid models that generalize. In International Conference Learning Representations, 2021. URL /attps://ppenreveee
Oliver Wett-Meyer, Brian Henn, Jeremy McGibbon, Spencer K Clark, Anna Kwa, W Andre Perkins, Elynn Wu, Lucas Harris, and Christopher S Bretherton. Ace2: accurately learning subseasonal to decadal atmospheric variability and forced responses. npj Climate and Atmospheric Science, 8(1): 205, 2025.
12

---

Yogesh Verma, Markus Heinonen, and Vikas Garg. ClimODE: Climate and weather forecasting with physics-informed neural ODEs. In The Tweifth International Conference on Learning Representations, 2024. URL
Zijie Li, Dule Shu, and Amir Barati Farimani. Scalable transformer for pde surrogate modeling. Advances in Neural Information Processing Systems, 36:28010-28039, 2023.
Louis Serrano, Thomas X Wang, Etienne Le Naour, Jean-Noël Vittaut, and Patrick Gallinari. Aroma: Preserving spatial structure for latent pde modeling with local neural fields. Advances in Neural 2024.
Information Processing Systems,
Benedikt Alkin, Andreas Flist Simon Schmid, Lukas Gruber, Markus Holzleitner, and Johannes Brandstetter. Universal physies transformers: A framework for efficiently scaling neural operators. Advances in Neural Information Processing Systems, 37:25 155-25194, 2024.
Tailin Wu, Takashi Maruyama, and Jure Leskovec. Learning to accelerate partial differential equations via latent global evolution. Advances in Neural Information Processing Systems, 35:2240-221 2253, 2022.
David M Knigge, David R Wessels, Riccardo Valperga, Samuele Papa, Jan-Jakob Sonke, Efstratios Gavves, and Erik J Bekkers. Space- time continuous pde forecasting using equivariant neural fields. Advances in Neural Information Processing Systems, 37:76553-76577, 2024.
Steffen Wiewel, Moritz Becher, and Nils Thuerey. Latent space physics: Towards learning the temporal evolution of fluid flow. In Computer graphics forum, volume 38, pages 71-82. Wiley
Online Library, 2019.
XU HAN, Han Gao, Tobias Pfaff, Jian-Xun Wang. and Liping Liu. Predicting physics mesh- reduced space with temporal attention. In International Conference Learning Representations, 2022. URL https://////peereeiie
wet./forum/.dd-xctollltttc CmP
Yuan Yin, Matthicu Kirchmeyer, Jean-Y Franceschi, Alain Rakotomamoniy. and patrick gallinari. Continuous PDE dynamics forecasting with implicit neural representations. In The Eleventh International Conference on Learning Representations, 2023. URL htt.::ttt://// //ttprev//oeeeevveew .ntt/
forum?id-B77 Borumiidd-73nn1]JPP
Zijie Li, Saurabh Patil, Francis Ogoke, Dule Shu, Wilson Zhen, Michael Schneier, John R Buchanan Jr, and Amir Barati Farimani. Latent neural pdc solver: A reduced-order modeling framework for partial differential equations. Journal of Computational Physics, 524: 137005, 2025.
Randall J. Veque. Finite Volume Methods for Hyperbolic Problems. Cambridge Texts in Applied Mathematics. Cambridge University Press, Cambridge, 2002. ISBN 978-0-521-00922-9, doi: 10. 10.10077CC00978055179922333 hww.:hoddd--.ooo.eeeeood------oo hyperbolli--poobeem..
URI. https: :/wpw/
cambridge org/ http://ore//booooo www.booooofoiittto/eeeeee// 9705D444012220011444333111222223
for
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Nassir Navab, Joachim Homegger. William M. Wells, and Alejandro F. Frangi, editors, Medical Image Computing and Computer-Assistt Intervention - MICCAI 2015, pages 234 241, Cham, 2015. Springer International Publishing. ISBN 978-3-319-2457444, doi: 0-11:/978---31---5---- 28. 9999 citations (Semantic Scholar/DOI) [2024-06-06] 29178 citations (Crossref) [2024-06-05].
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus. S. V. N. Vishwanathan, and Ro- man Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Confer- Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA. USA, pages 199986008 2017. wtw htppp//dd////ccoppp/eeeeo neurips cc/ htt//ppppr/////// 2017 20007//har /hash/ Abstratt..attttttaa.aa.aa.. html
Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Im- plicit neural representations with periodic activation functions. Advances neural information processing systems, 33:7462-7473, 2020.
13

---

Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich- Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains. In Ad- vances Neural Information Processing Systems, volume 33, pages 7537-7547. Curran Associates, Inc., 2020. URL
wwwpediiinnnnnnnniinnee nww.ierrisp. cc httpp/ppppppp////eee/ 55003999657777/354tafff222e12 Abssaaaraattctttttaaatatttt...t .tml
Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurlPS 2018, December 3-8, 2018, Montréal, Canada, pages 1577265882 2018. URL httpppps//poccceennngg.. htw.ouuurdnnnn..uu........ cc/ pappp/2011///////// /ttp:///8866600f6666466666666660
html
Paula Harder, Alex HemandezzGGaria. Venkatesh Ramesh, Qidong Yang. Prasanna Sattegeri, Daniela Szwareman, Campbell Watson, and David Rolnick. Hard-constrained deep learning for climate downscaling. Journal of Machine Learning Research, 24(365): 1-40, 2023.
Karen Stengel, Andrew Glaws, Dylan Hettinger, and Ryan N. King. Adversarial super-resolution of climatological wind and solar data. Proceedings of the National Academy of Sciences, 117(29):
16805-16815, 2020. doi: 10. 10.1073/pnas. 1918964117 http:::::// pnas . org.ooogoo..o.ooooo.oo /do//0 1073/pnas. 1ww.aaas..999966 1918964117
Publisher: Proceedings of the National Academy of Sciences.
Soheil Esmaeilzadeh, Kamyar Azizzadenesheli, Karthik Kashinath, Mustafa Mustafa, Hamdi A Tchelepi, Philip Marcus, Mr Prabhat, Anima Anandkumar, et al. Meshfreeflownet: A physics- constrained deep continuous space-time super-resolution framework. In SC20: International conference for high performance computing, networking, storage and analysis. pages 1-15. IEEE, 2020.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. 7th International Conference on Learning Representations, ICLR 2019, New Orleams, LA, USA, May 6-9, 2019. OpenReview.net, 2019. URL htttps:// wtt:/oorrmmdddddeooRo.....
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL Sttt://arxivg org/ wws//412....0
Ilya Loshchilov and Frank Hutter. SGDR: stochastic gradient descent with warm restarts. In 5th International Conference on Learning Representations, ICIR 2017, Toulon, France, April 24-26. 2017, Conference Track Proceedings. OpenReview.net, 2017. URL /tttps://oppnreviiie net/
forum?id=Skq39Sczz
Yair Schiff, Zhong Yi Wan, Jeffrey B Parker, Stephan Hoyer, Volodymyr Kuleshov, Fei Sha, and Leonardo Zepeda-Núñez. Dyslim: dynamics stable learning by invariant measure for chaotic systems. In Proceedings of the 41st International Conference on Machine Learning, pages 43649-43684, 2024.
Bjoern List, Wi- Chen, Kartik Bali, and Nils Thuerey. How Temporal Unrolling Supports Neural Physics Simulators. ArXiv preprint, 12971, 2024. URL httth::/pss:xiii oor.aaiii.oo.oooooooooooooo /bs/ 2402. 12971
Luke Metz, C. Daniel Freeman, Samuel S. Schoenholz, and Tal Kachman. Gradients are Not All You Need. ArXiv preprint, abs/2111.05800 2021. httptts://axxii..rrggr/// Borg/abs/21111 05803
Jonas Mikhaeil, Zahra Monfared, and Daniel Durstewitz. On the difficulty of learning chaotic dynamics with mns. Advances in neural information processing systems, 15:1129711111,, 2022.
Bjoern List, Li- Li-Wei Chen, Kartik Bali, and Nils Thuerey. Differentiability in unrolled training of neural physics simulators on transient dynamics. Computer Methods in Applied Mechanics and Engineering, 134:11 2025. 2025. ISSN 0045-7825. doi: 10. .0.. URL
14

---

R Wang. Incorporating symmetry into deep dynamics models for improved generalization. In International Conference on Learning Representations (ICLR), 2021.
Ricardo Buitrago, Tanya Marwah, Albert Gu, and Andrej Risteski. On the benefits of memory for modeling deneeddenddntt PDEs. In The Thirteenth International Conference on Learning Representations, 2025, URL https://openreviiet
Jonathan Tompson, Kristofer Schlachter, Pablo Sprechmann, and Ken Perlin. Accelerating eulerian fluid simulation with convolutional networks. In Doina Precup and Yee Whye Tch, editors, Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pages 3424-3433, PMLR, 2017. URL http:::tttccceeeeeeeeee mmr. http://wwoerrraaaaaoooooooeoooaa wttpp///////topsooiiiiiio
html
M. Raissi, Perdikaris, and G. E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686-707, 2019. ISSN 0021-9991. doi: 10.
1016//..ep.2018. 10.045. URL https://wwwt wwt.weeeeeencrrrt....... http///tteeecceeeeeeeecc//rr/cae $0021999118307125
Kiwon Um, Robert Brand, Yun Raymond Fei, Philipp Holl, and Nils Thuerey. Solver-in-the-loop: Learning from differentiable physics to interact with iterative pde-solvers. Advances in neural information processing systems, 33:61 11-6122 2020.
Jaideep Pathak, Mustafa Mustafa, Karthik Kashinath, Emmanuel Motheau, Thorsten Kurth, and Marcus Day. Using Machine Learning to Augment Coarse-Gria Computational Fluid Dynamics Simulations. ArXiv preprint, abs/2010.00072, 2020. URL https:://axiv..o 000022
Chloe Paliard, Nils Thuerey, and Kiwon Um. Exploring Physical Latent Spaces for High-Resolution Flow Restoration. ArXiv preprint, abs/2211, 11298, 2022. URL
arba///221
11298
Yohai Bar-Sinai, Stephan Hoyer, Jason Hickey, and Michael P Brenner. Learning data-driven discretizations for partial differential equations. Proceedings of the National Academy of Sciences, 116331)15344415544 2019.
Boris Bonev, Thorsten Kurth, Christian Hundt, Jaideep Pathak, Maximilian Baust, Karthik Kashinath, and Anima Anandkumar. Spherical fourier neural operators: Learning stable dynamics on the sphere. International conference on machine learning, pages 2806-2823. PMLR, 2023.
Lu La, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Kamiaddkis.. Learning non- linear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence, ((3):318-2222) 2021. ISSN 2522-5839, doi: 103888442556002100000000
URL https:: ://::://wwwww www.raat.ee.... wttp////r..ceeee////22------2 02004444222222002-202000332 lishing Group.
Publisher: Nature Pub-
Jan Hagnberger, Marimuthu Kalimuthu, Daniel Musekamp, and Mathias Niepert. Vectorized conditional neural fields: A framework for solving time-dependent parametric partial differ- ential equations. In Forty-firt International Conference on Machine Learning, 2024. URL hhtpp:: ftttp:://openrevees peet/foorummid==sSSppkkku
Maximilian Herde, Bogdan Raonic, Tobias Rohner, Roger Käppeli, Roberto Molinaro, Emmanuel de Bézenac, and Siddhartha Mishra. Poseidon: Efficient foundation models for pdes. Advances in Neural Information P Processing Systems, 37:72525-7762. 2024.
Dmitrii Kochkov, Janni Yuval, Ian Langmore, Peter Norgaard, Jamie Smith, Griffin Mooers, Milan Klöwer, James Lottes, Stephan Rasp. Peter Düben, al. Neural general circulation models for weather and climate. Nature, 632(802)):1000-1000 2024.
Ryan Keisler. Forecasting Global Weather with Graph Neural Networks. ArXiv preprint, abs/2202.07575, 2022, URL httts::/t//attt org/ http/bbb/222.007//
15

---

Oliver Watt-Meyer, Gideon Dresdner, Jeremy McGibbon, Spencer K. Clark, Brian Henn, James Duncan, Noah D. Brenowitz, Karthik Kashinath, Michael Pritchard, Boris Bonev, Matthew Peters, and Christopher S. Bretherton. ACE: A fast, skillful learned global atmospheric model for climate prediction. ArXiv preprint, abs/2310.00074 2023. URL.httpe //arsiv///xvvv// org/ abs/ 2310. 020040
alan Price, Alvaro Sanchez- Sanchez-Gonzaler Gonzalez, Ferran Alet, Tom R. Andersson, Andrew El-Kadi, Dominic Masters, Timo Ewalds, Jacklynn Stott, Shakir Mohamed, Peter Battaglia, Remi Lam, and Matthew Willson. Probabilistic weather forecasting with machine learning. Nature, pages 1-7, 2024. ISSN 1476-4687. doi: 10. 0..1038//41886 028-6-022--022---- URL URLpttps:: htttts://ww/ nature. com/ coomartticls// 151586-024-082522
Publisher: Nature Publishing Group.
Tung Nguyen, Johannes Brandstetter, Ashish Kapoor, Jayesh K Gupta, and Aditya Grover. Climax: A foundation model for weather and climate. In International Conference on Machine Learning, pages 25904-25938, PMLR, 2023.
Tom Beucler, Michael Pritchard, Stephan Rasp, Jordan Ott, Pierre Baldi, and Pierre Gentine. Enfore- ing analytic constraints in neural networks emulating physical systems. Physical review letters, 12669):098302, 2021.
Kookjin Lee and Kevin T. Carlberg. Deep Conservation: A Latent- Matent-Dynamics Model for Exact Satisfaction of Physical Conservation Laws. Proceedings of the AAAI Conference Artificial Intelligence, 35(1 2:27-285, 2021. ISSN 2374-3468. doi: 10. 10906669.iiivvaaaa www.ii...1...1111. URL
https://ojs aaai org wergiinex.... pbtp://AA///arttillll/tee///////
Number: 1.
Janni Yuval and Paul A. O'Gorman. Stable machine-learning parameterization of subgrid processes for climate modeling range of resolutions. Nature Communications, 11(1):3295, 2020. ISSN 2041-1723. doi: 10.1038/s4 1000///41467--20 167--000-1744---3 URL httthttt:///wwe htttt::://ww.naturee nature. com/ www./ootiit/eeeeeee http/rrtilles/////
-21-477-00--7142- citations (Semantic Scholar/DOI) [2023- 12-01] 89 citations (Crossref) [2023-12-01] Number
Publisher: Nature Publishing Group.
Johannes Brandstetter, Max Welling, and Daniel E. Worrall. Lie Point Symmetry Data Augmentation for Neural PDE Solvers. In Proceedings of the 39th International Conference on Machine Learning, pages 1241-2255 PMLR, 2022b. URL htttps://proceedings mlr pww..//esssvv..
brandstetter22a. html ISSN: 2640-3498.
Joel Oskarsson, Tomas Landelius, Marc Deisenroth, and Fredrik Lindsten. Probabilistic weather forecasting with hierarchical graph neural networks. Advances in Neural Information Processing Systems, 37:41577-41645 2024.
Anthony Zhou, Zijie Li. Michael Schneier, John R Buchanan and Amir Barati Farimani. Text2pde: Latent diffusion models for accessible physics simulation. arXiv preprint arXiv:2410.01 153, 2024b.
Yunfei Huang and David S. Greenberg. Geometric and physical constraints synergistically enhance neural PDE surrogates. In Forty-second International Conference on Machine Learning. 2025. URL https:://peenrevier httpre/wwweeeeeeeeeooorr////e.s HxxeeneF
Zhuoran Shen, Mingyuan Zhang, Haiyu Zhao, Shuai Yi, and Hongsheng Li. Efficient attention: Atten- tion with linear complexities. In Proceedings of the IEEE/CVF winter conference applications of computer vision, pages 3531-3539, 2021.
Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In So Kweon, and Saining Xie. Convnext v2: Co-designing and scaling convnets with masked autoencoders. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16133-16142, 2023.
Jean Kossaifi, Nikola Kovachki, Zongyi Li, David Pitt, Miguel Liu-Schiaffini, Robert Joseph George, Boris Bonev, Kamyar Azizzadenesheli, Julius Berner, and Anima Anandkumar. A Library for Learning Neural Operators. ArXiv preprint, abs/2412, 10354, 2024. httpsssss:///aiitt abs / /212/22122.1 10354
16

---

Makoto Takamoto, Timothy Praditia. Raphael Leiteritz, Daniel MacKinlay, Francesco Alesiani, Dirk Pflüger, and Mathias Niepert. Pdebench: An extensive benchmark for scientific machine learning. Advances in Neural Information Processing Systems, 15:1596-1611, 2022.
R. Courant, K. Friedrichs, and H. Lewy. Über die partiellen Differenzengleichunnenn der math- ematischen Physik. Mathematische Annalen, 100(1):32-74, 74, 1928. ISSN 1432-1807. doi: (0.1007/BF0144883) URL BtLhhtts:://// //doi///doo./ 1ttp//00..007//F1444483333
17

---

A Detailed model and training descriptions
A.1 Additional training details
Downssmmpling Staggered
Factor of
Downsammplingg
Factor of 2
Collocated
Vector potentials az
Conserved variables {p, Mg My. E}
Horiz. velocities
Vert.
velocities v
Figure 6: Coarsening operations for staggered and collocated grids. For staggered grids, we em- ploy the face averaging approach for velocities described in [Kochkov et al., 2021]. This method conserves momentum and maintains incompressibility. In contrast, for collocated grids, we apply non-overlapping averaging that adheres conservation laws.
We provide the summary of the training parameters in Table 2
Training of our decoder and Super-resolution (SR) models We train the SR models separately for upsampling tasks without unrolling time. Since the rolled-out auxiliary variables Č++ are not expected to match the encoded variables (Xe+k)) our decoder is trained on rolled-out trajectories of (0,, 2.) However, we stop gradient propagation to both the processor and encoder during decoder training, and thereby avoid propagating gradients backward in time. We note that there are framework as in eq. 4 found this to be more costly and less effective in initial experiments.
technical obstacles to training the HRE end-to-end in an
A.2 Additional model details
In this section, we give additional model details that are not present the main text or Table 2 We also describe our modifications to the official implementations of FactFormer and DINo.
18

---

agk
Up
Uk
Ma
Ue
2
2
Uud
Me
1/s
iss
Upsampling Factor of 2 Staggered
Upsampling Factor of Collocated
Uk
Mc
High res. 0₂
Zero a, values
Figure 7: Upsampling methods for staggered variables and collocated grids. For staggered variables, face values are copied and velocity values in inner cells are linearly interpolated, preserving momen- tum conservation and incompressibility. Corrections are learned through vector potential ag. We either fix the corner values of ag to satisfy Equation [1 or relax this assumption obtain continuous upsampled velocity fields. For collocated grids, upsampling is performed by copying pixels from the low-resolution input or by bilinearly interpolating them.
mUnet Our implementation of mUnet uses initial convolution layer to lift the channel dimension to 32. The 3 up/downsampling blocks contain 2 ConvNeXt blocks, linear attention (Shen et al. 2021] and residual connection from the block inputs. The bottleneck layer includes multi-head self-attention [Vaswani et al. 2017]. and skip connections join downsampling outputs to upsampling inputs at each resolution. At each level of the downsampling, two consecutive ConvNeXt layers [Woo et al.) 2023 are followed by 2D convolution using a stride and a group norm with 32 groups The channel dimensions for each level of downsampling
[32, 64, 128, 256). Each skip connection incorporates linear self-attention operations. The bottleneck of the mUnet consists of XonvNeXt layer followed by full self-attention and another ConvNeXt layer. The upsampling blocks are designed to be symmetric to the down blocks (We use transpose convolutions for upsampling).
HRE ours Our implementation uses the up and down blocks of the mUnet without the skip connections. The encoder & channel counts are [32, 64, 128, 16] with corresponding norm groups of (32, 32, 32, 8). The processor maintains the same hyperparameters as the mUnet but uses different initial lifting convolution that operates on (It, e,) pairs. o fields are passed through ConvNeXt layers before and after the processor, while Ie fields passed through physical constraint layer after
19

---

the processor. Since in our high-resolution tasks the encoding and target resolutions are the same, the decoder symmetric the encoder except that the inputs are (ax, pairs and the first group normalization uses 9 groups for ID, KF and KFHR tasks and 10 groups for the CD task. Code is available at atttps///githu: cont al caneekar/hees
FactFormer We use the official FactFormer implementation, with a hidden dimension of 128, 4 layers, 8 heads, and kernel dimension 128. The input encoder is 2D convolutional layer and the output decoder is an MLP. We increase the input channel count to 2. For rollout resolutions of 256 and 512", to prevent out-of-memory errors, we decrease the number of latent steps from 4 to 2, the output window from 16 to 10 and the batch size from 32 to 8. We also decrease the maximum rollout length for the pushforward trick in our training curriculum from
2. When evaluating accuracy for FactFormer-256 in Fig. S we compare to reference simulations coarsened to 256 We note that despite this unfair advantage, FactFormer-223 remains less accurate than mUnet-64+HR and HRE-64.
DINo We use 3 layer decoder with 64 channels and latent size of 100. ODE tendencies are computed with a 4-layer MLP with 512 hidden units. We adjust the number of input channels based on the emulation task. We observed that the performance of DINo after 12K epochs is not competitive, and instead trained it for 60K epochs with early stopping patience of 20K epochs.
LatRoll-FAE and LatRoll-E2E Both models have the same setup ours for rollout resolutions of 32 and 642. The only difference is that we remove the coarse 24 fields and corresponding encoder-decoder pairs, and lift the latent fields to the same size before passing them to the propagator. During training and inference, these models take high-resolution inputs (512²) and encode them into the latent space as c {Xx). The propagator advances the latent representation as + MM(())) and the decoder maps the propagated fields to a high-resolution vector potential A = D(ē++)). A curl layer at the output then calculates the high-resolution velocity fields 512² as X = V x A'+b, satisfying the incompresiibility condition. This approach maintains zero momentum in the output field, which is consistent with the initial conditions in the ID and KF cases where momentum is nearly zero, For the FAE model, we train the encoder-decoder pairs in autoencoder mode on the entire training set, then freeze these weights during unrolled training. All parameters of the E2E model are trained at each rollout step. We only minimize a loss between the network predictions and the ground truth at the final rollout step to keep the models trainable in a reasonable time. We use the same curriculum K [1, 8, 16). with each curriculum step trained for 100 epochs. As a result, after the last curriculum step is completed, RMSE decreases for this model until 16 steps, then increases thereafter.
DPOT-S
This model uses Fourier attention on 1024-dimensional internal representation at rollout resolution 32 or 64?, with 6 layers and 8 heads. We use the pretrained model and finetune it for the downstream ID and KF tasks using batch size 32 on a single H100 GPU. We train the model for 500 epochs on the same set of simulation trajectories used for the other baselines. In each epoch, each trajectory contributes a sample to one element of one batch, starting on a random time step of the trajectory. This results in training on total of 960000 samples for KF and 640000 samples for ID in total, which is roughly equivalent to training for 35 epochs for ID and 20 epochs for KF on the whole dataset. We consider this a reasonable (or somewhat generous) quantity of data when finetuning a foundation model for few-shot inference. Attempts to use DPOT-S in zero-shot mode did not produce useful results beyond 5-10 time steps.
DilResNet The architecture consists of a linear projection that maps PDE fields onto 48 channels, followed by 4 blocks, of 7 convolutional layers with 3 x 3 kernels. Each block uses kernel dilations of 1,, 2, 4, 8, 4, 2 and 1. A final convolution maps onto output predictions. We use Swish activations, preceded by group normalization layers. For unrolled training, we use the same architecture on our standard curriculum K E [1, 2,4, 8, 16). The batch size per GPU is fixed at 32, and the number of GPUs for each curriculum step is (4, 6, 6, 6, 16). All other parameters match the DilResNet baseline with input noise training.
FNO We use the official FNO implementation from Kossaifi et al. 2024 (version 0.3.0), with 4 Fourier layers including lifting, filtering and projection, and hidden layers with 128 channels. We use 16x x- and y-modes for parameter count close to the mUnet.
20

---

A.3 Physics-constrained decoders for hybrid representations
Here we describe the construction of decoders with hard physical constraints. We consider two cases: conservation of multiple quantities on a collocated grid, and incompressibility with conservation of momentum on a staggered grid. each case, we learn a correction B, that added to an interpolation operator D.. D2 is the coarse-to-fine counterpart of &, (Fig. 7 and obeys all relevant physical constraints for each PDE.
A.3.1 Collocated grid
We enforce conservation of the field variable for KS as well as conservation of mass, energy and momentum for compressible flow on collocated grids by using an r x r local average as Ex, and bilinear interpolation as D.. The network learns additive correction the updated fields BR, from which the local r x r average removed to produce a conservative update Bg. To remove the local average, this average is first computed using &g, then removed at high resolution using Dz:
(B, (.) B (R,, (3) Dr o & (5,, 6e)
(14)
Here Da, a fixed interpolation operator respecting physical constraints (e.g., bilinear or nearest neighbor interpolation), is the coarse-to- to-fine counterpart of & (See Fig. 6 -rrrght). By is an unconstrained mapping from (āz, ōt) onto an additive correction for X, In eq. 14 the second term ensures that B2 corrects the physically constrained quantity (e.g. density, momentum or energy) in way that respects physical constraints, and ensures the decoder satisfies the consistency condition:
Vaa, Cr
E DUR (+) = St.
(15)
For example, if & averages density field over nonoverlapping x pixel squares, and Dr copies each pixel r x times, then Be has O mean on each r x square of outputs.
A.3.2 Staggered grid
Our KF, KFHR and ID tasks are based on incompressible flow on a staggered grid. In this case, & applies face-averaging (Fig- ()) as in Kochkov et al. [202T]. To satisfy the divergence-free condition, the decoder compute the curl of a learned high-resolution vector potential:
DHR 2x) V (ā), (,.1)))
(16)
Here, i and j pixel indices of the high-resolution output. Function (r fixes the corners of each r x square on the learned vector potential to zero (Fig. left). This guarantees that the decoder satisfies the eq. and is the counterpart of face-averaging & in Fig. left. The function (,((i,) is defined as:
(1, {
if mod and j mod r 0,
(17)
otherwise.
DD, thus interpolates horizontal velocities horizontally and vertical velocities vertically (Fig. 7 Both En and Dr trivially conserve the incompressibillity condition (consider the divergence of each cell in Fig. 7 upper left).
We observe that fixing the corner vector potential values introduces discontinuities that the correction term cannot rectify. Therefore, we relax the assumption of Equation 15 and allow the decoder to output continuous vector potential fields. This relaxation does not violate the conditions of incompressibility or momentum conservation. However, the encoding and decoding processes compromise the immutability of the coarse field variables, meaning that repeated decoding-encoding changes the coarsened fields Zt,
B Measurement of inference speed
We analyze inference times across varying batch sizes for the and KF cases at rollout resolutions of 32 and 642 on A100 GPU with 40GB memory. We exclude the data transfer and encoding costs for DINo and our model to obtain the marginal rollout cost per time step. To obtain the scaling behavior, we fit a linear regression using least squares to the largest three batch size measurements
21

---

for each model and calculate the resulting slope (However, we use the inference speed for batch size of 32 in Figure [I due GPU memory limitations). This slope approximates the per-batch- element inference time under GPU saturation conditions (considering that the last three points lie approximately on a line), while the y-intercept represents the computational overhead inherent to each model. The complete results are presented in Figures 23 and 24] In both cases, DilResNet is the slowest and DINo is the fastest model. Also, DINo has the smallest overhead while DilResnet has the highest. Our model is slightly slower than mUnet and slightly faster than FactFormer. All models are faster than the numerical solver operating in the native resolution, which takes 19.5 seconds to solve the system. However, the numerical solver operating in 512² input size takes = 0.6 seconds, meaning that models operating at 642 rollout resolution with batch size can be slower than this low-resolution numerical solver.
C Additional dataset details
In this section we give full details on the datasets used for each emulation task. All datasets are stored in float32 precision. Additional information and dataset parameters are summarized in Table 4 Initial conditions are taken from [Kochkov et al. [202]] for ID, and from [Takamoto et al.) 2022] for CD. For KF and KFHR, we take independent samples from the stationary distribution. Training, validation and testing data consisted of nonoverlapping sets of samples drawn from the same distributions (we use non-overlapping sets of random initialization keys for the training, validation, and testing datasets when generating data from JAX solvers), except in cases where we specifically measure generalization of trained emulators to out-of-distributtin data (Table 9).
C.1 2D Kolmogorov Flow (KF)
We use the staggered conservative DNS solver from the jax-efd package [Kochkov al., 2021 to integrate this system with periodic boundary conditions. We generate 128 experiments using 2048 x 2048 cells for the solver. We set the numerical solver's CFL [Courant al.) 1928] condition to 0.5 and then uniformly subsample its solution, obtaining = 300 points in the interval [40, 80). We then coarsen the data (velocities) maintain incompressibility and momentum conservation. This 512 x 512 data then comprises the high-resolution inputs Ar (X.) for HREs and other methods operating on inputs 5122 resolution for our emulation tasks, or are coarsened further as needed by other emulators. These spatial and
512 512 grid using the face-averaging approach to
temporal coarsening steps result in a CFL number of 9.4 for the training dataset, far exceeding that of the solver.
C.2 2D Kolmogorov Flow with Re 4000 (KFHR)
The Reynolds number of KF scales as Re x VXL2 /v. x is the forcing strength scale, L is the domain size and is the kinematic viscosity. Following [Kochkov al. 2021] we scale L 2L, v + v/2 and x + x/2 to increase the Reynolds number to 4000. Since the smallest eddy size scales with 1/vRe, Re, we get smaller eddy structures in the solution. We use a 4096 x 4096 grid for the solver and downsample the solution to 512 x 512 grid. All other parameters match KF.
C.3 2D Incompressible Decaying Turbulence (ID)
This dataset, like the Kolmogorov flow, is governed by the incompressible Navier-Stokes equations and solved with the same jax-cfd package. We generate 200 experiments using the same parameters and spatial resolutions as KF except for the forcing term, which is set to f = O. Time is uniformly discretized n, = 166 points in [4.5, 25). The spectral density of the initial conditions sampled from a log-normal distribution with variance 0.25 that peaks at a wavenumber of umax = 4.2. We then generate random initial conditions from a normal distribution and filter the fields using FFT to match the desired spectral density.
C.4 Compressible Decaying Turbulence (CD)
We generate the dataset using direct numerical simulation (DNS) software provided by (Takamoto) al. 2022 Time integration uses mass, momentum and energy-conserving finite volume scheme,
22

---

whose implementation we slightly modified to avoid rounding errors when converting between floating point representations of momentum and velocity. Our dataset consists of 200 different simulations having varying initial conditions of density, momentum and energy. Space is uniformly discretized to n By 512 cells in [0, 1] x [0, 1] with periodic boundary conditions and time is uniformly discretized to n 100 points in (0.
Initial velocities are generated
superposition of sinusoidal waves, v(x,t 0)
Ei=1 A sin(kx + d4) where N = 4, A = 8/ 8/|k], i is maximum initial velocity determined us- ing the Mach number 8 caM, where c, is the speed of sound. Wavenumbers are given by = 2 L, where L is domain length, and the phases Φ; ~ 24[0,2z] Density and pressure are initialized uniformly.
C.5 1D Kuramoto-Sivaahinsee Equation
We generate the dataset using the jax-efd package. The spatial domain (with length = 256) is discretized into 2048 points, while the temporal domain E [0, 150] is discretized into nt 3000 time steps. Initial conditions {Am: em 2m are random parameters sampled as in [Lippe et al., 2023). Boundary conditions are specified periodic. We use a spectral solver with an RK4 time integrator. To ensure statistical stationarity, we discard solutions up to / = 80 as a burn-in period. We generate 1000 trajectories and downsample the solutions by a factor of 5 along the temporal axis. Rollout resolution is chosen as 32² meaning that r = 64.
specified (a(x)) Σ m= A. sin sin(2mmmm//
Φm), where
D Supplementary experimental results
KS and ID with multiple seeds To test the robustness of our HRE method different initial seeds, we initialized the network used in the KS case on a coarse version of the dataset using 5 different random seeds and trained it on next-step prediction task. We observed that the model's performance was consistent over seeds, with mean validation MSE of 5e - 8 and standard deviation of 1e - 8. We also repeated the ID experiment on a rollout resolution of 322 with an additional seed, resulting in similar accuracy. The additional experiment is conducted with seed 44 and resulted in a validation rollout loss of 0.29, In comparison, the experiment we report was run with the seed 43 and achieved rollout error of 0.31.
TKE spectra We report the TKE spectra of PDE fields predicted by the trained models at rollout
resolution of 32 in Figures 18 19 We observe that the high-resolution information encoded in ca fields enables our model to capture high-frequency features in the flow field. On the KF task, our model consistently outperforms others in reproducing the TKE spectrum of the reference solver. However, it occasionally produces unwanted oscillations at high frequencies for the case. For KF rollout resolution 32 use the mUnet model trained on rollout length 16. The model trained on rollout length 2 overshoots some TKE fields by a large margin; we believe that training on longer rollouts helps the model in this case. However, for the stability plots on KF with rollout resolution 322, we use the model trained on 2 steps since achieves the highest correlation with the ground truth after 64 timesteps.
E Training Costs
For the ID case at rollout resolution 642 and rollout length 16, mUnet requires R 7 hours to train versus 9.5 hours for HRE on 8 A100s with batch size 16 per GPU. For long rollouts, most HRE computations occur at coarse resolution, but for short rollouts, mUnet is considerably faster: at rollout length 1, mUnet needs A100s. ENO takes ~ 20 minutes to train on 4 A100s for rollout length and ~ 90 minutes for rollout length 16. DINo trains in hours on single V100 at the same input resolution with batch size 32. FactFormer takes ~ 90 minutes on single H100 with batch size 32; however, it requires z 106 hours to train at rollout resolution 5122 with batch size 8. DilResNet takes Rs 13 hours to train for 500 epochs on 4 H100s in noise training mode with batch size 32. Finetuning for DPOT-S takes x 1 hour on an H100. LatRolI-FAF requires Rs 9 hours on 100Ss for the KF case at rollout resolution 16 when training on a single timestep and x 15 hours on 16 timesteps, while Lat EEE takes se 15
hour for 100 epochs on 4 A100s, while HRE requires 5 hours on
23

---

hours and = 17 hours when training on rollouts of and 16 timesteps, respectively. All batch sizes are per GPU.
(onrs)
DilResNes
FastFormes
Latrol-tE-PAA
(ouns)
efa,
(sccud))
(woond))
etx.
0.00
1,7
2.4 5.0 7.87 9.54 11.1
0.00
1.97
5.4
5.9
7.7 1.44 11.82
0.00
157
XM4 E92 7.87 9,84
11.81
0.00
LOR
304
530 7S5 9.4 11.1
1.5
3.
1.0
0.5
0.5
0.0
1.0
1.0
1.0
Comitaton
0.0
0.0
54
0.5
0.4
100
190
600
000
[|
100
200
16
50
65
Tion
Ti0e
Figure 8: Rollout performance on the ID task at rollout/encoding resolutions 32 (left), 642.
HRE (onas)
mitlsct
FNO
DilResNett
Foemerr
DING
DPOT-S
Latroll-FAR
+1s.0) (sconds)
(wacnd))
Time (eooii))
Time (ancaad))
0.00
1.28
9.51
2:13
0.20
2.15
4.35
6.15
351
1.5
3.5
1.5
2.0
1.0
0.5
0.0
0.0
0.0
1.0
3.6
1.0
1.0
0.8
0.6
0.8
0.5
0.2
0.7
140
1.90
560
520
100
020
110
500
400
10
Tinon
Thas
Tiae
Figure 9: Rollout performance on the KF task at rollout/encoding resolutions 32 (left) and 642
24

---

HRE (ouss]
milinet
PNO
DiIRoNNer
Fantermee
INN
SETER M6G.
Tirs (ewenda))
KEEE vtc.
Thrs (woomta)
KFHR sia.d
The (moord))
REHE v(x.d)
Then (prorsdg)
2.8
18
180
80
34
0.
0.0
1/
2.0
1.0
Chovilato
0-8
3.8
0.
08
0.
34
0.8
0.0
80
08
200
2001
55
53
2.0
45
48
22
48
36
32
61
16
Tiace Stage
These Strpe
Titae Strpp
Tita Strpp
Figure 10: Rollout performance on the KFHR task at rollout/encoding resolutions 32 (left) and 64
HRRE (owr)
muset
D:IRoeNee
Time (icoodda)
Thnn (poosds))
(.(a.))
Time (scoodd))
M,(s.t)
Time (ecoonds)
6.00
0.00
1.00
0.00
0.50
1.50
0.00
0.90
1.00
0.00
0.90
110
0.30
RAMND
0.00
02
62
10
1.0
1.0
3.0
Claroittoz
0.8
0.5
6.6
4000
0000
200
250
3000
Thee
Tiaa Strpe
Taa Shaps
Figure 11: Rollout performance for 322 CD.
25

---

(2000 EP.)
1 VI00
Table 2: Training details. Our model trains with batch size 16 per GPU on KFHR; all other experiments use batch size 8. For mUnet, we maintain fixed effective batch size across configurations by increasing the number of GPUs when the per- GPU batch size is decreased. ENO and mUnet models trained on either A100s or H100s.
DINO
KF KF/ID
ADAM
ESS [16]
60000
MSE
SILU
32 le
PUSHFORWARD 21,
RELATIVE MSE
FACTFORMER KP /IDDD//// SE/ID/XPHI /D//XPRR
ONECYCLE
1 H100 GPU
8/32
ADAMW
1e 4
50
GELU
No
1234
COSINEWR - le -6- 6
DILRESNET ////KPH/// ID//CD/CCCDD
4 GPUs
# O. 7= 10
ADAM
CIRCULAR
32
0.01
500
MSE
SILU
YES
42
AA100/H100 GPUS
COSINE + Le - 6
/E///PR/// ID/ CD
[1,2,4,8, 16)
100 x CIRCULAR MSE
ADAMW
FNO
32
0.01
SILU YES
42
8/4 A100/H100 GPUS
COSINE + le - 6
/P/KKPR//// /ID/CD/CDDD
[1,2, 4, 16)
100 5 CIRCULAR
MUNET
ADAMW
16/32
0.01
MSE
SILU
YES
42
100
/KPKK/K///AA ID CCD
COSINE - le - 6 8 A100 GPUS
HRE (OURS)
100 5 CIRCULAR
8/16
-44 4
ADAMW
0.01
11,2,4,8,10
MSE
SILU
YES
42
100
EARLY STOPPING (75 EP.)
INITIAL LEARNING RATE
BATCH SIZE (PER GPU)
DEC./SR DEC. EPOCHS
TRAINING EPOCHS
TRAINING NOISE
WEIGHT DECAY
LR SCHEDULER
CURRICULUM UM
OPTIMIEER
RESOURCES
ACTIVATION
PADDING
LOSS
SEED
26

---

Table 3: Number of parameters per model.
KF/ID
KFHR
CD
OUR PROCESSOR
19.492M
19.492M
19.493M
MUNET
19,430M
19.430M
19.433M
FNO
18.974M
18.974M
18.974M
DISREST
583680
583680
588096
FACTFORMER
3464834
3464834
DINo
769420
769420
DPOT-S
30M
LATROLL (FAE-E2E) (64")
23.8M
LATROLL (FAE-E2E) (32²)
37.1M
ENCODER (642)
2.156M
2.156M
OUR ENCODER (32*)
8.780M
8.780M
OUR DECODER (64²)
2.204M
2.204M
OUR DECODER (32
8.833M
8.833M
SR DECODER (642)
2.153M
2.153M
SR DECODER (32?)
8.743M
8.743M
HRE (ours)
mUnet
ENO
DilRonNes
Time (eccoud))
4
Tlue (pcossds)
Ma(a.))
Tine (pccouds)
M,(x.t)
Thme (scondss)
6.00
3.50
1.80
0.00
0.30
1.00
0.00
0.50
3.60
0.00
0
1.00
0.6
0.4
0.05
0.2
0:00
0.0
1.0
3.0
1.0
Caedaaa
0.8
0.8
0.8
0.8
0.6
0.4
0.4
0.4
17500
10000
12500
18
18
10
Time Skopa
Tim Skepa
Time Stape
Tine Strpe
Figure 12: Rollout performance for 642 CD.
27

---

INCOMP. DECAYING
Reini = 1000
[0, 2T. x [0, 2m] 2n 02/22048
3.652 10
6.25/12.5
Ra: 2y
0
2 /512
1, 16
0.123
340
0.5
4.2
166
200
0.1
(p. Re, v) = (1,4 x 10 , x 10
KOLMOGOROV (RE=4000)
0.5 (4y) - -.1u
[0: 4m] x (0, [z]
2.191 10-4
4. 4.687/9.38 .375
2
2
2x 07/4
2n /512
8, 16
0.133
~ 600x
0.5
40
80
300
128
0.1
16
(p, Re, 1) = (1. 10 10-3)
sin (4y))e 0.1u
KOL COLMOGOROV
[0, 2x/ [0, 2n]
2.191 10-1
Table 4: Dataset details.
Va: by
2T/ /2048
2T 2512
1,8, 16
0.133
600
0.5
9.375/18.75
300
128
0.1
(G.).M) = (10 101000000,0
COMP. DECAYING
Ps Mr. My E
[0, 1] x [0.1]
R 1.1 10-1
28. 125/56.255
2
0
1/512
1/512
8, 16
0.05
2 x50
0.3
100
200
0.1
20
# SPATIAL DIMENSIONS
SPATIAL COARSENING
EMULATOR TIMESTEP
TEMPORAL COARSENING
COARSE DATASET CFL
TRAJECTORY LENGTH
# TR TRAJECTORIES TRAIN
VALIDATION ERACTOON
TRAJECTORIES TEST
DATASET SIZE
SOLVER GRID SIZE
SOLVER TIMESTEP
TERMINAL TIME
FEATURES
PARAMETERS
DOMAIN SIZEE
SOLVER CFL
BURN-IN TIME
FEATURES
FORCING
TEMP. INFO
PARTITIONS
GRID
28

---

Table 5: Rollout performance across emulators and experiments. We report MSE and correlation (Corr.) of field variable u for each method after 64 rollout steps (52 steps for KFHR at 32²). Best model performance bold and second underlined.
KF
ID
KFHR
RMSE
CORR.
RMSE
CORR.
RMSE
CORR.
ROLLOUT RESOLUTION 32 x 32
OURS
0.71 2:23
0.68 2.21
0.29 4.08
0.96 #.03
0.50 0.04
0.74 1.05
MUNET
+.07
0.35
0.58
0.78
0.52 +.05
0.694.05
FNO
1.29
0. 19
0.7 2:27
0. 0.74
0.66
0,48
DILRESNET
1. 1.16
0.26
1.77
0.50 1.50+..7
0.5
0.63
FACTFORMER
1.12
0.28 2.2.0.2
0.58
0.79 +: .61
0.53 45.0
0.66
DINo
1.000.10
0.20
1.10
0.30 2: 3:30
0.99..04 1.00
0.21 1:04
DPOT-S
1.85 2.24
0.20
1.65
0. 8.188:20
DILRESNET-UT
1.19 #.22
0.26
0.84
0.72
LATROLL-32-FAN
1.15
0. 0.14
0.964.22
0.39
LATROLL-32-B2E
1.23
0.20
1.20
0.27
ROLLOUT RESOLUTION 64 64
OURS
0.43
0.88
0.28
0.93
0.44 4.05
0.79+.05
MUNET
0.69 +. 28
0.67
0.28
::.5
0.41 4.04
0.81 4.00
FNO
1,27 2:22
14
0.78
0.67 2.20
0.76 :5.5
0.35
DILRESNET
7.05 13.22
0. 1..13.11
20.80 311.3
0.04
0.52 0.06
0.70...10
FACTFORMER
1,15
0.23
0.664.36
0.7 0.71+.87
0.58 +.
0.644.05
DINo
1.07
0.15 0.15
1.094.28
0.31
1.67
0.06
DPOT-S
1.80 1.802..4
0.09
1.5 1.5.
0.1 0.17
DILRESNET-UT
3.32
0.07 #.17
6.5623.58
0.05
LATROLL-64-FAE
2.23
2.212.18
LATROLL-64-E2E
1.07-.16
0.32
29

---

Table 6: Rollout performance across different experiments for the field variable for each method.
Model parameters are the same as Table
METHOD
KF
ID
KFHR
RMSE +
CORR. 1
RMSE
CORR.
RMSE
CORR.
ROLLOUT RESOLUTION 32 32
OURS
0.764.27
0.67 #.20
0.30
0.96
0.50 4.04
0.70
MUNET
1.10+.24
0.34 22
0.51+.21
0.86
0.53
0.62
FNO
1 1.22
0.20
0.73
0.76
0.68
0.37
DILRESNET
1 1.244.26
0.21
2.18
0.40
0.57:.006
0.55
FACTFORMER
1.174.23
0.25
0.54
0.85
0.55
0.58
DINo
1.1 1.10
0.00
1.27
0.05
0.98
0.06
DPOT-S
2.1 2.11
a..25
9.23 #2.19
0.31 #:27
DILRESNET-UT
1.22#.19
0.20
0.77+.19
0.80
LATROLL-32-FAE
2.104.31
0. 0.10
1.25
0.34
LATROLL-32-EDE
1.29
0.13
1.244.36
0.43
ROLLOUT RESOLUTION 64 x 64
OURS
0.45
0.87
0.27
0.97..03
0.46
0.76+.05
MUNET
0.70+.25
0.72+.17
0.294.10
0.97=.02
0.43+.04
0.77+.05
FNO
1.22
0. 1..18.2
0.79
0.72 23
0.794.05
0.2 :.10
RESNE
8.114
0. #:15
12.23 13.91
0.08 1::12
0.53
0.64 1:.0
FACTFORMER
1.194.21
0.23 #.23
0.54
0.87 +: 1..08
4..4
0.54
DINo
1.12
0.03
0.99
0.42+.30
1.40
0.04
DPOT-S
3.63
0.03
9.42
0.32
DILRESNBT-UT
2.60
0. 0.122.14
1.40
0.08
LATROL1-64-FAE
1.38
0. 0.17
LATROLI-64-E2E
1.02
0.33
Table 7: Rollout performance across different numbers of learned latent fields and coarsening
factors r. We report MSE and correlation (Corr. of field variable for each method. The rollout length is steps for and 128 for ID. Values bold show best result at each rollout resolution.
KF
ID
RMSE +
CORR. 1
RMSE
CORR.
ROLLOUT RESOLUTION 32 32
1.263 + 0.23
0.143 + 0.157
0.6 + 0.319
0.766 + 0.332
n 16
1.295 1 0.188
0.147 1 0.118
0.192 1 0.227
0.658 1 0.33
ROLLOUT RESOLUTION 64
T1
1.23 +: 0.21
0.214 + 0.215
1.17 +.026
0.868 4: 0.192
n = 16
0.69 + 0.25
0.69 + 0.24
0.28 + 0.126
0.95 + 0.039
30

---

Figure 13: A snapshot of extracted ea fields for KFF.
Figure 14: A snapshot of extracted c fields for ID.
31

---

HRE (ouns
unUnet
mfc 0
The (sooudss)
0.50
4,00
5.00
12:00
1.00
0.75
10066
0.50
0.25
1.0
Oonclatos
0.6
0.5
0.6
Stepo
Figure 15: Comparison of mUnet emulators and HREs on the KS task. The reference resolution
was 2048, and we used r = 64 for a rollout resolution of 32. We used r' = I, meaning our HREs encoded full-resolution PDE fields from the reference simulation.
HRE (ours)
musst
DilResNes
DINo
Letroll-32-FAB
StowNet
PNO
PactFormer
DPOT-S
K3 m(x.
Tine (ssornde)
KP
Thoe (swondg
11
2.13
4.38
00208
831
0.00
2:33
6:55
551
2.5
1.5
3.0
0.0
1.2
0.5
0.4
130
130
119
110
45
Figure 16: Comparison of SineNet, mUnet and HRE emulators on KF (32²). We use the SineNet-8 model with the same training parameters for their INS task [hang al. 2024]. We excluded this baseline from further analysis due to its unfavorable cost-accuracy trade-off. While provides slight improvement accuracy, is several orders of magnitude slower than mnet.
32

---

Table 8: Physics and c field ablations. We measure the rollout performance of the models KFF (32²) for 64 timesteps and compare the final RMSE and correlation values.
KF
RMSE
CORR.
OUR MODEL
0.71 + 0.23
0.68 +:: 0.21
MUNET W PHYSICS
0.989 + 0.208
0.455 = 0.213
OURS W/O PHYSICS
0.881 0.21
0.563 0.215
MUNET
1.085 0.22
0.345 0.21
HRE (ours)
In Ctet tr plasies
Ours w/o plysins
K7
(wooodd)
Thoo (peoodd))
0.00
2:13
4.25
6,36
R.1
0.00
2.1
4.6
0.88
8.5
1.00
3.00
0.75
0.72
0.25
0.25
Orevbation
140
130
110
130
Stepe
Stape
Figure 17: Performance of ablated models KF 32
Table Rollout performance of HRES vs. mUnet for varying IC distributions. The in-distribution training data were generated with Umax 4.2, and we vary this initial peak wavenumber to evaluate the generalization performance of our model versus mUnet. The burn-in time is calculated by scaling the default value for Umax 4.2 by the inverse ratio of the chosen peak wavenumber; all other parameters remain the same. Rollout and target resolutions are 322. We report RMSE and correlation (Corr.) of the field variable u after 64 rollout steps across increasing umax values, Best performance in bold.
4.2
6
METHOD
RMSE 1
CORR. f
CORR.
RMSE
CORR.
RMSE
CORR.
HRE (OURS)
0.15
0.98 1.
0.292.0e
0.96
0.65
0.85
1.01
0.71
MUNET
0.30
0.90...00 .00
0.58+..4
0.78
0.74
0.75
1.054.68
0.61
33

---

Ground Truth HRE-32 (ours)
mDnnet
FNO
DilResNet
FaetFormer
10
1KE "
10
104
1HE 9-
20
23
2
22
24
29
21
25
Figure 18: Additional TKE analysis for ID rollouts of 96 (Resolution 32
Ground Truth HRE-32 (ours)
mUnet
FNO
DilResNet.
FactFormer
10a
104
107
10a
10
136
102
2
21
2
20
2
29
Figure 19: Additional TKE analysis for KF rollouts 64 (Resolution 322).
34

---

alfari-512 LaidEFFA
Latad-E2E
PasPPemmed112
PastPormess
(wendds)
X3
(peordd
(wncands)
(eeoodd))
3.00
6:38
0.03
2.13
4.20
8.36
8.31
0.00
1.0
0.00
1.97
3.00
5.00
7.87
1.3
1.5
1.0
1.0
1.0
1.0
$4000
40000
40000
320000
20000
38000
30000
10000
10000
Stogg
Theee Strpe
Stops
Throe Stege
Figure 20: Comparison of HRE with mUnet and FactFormer operating on high-resolution data and mUnet with super-resolution decoder for the KF (left) and ID (right) tasks. Rollout resolutions 642 for KF and 32 for ID for model, mUnet+SR and Latroll models.
35

---

Prediction
Prediction
Prediction
1.0
u(x, t)
1.0
u(x, t)
1.0
u(x, t)
0.8
0.8
1
0.6
0.6
0.2
0.2
2
2
0.0.0
0.4
0.8
0.0.0
0.2
0.4
0.6
1.0
0
0.25
0.50
0.75
1.00
x
Ground Truth u(x, t)
Ground Truth u(x, t)
Ground Truth
1.0
1.0
u(x,) t)
0.8
0.8
1
0.6
0.6
1
0.2
0.2
2
-2
0.0.0
0.2
0.4
0.6
0.8
0.0.0
0.2
0.4
0.6
1.0
0
0.25
0.50
0.75
1.00
Difference
Difference u(x, t)
Difference u(x, t)
1.0
u(x, t)
x10
x10
1.0
0.8
0.8
0.6
0.4
0.2
0.2
1
0.0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
-2
0
0.25
0.50
0.75
1.00
x
Figure 21: HRE-based prediction high-resolution KF fields with rollout resolution 642 and target resolution 5122 with 32 timestep intervals.
36

---

Prediction u(x, t)
Prediction
Prediction
1.0
1.0
u(x, t)
1.0
u(x, t)
0.8
2
0.8
0.6
0.5
-1
0.2
0.2
-2
-2
0.0.0
0.2
0.4
0.6
0.8
0
0.25
0.50
0.75 1.00
0.0
0.2
0.4
0.6
0.8
1.0
x
x
Ground Truth u(x, t)
Ground Truth u(x, t)
Ground Truth
u(x, t)
1.0
1.0
1.0
0.8
2
0.8
1
0.6
0.6
0
-1
-1
0.2
0.2
-2
-2
0.0.0
0.2
0.4
0.6
0.8
0
0.25 0.50 0.75
1.00
0.8.0
0.2
0.4
0.6
0.8
1.0
Difference u(x,
Difference
Difference
1.0
x10
1.0
u(x, t)
x10-1
1.0
u(x,
0.8
0.8
1
0.6
2
0.6
0.4
-2
11
-1
0.2
0.2
-2
0.0.0
0.2
0.4
0.6
0.8
1.0
0
0.25
0.50
0.75
1.00
-3
0.8.
0.2
0.4
0.6
0.8
1.0
x
x
Figure 22: HRE-based prediction of high-resolution ID fields with rollout resolution 642 and target resolution 5122 with 32 timestep intervals.
37

---

ours (19.5M params)
FNO (18.7M params)
mUnet (19.4M params)
0.012196x 0.1029
0.005894x + 0.1973
y= 0.010496x + 0.2163
Inference Time (8)
Inference Time (5)
Time
2.5
2.0
2
1.5
1.0
0.5
Y-Intercept: 0.1029
Y-Intrceep:: 0.1973
Y-mteroept: 0.2163
0.0
100
200
200
400
100
200
Batch Size
Batch
Batch Size
DilResNet (0.6M params)
DINo (0.7M params)
FactFormer (3.5M params)
0.532771x 14.1634
0.8
0.005203x 0.0870
0.012000x 0.1280
Inference Time (s)
150
Inference Time (s)
Inference Time (s)
1.5
125
0.6
1.0
75
0.4
50-
0.2
0.5
25
**
Yintercopt:: 14.1634
Yinterropp::
0.0870
Y-inerrepp:: 0.1280
0
0.0
100
200
50
100
50
100
Batch Size
Batch Size
Batch Size
Figure 23: Inference times for varying batch sizes. We choose rollout resolution (32*) for ID, KF and KFHR tasks and computed rollouts over 64 time steps.
HRE ours (19.5M params)
FNO (18.7M params)
mUnet (19.4M params)
2.5
0.038717x + 0.5240
12
0.021890x + 0.2045
= 0.035084x 0.6697
Inference Time (s)
Inference Time (6)
Inference Time (6)
2.0
8
1.5
10-
1.0
4
0.5
Yontercept:: 0.5240
Y-inteccet:: 0.2045
Y-ntercept: 0.6697
0.0
20
200
100
200
400
Batch Size
Batch Size
Batch Size
DilResNet (0.6M params)
DINo (0.7M params)
FactFormer (3.5M params)
1.2
12
0.634047x 1.2601
0.020187x 0.0773
0.038914x 4x 0.7680
Inference Time (+)
1.0
Inference Time (5)
Inference Time
60
0.8
0.6
0.4
4
0.2
2-
Yintercept: 1.2601
Yintercept: 0,0773
Yinterepl:: 0.7680
0.0
50
100
20
40
100
200
Batch Size
Batch Size
Batch Size
Figure 24: Inference times for varying batch sizes. We choose rollout resolution (6442) for ID, KF and KFHR tasks and computed rollout for 64 time steps.
38

---

KS
322
322
642
642
CD
Table 10: Summary of the trained models. We train models for specific rollout resolutions (encoding resolution for DINo and FactFormer) and target resolutions. When the target resolution exceeds the rollout resolution, we train separate super-resolution decoder, indicated by "SR" in model suffixes (except for HRE models, which have their own decoders). When the target resolution smaller than the rollout resolution, we coarsen the outputs to the target resolution using conservative coarsening operators. For clarity, models in the main text are named using their rollout resolution, e.g. mUnet-333 32+SR indicates that the model rolls out at a resolution of 32 Underlined cases are used for the cost-accuracy scatter plot in Fig. [ testing data.
We use a rollout length 96 for ID and 64 for KF cases on in diddsributtion held-our
322
322
/
642
642
322
322
KFHR
642
100
322
5122
5122
7
5122
64
642
7
7
ID
5122
/
V
322
322
642
L
100
322
5122
5122
V
V
2562
256
5122
642
AAA 4
64
7
KF
5122
322
AR I
322
7
7
7
7
642
322
1
Rollout/Enc. Res.
DilResNet-NT
Benchmark
Target Res.
HRE (Ours)
DilResNet-UT
FaetFormer
LatRoll-FAE
LatRoll-F2E
DPOT-S
mUnet
FNO
DINo
39

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We give a detailed description of the state of the field, how our new hybrid representations provide an advance, and what evidence demonstrates the improvements achieved. We clearly state the scope of our achievements, which include 5 challenging tasks.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We explicitly include a section on limitations and future work. We discuss the need for more tasks, other grids, other PDES, etc.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA].
Justification: We do not introduce new theoretical results.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We give all hyperparameters for all methods and data generation procedures.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: release the full source code and configuration files to run the experiments.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: all details are specified in the main and/or supplementary text. We use established benchmark datasets most cases.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We provide error bars initial conditions and random seeds in several instances, and indicate the nature of these in each case. We do not carry out statistical tests of any null hypotheses.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
40

---

Answer: [Yes]
We provide training times and inference speeds, and specify the hardware resources used.
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http://ppw...aaeep..c..//aaa//aa
Answer: [Yes]
Justification: We reviewed and abided by the NeuRIPS code of ethics.
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA].
Justification: We foresee negative impacts specific to our work on hybrid PDE emulation, in this sense the same position relative potentila impacts as a generic optimzation algorithm for neural networks mentioned an example in the guidelines.
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA].
Justification: There are no such risks. We generate our own data by simulation.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cite the relevant papers for software packages and algorithms we used.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA].
Justification: We do not introduce new assets.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA].
Justification: No crowdsourcing or human subjects.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country or institution) were obtained?
Answer: [NA].
Justification: No IRB approval required.
16. Declaration of LLM usage
41

---

Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]- Justification: No LLMs used.
42


Reviews:
[{"review_id": "JdxqXtVIrC", "reviewer": "Reviewer_kTab", "summary": "In this paper, the authors propose a hybrid latent representation for learning based PDE emulations. The proposed representation enhances the coarsened PDE fields with structured latent variables extracted from high-resolution inputs. The hybrid representation is validated on multiple PDE emulation tasks and the experiments results show the representation can improve the skills of the PDE emulators.", "questions": "- How is the generalization ability of the proposed method?\n- Regarding to the datasets, what are the differences between the training and test trajectories?\n- How would the proposed method be applied to scenarios with more complex geometries (e.g., arbitrary geometries in real-world cases)/boundary conditions? \n- The test cases shown in the paper are mainly fluid dynamics problems. How would the proposed method perform on more broad of problems e.g, deformable body simulations, fluid-solid coupling problems?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "- Strengths\n\nThe paper is well written and organized. The latent representation is an important topic for PDEs learning. In this paper, the authors discuss the existing fixed resolution and unstructured learned representations in detail with qualitative analysis. The proposed hybrid representation is a straightforward combination of these two, i.e., construct two branches at both the encoder and decoder for fixed and unstructured representations. The two branches merged in the processor for multistep rollout. In general, the idea is sound and intuitive, the experiment results show the proposed method can keep lowest RMSE and highest correlation during the long rollouts among various baselines.\n\n- Weaknesses\n\nThe test cases are restricted to 1D and 2D cases with simple geometries. It would be more convincing if there were more large-scale/complex 3D scenarios.", "was_revised": true, "final_justification": "The authors responses address my concerns, I will keep my scores unchanged.", "rebuttal": "> The test cases are restricted to 1D and 2D cases with simple geometries. It would be more convincing if there were more large-scale/complex 3D scenarios.\n\nWe agree this limitation is worth mentioning and have revised our discussion accordingly. We do wish to point out that our compressible Navier-Stokes example is, despite its simple boundary conditions and geometry, more complex than most PDEs considered for emulation.\n\n>How is the generalization ability of the proposed method?\n\nWe have added new experiments to measure the generalization performance of HREs and baseline emulators to out-of-distribution initial conditions. These experiments show that HREs continued to outperform baselines as the distribution of initial conditions used for testing moved further away from the training distribution. Full details are given in the response to GR4j.\n\n>Regarding to the datasets, what are the differences between the training and test trajectories?\n\nInitial conditions for training and test trajectories are simply different samples from the same distributions, as is standard practice when training and evaluating neural PDE emulators, except when specifically measuring generalization performance (as we now do as well). We now clarify this point in Appendix C, which also contains full details of the IC distributions for each task.\n\n>How would the proposed method be applied to scenarios with more complex geometries (e.g., arbitrary geometries in real-world cases)/boundary conditions?\n\nThe incorporation of complex geometries and boundary conditions is an important aim for neural emulation, but is essentially orthogonal to our aim of learning PDE system state representations that improve rollout stability and accuracy. Previous studies have provided boundary/geometry information as additional network inputs for PDE emulation on grids [1] or irregular meshes, or for weather and climate modeling [2-3]. The same can easily be done for HREs, with a trained encoder used to transform high-resolution boundary information into coarse-resolution inputs for the processor network. In the case of a GNN backbone architecture, [4] proposes a conservative GNN message passing method to satisfy conservation laws; combining this with suitable downsampling would be sufficient to implement HRE for problems with complex geometries. We now mention this in our discussion.\n\n[1] Wandel et al. \"Learning Incompressible Fluid Dynamics from Scratch-Towards Fast, Differentiable Fluid Models that Generalize.\" ICLR, 2021.\n\n[2] Lam, et al. \"Learning skillful medium-range global weather forecasting.\" Science, 2023.\n\n[3] Watt-Meyer et al. \"ACE2: accurately learning subseasonal to decadal atmospheric variability and forced responses.\" npj Climate and Atmospheric Science, (2025).\n\n[4] Horie et al. \"Graph Neural PDE Solvers with Conservation and Similarity-Equivariance.\" ICML, 2024.\n\n> The test cases shown in the paper are mainly fluid dynamics problems. How would the proposed method perform on more broad of problems e.g, deformable body simulations, fluid-solid coupling problems?\n\nWhile beyond our present scope, this is an interesting direction and there are no fundamental limitations to applying HREs outside fluid dynamics. The only real requirement is the availability of a coarsening operator that respects the desired conservation laws, and clearly this is the case for many solid-body and mixed-phase problems. In addition to the proposed examples, we note that biological or chemical systems described by kinetic rate equations coupled to advection and diffusion processes are ideal candidates, as these systems are naturally endowed with conservation laws encapsulated in the stoichiometric matrix of the rate equations. We now mention this as a possible future direction."}, {"review_id": "keBPu1BBRF", "reviewer": "Reviewer_7kAd", "summary": "This work introduces a novel paradigm for neural operator modeling that integrates the strengths of fixed-resolution and unstructured latent-space emulation. The central idea is to enable the enforcement of physical principles—such as conservation laws—even within the latent space. The approach is validated through comparative studies against several operator learning baselines in two-dimensional settings, demonstrating promising performance.", "questions": "1) Regarding the training procedure described on pp. 3–4, and specifically the loss function in Eq. (8): it would be helpful to explicitly clarify which components are being trained and which are held fixed during optimization.\n\n2) Could the authors provide additional details on the decoder's training strategy? Is it trained independently of the loss in Eq. (8), or is there interaction between the two stages? Presenting a decoder-specific training objective in the form of an explicit loss function, akin to Eq. (8), would significantly improve transparency.\n\n3) Further explanation is needed to understand how the decoder, as defined in Eqs. (13) or (15), ensures the conservation of physical quantities. In particular, how are the operators $\\mathcal{D}_x$, $\\mathcal{E}_x$ from Appendix A.3.1 chosen? How is the vector potential $a$ (Appendix A.3.2) learned, and how is it utilized in subsequent stages? How is the obtained information used to benefit the encoder? A more formal treatment, such as specifying loss terms that enforce these properties, would be highly beneficial.\n\n4) Section 6 would benefit from a clearer specification of the PDE setups: What types of domains, boundary conditions, and initial conditions are employed?\n\n5) Could the authors comment on the computational costs (in terms of time and memory) of HRE compared to FFRNE or ULSE in the training phase? Are the costs comparable, or is there a noticeable trade-off?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* Novelty: The proposed hybrid latent representation combines elements of structured and unstructured modeling in a creative and original way.\n* Significance: The paper engages thoroughly with recent literature, situating its contributions in the context of current trends in operator learning.\n* Experimental validation: The experimental section is well-designed, with clearly documented ablation studies that provide useful insights into the method’s components.\n\nWeaknesses:\n* Clarity: Several parts of the paper would benefit from more detailed exposition, particularly regarding the algorithmic implementation. These points are elaborated upon in the questions below.\n* Missing comparisons: Given recent developments in operator learning (OL), it would be valuable to compare the proposed method against foundation models for OL, e.g., [1,2], even if these are pretrained on different tasks. Can such models still capture relevant physics (e.g., in the KS case)? Furthermore, comparisons to Transformer-based methods tailored for fluid dynamics, such as [Alkin et al., 2024], are notably absent for the Navies-Stokes examples and should be addressed.\n\nOverall, the paper presents an original idea and is generally well written. If the authors can improve clarity and provide more comprehensive comparisons to recent OL approaches, I would be inclined to revise my evaluation in favor of acceptance.\n\n[1] Herde, Maximilian, et al. \"Poseidon: Efficient foundation models for PDEs.\" Advances in Neural Information Processing Systems 37 (2024): 72525–72624.\n\n[2] Hao, Zhongkai, et al. \"DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training.\" Proceedings of the 41st International Conference on Machine Learning, PMLR 235:17616–17635, 2024.", "was_revised": true, "final_justification": "The authors have convincingly addressed my questions regarding architectural details and have provided additional comparisons with recent baseline models. Consequently, I have raised my score accordingly.", "rebuttal": ">Clarity...\n\nWe agree and have revised (see below).\n\n>Missing comparisons: ...\n\nTo include comparisons to OL and FMs as suggested, we added DPOT on our ID and KF tasks. Starting from DPOT-S, whose 31M parameters make it comparable to our HREs, we fine-tune on either a smaller dataset (DPOT-S(FTS), 58 ICs for KF/90 for ID) or a larger one (DPOT-S(FTL), 116/180), for 500 epochs (learning rate 1e-4, other hyperparameters as in official implementation).\n\nDPOT's framework and pretraining data confer several advantages. For KF, pretraining on incompressible NS with forcing could improve short-rollout performance, while pretraining on compressible NS with decaying turbulence could aid long rollouts for ID and CD tasks (e.g., 64 steps for ID). Conditioning on 10 previous steps could compensate for coarsening.\n\nNonetheless, our task-specific HREs outperform DPOT:\n    \n**HREs vs Finetuned DPOT for KF u field**\n\n| KF | Tsteps=16|  | Tsteps=32 |  | Tsteps=64 |  |\n|--|----|----|----|----|----|----|\n|  | RMSE ↓ | Corr. ↑ | RMSE ↓ | Corr. ↑ | RMSE ↓ | Corr. ↑ |\n| **Rollout Resolution 32 × 32** |  |  |  |  |  |  |\n| Ours | **0.03**±.01 | **0.99**±.00 | **0.13**±.04 | **0.99**±.01 | **0.71**±.10 | **0.69**±.27 |\n| DPOT-S(FTL) | 0.14±.02 | 0.99±.01 | 0.88±.12 | 0.75±.06 | 1.63±.30 | 0.15±.28 |\n| DPOT-S(FTS) | 0.17±.02 | 0.99±.01 | 0.95±.16 | 0.70±.07 | 1.62±.27 | 0.10±.23 |\n| **Rollout Resolution 64 × 64** |  |  |  |  |  |  |\n| Ours | **0.02**±.01 | **0.99**±.00 | **0.06**±.03 | **0.99**±.00 | **0.43**±.17 | **0.88**±.07 |\n| DPOT-S(FTL) | 0.26±.03 | 0.99±.01 | 1.33±.14 | 0.57±.07 | 1.67±.28 | 0.0±.22 |\n| DPOT-S(FTS)| 0.27±.03 | 0.98±.01 | 1.27±.17 | 0.56±.08 | 1.60±.28 | 0.0±.11 |\n   \n**HREs vs Finetuned DPOT for ID u field**\n| ID | Tsteps=16|  | Tsteps=32 |  | Tsteps=64 |  |\n|--|----|----|----|----|----|----|\n|  | RMSE ↓ | Corr. ↑ | RMSE ↓ | Corr. ↑ | RMSE ↓ | Corr. ↑ |\n| **Rollout Resolution 32 × 32** |  |  |  |  |  |  |\n| Ours | **0.05**±.01 | **0.99**±.00 | **0.12**±.03 | **0.99**±.00 | **0.30**±.09 | **0.96**±.03 |\n| DPOT-S(FTL) | 1.17±.19 | 0.97±.01 | 2.25±.39 | 0.53±.19 | 2.24±.6 | 0.17±.21 |\n| DPOT-S(FTS) | 1.13±.16 | 0.96±.02 | 2.34±.39 | 0.48±.21 | 2.31±.64 | 0.12±.20 |\n| **Rollout Resolution 64 × 64** |  |  |  |  |  |  |\n| Ours | **0.03**±.01 | **0.99**±.00 | **0.09**±.03 | **0.99**±.00 | **0.28**±.16 | **0.93**±.12 |\n| DPOT-S(FTL) | 1.15±.19 | 0.96±.02 | 2.13±.37 | 0.56±.2 | 1.9±.51 | 0.16±.22|\n| DPOT-S(FTS)| 1.13±.17 | 0.96±.02 | 2.31±.4 | 0.5±.20 | 2.09±.59 | 0.14±.22 |\n\nWe revised sections 4, 7, 8 and 9 to include DPOT, and describe how OL/FMs relate to our approach. An HRE FM is possible, though training an encoder across domains, grids/meshes and variable fields would require careful design choices.\n\n>Comparisons to Transformer-based methods tailored for fluid dynamics...\n\nWe agree UPT is a promising approach, but now already have 9 baselines (including DPOT and the two from 6mAC's review). Regarding transformer-based methods tailored for fluid dynamics, we note FactFormer and DPOT fit this description. Despite UPT's impressive results on Lagrangian flows and adaptive-mesh Eulerian pipe flows, it has not yet produced competitive results for Navier-Stokes on a regular grid in a closed system.\n\n>Regarding the training procedure described on pp. 3–4, and specifically the loss function in Eq. (8): it would be helpful to explicitly clarify which components are being trained and which are held fixed during optimization.\n\nWhile we previously stated on line 131 \"We therefore jointly train the encoder and processor...\", this did not clarify whether the decoder was also trained using this loss. We revised to emphasize that:\n* The decoder is not trained (or used) when minimizing $\\mathcal L_c$ in eq. 8.\n* All encoder/processor parameters are optimized when minimizing $\\mathcal L_c$ in eq. 8.\n* The processor and encoder are frozen when training the decoder (see below).\n\n>additional details on the decoder's training strategy...\n\nThe decoder is trained independently with frozen encoder/processor (L157), for efficiency and so reconstructing high-res outputs doesn't conflict with long-rollout accuracy. We include the decoder loss:\n\n$$ \\mathcal L_{\\mathcal D} = \\sum_{k=1}^K \\left\\lVert X_{t+k} - \\mathcal D_c(\\tilde x_{t+k}, \\tilde c_{t+k}) \\right\\rVert_2^2 = \\sum_{k=1}^K \\left\\lVert X_{t+k} - \\mathcal D_c \\circ \\mathcal M_c^{k}\\circ \\mathcal E_c(X_t) \\right\\rVert_2^2$$\n\nWe also include additional details in Appendix A.\n\n>Further explanation is needed to understand how the decoder, as defined in Eqs. (13) or (15), ensures the conservation of physical quantities. In particular, how are the operators $D_x, \\mathcal E_x$ from Appendix A.3.1 chosen?\n\nWe revised the main text as well as A.3.1 as follows:\n\nThe fixed interpolation operator $\\mathcal D_x$ and the form of $\\mathcal B_c$ are PDE-specific, and chosen to enforce conservation laws and/or incompressibility as hard constraints.\n\nFor KS and compressible flow, conservation of mass, energy and momentum are enforced by using an $r\\times r$ local average as $\\mathcal E_x$, and bilinear interpolation as $\\mathcal D_x$. The network learns an additive correction to the updated fields $\\mathcal B_\\psi$, from which the local $r \\times r$ average is removed (eq. 13) to produce a conservative update $\\mathcal B_c$. To remove the local average, this average is first computed using $\\mathcal E_x$, then removed at high resolution using $\\mathcal D_x$.\n\nFor incompressible flow, $\\mathcal E_x$ applies face-averaging (Fig. 6) as in [1], while $\\mathcal D_x$ interpolates horizontal velocities horizontally and vertical velocities vertically (Fig. 7). Both operations trivially conserve the incompressibility condition (consider the divergence of each cell in Fig. 7, upper left).\n\n>How is the vector potential (Appendix A.3.2) learned...\n\nWe heavily revised the \"decoder\" section and A.3.1-A.3.2, adopting new notation and giving clear, explicit definitions. To summarize:\n\nVector potentials are used only for incompressible flow, where they are simply output channels of the processor and decoder networks. These vector potentials are immediately used to compute divergence-free additive updates to the velocity fields and not used for any further computations (except backpropagation). There is no loss term for enforcing incompressibility, as this is a hard constraint (in [2], which used the same trick, loss-terms were to be less effective). Concretely, the processor outputs latent variables $\\tilde c_t$ as well as a vector potential $a_t$, and we use the curl operator to compute $\\tilde x_t = (\\tilde u_t, \\tilde v_t) = \\nabla \\times a_t + (\\tilde u_{t-1}, \\tilde v_{t-1})$. The decoder outputs only a vector potential $A_t$, from which we compute $X^\\prime_t = (U_t, V_t)^\\prime_t = \\mathcal D_x(\\tilde x_t) + \\nabla \\times A_t$, where $\\mathcal D_x$ is fixed interpolation operator showin in Fig. 7. For periodic boundary conditions, it follows trivially that a periodic vector potential gives a zero-mean velocity update, so momentum conservation is also hard-constrained.\n\n> clearer specification of the PDE setups\n\nWe added BCs, domain geometry and ICs for all tasks (in the appendix due to space constraints):\n\nL866: The spectral density of the initial conditions is sampled from the log-normal distribution that peaks at wavenumber of 7, then we generate random initial conditions from the normal distribution and filter the fields with FFT to match the spectral density.\n\nL867: We generate 128 experiments using 2048 × 2048 cells in $[0, 2\\pi]\\times[0, 2\\pi]$ for the solver. \n\nL883: We generate the initial conditions using the same procedure as the KF case, with two differences: the peak wavenumber is chosen as 4.2, and the forcing term is set to f = 0. We generate 200 experiments using the same parameters and spatial resolutions as KF, except for these modifications.\n\nL888: Initial velocities are generated as a superposition of sinusoidal waves,\n    $\\mathbf{v}(\\mathbf{x}, t = 0) = \\sum_{i=1}^{N} \\mathbf{A}_i \\sin(k_i \\mathbf{x} + \\phi_i)$\n    where $N=4$, $\\mathbf{A}_i=\\bar{v}/|k_i|$, $\\bar{v}$ is max. initial velocity determined using the Mach number as $\\bar{v}=c_s M$, where $c_s$ is the speed of sound. Wavenumbers are given by $k_i=2\\pi i/L$, where $L$ is domain length, and the phases $\\phi_i \\sim \\mathcal{U}[0, 2\\pi]$. Density and pressure are initialized uniformly.\n\n> computational costs (in terms of time and memory) of HRE compared to FFRNE or ULSE\n\nWe now report this for all models in Appendix E. HREs have similar training costs to mUnet for longer rollout training. For ID at rollout resolution of $64^2$, mUnet requires ≈7 hours to train vs. ≈9.5 for HREs on 8 A100s with batch size 16/GPU. For long rollouts, most HRE computations are at coarse resolution, but for short rollouts mUnet is considerably faster: at rollout length 1 mUnet needs ≈1 hour for 100 epochs on 4 A100s, and HREs ≈5 hours on 8 A100s. DINo trains in 7 hours on a single V100 at the same input resolution, but lacks accuracy. To summarize, HRE's superior cost-accuracy tradeoff incurs moderate training costs, which we now explicitly report.\n\n> formatting of the document\n\nWe fixed compression errors. The PDF is searchable while images remain high-quality.\n\n> update the bibliography\n\nAdjusted along with others (Stachenfeld 2022, Li 2021, Wang 2020, Rahaman 2019, Lippe 2023, ...).\n\n[1] Kochkov et al. \"Machine learning–accelerated computational fluid dynamics.\" PNAS, 2021.\n\n[2] Wandel et al. \"Learning Incompressible Fluid Dynamics from Scratch-Towards Fast, Differentiable Fluid Models that Generalize.\" ICLR, 2021."}, {"review_id": "Z6yejQlnFY", "reviewer": "Reviewer_Gr4j", "summary": "This paper introduces Hybrid Representation Emulators (HREs), a novel architecture for emulating partial differential equations (PDEs). The core idea is to augment the low-resolution physical fields typically used in neural emulators with a set of spatially-structured latent variables. These latent variables are generated by a separate encoder network that processes higher-resolution input data, aiming to capture fine-scale information that is lost during simple coarsening. \n\nThe paper demonstrates that this hybrid approach can achieve a more favorable cost-accuracy trade-off compared to several baselines, including convolutional (mUnet), Fourier-based (FNO), and other latent space models, across a range of benchmarks. The decoder uses a curl formulation for incompressibility of fluids flows.", "questions": "My evaluation of this paper hinges on whether the authors can convincingly argue that the HRE provides a fundamentally more robust and generalizable approach, rather than being a more complex model that overfits to an in-distribution data set.\n\n- Can the authors provide examples showing out-of-distribution generalization? This could involve testing on a trajectory with a different Reynolds number, forcing term, or other physical variance than seen during training. \n\n- Which physical constraints are actually used in each of the test cases? The curl is only applicable for divergence free flows, which other constraints are the authors using?\n\n- Figure 17 shows the unconstrained HRE model producing large spikes in the L1 norm, while the mUnet baseline remains stable. This suggests that the hybrid representation may be prone to overfitting. Can the authors comment on this behavior?\n\nSome additional minor suggestions :\n- Plot legends: The dark dotted line representing the ground truth in the L1 norm plots (e.g., Fig. 2, Fig. 15) is a crucial reference but is missing from the legends. \n- Decoder loss: The loss function used to train the decoder is not explicitly defined in the main text. \n- L180 \"an ULSE\" should be \"a ULSE\"as in L181\n- the PDF format is strange: it doesnt allow for search or text selections", "limitations": "The last section gives an outlook, but does not properly discuss limitations of the method. This should be added.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n-  The core concept of a hybrid, spatially-structured latent representation and is novel as far as I can tell. It directly addresses a known limitation of standard fixed-resolution emulators (loss of fine-scale information) while avoiding the scalability issues of fully unstructured latent space models. \n\n-  The authors conduct an interesting set of experiments on multiple fluid systems (Kolmogorov flow, decaying turbulence, etc.) and a KS case. They compare their method against a set of state-of-the-art baselines, and provide a good view of its performance.\n\nWeaknesses:\n\n- A primary concern is that the HRE architecture, with its additional encoder and slightly larger parameter space, is essentially acting as a powerful data augmentation that overfits to the dynamics present in the training distribution. The following points are related to this.\n\n- As detailed in Table 3 and acknowledged in the limitations, the test set is drawn from the same data-generating distribution as the training set (technically, this should be called a validation set). This setup does not measure the model's ability to generalize to out-of-distribution scenarios (e.g., different Reynolds numbers or forcing terms), which is a key goal for neural emulators. Without such testing, it is difficult to disentangle true modeling improvements from overfitting to the specific characteristics of the training distribution. The strong performance reported could be a result of the encoder learning features highly specific to the training data.\n\n- The stability of the emulators appears to be limited by the training horizon. In Figure 2, the HRE's performance visibly degrades after approximately 32 steps. Given that the longest training curriculum uses rollouts of K=16, this suggests the model has learned stability primarily within its training horizon. This would be especially important for the 2D cases (1D KS is shown for more steps in Figure 8).\n\n- The ablation study in Figure 17 is interesting but also raises questions. The unconstrained HRE (\"Ours w/o physics\") shows larger spikes in the L1 norm. In contrast, the unconstrained mUnet (the \"MUNET\" baseline, which is also unconstrained) is far more stable. Since both processors are based on a U-Net architecture, the addition of the latent channels lead to such dramatic spikes acts similarly to overfitting patterns. This suggests the hybrid representation may be inherently prone to overfitting.\n\n- The \"physical hard-constraints\" seem to be directly based on previous work, and are not properly explained in the text. It is implied that all fluid flow test cases use divergence freeness, but while the paper initially talks about general PDEs, the final method seems to be tailored to 2D fluid flow cases. I would recommend to make clear in title, intro and intro ...", "was_revised": true, "final_justification": "The authors have provided additional information and clarifications. I have updated my score accordingly. I don't think it's necessarily a strong accept, but the paper makes the case for an interesting latent-space variant in the context of PDE learning.", "rebuttal": "> A primary concern is that the HRE architecture, with its additional encoder and slightly larger parameter space, is essentially acting as a powerful data augmentation that overfits to the dynamics present in the training distribution. The following points are related to this.\n\nWe agree that generalization capability is an important aspect of PDE emulation. However, we disagree with two aspects of how this question is posed:\n1. It is not necessary for an emulator or other ML model to generalize outside its training distribution to be useful. There are many cases in scientific and industrial ML applications where efficient, accurate processing of novel in-distribution inputs is precisely what is required, both for PDE emulation (e.g. choosing subtle optimizations for an airfoil design) and for other problems in ML (facial recognition under controlled conditions). Generalization is therefore indeed a \"key goal,\" but not the only goal.\n2. This is not really the accepted use of \"overfitting.\" If a model performs well on held-out, in-distribution data it is not overfitted.\n\nTo measure generalization capabilities of HREs,  we carried out new experiments to evaluate generalization performance. We used models trained on the ID task at rollout resolution of $32^2$. We changed the initial peak wavenumber for the log-normal distribution, which changes the distribution from which the initial conditions are sampled. Changing the initial peak wavenumber $u_\\text{max}$ from the value of 4.2 used during training changes the decay rate of turbulence in the system. We rolled out for 64 timesteps and calculated the RMSE and correlation values for these modified ICS, for our HREs and for mUnet, a frequently employed FRNE baseline with strong performance across tasks. These experiments thus tested whether the advantage of HREs over baseline methods would persist when testing generalization to new ICs never seen during training.\n\nUnsurprisingly, the performance of both models began to degrade as the distribution of ICs used for testing moved further from the training distribution. However, our HRE approach continued to consistently outperform mUNet. These results show that the spatially structured latent variable fields learned by HREs do not overfit or overspecialize to the training distribution in a way that reduces performance on out-of-distribution inputs, compared to FRNEs that rely only on coarse features.\n\n**Rollout performance of Ours vs mUnet for changing IC distributions - field variable u, 64 steps**\n| ID | u_max=3|   | u_max=4.2 | | u_max=5 |  | u_max=6 |  |\n|--|----|----|----|----|----|----|----|----|\n|  | RMSE ↓ | Corr. ↑ | RMSE ↓ | Corr. ↑ | RMSE ↓ | Corr. ↑ | RMSE ↓ | Corr. ↑ |\n| Ours | **0.15**±.05 | **0.98**±.02 | **0.29**±.08 | **0.95**±.01 | **0.65**±.31 | **0.85**±.13 | **1.01**±.55 | **0.71**±.34 |\n| mUnet | 0.30±.12 | 0.90±.06 | 0.58±.34 | 0.78±.25 | 0.74±.42 | 0.75±.35 | 1.05±.68 | 0.61±.50 |\n\nWe also carried these experiments out for the DilResNet baseline, yielding significantly worse results. We omit these here for brevity but will add them to the supplementary materials.\n\nWe appreciate the reviewer raising this point, as we believe the inclusion of these generalization tests as a new supplementary figure improves our manuscript.\n\n>The stability of the emulators appears to be limited by the training horizon. In Figure 2, the HRE's performance visibly degrades after approximately 32 steps. Given that the longest training curriculum uses rollouts of K=16, this suggests the model has learned stability primarily within its training horizon. This would be especially important for the 2D cases (1D KS is shown for more steps in Figure 8).\n\nIn general, existing neural PDE emulators can match the reference solution only for a finite time horizon, especially in chaotic systems such as KF. This is true for PDE emulators, weather prediction models, and other dynamical systems. While it would be desirable to have a method that trains on a finite time horizon but stays correlated for far longer, in the general case this remains an open problem and we do not claim to solve it. Rather, we show that HREs can continue to improve from longer training rollouts, while the benefits of these saturate early for previous approaches such as FRNEs (Fig. 4). Based on our results and many other studies, there is no reason to expect that rollouts during training can be avoided. HREs provide a way of making these rollouts computationally affordable at low resolution while still learning to extract and incorporate relevant features from high-resolution input fields.\n\nAlso, while the predicted and reference solutions do eventually decorrelate for all known neural PDE emulators, this does not mean they all become unstable after this occurs. An unstable method would show a diverging RMSE (e.g. DilResNet in Fig. 2, mUnet-512 in Fig. 20), but this is not the case for HREs for any rollout lengths we tested.\n\n> The ablation study in Figure 17 is interesting but also raises questions. The unconstrained HRE (\"Ours w/o physics\") shows larger spikes in the L1 norm. In contrast, the unconstrained mUnet (the \"MUNET\" baseline, which is also unconstrained) is far more stable. Since both processors are based on a U-Net architecture, the addition of the latent channels lead to such dramatic spikes acts similarly to overfitting patterns. This suggests the hybrid representation may be inherently prone to overfitting.\n\nWe agree that the unconstrained HRE exhibits these tendency L1 norm oscillations. Nonetheless, it is more accurate than unconstrained mUnet, not only RMSE and correlation, but also in the tendency L1 norm, especially after 48 steps. The fact that the HRE, with more channels and parameters, is more susceptible to these oscillations is perhaps not unexpected, but the point we wish to emphasize is that the hard physical constraints HRE can incorporate totally suppress these.\n\nWe don't see a connection with overfitting, as these oscillations in the tendency L1 norm do not occur more for validation than for training data. Our new generalization tests (see above) show that HREs are not more prone to generalization failures than the baselines we aim to improve on.\n\nOverall, we agree this point bears further discussion, and now acknowledge in further detail these oscillations and the role of physical constraints in suppressing them in the \"Ablations\" section of our experimental results (starting L343).\n\n> The \"physical hard-constraints\" seem to be directly based on previous work, and are not properly explained in the text. It is implied that all fluid flow test cases use divergence freeness, but while the paper initially talks about general PDEs, the final method seems to be tailored to 2D fluid flow cases. I would recommend to make clear in title, intro and intro ...\n\nWe agree and have thoroughly revised and expanded the explanations of these constraints and the way we impose them on HREs, using improved notation (see response to 7kAd). As noted above, we consider both the divergence-free constraint of incompressible flow, as well as conservation of momentum (all tasks), and of mass and energy (CD).\n\n>Plot legends: The dark dotted line representing the ground truth in the L1 norm plots (e.g., Fig. 2, Fig. 15) is a crucial reference but is missing from the legends.\n\nRevised as suggested.\n\n> Decoder loss: The loss function used to train the decoder is not explicitly defined in the main text.\n\nRevised as suggested, see response to 7kAd.\n\n> L180\n\nCorrected.\n\n> PDF format\n\nCorrected."}, {"review_id": "biEyKLB5OK", "reviewer": "Reviewer_RrQR", "summary": "The paper proposes a new paradigm for constructing neural PDE solver based on coarsened discretization, more specifically, a decoupled encoding framework with physics constraints. Numerical experiments on several turbulent systems demonstrate the effectiveness of the proposed framework.", "questions": "Can the decoder be trained simultaneously with the encoder part, while the processor is trained after the autoencoding model has been trained?", "limitations": "* In current experiments, only problems with structured grid (i.e. uniform equispaced grid) are studied, and it remains unknown how the proposed framework performs on unstructued grid.\n\n* (Minor) Even with the learned encoder/decoder and learned encoding c_t, the information of the state function will inevitably be lost after downsampling from a fine discretization, and sometimes the tiny scale physics are still highly important to the dynamics, e.g. global weather, 3D turbulence", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strength:\n\n1. The method is clearly written and easy to follow.\n\n2. The integration of physics constraints (conservation and divergence-free) to the decoder part is straightforward but effective, which can be helpful for a wide array of other relevant tasks like super-resolution.\n\n3. The idea of combining (non-learned) coarsened field with a learned latent encoding is novel. The empirical performance of the proposed model is strong across various benchmarks.\n\nWeakness:\n\n1. It is not surprising that truncating higher frequency modes promotes global stability when simulating a chaotic system. RMSE might also favor \"blurrier\" results. It would be interesting to study the pattern (e.g. spectrum of prediction) on high-resolution simulation in both short-term and long-term.\n\n2. The paper ignores many prior works on building reduced-order representations and does not compare with any state-of-the-art frameworks simulating PDEs on reduced-order representations, for example, [1], [2], [3], [4] have investigated simulating PDEs in a mesh-reduced space.\n\n[1] Han X, Gao H, Pfaff T, et al. Predicting physics in mesh-reduced space with temporal attention\n\n[2] Zhou A, Li Z, Schneier M, et al. Text2pde: Latent diffusion models for accessible physics simulation\n\n[3] Li Z, Patil S, Ogoke F, et al. Latent neural PDE solver: A reduced-order modeling framework for partial differential equations\n\n[4] Wang T, Wang C. Latent neural operator for solving forward and inverse pde problems", "was_revised": true, "final_justification": "The authors have provided sufficient clarification to address my concerns. Overall, the proposed model demonstrates strong performance and could be of significant interest to the reduced-order modeling community.", "rebuttal": "> physics constraints...can be helpful for a wide array of other relevant tasks like super-resolution.\n\nWe thank the reviewer for making this point, which had escaped our attention. Indeed, a similar approach of employing hard physical constraints can be used for super-resolution without any prediction of temporal dynamics, and we now call attention to this fact.\n\n>It is not surprising that truncating higher frequency modes promotes global stability when simulating a chaotic system...\n\nWe agree, but emphasize the difference in accuracy between our HREs and previous FRNEs operating at the same resolution. While both are stable, ours remain correlated to the reference solutions for significantly longer. We also note that HREs, despite operating at low resolution, do not really remove all information from higher frequency modes, as this can be encoded in the c-fields at coarse resolution. This seems to be the case, given the decoder's ability to reconstruct high-resolution fields.\n\n>RMSE might also favor \"blurrier\" results.\n\nWe agree. This is a well-known limitation of RMSE-based losses, and more generally for deterministic prediction of chaotic systems, that the result becomes blurry when training on long prediction horizons. This is not specific to our method, and indeed, HREs maintain accuracy for longer horizons when evaluating at coarse or fine resolution than comparable baselines (Fig. 2, 5). While the challenge of probabilistic forecasting is essentially orthogonal to the problem we address with HREs, our approach could potentially be used to provide input to a probabilistic decoder (e.g. a diffusion model) and maintaining physical hard constraints while estimating the denoised target signal in a diffusion training loss. We now note this in our discussion.\n\n> The paper ignores many prior works on building reduced-order representations and does not compare with any state-of-the-art frameworks simulating PDEs on reduced-order representations, for example, [1], [2], [3], [4] have investigated simulating PDEs in a mesh-reduced space.\n\nWe agree these references are relevant and now cite them all as suggested. We also carefully considered them as possible additional baseline methods for our experiments. We ultimately decided against directly including them, both due to the already high number of baselines we consider (now 9, including DPOT and the two addressing 6mAC's review), and for additional reasons specific to each method. However, some of our new and previous baselines do incorporate the ideas presented by these studies:\n* [4] represents an ULSE approach similar to DINo, which we already include.\n* [2] is a probabilistic ULSE approach with a diffusion model and conditioning on text inputs, complicating potential comparisons to our deterministic HREs and baselines. Our study's aim is to examine the potential of hybrid latent representations for long-term accuracy and stability in PDE emulation, while probabilistic forecasting addresses a different set of important concerns.\n* [1] proposes an encode-process-decode type of GNN that uses all-to-all temporal attention among the latent variables to propagate in time. Authors train the encoder-decoder and processor separately. Similar to [1], [3] uses the strategy of training an autoencoder first and freezing it to train the propagator. We now include the training mode from [1] and [3] with a frozen autoencoder operating in pure latent space (LatRoll-FAE) as an additional baseline (see response to 6mAC which includes the results for ID and KF cases for rollout resolutions 32^2 and 64^2). While this pure latent representation approach performs comparably to our model for short rollouts, it quickly decorrelates from the reference solution as rollout length increases. We attribute this to two main factors. First, when high-frequency features are present in the data, the lower-dimensional manifold learned by the autoencoder (which is trained without regard to the system's temporal evolution) may not represent the optimal embedding space for long-term emulation accuracy and stability. Second, rollout training helps the propagator correct its predictions and improve accuracy over longer time scales. However, when the system state representation consists only of learned latent variables without coarsened versions of the original PDE fields, the high-dimensional latent representation can diverge from the distribution of inputs the decoder encounters during training, reducing prediction accuracy.\n\n> Can the decoder be trained simultaneously with the encoder part, while the processor is trained after the autoencoding model has been trained?\n\nYes, nothing prevents end-to-end training of HREs, or initial pre-training with an encoder/decoder pair. However, we chose to train the processor and encoder without a decoder, both to benefit from the efficient loss formulation at coarse resolution (eq. 8), and since we wanted to prioritize long-rollout performance over reconstruction of high-resolution features (see L157). We did experiment with alternative training modes (see L136-7), which we found to be less effective, and now describe these possibilities in greater detail, emphasizing there are no technical obstacles to employing these additional loss formulations and training modes with HREs.\n\n> In current experiments, only problems with structured grid (i.e. uniform equispaced grid) are studied, and it remains unknown how the proposed framework performs on unstructured grid.\n\nWe agree that extension of this approach to unstructured meshes would be promising, and nothing in our approach is incompatible with message passing or (cross-) attention on graphs. We now acknowledge this as a limitation of our experimental results so far, and as a promising future direction.\n\n> Even with the learned encoder/decoder and learned encoding c_t, the information of the state function will inevitably be lost after downsampling from a fine discretization, and sometimes the tiny scale physics are still highly important to the dynamics, e.g. global weather, 3D turbulence.\n\nWe agree that the encoder must discard significant information from the high resolution inputs $X_t$, due to the smaller number of degrees of freedom for $(x_t, c_t)$ compared to $X_t$. Nonetheless, it remains an open question whether/how much this actually has to degrade prediction accuracy. For most physical systems, the manifold or attractor of \"realistic\" system state trajectories will be lower than the dimension of the ambient space containing $X_t$, so it should be possible for the encoder to learn an effective representation that is more compact than the original high-resolution fields. At least for the tasks, dataset sizes, resolutions and time scales we examined, HREs seem to be the most effective approach at predicting future high-resolution fields. In comparison, emulators trained only on high- or low-resolution fields yielded inferior performance (Fig. 5)."}, {"review_id": "7nSQMfIhfG", "reviewer": "Reviewer_6mAC", "summary": "The authors present a method for training/rolling out neural emulators that combines the downsampled input field and an encoded representation of the original field. Experiments on coarse grids show that the additional context from high-resolution inputs aids in rollout performance, and operating in the low-resolution regime allows for fast inference.", "questions": "- As stated earlier, I would be happy to improve my rating if the authors can show that their method is better than using an equivalent, but purely latent approach.\n- For Figure 5 (experiments on high-res data), how is the error calculated for Factformer-256 if the training data is 512x512?", "limitations": "Many engineering simulations are mesh-based, which wouldn’t really admit a good averaging operator or perhaps even need coarsening. Some emulators can work with a full-resolution field/mesh, which would theoretically be better than an HRE approach.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n- The problem is framed very well.\n- The proposed method is explained well and makes a lot of intuitive sense.\n- The experiments are executed well.\n- It is good that the authors bring attention to details that are usually overlooked in PDE studies, in particular, the downsampling of simulation data.\n\nWeaknesses\n- I would say the main weakness is a lack of comparison to a convincing ULSE baseline. In particular, the speed/resolution of the HRE method can be achieved by ULSEs, and their proposed limitations don’t seem to be justified. Physics-based losses can be applied after decoding and spatial/structured inductive biases don’t seem to be necessary for PDE learning, as attention-based emulators are nearly SOTA (Transolver, UPT, etc.). If the authors could run a test on comparing the HRE w/ an equivalent architecture where the downsampled field is not used, specifically, use the same encoder to produce c = E(X), use a processor that propagates c instead of both (c, x), and use a decoder that only takes in c to produce X’ = D(c), this would be beneficial. Increasing the latent channels n only improves HRE performance, and at 16 latent channels and 3 physical channels (assume 2D velocity + pressure) is the original downsampled field needed anymore?  \n- At the rollout resolutions that HREs work the best (32^2), the resolution may be too coarse for practitioners to really use it, as many details are lost that may be important. At more useful rollout resolutions (64^2 or 128^2), the benefits are more incremental.", "was_revised": true, "final_justification": "- The perspective taken about downsampling and the resolution of PDE inputs is one that is not typically considered (usually left as a detail in the dataset generation or prior PDE benchmark datasets).\n- The performance gain of using both a latent variable and the coarsened input is good.\n- The clarity of the work could be improved, as it is currently difficult as a reader to understand the main message or the comparisons presented in the figures.", "rebuttal": "> I would say the main weakness is a lack of comparison to a convincing ULSE baseline. In particular, the speed/resolution of the HRE method can be achieved by ULSEs, and their proposed limitations don’t seem to be justified.\n\nWe agree that ULSEs are the fastest methods, since their lack of spatial structure leads to very compact latent spaces and inexpensive rollouts. We now better emphasize this in section 2.2.\n\nHowever, we were surprised by the assertion that a convincing ULSE baseline has not been included. DINo and FactFormer are ULSEs, recently published with multiple strong results.\n\nWe are unaware of ULSE methods likely to be more effective than these on our tasks. Transolver and UPT (see response to 7kAd) are promising and interesting methods, but have not shown strong results on closed NS systems simulated at high resolutions for more than 10 time steps (UPT used a pipe flow, in which errors can be \"washed out\" by the flow, while Transolver only predicted 10 steps of NS). Given our many baselines (now 9) and finite compute, we chose DINo and FactFormer, which already showed strong results on tasks similar to ours.\n\n> Physics-based losses can be applied after decoding and spatial/structured inductive biases don’t seem to be necessary for PDE learning, as attention-based emulators are nearly SOTA (Transolver, UPT, etc.).\n\nPhysics-derived losses are usually proven less effective than hard-constraints [1].\n\n> If the authors could run a test on comparing the HRE w/ an equivalent architecture where the downsampled field is not used, specifically, use the same encoder to produce c = E(X), use a processor that propagates c instead of both (c, x), and use a decoder that only takes in c to produce X’ = D(c), this would be beneficial. Increasing the latent channels n only improves HRE performance, and at 16 latent channels and 3 physical channels (assume 2D velocity + pressure) is the original downsampled field needed anymore?\n\nWe trained these c-field-only models as suggested. To match the inference speed with our HREs, we used the same latent space resolution and total channel counts, and similar parameter counts. We removed the physical constraints from the processor and placed it at the output layer of the network as recommended.\n\nDuring training and inference, this new model takes high-resolution inputs (512x512) and encodes them into the latent space as $c_t = \\mathcal E(X_t)$. The propagator then advances the latent representation as $\\tilde c_{t+k} = \\mathcal M^k(c_t)$, and the decoder maps these propagated fields to a high-resolution vector potential $A^{t+k} = \\mathcal D(\\tilde c_{t+k})$. A final curl layer then calculates the high-resolution velocity fields (512x512) as $X^\\prime_{t+k} = \\nabla \\times A^{t+k}$, satisfying the incompressibility condition. Conservation of momentum is also satisfied since for our tasks the mean momentum is always zero, and this is achieved (as a hard constraint) by using a periodic vector potential. We train by minimizing MSE on high resolution predictions.\n\nWe use the same curriculum $K \\in [1, 2, 4, 8, 16]$, with each curriculum step trained for 100 epochs. We implemented two training strategies:\n1. Train an autoencoder on $X_t$, freeze it then train the propagator with our rollout curriculum (LatRoll-FAE)\n2. Train the entire network end-to-end at each curriculum step (LatRoll-E2E).\n\nFor short rollouts, the performance of these models is comparable to HREs. For longer rollouts, HREs increasingly outperform them.\n\nWe trained LatRoll-FAE and LatRoll-E2E on two datasets (ID and KF) with rollout resolution 32×32. The following tables show the RMSE and correlation values for rollout timesteps 16, 32, and 64 for our HREs versus these models.\n        \n**Rollout performance of Ours vs Latent Rollout Models without coarse fields for KF dataset field variable u**\n| KF | Tsteps=16|  | Tsteps=32 |  | Tsteps=64 |  |\n|--|----|----|----|----|----|----|\n|  | RMSE ↓ | Corr. ↑ | RMSE ↓ | Corr. ↑ | RMSE ↓ | Corr. ↑ |\n| Ours | **0.03**±.01 | **0.99**±.00 | **0.13**±.04 | **0.99**±.01 | **0.71**±.10 | **0.69**±.27 |\n| LatRoll-E2E | 0.09±.02 | 0.99±.01 | 0.80±.16 | 0.66±.08 | 1.23±.18 | 0.20±.17 |\n| LatRoll-FAE | 0.09±.02 | 0.99±.00 | 0.87±.18 | 0.52±.11 | 1.15±.18 | 0.14±.17 |\n\n\n**Rollout performance of Ours vs Latent Rollout Models without coarse fields for ID dataset field variable u**\n| ID | Tsteps=16|  | Tsteps=32 |  | Tsteps=64 |  |\n|--|----|----|----|----|----|----|\n|  | RMSE ↓ | Corr. ↑ | RMSE ↓ | Corr. ↑ | RMSE ↓ | Corr. ↑ |\n| Ours | **0.05**±.01 | **0.99**±.00 | **0.12**±.03 | **0.99**±.00 | **0.30**±.09 | **0.96**±.03 |\n| LatRoll-E2E | 0.12±.03 | 0.99±.01 | 0.85±.15 | 0.63±.22 | 1.20±.39 | 0.27±.44 |\n| LatRoll-FAE | 0.13±.04 | 0.99±.01 | 0.78±.11 | 0.67±.22 | 0.96±.23 | 0.39±.36 |\n\nAs mentioned (L133, L137, L157), we find training a latent space rollout model by directly minimizing MSE on high-resolution targets creates a conflict between long-term accuracy (our main aim) and high-resolution reconstruction. Our simple loss on $x_t$ (eq. 8) avoids this, prioritizing long-rollout accuracy as desired. These new results provide key empirical evidence to support these arguments.\n\nAdditionally, training on MSE for high-resolution predictions is costlier than our HREs, since we use low-resolution targets.\n\nWe thank the reviewer for raising this important question, and helping us to avoid overlooking an important ablation experiment. We now add this ablation experiment to the manuscript, including for the KFHR and CD cases. Overall, we find these new results significantly strengthen our manuscript and underline the importance of the hybrid nature of our learned representations for PDE system states.\n\n>At the rollout resolutions that HREs work the best (32^2), the resolution may be too coarse for practitioners to really use it, as many details are lost that may be important. At more useful rollout resolutions (64^2 or 128^2), the benefits are more incremental.\n\nWe see no fundamental reason why learned latent representations must lose important details at coarser rollout resolutions, so long as the number of latent channels is sufficient and the original PDE fields $X_t$ exist on a manifold/attractor of sufficiently low dimension in their original ambient space. Our results show that--at least for the tasks we investigated--HREs can indeed operate efficiently and accurately at coarse resolutions, while nonetheless producing accurate outputs at high resolution (Fig. 5, 21-22). Our KS experiments (Fig. 15) also show that far from being useless, a rollout at resolution 32 can accurately predict future states at resolution 2048, but only if learned representations are used (compare HRE and mUnet performance). Overall, the key requirement seems to be a high ratio between the rollout and input resolutions rather than a low rollout resolution per se, but further experiments should investigate this more systematically.\n\nULSEs can also be considered an extreme case of spatial coarsening with 1x1 rollout resolution (excluding the channel dimension), but clearly these can produce useful and accurate outputs at high resolution.\n\nWe agree that HREs' dependence on rollout resolution, input resolution, and domain size bears further investigation. Because our models are convolutional, we can apply them to larger physical domains at the same resolutions, e.g. $1024^2$ input / $64^2$ rollouts vs. $512^2$ /$32^2$. While time, space and computational constraints prevent us from fully answering these questions in the present study, we now better emphasize them in our discussion.\n\n> For Figure 5 (experiments on high-res data), how is the error calculated for Factformer-256 if the training data is 512x512?\n\nWe calculate RMSE for FactFormer-256 after coarsening reference fields to $256^2$. This essentially gives FactFormer-256 an unfair advantage over other methods whose predictions were evaluated at the higher resolution of $512^2$, though its results are still not competitive with HREs or with mUnet + superresolution (Fig. 5). We now explicitly describe how RMSE was computed for this model.\n\n> Many engineering simulations are mesh-based, which wouldn’t really admit a good averaging operator or perhaps even need coarsening. Some emulators can work with a full-resolution field/mesh, which would theoretically be better than an HRE approach.\n\nIndeed, state-of-the-art weather forecasting networks use spherical mesh representations of atmospheric states [2-4]. These methods use coarse-resolution internal representations of original mesh, which are computed using well-established procedures for graph-based message passing between meshes of different resolutions. However, unlike our HREs, mesh-based weather-prediction models have not yet included coarsened input fields, hard physical constraints, or multistep rollouts within a latent space. Similarly, UPT [5] and related methods use a cross-attention-based perceiver layer to drastically decrease the number of tokens relative to the input dimensionality. Given the ability of these \"coarsening-like\" operators for meshes and point clouds, we see no fundamental technical obstacles preventing the implementation of HREs for these spatial discretizations. We now note this in the \"Limitations and Future Work\" section of our discussion.\n\n> Line 184\n\nCorrected.\n\n[1] Wandel et al. \"Learning Incompressible Fluid Dynamics from Scratch-Towards Fast, Differentiable Fluid Models that Generalize.\" ICLR, 2021.\n\n[2] Lam, et al. \"Learning skillful medium-range global weather forecasting.\" Science, 2023.\n\n[3] Oskarsson, et al. \"Probabilistic weather forecasting with hierarchical graph neural networks.\" NeuRIPS, 2024.\n\n[4] Price et al. \"Probabilistic weather forecasting with machine learning.\" Nature, 2025.\n\n[5] Alkin et al. \"Universal physics transformers: A framework for efficiently scaling neural operators.\" NeuRIPS, 2024."}]

Author rebuttal:

Meta-review: This paper introduces a hybrid latent representation for PDE emulation. The hybrid latent representation consists of coarsened PDE fields and spatially structured latent variables (encoded by mUnet). The method is validated on multiple PDE emulation tasks and demonstrate that it can achieve a more favorable cost-accuracy trade-off compared to several baselines, including convolutional (mUnet), Fourier-based (FNO), and other latent space models.

The reviewers generally recognize novelty, significance, and the clarity of the paper. The reviewers also raised several concerns and questions. During the rebuttal, the authors addressed these concerns by including comparison with finetuned DPOT, novel IC distributions, an ablation where downsampled field is not used, etc. The reviewers are generally satisfied with the rebuttal. During the camera-ready, the authors are required to incorporate these additional experiments and suggested improvements into the camera-ready version of the paper. In addition to the reviewers' suggestions, I suggest that the authors put more experiment results (including some results in the appendix and the additional experiments in the rebuttal) into the main text of the paper, while some parts can be written in a more concise way (e.g., baseline description and task description).
Decision: Accept (poster)