EddyFormer: Accelerated Neural Simulations of Three-Dimensional Turbulence at Scale
Authors: Yiheng Du
Aditi S. Krishnapriyan
Venue: NeurIPS 2025 poster ()
Keywords: ML for PDEs
fluid dynamics
turbulence
simulation
large-eddy simulation
Transformer
large-scale
spectral methods
Primary area: machine_learning_for_sciences
TL;DR: We develop an architecture, EddyFormer, to accelerate the resolution of three-dimensional turbulent fluid flows.

Abstract: Computationally resolving turbulence remains a central challenge in fluid dynamics due to its multi-scale interactions. Fully resolving large-scale turbulence through direct numerical simulation (DNS) is computationally prohibitive, motivating data-driven machine learning alternatives. In this work, we propose EddyFormer, a Transformer-based spectral-element (SEM) architecture for large-scale turbulence simulation that combines the accuracy of spectral methods with the scalability of the attention mechanism. We introduce an SEM tokenization that decomposes the flow into grid-scale and subgrid-scale components, enabling capture of both local and global features. We create a new three-dimensional isotropic turbulence dataset and train EddyFormer to achieves DNS-level accuracy at 256^3 resolution, providing a 30x speedup over DNS. When applied to unseen domains up to 4x larger than in training, EddyFormer preserves accuracy on physics-invariant metrics-energy spectra, correlation functions, and structure functions-showing domain generalization. On The Well benchmark suite of diverse turbulent flows, EddyFormer resolves cases where prior ML models fail to converge, accurately reproducing complex dynamics across a wide range of physical conditions.

Eddy Accelerated Neural Simulations of Three-Dimensionall Turbulence at Scale
Yiheng Du UC Berkeley yihhenguu@berkeley.y edu
Aditi S. Krishnapriyan UC Berkeley, LBNL aditik10berkeley edu
Abstract
Computationally resolving turbulence remains central challenge in fluid dynamics due to its multi-scale interactions. Fully resolving large-scale turbulence through direct numerical simulation (DNS) is computationally prohibitive, motivating data- driven machine learning alternatives. In this work, we propose EddyFormer, a Transformer-based spectral-element (SEM) architecture for large-scale turbulence simulation that combines the accuracy of spectral methods with the scalability the attention mechanism. We introduce an SEM tokenization that decomposes the flow into grid-scale and subgrid-scae scale components, enabling capture of both local three-dimensional isotropic turbulence dataset
and global features. We create and train EddyFormer to achieves DNS-level accuracy at 256 resolution, providing DNS. When applied to unseen domains up 4x larger than in training. Eddy Former preserves accuracy on physics-invariant metrics- spectra, correlation functions, and structure functions-shone alization. On The Well benchmark suite diverse turbulent flows, EddyFormer resolves cases where prior ML models fail converge, accurately reproducing complex dynamics across a wide range of physical conditions, Code and project page are available
30x speedup
domain gener-
wtt.:778...iittttti github..hh..... wtt.tt......ddyyr..e.e.ee...eeee io/eddyformer
Introduction
Numerical simulation of fluid systems essential for understanding and predicting complex flow phenomena (Van Dyke, 1982 in engineering and the natural sciences. Many critical applications, such as acrodynamics and weather forecast, rely on computational fluid dynamics (CFD) to simulate these flows. However, capturing flow patterns in turbulent flows (Pope 2001 remains fundamental challenge. In turbulence, eddies-cohent energy across wide range of length scales. The extent of these scales is determined by the Reynolds number, Re, a dimensionless quantity which represents the ratio of inertial to viscous forces. Fully resolving a turbulent flow requires R/ resolution, making DNS prohibitively expensive at high Reynolds number [Moin and Mahesh| 1998), even with supercomputers (Yeung et al.) 2012 2015].
swirling structures contiuuuusly interact and transfer
Turbulence exhibits a characteristic cascade: energy is injected large scales, transferred through the inertial range, and dissipated at the smallest scales where viscosity dominates. Statistical descriptions of turbulence assumes that the smallest scales become independent of the large-scale flow structures, which is known as the Kolmogorov's 's similarity hypothesis [Pope] 2001]
sufficiently high Reynolds number, the statistics of small-scale motions have a universal form that is uniquely determined by viscosity and energy dissipation rate.
Based on this principle, large-eddy simulation (Smagorinsky) 1963 (LES) resolves only the large- scale structures while using theoretical models on smaller, subgrid scales (SGS). By filtering the governing equations, LES captures dominant flow features while relying on "closure" models to ap- proximate unresolved dynamics at SGS. Despite its successes, LES faces challenges in wall-bounded and anisotropic turbulence [Piomelli 1999], particularly in balancing accuracy and efficiency.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Simulation time: 152 sec.
Simulation time: 4.86 sec.
2.50
2.25
2,00
1.75
1.50
1.00
0.75
0.50
(a) 256 numerical solution (16.3% L2 error).
(b) EddyFormer's prediction (18.2% L2 error).
Figure I: Vorticity magnitude of forced turbulence with Reynolds Re 94 at 5. EddyForme reproduces the accuracy level and invariant flow statistics of a direct numerical simulation (DNS) at 256 resolution, computed with highly efficient seeudo-spectral schemes (Canuto, 2007). The reported L2 error is measured against a reference DNS at 384 resolution. When benchmarked on single NVIDIA A100 GPU, EddyFormer is 30x faster, achieving essentially real-time simulations.
Recent trends in ML have explored data-driven turbulence simulation [Li et al., 2020 Kochkov al. 202T], with spectral-basee based neural network surrogates showing promising accuracy. Similar to spectral methods (Canuto 2007]. they approximate the solution using spectral bases and parameterize example, Fourier neural operators (FNOs) [Li et al. 2020] provide expressive model for learning mappings on structured grids. However, scaling them to large problems with a wide range of fine-scale eddy dynamics remains challenging, parameterizing large Fourier kernels is inefficient [Qin et al. (2024]. Likewise, Transformer architectures [Vaswani et al., 2017] face certain obstacles in fluid systems, as the quadratic growth of attention with mesh resolution may constrain their scalability in practice. These limitations suggest exploring architectures that balance efficiency with scalability to handle large-scale turbulence.
dynamics through spectral transformations.
Our contributions. We propose EddyFormer, machine learning model for large-scale fluid flow simulation that captures multi-scale interactions through learned representations the flow. The input is tokenized using the spectral/hp element method (SEM) [Karniadakis and Sherwin 2005), which partitions the domain into coarse elements with spectral expansions in each subdomain. This tokenization combines the strengths of spectral modeling and scalability: local spectral expansions provide highly expressive representations, while each coarse element, i.c., a SEM token, yields compact sequence that makes the attention mechanism efficient for capturing long-range interactions.
EddyFormer consists of two complementary streams: an SGS stream ($3.I), which captures localized eddy interactions; and an LES stream ($3.2). which models global, coherent structures. The LES stream is parameterized using multi-hean attention applied the filtered SEM tokens, significantly reducing computational complexity compared to self-attention on the full mesh. For the SGS stream, Kolmogorov hypothesis suggests that small-scale dynamics can be modeled universally. We use spectral convolutions with localized kernel to model the subgrid-scale dynamics.
EddyFormer demonstrates strong performance across wide range of fluid flow problems, outperform- ing state-of-the-art ML models, as well as efficient numerical solvers at the same level of accuracy. For three-dimensional isotropic turbulence ($4.1 Fig. M. EddyFormer achieves the accuracy of direct numerical simulation (DNS) at 256 resolution while running 30x faster, reducing error by 30% compared to neural operator and Transformer baselines. When trained on two-dimensional turbulence ($4.2). it generalizes to domains up to four times larger than those seen in training by applying attention masking at test time, suggesting learns domain-inen nomain-independent flow dynamics that can be adapted to general domain geometries. Benchmarks on other turbulent flows from The Well Ohana al. [2025] show that EddyFormer accurately resolves turbulence driven by a wide range of physical conditions, including cases where other ML baselines fail to converge.

---

Related works. We briefly review ML approaches for turbulence here, with more comprehensive
survey of TTansformer-basee models and application domains presented in ŞA Early attempts focused
on learning data-driven turbulence closures for Reynolds-averaged Navier-Stokes (RANS) | Tracey
al. 2015| Ling al,, 2016 and large-eddy simulation (LES) [Beck et al. 2018 Font et al. 2021
List et al. [022]. Wang et al. (2020) adopted the LES decomposition and modeled the mean and
fluctuating fields separately. More recent works instead focus on direct simulation of turbulent
flows using advanced neural networks architectures (Li et al. 2020 Tran et al.) 2021 Wen et al.
2022 Kossaifi et al. 2023], showing promise but not yet consistently matching the accuracy of numerical solvers [Spalart] 2023 McGreivy and Hakim 2024]. - A promising alternative, "learned correction" [Kochkov et al. 2021| Shankar et al., 2025]. uses low-accuracy solvers to improve
accuracy. However, most previous works remain limited to small-scale wwo-dimensional flows.
2
Background
The Navier-Stokes (NS) equations [Kundu al. 2024] describe the conservation of mass, momentum, and energy in fluid systems, capturing flows in wide range of physical conditions. We focus on the incompressible flow described by the Boussinesq approximation for negligible density variations: ou
+ u Vu = v 1 Vv.
v 0,
where utt; 2) E TR33 is the velocity field on domain Q2 p is the fluid density, Vp is the pressure gradient, and и is the kinematic viscosity. Given the initial velocity field u(0;) .) and certain boundary conditions on u and p, the task is to simulate the field u(t;-) at a later time step t.
Large-eddy simulation. We adopt the methodology of large-eddy simulation (LES) in our model architecture. Classical LES aims to directly simulate the large-scale unsteady turbulent motions, while modeling the effects of small motions in theory. Given a (spatial) filter G(:,)) on Ω the velocity field u(t:-) is decomposed into its filtered field n(t:) -) = G u(t;) )) and the corresponding subgrid-scale field (SGS) u' = u ú. The filtered velocity field i is governed by the filtered momentum equation,
du Ot
1
+ ú Vū 2 V
Vp. Po
where Tay u.u ū, is the residual stress (or SGS) tensor. This filtered equation is unclosed, in terms of the SGS tensor being approximated based on the filter G and the filtered field u. Many an eddy viscosity Taj R -2v,Sej. where S is the resolved
classical turbulence models express strain rate tensor and ve is the eddy viscosity to be modeled by the corresponding SGS model.
Spectral element method. We use the standard spectral element method (SEM) parameterize the solution field -tt:-). The physical domain Z is discretized into H structured elements Ω = U HSn where each element Q2 is mapped to an unit cube through a geometrie transformation. We assume a uniform mesh with cell size A and denote the local coordinates on Ω4 as 0 S 5% 5% Ex. S 1.
The local field un (t: { on 22 is expanded using higher-ordes bases. Let { hm be set of one- dimensional basis on [0, 1]. their full tensor product forms a multi-dimennional basis set (E()) = Th (Ex). The local field (t; :) is then approximated as,
un u, (t: Ex) = Σ üh (EA),
PPlSM
where ú/, E 1RM are the expansion coefficients of the h'th element. In contrast to spectral methods, the standard spectral basis must be modified to enforce certain order of continuity across the element boundaries. In this work, we consider CtD -continuous boundary-inters interior decomposition. Given a set of orthogonal polynomial [1 (Pm Jmn their corresponding modal p-type basis is defined as,
m = 0, < m < M, @m (E) = (5- Pm-
1-
E
m- = M,
where do and Φa are the shared left and right boundary modes, respectively.
In this work, we implement the (shifted) Chebyshev polynomials Tm, (x) cos(m arccos 1)), and 4
the Legendre polynomials Pm (2) mm dem (x=2 - 2x)"" as the orthogonal polynomial set pm.
3

---

(a) EddyFormer architecture
(b) SEMConv block
(c) SEMAttn block
ULES
UgGs
Linear
Linear
Concat
Linear
Gaussian quadrature
Self-attention local coords.
weights
Eleeent-- wise MLP
Element- wise MLP
b(x)
Numerical convolution
SEMConv
SEMConv
Rotary PE
LayerNorm
SEMAttn
y.
Kernel coef.
Global coords.
SEMConv
Large-eddy scale
Sub-grid scale
Our proposed architecture: EddyFormer. (a) The PDE initial conditions are interpolated using spectral clement methods (SEM). The large-eddy simulation (LES) field and subgrid-scale (SGS) field are transformed through the LES stream ($3.2) and SGS stream ($3.1). respectively. The SGS stream consists of (b) SEM-based convolution blocks model local eddy dynamics, while the LES stream consists of (c) SEM- based attention blocks to capture long-range dependencies.
Figure 2:
3 Model design choices for EddyFormer
We introduce the EddyFormer architecture (Fig. 2). EddyFormer parameterizes the solution field using SEM bases defined in Eqn. (3). For a fixed domain discretization f = U with H N elements and P = 3 modes per element, EddyFormer maps the spectral representation of the initial condition un to the prediction u(t; -) at fixed time based on chosen SEM basis c
u(0;)
Σ upp &
uf. L u(t;-) on Uh.
(5)
PI
PlEM
where n' E R3N M the output of the model.
EddyFormer decomposes the output as the sum of an LES- part ULES and the residual field USGS. The residual coefficients úsGs are unfiltered and parameterized by an SGS stream ($3.1). For the LES filter, we simply use a spectral cutoff filter with kmax < M modes on each axis. parameterizing the I.ES-fittred coefficients ÚLES E R3N by an LES stream ($3.2) In C.2.1 we perform an ablation study on the LES-SGS splitting with different mmxx to verify effectiveness.
SGS stream:
(1) - (1) - usGS
ULES + USGS.
LES stream:
u
(1)
(1) - uLES
3.1 SGS stream for local motions
The input is first interpolated onto the chosen SEM basis o and then projected to the initial feature, ( Winu(0; :), and the output is the projection of the final feature, UsGs Wscsu((.) (?)) In each layer the locals fields u receive information from the LES stream and then perform convolutions:
((+1) u
u ( + FFN(SEMConv, (u + (())
where FFN is two-layer feed-forward network applied the collocation points independently. and é is a learnable scalar. SEMConv is our SEM-based convolution block, as introduced below.
SEM-based convolution. We extend the powerful spectral convolution et 2020) as the basic block in our architecture. The SEMConv block parameterizes convolution in SEM bases. For a given

---

field u with d components, we parameterize its convolution with compact kernel k that is supported on a window of size s,
-f J
SEMConv(u)(x')
k(x)u((x= x)" dx.
(8)
Note that 0 is hyperparameter to be determined. We found that small value of (compared to the domain length-scale) is usually expressive enough. Due to the success of Fourier factorization techniques (Tran et al. 2021]. we also restrict the convolution kemel to axial form,
k(x, y.
Σ km [e/mm//s clmmpls einmz/et?"
where km E Cdxdx3 are the learnable Fourier coefficients. The convolution computed by directly evaluating Eqn. (8) using numerical quadrature rules (see Algorithm [I] Since the number of modes per element is small compared to the total number of modes, the computational cost of SEMConv remains comparable to that of FFT-based convolution with global kernels (see Tab. I for comparison).
Inclusion of LES features. The inclusion of LES features the SGS stream, Eqn. 7 is justified by the turbulence cascade. Large eddies transport the turbulent kinetic energy downwards, while smaller eddies primarily dissipate that energy. Classical closures- -uch Smagorinsky [1963] and Spalart and Allmaras [192]] -99fect using the resolved large-scale flow field. Similarly, we also find that the inclusion of LES features into the SGS features is crucial for EddyFormer's performance, but not vice versa.
reffeect this hierarchy by parameterizing subgrid stress T in Equ. (2)
3.2 LES stream for global motions
The initial LES feature (") is spectrally truncated to kmax modes, and the output is the projection of the final feature, ULES = (L), The local fields n are regarded as SEM tokens. In each layer 1, the SEM tokens are globally mixed with other tokens first, and then locally transformed. Specifically,
(1) (1)
u + SEMAttn, (u ())
(10)
(1+1)
0% + FFN(SEMCOnA, (a )),
where SEMAttn is our SEM-based self-attention block, which introduce below.
SEM-based attention. We utilize the attention mechanism to model interactions between SEM tokens. In SEMAttn, the SEM tokens új are mixed based on their local coordinates. Specifically, these features are projected using convolutions, /\/k/V = SEMConv O/K/////////V (ú("), and the SEMAttn defined by mixing all features on the same local coordinate & using self-attention mechanism, SEMAttn, (E)) = Σ SoftMaxx ( (E) u (E)) (l)(E)),
(11)
R'SH
which is implemented by performing self-attention on each collocation point independently. Note that SEMAttn strictly preserves the continuity of the input features, regardless of the discretization scheme of SEM representation. Similar standard Transformers, we also add learnable bias terms, apply layer normalization and perform position encoding to the key and query features.
In contrast mesh-based Transformers, the sequence length, or the number of SEM tokens, equals the number of spectral elements N which is orders of magnitude smaller than the number of resolved modes 3 3 This avoids the unfavorable quadratic scaling of standard attention mechanism.
Rotary position encoding. While vision Transformers Dosovitskiy et al. 2020 are insensitive to position encoding, we find that turbulence simulation relies on substantial positional information in SEMAttn. Inspired by [Su et al. [2024), we rotate the key and query fields based on their global coordinates, namely, for the k thth channel, the rotation along x axis is defined by,
Q/K u h.2k (z, y, 2) + cos(kz)u u h.2k (I. 2) sin (kx) u h,2k+ 1 (I. 2,2,,
Q/K
Q/K
(12)
Q/K U 2,2k 1 (x. y, 2) +- sin(kx)
Q/K ( 2,2k (x. y. 2) + cos (kar) i 1,2k+ (I. v)
Q/K
The rotation along y and z axes are similar, and the final results are concatenated together. Due the properties of rotation, it's apparent that the SEMAttn layer is also strictly translational invariant.
We don't 't apply normalization on other hidden features, since it was found to hurt model performance. In order regularize gradients, we initialize all convolution kernels with a random variable of magnitude 10-?
5

---

(3)
Relative error (%) 4
(b)
Point-wise correlation t
(c)
Structure function Sy
1.0
-0.6
30
-0.4
0.8
20
0.6
-0.1
10
0.2
1
2
9.
10
Flow simulation time (sec)
Radical distance r (m)
DNS 963
DNS 1283
DNS 1923
DNS 256
F-FNO
EddyFormer
Figure 3: Performance of models and DNS on Re94 ($4.1) 3D homogeneous isotropic turbulence fluid flow. The models learn to correct DNS at 96 resolution, and we report evaluation metrics relative to a reference DNS at 384 resolution. EddyFormer achieves the accuracy of DNS at 2563 resolution in terms of (a) relative L2 error for S 5 and (b) correlation for < 10 which measures field similarities; for (c) longer prediction rollouts up to 20 seconds, EddyFormer also captures the third-order structure function, Eqn. (18), which measures the velocity skewness and indicates the energy transfer across different scales. In contrast, the baseline F-FNO shows higher rollout error and a mismatched structure function.
4 Experiments
implementation of EddyFormer available at lattps://github com/ASK con/ASK-Berkell Bew.eeeeeeeeeeyrrrrrreeeeee and Py Torch [Paszke al. 2019) version is provided via NVIDIA PhysicsNeMo bectps://githing com/ NVIDIA A physicsnemo) three-dimensional turbulent flows that are maintained by an artificial forcing. We evaluate the models using a set of comprehensive metrics (see [ for details). In $4.3 we benchmark EddyFormer on The Well [Ohana al. 2025), large-scale fluid simulation dataset under diverse flow conditions.
The JAX BBaddbury et al. 2018
$4. and $4.2 we consider two- and
4.1 Homogeneous isotropic turbulence
We consider three-dimensional homogeneous isotropic turbulence, Re94, We use an isotropic forcing scheme the lowest Fourier modes, which has been extensively used in turbulence modeling [Lang- ford and Moser 1999, Bazilevs et al.) 2007 Yeung et al.) 2015). Specifically, constant power forcing f(x) is applied on the lowest velocity modes [k]
1, in order to maintain the turbulence:
Ou
P
u Vu = vV f(x), f(x)
Σ
e
kx
(13)
Ōt
E
where = 0.01 is the kinematic viscosity, Pin = 1.0 is the input power, and E =1=12 = ûk th the kinetic energy in lowest forced modes where |k| The task predict the velocity field after At = 0.5. This flow (see Fig. [ for vorticity rendering) at equilibrium has Reynolds number Re at Taylor microscale. The dataset is recorded 96* resolution. We generate the dataset using a high-ace accuracyy pseudo- spectral direct numerical simulation (DNS) at 384 resolution, which corresponds to a Kolmogorov scale of 0.5n. Detailed dataset and flow statistics can be found in $B
For FdryFrrmer, we use 83 elements and 133 modes per element, resulting in approximatly ~ 0.1% interpolation error Re94. Detailed model architecture is described in Tab. 7 The L.ES filter size is set to kima 5 in order to balance accuracy and efficiency (see C.2.1 for ablations on kima)).
Direct prediction ("pure ML") benchmark. We first benchmark EddyFormer in purely learned setting, where the task is to directly predict the one-step velocity field u without using any coarse initial guess from numerical solver. Baseline models include the classic ResNet [He et al., 2016), TF-
Net Waan et al. 2020 which performs filter-based splitting, the popular neural operators (FNO LLi jet al. 2020) and F-FNO Tran al. [202T)) I),, and competitive Transformers for partial differential
equations (GNOT Hao et al. 2023 and AViT McCabe et al. 2023 )

---

0.53
0.25
0.30
0.00
(a) 384 numerical solution.
(b) EddyFormer's prediction.
(c) F-FNO's prediction.
Figure 4: Visualization of the Q-criterion for Re94 ($4.1) test sample at 5. The Q-criterion. Eqn. (99), identifies vortex structures where rotational motion dominates over strain, with the red volume in (a) reference DNS solution highlighting the vortex cores; (b) EddyFormer successfully captures both vortex cores, while (c) F-FNO fails to resolve the vortex at the center of the domain.
Every model is trained for 15k steps using the time-averaged one-step relative error,
Lg = E, uo U fa (u,) ( |u++all.
(14)
where fo is neural network parameterized by 0. Training details and loss curves for different models can be found in $C.2 Tab. T compares the one-step error and the training speed each model.
EddyFormer achieves lower errors, combined with compact model size and competitive training speed, outperforming all other baseline models.
Table 1:
Direct prediction on Re94 model size, training time per iteration, and one-step relative emro. EF (cheb) and EF (leg) stands for our model EddyFormer with Chebyshev and Legendre basis.
ResNet
TF-Net
FNO
F-FNO
GNOT
AVIT
EF (cheb)
EF (leg)
# of params.
2.3M
6.3M
17.6M
1.7M
6.2M
6.0M
2.3M
2.3M
Train speed
1.2s/it
3.0s/it
1.5s/it
1.5s/it
7.4s/it
1.0s/it
2.7s/it
1.8s/it
Test error
87.2%
27.7%
23.1%
22.5%
69.7%
26.6%
8.61%
8.52%
Learned correction from a coarse numerical solver. Next, we use the learned correction ("hybrid model") approach [Kochkov al. 2021). which uses the output of low-resolution DNS efficient initial solution and trains the model to predict the missing fine-scale information. This provides good starting point for ML. The model additionally receives the output, U of coarse solver, and predicts its error with respect to the high-resolution reference data U At. Similarly, the loss function is also defined the time-averaged relative 1.2 error,
(15) where LCa(u, u²) u' + affo (u, u*) is the correction the coarse solution u*. Here, we choose = 10 which corresponds to an priori estimate of the coarse solver's error. In practice, we find that model performance is not very sensitive the value.
Lg = E, |u LCo(u,, ui 1l)ll u
For baseline comparison, we only consider F-FNO since it was the best baseline in the direct prediction benchmark. All models learn to correct DNS at 963 resolution, which under-resolves the Kolmogorov scale y by a ratio of Each model trained for 15k steps on four NVIDIA A100 GPUs, taking approximately one day wall-clock time to converge.
Results. Fig3 shows the errors and invariant measures for both models, alongside DNS results different resolutions. On the test set, EddyFormer matches the accuracy of DNS at 2563 resolution both the within trained window (Fig. 3a) and later time steps (Fig. (3b). while accurately capturing the invariant structure functions over longer rollouts (Fig. 3c). EddyFormer also successfully predicts all characteristic vortices (Fig.) preserving both the shape and spatial coherence of highly rotational eddies.

---

(a)
2x domain expansion
(b)
4x domain expansion
10a
102
k- ((k)
1 E(b)
10
100
Attention
10-2
DNS 5122 DNS 10242
10-2
DNS 10242
DNS 20482
Trained domain
10-4
EddyFormer
10-4
EddyFormer
F-FNO
F-FNO
10
6
10-6 G
101
101
102
Tested domain
Wavenumber k
Wavenumber
Figure 5: (Left) Illustration of how EddyFormer generalizes to larger domains. Trained on smaller domain, EddyFormer predicts the flow field on a larger domain by applying an attention mask that ensures consistency with the training setup. The context window of each SEM token is constrained to square of length 2x, maintaining consistency between training and test times. (Right) Model evaluation on both 2x and 4x larger domains of KF4 (.2.) The scaled energy spectra, Eqn. (7),. of the EddyFormer's s predictions align closely with the reference high-resolution DNS, while F-FNO does not capture this and shows deviation in high-frequency modes.
In addition its accuracy, EddyFormer offers computational benefit. At test time, it only adds 50% extra computational
Table 2: Simulation times of Re94 and errors w.r. the reference DNS at 384 resolution. EddyFormer corrects the pseudo-spectral DNS 963 resolution.
the base DNS at 96 resolution, accelerating the simulation by 30-fold on single GPU (includ- ing the cost of the coarse DNS, as shown in Fig. [ and is comparable to a high-accuracy DNS at 256 resolution. This speed-up allows for almost real-time high-accuracy simulations on Re94, enabling long-time ensemble rollouts that would otherwise be computationally expensive with high-resolution DNS.
DNS 96*
DNS 256
EF (leg)
Error = Inference time
55.9%
16.3% 152s
18.2%
3.51s
4.86s
4.2 Kolmogorov flow
We also consider the "Kolmogorov" flow KF4, another very popular model for studying turbulence in two dimensions. We use the scalar vorticity form of the Navier-Stokes equation:
Ow
+ Vu = =~= + l (kxr) 0.1w,,
(16)
σt
where is the vorticity on z-axis. The energy is injected by a sinusoidal forcing f(r, yy sin (knkk))) on periodic domain of length 2τ. Following Kochkov et al. [202T), we set the forcing mode k = 4. the viscosity v 10 and add damping term -0. prevent energy accumulation the lowest modes [Chertkov et al. 2007]. The task is to predict the vorticity field w after At =.
We the forcing mode i- 4 and use the vorticity form of the equation. The resulting flow equilibrium has a Reynolds number Re se 800 at Taylor microscale. The dataset is generated using a 40962 pseudo-spectral DNS, which is more accurate than Kochkov et al. [002 and Dresdner et al. 2022 See $B for dataset details and simulation statisties.
All models are trained to correct DNS at 256 resolution. To control the propagation of errors over time, we incorporate 5-step roll-out training approach, which minimizes accumulated error by differentiating through the DNS steps [Kochkov et al.) [022]] The roll-out loss function is applied during the fine-tuning stage, after the one-step loss has converged. See $C.1 for training details.
Results. In Fig. 8a and 8b EddyFormer matches the accuracy of DNS at resolutions between 5122 and 1024a, both within the training window and beyond, terms relative error and correlations with the reference DNS. Over long rollouts (Fig. 8c). EddyFormer also accurately captures the invariant structure function whereas the baseline F-FNO shows significant deviation. Tab. repoorts the kinetic energy, Eqn. 23 showing that EddyFormer conserves energy extended time horizons.

---

(a) Reference solution at t 30. (b) EddyFormer's prediction.
(c) FNO's prediction.
case. EddyFormer successfully predicts the structure of plumes due to buoyancy rise up to a long rollout of = 30, while the baseline FNO fails converge physical solution. See Fig- for the full density evolution over time.
Figure 6: Density of Rayleigh-Taylor instability
Domain generalization. We evaluate EddyFormer's ability to generalize beyond the training domain by testing it on domains that are 4 times larger. EddyFormer requires minimal modification (Fig- 5): we tile more elements proportionally
the domain expansion rate and apply fixed 16 x 16 attention window to the SEMAttn block. This ensures that each spectral element remains the same size between training and test time, while still exchanging information within fixed [0, 2T)² neighborhood. This also ensures that the computation cost only scales linearly with domain size.
Fig. 5 shows the scaled energy spectra both 2x and 4x larger domains. EddyFormer consistently matches the energy spectrum of high-resolution DNS across all length scales, generalizing to larger scales without degradation in accuracy. In contrast, F-FNO, while performing well on the original domain, shows noticeable deviations in the high- sequurryyy modes on the larger domains, indicating a loss in consistency the domain size increases.
4.3 Turbulence beyond homogeneous isotropy
The Well Ohana et 2025 is large-scale physics simulation dataset that contains fluid flows under diverse set of conditions. We benchmark EddyFormer on turbulence-relates subsets, which include nagnetohydrodynamee
shear layer, thermal convection, and buoyancy-drt -drive turbulence. We follow their training protocols and describe experiment details in
Results. The VRMSE scores (Eqn. 25 ) reported in Tab. Across all flow cases, Eddy achieves lower errors and consistently outperforms the baselines across wide range of physically- driven turbulent flows. Fig. visualizes the model predictions the Rayleigh-Tayle instability problem. EddyFormer able to accurately capture the development of turbulent plumes, while all baseline methods fail to converge.
Table 3: Performance The Well dataset: time-averaged one-step VRMSEJ scores test sets. EF (cheb) and EF (leg) denote our EddyFormer model with Chebyshev and Legendre basis, respectively.
Dataset name
FNO
T-FNO
U-net
CNextU-net
EF (cheb)
EF (leg)
MHD_64 shear_flon flow byyleigh_bemaree Beriiihhtayllrraiee instab. 10
0.3605
0.3561
0.1798
0.1633
0.1160
0.1240
1.189
1.472
3.447
0.8080
0.1123
0.0837
0.8395
0.6566
1.4860
0.6699
0.2527
0.1854
10
10
10
0.1115
0.1243
Conclusion. We introduced EddyFormer, an architecture inspired by the principles of large-eddy simulation, and designed for accurate and efficient turbulence simulation. EddyFormer demonstrates high accuracy with a significant acceleration on three-dimensional isotropic turbulence, as well as on flows driven by wide range of conditions, ensuring stable rollouts over extended time periods. Future work will focus on extending EddyFormer more complex flow domains and further scaling it for extreme-scale turbulence.

---

Acknowledgements. This work was supported by Laboratory Directed Research and Development (LDRD) funding under Contract Number DE-AC02-0SCHI This research used resources of the National Energy Research Scientific Computing Center (NERSC), Department of Energy User Facility. We thank Ningxiao Tao and Eric Qu for helpful discussions and feedback on earlier drafts.
Broader impacts. The development of EddyFormer has the potential to impact both scientific research and practical applications. By accelerating turbulence simulations, it can aid solving complex fluid dynamics problems across fields such climate modeling, aerospace engineering, and energy systems, where understanding turbulence is critical. Additionally, the use of machine learning to improve simulation efficiency opens new possibilities for real-time simulations, making feasible to explore more dynamic and complex systems. However, as with any Al-driven technology, it is important consider potential risks, such over-reliance cient validation or unintended biases training data. Moreover, as simulation tools become more accessible, there is responsibility to ensure their ethical and equitable use, particularly in areas like environmental modeling and resource management, where decisions based on inaccurate simulations could have widespread societal impacts.
model accuracy without suffi-
References
Benedikt Alkin, Andreas Fürst, Simon Schmid, Lukas Gruber, Markus Holzleitner, and Johannes Brandstetter. Universal physics transformers. arXiv e-prints, pages arXiv-2402, 2024.
Yuri Bazilevs, Victor M Calo, John A Cottrell, Thomas JR Hughes, Alessandro Reali, and Guglielmo Scovazzi. Variational multiscale residual-ban basee turbulence modeling for large eddy simulation of incompressible flows. Computer methods in applied mechanics and engineering, 19731-4):173- 2717201,112220 2007.
Andrea D Beck, David G Flad, and Claus-Dieted Munz. Deep neural networks for data-driven turbulence models. arXiv preprint I 04482, 2018.
Gal Berkooz, Philip Holmes, and John L Lumley. The proper orthogonal decomposition in the analysis of turbulent flows. Annual review of fluid mechamics, 25(1):539-575, 1993.
Anudhyan Boral, Zhong Wan, Leonardo Zepeda-Núñez, James Lottes, Qing Wang, Yi-fan Chen, John Anderson, and Fei Sha. Neural ideal large eddy simulation: Modeling turbulence with neural stochastic differential equations. Advances in Neural Information Processing Systems, 36:69270-692222 -62283,, 2023.
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URL http: ///ggtttubbbba com/ jax-ml/jax
C Canuto. Spectral Methods: Evolution to Complex Geometries and Applications to Fluid Dynamics. Springer- Verlag. 2007.
Shuhao Cao. Choose a transformer: Fourier galerkin. Advances in neural information processing systems, 34: 24924-244 24940, 2021.
Nithin Chalapathi, Yiheng Du, and Aditi Krishnapriyan. Scaling physics-informed hard constraints with mixture-of-experts. arXiv preprint arXiv:2402. 234112 2024.
M Chertkov, C Connaughton, I Kolokolov, and V Lebedev. Dynamics of energy condensation in two-dimensional turbulence. Physical review letters, 99)))08450,, 2007.
Yuchen Dang, Zheyuan Hu, Miles Cranmer, Michael Eickenberg, and Shirley Ho. Tnt: Vision transformer for turbulence simulations. arXiv preprint arXXiv2207.04611 2401. 2022.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenbom, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1199, 2020.
Gideon Dresdner, Dmitrii Kochkov, Peter Norgaard, Leonardo Zepeda-Nunez, Jamie A Smith, Michael Brenner, and Stephan Hoyer. Learning to correct spectral methods for simulating turbulent flows. arXiv preprint wrXivv2207.0055:: 2022.
Yiheng Du Nithin Chalapathi, and Aditi Krishnapriyan. Neural spectral methods: Self supervuper learning in the spectral domain. The Twelfth International Conference on Learning Representations, 2024.
10

---

Vladimir Sergeevich Fanaskov and Ivan V Oseledets. Spectral neural operators. In Doklady Mathematics. volume 108. pages 9226-S232. Springer, 2023.
Bernat Font, Gabriel D Weymouth, Vinh- Tan Nguyen, and Owen R Tutty. Deep learning the spanwise- averaged navier-stokes equations. Journal of Computational Physics, 434:110199, 2021.
Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying, Yiapeng Dong. Songming Liu, Ze Cheng, Jian Song. and Jun Zhu. Gnot: A general neural operator transformer for operator learning. In International Conference on Machine Learning pages 12556-12569. PMLR, 2023.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
Jacob Helwig, Xuan Zhang, Cong Fu, Jerry Kurtin, Stephan Wojtowytsch, and Shuiwang Ji. Group equivariant fourrer neural operators for partial differential equations. arXiv preprint mxXiv22366.0569, 2023.
Maximilian Herde, Bogdan Raonic, Tobias Rohner, Roger Käppeli, Roberto Molinaro, Emmanuel de Bézenac. and Siddhartha Mishra. Poseidon: Efficient foundation models for pdes. Advances Neural Information Processing Systems, 37:72525- 726224, 2024.
Thomas Hou and Ruo Computing nearly singular solutions using pseudo-spectral methods. Journal of Computational Physics,
2007.
Jinhee Jeong and Fazle Hussain. On the identification of vortex. Journal of fluid mechanics, 285:69-94, 1995.
George Karniadakis and Spencer J Sherwin. Spectral/hp element methods for computational fluid dynamics. Oxford University Press, 2005.
George Em Karniadakis, Ioannis G Kevrekidis. Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physics- informed machine learning. Nature Reviews Physics, 3(6) 440, 2021.
Youngkyu Kim, Youngsoo Choi, David Widemann, and Tarek Zohdi. A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder. Journal of Computational Physics, 451:1 10841, 2022.
Dmitrii Kochkov, Jamie A Smith, Ayya Alieva, Qing Wang, Michael P Brenner, and Stephan Hoyer. Machine leaming-aacelerateed computational fluid dynamics. Proceedings of the National Academy of Sciences, 118 (21):c210178411 18, 2021.
Jean Kossaifi, Nikola Kovachki, Kamyar Azizzadenesheli, and Anima Anandkumar. Multi-grid tensorized fourier neural operator for high-resolution pdes. arXiv preprint arXiv:2310.00120, 2023.
Robert H Kraichnan. Inertial ranges in two-dimensional turbulence. Physics of fluids, 10(7):1417, 1967.
Robert H Kraichnan David Montgomery. Two-dimensional turbulence. Reports on Progress Physics, 43 (5):547, 1980.
Aditi Krishnapriyan, Amir Gholami, Shandian Zhe, Robert Kirby, and Michael W Mahoney. Characterizing possible failure modes in physics-informed neural networks. Advances in neural information processing systems, 34:26548-26560, 2021.
Pijush Kundu, Ira M Cohen, David Dowling, and Jesse Capecelatro. Fluid mechanics. Elsevier, 2024.
J Nathan Kutz, Steven Brunton, Bingni Brunton, and Joshua L Proctor. Dynamic mode decomposition: data-driven modeling of complex systems. SIAM, 2016.
Jacob A Langford and Robert D Moser. Optimal les formulations for isotropic turbulence. Journal of fluid mechanics, 398:321-346,, 1999.
Kookjin Lee and Kevin T Carlberg. Model reduction of dynamical systems nonlinear manifolds using deep convolutional autoencoders. Journal of Computational Physics, 404:108973, 2020.
Zijie Li, Kazem Meidani, Amir Barati Farimani. Transformer for partial differential equations' operator learning. arXiv preprint 2205: 13611, 2022.
Zijie Li, Dule Shu, and Amir Barati Farimani. Scalable transformer for pde surrogate modeling. Advances in Neural Information Processing Systems, 00-220100280330 2023a.

---

Zongyi Li. Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895 2020.
Zongyi Li, Daniel Zhengyu Huang. Burigede Liu, and Anima Anandkumar. Fourier neural operator with learned deformations for pdes on general geometries. Journal of Machine Learning Research, 24(388):1-26, 2023b.
Zongyi Nikola Kovachki, Chris Choy, Boyi Li, Jean Kossaifi, Shourya Otta, Mohammad Amin Nabian, Maximilian Stadler, Christian Hundt, Kamyar Azizzadenesheli, et al. Geometry-informed neural operator for large-scale 3d pdes. Advances in Neural Information Processing Systems, 36:35836-35854, 2023c.
Marten Lienen, David Lüdke, Jan Hansen- Palmus, and Stephan G'innemann. From zero to turbulence: Generative modeling for 3d flow simulation. arXiv preprint
2023.
Julia Ling, Andrew Kurzawski, and Jeremy Templeton. Reynolds averaged turbulence modelling using deep neural networks with embedded invariance. Journal of Fluid Mechanics, 807:155-166, 2016.
Bjöm List, Li-Wei Chen, and Nils Thuerey. Learned turbulence modelling with differentiable fluid solvers: physics-based loss functions and optimisation horizons. Journal of Fluid Mechanics, 949:A25, 2022.
Huakum Luo, Haixu Wu, Hang Zhou, Lanxiang Xing, Yichen Di, Jianmin Wang, and Mingsheng Long. Tran- solver+:: An accurate neural solver for pdes on million-scale geometries. arXiv preprint arXiv:2502.02411 2025.
Michael McCabe, Bruno Régaldo-Saint Blancard, Liam Holden Parker, Ruben Ohana, Miles Cranmer, Alberto Bietti, Michael Eickenberg, Siavash Golkar, Geraud Krawezik, Francois Lanusse, et al. Multiple physics pretraining for physical surrogate models. arXiv preprint urXiv:2310.02994. 2023.
Levi D McClenny and Ulisses M Braga- Neto. Self-adaptive physics infooseed neural networks. Journal of Computational Physics, 474:111722, 2023.
Nick McGreivy and Ammar Hakim. Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations. Nature Machine Intelligence, 6(10):1256-1269, 2024.
Parviz Moin and Krishnan Mahesh. Direct numerical simulation; a tool turbulence research. Annual review of fluid mechanics, 30(1):539-57) 578, 1998.
Geoffrey Négiar, Michael Mahoney, and Aditi S Krishnapriyan. Learning diffcrentiable solvers for systems with hard constraints. arXiv preprint vrXiv:2207.08677 2022.
Ruben Ohana, Michael McCabe, Lucas Meyer, Rudy Morel, Fruzsina Agocs, Miguel Beneitez, Marsha Berger, Blakesly Burkhart, Stuart Dalziel, Drummond Fielding, et al. The well: large-scale collection of diverse physics simulations for machine learning, Advances in Neural Information Processing Systems, 37:44989- 45037, 2025.
Pranshu Pant, Ruchit Doshi, Pranav Bahl, and Amir Barati Farimani. Deep learning for reduced order modelling and efficient temporal evolution fluid simulations. Physics of Fluids, 33(10), 2021.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
Benjamin Peherstorfer and Karen Willcox. Data-driven operator inference for nonintrusive projection based model reduction. Computer Methods in Applied Mechanics and Engineering, 306: 196-215, 2016.
Ugo Piomelli. Large-eddy simulation: achievements and challenges. Progress in aerospace sciences, 35(4): 335-362, 1999.
Stephen Pope. Turbulent flows.
Measurement Science and Technology, 12(11)20020 2021, 2001.
Shaoxiang Qin, Fuyuan Lyu, Wenhui Peng, Dingyang Geng, Ju Wang, Naiping Gao, Xue Liu, and Liangzhu Leon Wang. Toward a better understanding of fourier neural operators: Analysis and improvement from a spectral perspective. arXiv preprini arXiv:2404.00 072000, 2024.
Md Ashiqur Rahman, Robert Joseph George, Mogab Elleithy, Daniel Leibovici, Zongyi Li, Boris Bonev, Colin White, Julius Berner, Raymond A Yeh, Jean Kossaifi, et al. Pretraining codomain attention neural operators for solving multiphysics pdes. Advances in Neural Information Processing Systems, 37: 10:005 104064, 2024.
12

---

Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computatioonll physics, 378:686-707, 2019.
Varun Shankar. Dibyajyoti Chakraborty, Venkatasubramaniaa Viswanathan, and Romit Maulik. Differentiable turbulence: Closure as partial differential equation constrained optimization. Physical Review Fluids, 10(2): 024605, 2025.
Joseph Smagorinsky. General circulation experiments with the primitive equations: I. the basic experiment. Monthly weather review, 91(3)999 164, 1963.
Philippe Spalart. An ols-fosh framework for machine learning in turbulence modeling. arXiv preprint arXiv:2308. 00837, 2023.
Philippe Spalart and Steven Allmaras, one-equation turbulence model for aerodynamic flows. In 30th aerospace sciences meeting and exhibit, page 439, 1992.
Jianlin Su, Murtadha Ahmed, Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063, 2024.
Brendan D Tracey, Karthikeyan Duraisamy, and Juan Alonso. machine learning strategy to assist turbulence model development. In 53rd AIAA aeroxpace sciences meeting, page 1287, 2015.
Alasdair Tran, Alexander Mathews, Lexing Xie, and Cheng Soon Ong. Factorized fourier neural operators. arXiv preprint 11 2021.
Milton Van Dyke. An album of fluid motion, volume 176. Parabolic Press Stanford, 1982.
Ashish Vaswami, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Pauli Virtanen, Ralf Gommers, Travis E Oliphant, Matt Haberland, Tyler Reddy, David Cournaecau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, et Scipy 1.0: fundamental algorithms for scientific computing in python. Nature methods, 17(3):661-722, 2020.
Rui Wang, Karthik Kashinath, Mustafa Mustafa, Adrian Albert, and Rose Towards physics-informee deep learning for turbulent flow prediction. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1457-1466, 2020.
Sifan Wang, Shyam Sankaran, and Paris Perdikaris. Respecting causality is all you need for training physics- informed neural networks. arXiv preprint vrxiv:2203.07404 2022.
Sifan Wang, Shyam Sankaran, Panos Stinis, and Paris Perdikaris. Simulating fire-dimmessioal turbulence with physics-informed neural networks. arXiv preprint 2507:2250.. 2025.
Gege Wen, Zongyi Li, Kamyar Axizzadencshelli Anima Anandkumar, and Sally M Benson. U-fno-an an enhanced fourier neural operator- deep-learning model for multiphase flow. Advances in Water Resources, 163: 104180, 2022.
Matthew O Williams, Ioannis G Kevrekidis, and Clarence W Rowley. A data-driven approximation of the koopman operator: Extending dynamic mode decomposition. Journal Nonlinear Science, 2576):1307-1346, 2015.
Haixu Wu, Huakun Luo, Haowen Wang, Jianmin Wang, and Mingsheng Long. Transolver: A fast transformer solver for pdes on general geometries. arXiv preprint arXiv:2402.02366 2024.
Mingtao Xia, Lucas Böttcher, and Tom Chou. Spectrally adapted physics-informed neural networks for solving unbounded domain problems. Machine Learning: Science and Technology, 4(2):025024, 2023.
Jin-Han Xie and Oliver Bühler. Exact third-order structure functions for two-dimensional turbulence. Journal of Fluid Mechanics, 851:672-686, 2018
PK Yeung, DA Donzis, and KR Sreenivasan. Dissipation, enstrophy and pressure statistics in turbulence simulations at high reynolds numbers. Journal of Fluid Mechanics, 700:5-15, 2012.
PK Yeung, XM Zhai, and Katepalli Sreenivasan. Extreme events in computational turbulence. Proceedings the National Academy of Sciences, 112(41):12333-122 2015.
13

---

A Discussions on related works
We provide a broader overview of related work in ML. for fluid dynamics, covering physics-informed learning, spectral methods, Transformer-basel models, and reduced-order modeling.
MI for fluids. Except for the data-driven approaches discussed in $ another line of work focuses on developing large pretrained models that can generalize across a wide range of flow conditions, geometries, and Reynolds numbers [McCabe et al. 2023 Herde et al. 2024 Rahman et al. 2024). These models are trained on diverse simulation datasets and aim to serve as universal surrogates for fluid dynamics. In parallel, generative approaches have been proposed to capture the stochasticity and variability of turbulent flows. [ienen et al. [2023] apply diffusion models to synthesize physically plausible trajectories in zero-shot setting, while [Boral et al. [2023] introduce neural stochastic differential equations to model ensembles of turbulent trajectories.
Another popular direction involves physics-informed machine learning (Karniadakis et al., 202 which integrates prior knowledge problem during training. Physics-Infomin Neural Networks (PINNs) [Raissi et al. 2019] incorporates the residuals of the governing equations into the loss function. This framework allows for training with sparse or no supervision. Works in this direction seek to address challenges such training strategy KKrshhappriyan et al. 2021, Wang et al.) 2022), residual estimation [McClenny and Braga-Neto, 2023] Du et al. 2024 and constraint enforce- ment (Négiar et al., [2022] Chalapathi et (0024) Recent advances also demonstrated that PINNs able to simulate large-scale dimen-iimeniiona turbulence [Wang et al. 2025]. However, the training cost of PINNs, and solving for one problem at time, makes their efficiency worse compared to state-of-the-art numerical simulations.
Spectral-based models. Spectral methods [Canuto 2007] provide powerful inductive biases for
neural fluid simulation. Li et al. [0200 introduced Fourier neural operator (FNO), which is based on spectral convolutions in the Fourier domain. Neural operators have demonstrated accuracy and efficiency advantages over traditional convolutional networks. Subsequent works extended neural
operators by either improving the spectral kernel's design (Tran et al.) [021] Fanaskov and Oscledets
2023 Helwig al., 2023), or by learning completely in the spectral domain [Du al. 2024 Xia et al. 2023]. These models have shown promising results on low-dimensional problems; however. due to the inefficiency of parameterizing and learning large Fourier convolution kernels (Qin et al.
2024]. their application to large-scale diteeeiimani turbulence has been limited.
Moreover, they face challenges irregular domain geometries since spectral methods are inherently homogeneous. To overcome this, several extensions have been proposed that incorporate geometry more explicitly (Li et al. 2023ble] c). However, these methods still encounter limitations in accurately capturing complex geometries, as well as in applying to problems with a wide range of length scales.
Transformers for spatiotemporal modeling. There has been an increased interest in exploring the Transformer architecture for fluid simulation, and more generally, for solving partial differential
equations. One line of work [Dang et al. 2022 McCabe et al.] 2023 Herde et al., 2024) applied well-established Vision Transformer (ViT) architectures to this task. However, due the quadratic complexity of the self-attention mechanism, their scalability is limited by the spatial resolution of the solution field, especially for the resolution of three-dimensional turbulent flows.
Several approaches have been proposed to address this challenge: (a) simplified attention: Cao
[202T] draw connections between the attention mechanism and kernel convolution, providing insights
into reducing computational complexity. GNOT [Hao et al. 2023). FactFormer (Li et al. [202aa]
OFormer [Li et al., 2022), and AViT [McCabe et al. 2023 applied different simplifications to
the attention mechanism, enabling linear or almost-linear computational complexity; (b) grouped
tokens: this approach reduces the number of tokens involved in the attention mechanism by grouping
relevant mesh points together. UPT [Alki et al. 2024 uses sub- sampling supernodes in combination
with a perceiver-style architecture. while Transolver [Wu al. 2024 Luo al., 2025 operates
on alice-based physical tokens. While these approaches have shown promise for moderate-scale problems, their effectiveness and accuracy on large-scale turbulence problems unclear.
Reduced-order modeling. Reduced Order Modeling (ROM) provides a framework for approximat- ing high-fidelity simulations with lower computational cost. Classical methods such as Proper Orthog- onal Decomposition (POD) [Berkooz et al., 1993) have been extensively used as low-dimensional representations. Data-driven approaches, including Dynamic Mode Decomposition [Williams et al.
14

---

(3)
Energy spectrum ((k)
(b)
Velocity correlation f(x)
(c)
Structure function (r)))
10a
-1.4
1.25
-13
10 2
-0.2
10
-0.1
10
0.50
19-5
0.25
10
0.00
10
109
Wavenumber &
Longitudinal distance
Longitudinal distance r
Figure 7: Energy spectrum, velocity correlation, and structure function Popee 2001] of Re94 The energy spectrum follows the -5/3 scaling law, indicating inertial-range turbulence. The non -au-aassi velocity correlation shows intermittent behavior. Additionally, the structure function exhibits scaling consistent with Kolmogorov's ' theory, further confirming the presence of turbulent dynamics.
2015 Kutz et al., 2016) and Operator Inference [Peherstorfer and Willcox 2016]. have expanded the scope reduced-order modeling by constructing surrogate models directly from data. Recently, autoencoders has been employed to construct nonlinear reduced-order models (Lee and Carlberg 2020, Pant et al. [02T] Kim et al. 2022]. These approaches aim to learn low-dimensional manifolds that capture the essential dynamics of complex systems beyond linear methods. However, these methods often require complete information of the solution regime; consequently. their accuracy and generalization usually remain problem-dependent.
B Dataset details
In this section, we provide details on the two and direeddmmnsional turbulence dataset used in our experiments. Tab, 4 and Tab, contaann simulation parameters and flow statistics in the dataset. Fig. plots the energy spectrum of each flow, averaged across the whole test set.
Initial condition. The lowest Fourier modes |k| 16 of every initial velocity field are randomly initialized according to the -5/3 spectrum. The initial condition then burned in for fixed time to. which is chosen empirically that energy injection and dissipation reach equilibrium.
Solver details. We use pseudo-spectral method to generate the high-accuracy dataset as well as the low-resolution input for learned correction top of it. Smooth dealiasing [Hou and Li 2007 is used during advection in order to keep more modes than the common 2/3 rule. We use fourth-order Runge-Kutta, scheme for time-stepping. coupled with a second-order implicit diffusion step. The time step is determined dynamically by fixing Courant-Friedrrihhs-LLwwy (CFL) number below Cmax 1.
Dataset. The dataset is downsampled onto uniform grids using FFT intervals At, over a total duration t. The test set contains 16 longer trajectories: t 50s for KF4 and 20s for Re94.
Table 4: Simulation parameters of KF4 ($4.2) and Re94 ($4.1).
Table 5; Turbulence statistics
of KF4 ($4.2) and Re94 ($4.1)
Name
KF4
Re94
Name
KF4
Re94
Domain Ω
[0, 2m]²
[0, 2n]
Viscosity
0.001
0.01
Burn-in time to
40s
20s
Kinetic energy Eg
2.0061
3.6023
Record time At
0.125s
0.1s
Dissipation rate
0.0419
0.9749
Trajectory length t
10s
5.0s
Velocity r.m.s. (v)2
1.1539
1.5493
Solver resolution
4
3843
Taylor microscale À
0.6939
0.6090
Dataset resolution
256
963
Kolmogorov scale 7
0.0124
0.0318
Dataset size
89GB
1TB
Reynolds no. Rex
799.76
94.40
15

---

Evaluation metrics. We report standard error metrics, such as relative Lg error, well as other invariant flow statistics when evaluating all models and numerical solvers. Specifically, let be the reference solution and u' be the prediction, the following metrics are reported:
1. Relative L2 error (lu-ulll//l
ul/2 Relative error is used when and u' are highly correlated, i.e. during early steps in the rollout. is also used as the training objective (see Equ. (15)..
2. Point-wiss correlation (u, /l/ Correlation is used in later solution rollout, when and u' are statistically correlated, but relative error is too large to be a meaningful metric.
3. Energy spectrum E(k),
/
E(k)
a(k)/
(17)
|k|=
The energy spectrum describes how turbulent energy is distributed across different length scales. In the inertial range, E(k) is expected to follow Kolmogorov i 5/3 law, i.e., E(k oc 6-5/3, Accurately reproducing the energy spectrum is necessary for a model to yield physically meaningful predictions; even after the model's 's prediction becomes decorrelated from the reference, E(k) remains critical because it still reflects the turbulence cascade.
4. Two-point third-order longitudinal structure function Sr.
(r(r) ([lu(x + -x)) |r|=r-
(18)
The structure function (r) measures the statistical asymmetry of the velocity differences across a distance r, indicating the downscale energy transfer across turbulence scales. It is hypothesized follow the Kolmogorov's 4/5 law, i.e., Sy - where is the dissipation rate. However, in two-dimensional turbulence where inverse energy transfer, i.c., "backscattering". occurs, Sy exhibits more complex behavior [Kraichnan] 1967 Kraichnan and Montgomery 1980 Xie and Bühler 2018). Evaluating the two-point structure function on a model's prediction tests whether the preserves the correct inter-scale statistics.
5. The Q-criterion g.
q [1 -
(19)
where S = Vu + Vu²) and Ω (V Vu²² are the strain rate and rotation tensor, respectively. The Q-criterion is widely used diagmostic metric [Jeong and Hussain 1995] for identifying vortical structures in turbulent flows. Regions where q > 0 indicate local dominance of rotational motion over strains, which typically corresponds to coherent vortex cores. It provides a invariatt and physically interpretable way to visualize and quantify vortical structures in complex turbulent flows. We use the Q-criterion to visually compare the vortex cores in the reference DNS solution and EddyFormer's 's prediction.
C Experiment details
Hyperparameters. For 2D and 3D problems, EddyFormer uses the hyperparameters described in Tab. and Tab. respectively. We consider both Chebyshev and Legendre basis as the orthogonal polynomials in SEM basis for most problems, which is indicated by EddyFormer (cheb) and Eddy- Former (leg), respectively. We use the Adam optimizer with learning rate of 10-3, For homogeneous turbulence and 4.2
we use batch size of 16, while batch size of is used in $4.3
Table 6: EddyFormer architecture (2D).
Table 7: EddyFormer architecture (3D).
Stream
LES
SGS
Stream
LES
SGS
Number of layers T,
10
Number of layers T,
4
Hidden dimension d
32
Hidden dimension d
32
Mesh size H
162
Mesh size H
g3
Mode kmax and M
24
Mode kmax and M
5
13
Kernel mode
4
24
Kernel mode
5
13
Kernel size s
5 7/4
4/4
Kernel size s
π 7/4
T/4
Number of heads Theaad
Number of heads Theaad
Head dimension dhead
16
Head dimension dhead
32
16

---

Implementation details.
EddyFormer is implemented using the JAX framework [Bradbury et al. 2018). We implement our own daree-dimensional pseudo-spectral solver for all data generations and learned correction used in our experiments, We acknowledge the JAX-CFD [Dresdner et al. 2022 and NSM [Du al. 2024 codebases, which we referenced and built upon our implementation.
Implementation of SEMConv. The SEM-based convolution, Eqn. (8). evaluates the convolution between Fourier kernel and features on collocation points. For simplicity, here we only consider the implementation of ID convolution. Its extension to multi-dimensional case is straightforward due to the use of factorized kernels. Specifically. the SEMConv performed along æ axis is defined by,
- JniSS//2 k(z)u(x' - x)" dr.
SEMConv(u)(z))
(20)
We quadrature rules directly compute the convolution each collocation point. Specifically, Clenshaw- and Gauss-Lobatto- Legendr quadratures are use for Chebyshev and Legendre-
(m) based SEM bases, respectively. For each collocation point in the h a'th element, the integral is the sum of the quadratures in neighbor elements,
(EMConv(u)(o{"))
Σ
(m)) (m) (x0) wK k( Th )u(a!rl)
(21)
JN-RRSSS MEM
where S = - is the half-width of the neighbors within the convolution window and (w mm are the quadrature weights on element f2y, which can be pre-computed using routines from standard packages like QUADPACK Virtanen et al. 2020). Below is pseudo-code of our implementation.
(m))
Algorithm 1: SEMConv Implementation (ID)
Input:
Feature field u: Fourier convolution kernel k(x)
Output: Convolved feature field SEMConv(u)
S + s/2A
// Number of neighbors
Quadratureeieightts())
// Precomputed weights
foreach collocation point Th do
(n)
y 4 0; for h²' = h - to + S do
// Output z
for m to M do
(m)
t ww
// Quadrature weight
k + k(ujp' (mm) un (ni)
Evaluate convolution kernel
u(zn"' (m))
// Evaluate input feature
y += w u.
(EMConv(u) t
Computation cost of SEMConv. The computational cost of Algorithm | consists of two main components: evaluating the kernel k collocation points and performing quadrature over the input field. Under the assumption of a uniform mesh, the kernel only needs to be evaluated at MS distinct offsets, which is equivalent to performing an MS x S-sizee matrix multiplication. The quadrature step involves weighted dot product across neighbors, which costs ((M3S) per element. Therefore, the total cost scales with the number of resolved modes, with sublinear factor of O(MS).
C.1 Two-dimensional turbulence KF4
Learned correction. All models are trained to correct DNS at 256 resolution. Each model is first trained using the one-step loss, Eqn. (15). for 30k steps with a learning rate of 10-3 33 In order to control the roll-out error over time, we also differentiate through the DNS steps and minimize accumulated enror as Kochkov al. [202] did. Specifically. LC(") represents the n-step prediction roll-out using learned corrections, and the -tee loss function defined as follows:
uo
Σ |lu L N
(u:)ll/ httl///ennrllll
(22)
17

---

(a)
Relative error (%) 4
(b)
Point-wise correlation
(c)
Structure function Sy (r)
1.0
le-1
0.8
1
4
0.6
2
0.1
1m-5
0.2
2
4
10
12
14
16
18
20
0.1
Flow simulation time (sec)
Radical distance (m)
DNS 256
DNS 1024
F-FNO
ResNet
EddyFormer
DNS 5122
DNS 20482
FNO
Figure 8: Performance models and DNS at different resolutions on KF4 ($4.2 a two-dimensional fluid flow example ("Kolmogorov" flow). The models learn to corrrect DNS 256² resolution, and the metrics here are relative to a reference DNS at 40962 resolution. Fdryerrmer achieves the accuracy of DNS between 5122 and 10242 resolution in terms of (a) relative 1.2 error for / 10 and (h) point-wise correlation for 10 < t v 20; for (c) longer rollouts up to 50 seconds, EddyFormer also accurately captures the third-order structure function, Eqn. (18), which characterizes the forward and inverse energy transfer in fine length-scales.
However, as reported by Dresdner et al. 2022]. directly minimizing multi-ster steti-step loss is difficult and unstable. Therefore, choose N 5 and only fine-tuned using the 5-step loss for another 5k steps with reduced learning rate 10 after the mee-ttep training converged.
Baseline models. We consider ResNet [He et al. 2016), FNO [Li et al. 2020), and F-FNO Tran al., [222]] as baseline models for comparison. For ResNet, we adopt the standard ResNet-18 18 architecture. For both FNO and F-FNO, we use 642 modes, with 10 layers and 32 hidden dimensions, to ensure fair comparison with EddyFormer. All models, except for FNO, have a model size order of 10MB.
T 15
= 16
= 17
T 18
T 19
T 20
20
DNS 4096²
20
20
ForyFormer
0
20
20
F-FNO
0
20
Figure 9: Vorticity visualization of test sample on KF4 ($4.2) EddyFormer accurately tracks the dynamies of the vortex core (indicated by the red square) up 20, while the baseline F-FNO fails to resolve it over the long rollout.
18

---

(3)
Training loss (x10-*)
(b)
Test error (x10 at t= (c)
Test error (%) at t= 10
20
20
15
15
15
10
10
10
Training steps (k)
Training steps (k)
Training steps (k)
ResNet
FNO
F-FNO
EddyFormer (Chebyshev basis)
Figure 10: (a) Training loss, and (b) and (c) test error curves KF4 4.2
EddyFormer converges
significantly faster than all baselines, while achieving almost 50% reduction in converged loss.
For reference, we also include results reported by previous work (ochkov et al. [021] which were obtained by correcting we reproduced their result by correcting our high-accuracy pseudo-spectral solver using a ResNet.
finite- volume DNS 64 resolution. For fair comparison,
Fig. I0 shows the training loss well test errors different times. Tab. and Tab. report the relative L2 error and point-wise correlation metrics on test set. EddyFormer is able to converge faster than the baselines, consistently achieving lower test error. Fig. visualizes the evolution vortex core, showing that EddyFormer captures its development up to f 20.
Results.
Table Relative L2 error (%) on KF4 test set with learned correction of DNS 2562.
t=1
t=2
t=3
t=
f=5
t=
t=7
t=8
t 10
DNS 256
0.23
0.24
0.28
0.36
0.52
0.82
1.38
2.59
4.55
7.79
DNS 5122
0.09
0.10
0.12
0.16
0.23
0.38
0.64
1.21
2.13
3.71
DNS 1024
0.04
0.04
0.05
0.07
0.10
0.16
0.27
0.52
0.92
1.60
Kochkov et al.
0.63
1.38
2.53
4.54
8.09
14.7
24.3
36.5
56.4
74.5
FNO
0.17
0.20
0.28
0.40
0.71
1.19
2.06
3.58
6.59
11.4
ResNet
0.08
0.10
0.14
0.22
0.36
0.63
1.27
2.47
4.30
7.58
F-FNO
0.06
0.07
0.10
0.15
0.27
0.50
1.03
1.88
2.91
5.28
EddyFormer
0.03
0.04
0.06
0.09
0.14
0.24
0.49
0.86
1.44
2.61
Table 9: Point-wise correlation on KF4 test set with learned correction of DNS 256
t=
12
t=
15
t=
20
DNS 256
0.98
0.96
0.89
0.78
0.67
0.55
0.43
0.26
0.18
0.13
DNS 5122
1.00
0.99
0.96
0.90
0.83
0.71
0.59
0.43
0.31
0.19
DNS 1024
1.00
1.00
0.99
0.97
0.93
0.87
0.76
0.62
0.50
0.37
Kochkov et al.
0.69
0.51
0.36
0.24
0.15
0.09
0.09
0.02
-0.01
0.02
FNO
0.98
0.96
0.89
0.79
0.66
0.52
0.42
0.27
0.21
0.14
ResNet
0.98
0.95
0.87
0.78
0.66
0.50
0.39
0.27
0.22
0.12
F-FNO
0.99
0.98
0.95
0.89
0.79
0.67
0.53
0.39
0.31
0.20
EddyFormer
1.00
0.99
0.98
0.94
0.88
0.80
0.67
0.55
0.38
0.27
19

---

Discussions on domain generalization. super-resolution. While super-resolution fixed physical domain, domain expansion keeps the grid spacing and increases only the physical extent of the flow, testing whether the model has overfitted domain-specifie flow dynamics. Neural operators have no corresponding expansion mechanism. We tried two ad-hoc work-arounds: rescaling the appended spatial coordinates and stretching the learned convolution kernels. Neither produced comparable results: Fig. 5 shows that F-FNO, by rescaling the appended coordinates, fails to match the energy spectra on the expanded domains.
This expansion task (Fig. is conceptually different from [Li et al., 2020] aims recover finer-scale details on
Energy conservation. In addition to evaluating the statistical properties of model's long-term prediction, we also assess the model's ability to conserve total kinetic energy, Frst, over time:
ETor(T) = kl + /'' [(t) e(t)dt,
(23)
(V = Vu?) denoting the strain-rate tensor, and the injection rate / = (f u) given forcing term f. This cumulative energy accounts for the balance injection and dissipation and should remain constant in time. Even though energy conservation is necessary condition for physically correct model and implicitly present in the training data, the models are not explicitly informed of this constraint and must learn to conserve energy purely from data.
where the kinetic energy k == ( the dissipation rate 20(1|S|B) with S
Tab. reports the time-averaged total kinetic energy over a test trajectory. The results indicate that EddyFrrmer preserves the energy balance over long rollouts, whereas the baseline F-FNO exhibits significant and systematic energy dissipation over time.
Table 10: Time-averaged total kinetic energy Eior (t) of a trajectory in KF4 test set.
Time interval
[0,5]
[5, 10]
[10, 15]
[15, 20]
[20, 25]
[25, 30]
[30,35]
[35, 40]
[40, 45]
[45, 50]
EddyFormer
1.728
1.668
1.692
1.660
1.712
1.736
1.695
1.694
1.706
1.670
F-FNO
1.728
1.668
1.688
1.588
1.650
1.623
1.524
1.518
1.502
1.511
C.2
Three-dimensional turbulence Re94
The Re94 case is adopted from Bazilevs et al. [2007]. The forcing term fxx..
Pin
f(xx
Σ
uge
(24)
E1
injects energy uniformly across all directions. Here, ûn is the k'th Fourier coefficient of the velocity. Training loss (%)
Direction prediction ("pure MI.") benchmark. For baseline comparisons, adopt ResNet, FNO, and F. FNO in 3D without modifying their original architec- tures. For FNO, we reduce the number modes to 163 in order to avoid exceeding the model size constraint.
ResNet
AVIT Te-Net
FNO
GNOT EF(kg)
F-FNO
100
In addition, we also consider TF-Net [Qin et al., 2024
GNOT [Hao et al., 2023). and AVIT [McCabe et al. 2023) as additional baselines. We use their default ar- chitectures and adapt them to handle three- dineesiinnni
60
40
input fields. The number of trainable parameters for each model is reported in Tab.
20
All models trained for 15k steps using one-step loss. Fig. II shows the training loss of different baseline mod- els. EddyFormer converges significantly faster and con- sistently achieving lower test error than state-of-the-as neural operators and Transformers. In these baselines, F._ FNO is the best model, yet EddyFormer achieves almost 3x lower error.
0.0
2.5
7.5
10.0
Training steps (k)
Figure 11: Direct prediction loss curves
Re94 ($4.T). EF (leg) denotes FddyFormer with Legendre basis.
20

---

(a)
Training loss (x10->)
(b)
Test error (x10 at t= (c)
Test error (%) at
F. FNO
F. FNO
F FNO
50
13
EddyFormer
15
EddyFormer
EddyFormer
10
10
30
20
Training steps (k)
Training steps (k)
Training steps (k)
Figure 12: (a) Training loss, and (b) and (c) test curves Re94 ($4.1) with learned correction of DNS at 96 resolution. EddyFormer consistently achieves lower error on both training and test set.
Learned correction. Both F-FNO and EddyFormer are trained using one-step loss, Eqn. 15 for 15k steps with a learning rate of 10-3. Fig. 12 shows the training loss as well as test errors at different times. Tab. and Tab. 12 report the relative 1.2 error and point-wise correlation metrics test set.
Table Relative 1.2 error (%) on Re94 test set with learned correction of DNS 963.
0=5
1.0
1=5.
f=2.0
2-5
t= 3.0
t=5.
4=0.0
t= 4.5
t=5.0
DNS 963
1.53
3.34
5.64
8.74
13.0
18.7
26.0
34.8
44.8
55.9
DNS 128
1.04
2.28
3.88
6.08
9.1
13.3
19.0
26.2
35.0
45.5
DNS 1923
0.53
1.17
2.00
3.17
4.84
7.16
10.4
14.9
20.9
28.8
DNS 256
0.27
0.60
1.02
1.62
2.48
3.70
5.44
7.92
11.3
16.3
F-FNO
0.68
1.18
1.87
2.93
4.44
6.60
9.71
14.0
19.8
27.4
EddyFormer
0.46
0.78
1.23
1.89
2.80
4.20
6.23
9.27
13.1
18.2
Table 12: Point-wise correlation on Re94 test set with learned correction of DNS 963.
t= 5.5
t= 6.0
t=
6.5
{=
7.0
t=
7.5
1 t= 8.0
t= 8.5
(=9.0
t= 9.5
t= 10.
DNS 363
0.78
0.71
0.63
0.55
0.47
0.39
0.30
0.23
0.16
0.09
DNS 1283
0.84
0.78
0.70
0.62
0.54
0.45
0.36
0.28
0.20
0.13
DNS 192
0.93
0.88
0.82
0.75
0.67
0.60
0.51
0.43
0.34
0.25
DNS 256
0.97
0.95
0.91
0.86
0.80
0.74
0.67
0.59
0.51
0.43
F-FNO
0.93
0.89
0.83
0.76
0.69
0.62
0.54
0.46
0.38
0.30
EddyFormer
0.97
0.94
0.90
0.85
0.78
0.71
0.64
0.56
0.48
0.41
C.2.1
Ablation studies
In order to better understand Eddy and the right model design choices, we perform various ablation studies using the direct prediction ("pure ML.") benchmark on Re94 First, we consider the following modifications to isolate the model components:
EF (LES). EddyFormer using only the LES stream for large-scale scale dynamics.
EF (SGS). EddyFormer using only the SGS stream for small-scale dynamics.
EF (FFT), EddyFormer variant with SEMCony replaced by Fourier-based convolution.
A factorized kernel with the same number of modes as F-FNO is used for fair comparison.
EF (CNN). EddyFormer variant with SEMConv replaced by standard convolution.
A 3D convolution with the same kernel size is directly applied on the collocation points.
21

---

0.5
83
04
03
(a) Reference solution.
(b) EddyFrrmrr's prediction.
(c) T-FNO's prediction.
Figure 13: Visualization of the density field for MHD 64 test sample at EddyFormer is able to match the reference solution while the baseline T-FNO's prediction diverges after three steps.
Table 13: Direct prediction benchmark on Re94 one-step error of different ablation models.
EF
EF (LES)
EF (SGS)
EF (FFT)
EF (CNN)
F-FNO
ResNet
Test error 8.61%
12.6%
23.2%
12.8%
25.3%
22.5%
87.2%
Results. Tab. reports the one-step error of the ablation models. The results show that the LES stream alone performs worse without the support of the SGS stream, and the SGS stream alone is slightly less accurate than its F-FNO counterpart. This suggests that many global modes FNOs may be redundant for turbulence modeling. EddyFormer with CNN shows significant accuracy drop, consistent with the gap between ResNet and FNO. The FFT-based variant of EddyFormer also performs worse, highlighting that the SEM basis offers advantage over FFT and serves component of our architecture. Compared to the LES-only version, the full model demonstrates that (a) SEMAttn captures large-scale dynamics more effectively than F-FNO, and (2) SEMConv models small-scale dynamics that FFT-based spectral convolution fail to resolve.
Choice of the LES filter. In classical LES, the filter size defines the cutoff between resolved and subgrid-scale motions, and is typically set based the grid resolution or an associated spectral cutoff based on the energy spectrum.
Table 14: Data-only benchmark on Re94 one-step error across different LES filter sizes.
kmax
2
5 (ours)
6
8
Test 18.76%
8.61%
8.66%
8.34%
To assess the impact of the LES filter size, we vary kmax while keeping all other hyperparameters fixed. The results in Tab, 14 show that increasing kimax improves accuracy up to kmax 5, beyond which the performance saturates. This saturation reflects the limited resolution capacity of the SEMA module, which determines the range of large-scale structures the LES stream can effectively capture. When the spectral filter includes gighherffreuence modes beyond this capacity, the LES stream unable to utilize them cutoff adds computational cost without improving performance. These results validate our choice of kmax
under a fixed model size. In such cases, increasing the
balance between expressivity and efficiency.
C.3 The Well dataset
We benchmark EddyFormer on the nurbulence-relater subsets in The Well. We follow all training protocols described by Ohana et al. (225), with the exception of the training duration. EddyFormer trained for 10k steps on each problem, rather than the one-day training budget used in the original benchmark, to ensure reproducibility. The batch size is set to 64. We report the variance-scaled mean square error (VRMSE) scores on test sets as Ohana al. (2025) did, which is defined as:
VRMSE(u, u') = ((/u (2) x| + 222
(25)
where is the reference solution and u' is the prediction. The safety value 10-7 prevents extreme values on near-constant fields. All reported VRMSE scores are averaged across all time steps the test set. Baseline results in Tab. are taken from the original benchmark by Ohana et al. 20255.
22

---

0.0
T= 1.0
T= 1.5
T= 2.0
T= 2.5
T= 3.0
CNextU-nee Eddy Groundtruth
-1
Figure 14: Visualization of the tracer field for shear fllo test sample over time. EddyFormer predicts the development of two vorticies in the shear layer, while the baseline CNextU-net converged to over-dissipasive solution.
= 0.0
== 1.0
T= 2.0
T = 3.0
T= 4.0
Groundtruth
-2
EddyFormer
-2
2
T-FNO
0
-2
Figure 15: Visualization the density field for a rayleigh_ benardd test sample over time. Eddy-
Former predicts the growth of the mixing layer as well as the plume structures, while the baseline T-FNO predicts a non-physical solution (the solution collapsed into uniform region at = 4.0).
23

---

Results. Tab. 15 and Tab. 16 shows the time-averager averaged per-field VRMSE scores, and Tab. shows the field-average VRMSE scores. Fig. Fig. 14 Fig. [5] and Fig. [6 show visualizations of the MHD_ shear flow, rayleigh_ benard, and raylei taylor instab. cases, respectively.
Table 15: EddyFormer (Chebyshev basis) The Well: per-field VRMSEJ scores.
Table 16:
EddyFormer (Legendre basis) The Well: per-field VRMSEJ scores.
Field
VRMSE
Field
VRMSE
0.0944
0.10567
velocity
0.1007
velocity
0.1142
MHD_64
0.1076
MHD_64
0.1161
density
0.1647
density
0.1926
T0.13687
10.1611
magnetic
0.1052
magnetic
0.1219
0.1028
0.1161
[0.0071
0000044
velocity
0.3038
velocity
0.1908
Shear_flow
pressure
0.1193
shear_fll
pressure
0.1295
tracer
0.0189
tracer
0.0106
00.6211
0.62891
velocity
00.312
velocity
0.3063
rayleigh_benanl benard
pressure
0.0220
rayleigh_
pressure
0.0279
buoyancy
0.0555
buoyancy
0.0807
[0.1603T
0.114467
velocity
0.1280
velocity
0.1560
rayleigh_tayylor
0.0960
rayleigh_taylor
0.1284
density
0.0618
density
0.0679
D Limitations
While EddyFormer accelerates the simulation of three-dimensional turbulence, its application real-world turbulent flows in near-wall regions with fine features is left unexplored. In addition, scaling EddyFormer accelerate simulations of extreme-scale turbulence high Reynolds numbers is particularly challenging due to the complex nature of these flows. Future works include extending EddyFormer general domains and exploiting scalability.
24

---

(a) Reference solution at f
(b) EF's prediction at O.
(c) FNO's prediction at 0.
(d) Reference solution at / = 10.
(e) EF's prediction at / = 10.
(I) ENO's prediction at / = 10.
gg) Reference solution at = 20.
(h) EF's prediction at t 20.
FNO's prediction at f = 20.
ii) Reference solution at f 30. (k) EF's prediction at t 30.
()) FNO's prediction at = 30.
Figure 16: Visualization of the density evolution for rayleigh_taye tayloryllorr.ol _aisttat.... test sample. EddyFormer successfully predicts the development of instability and the plume structures up to t 30, while the baseline FNO fail to converge.
25

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: experiments show that EddyFormer significantly accelerates the resolution three-dimensional turbulence.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
we discussed that domain geometric effects are not explored in this work.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
26

---

Justification:
this work doesn't it include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: include all architecture and experiment details in appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
27

---

Answer:
[Yes]
Justification:
all datasets and source code will be open sourced.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ccpppp.ccppulli//uuuii wwwiiicgguie///eeeeeeeeic CoodeSubeiiieeee ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: all experiment details are in appendix. Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: error bars are not reported because it is expensive to repeat all experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
28

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: we provide information on how long does each experiment take to reproduce specific hardwares.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: gattps://neurips http:/pp....p....cchccccc.o.....
Answer: [Yes]
Justification: comply the NeurlPS code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: we discussed broader impacts of this work.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
29

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: this work poses no misuse risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: cite all software assets used in this work.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
30

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: we provide detailed explanation for the three-dimensional turbulence dataset introduced in this work.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification: this work does not involve crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: this work does not involve crowdsourcing.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
31

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: this work does not involve LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy hattap///aaarrpssscc///offeeeere for what should or should not be described.
32


Reviews:
[{"review_id": "MDLkdV0m3d", "reviewer": "Reviewer_pTWs", "summary": "This paper proposes EddyFormer for ML-based modeling of turbulence. The main idea is to take inspiration from large-eddy simulations and use two sub-networks: standard-attention for modeling large-scale and spectral convolutions for modeling subgrid-scale turbulence. EddyFormer is benchmarked on different turbulence problems from the Well dataset. Results show that EddyFormer outperforms neural operator (FNO, FFNO, TFNO variants) and transformer (GNOT, Transolver) baselines. The paper also reports 30x speedup over DNS simulations using efficient pseudo-spectrak solvers on a single GPU.", "questions": "Please see weaknesses listed above.", "limitations": "Data efficiency is not addressed. There is an underlying assumption that there is access to large training data for transformer models but this is not the case in turbulence (specifically DNS data as used in this paper).", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths: \n\n-- Builds on turbulence theory and well-established modeling methodologies (i.e., LES) motivating the dual-stream design underlying EddyFormer. \n\n-- Experimental evaluations consider challenging benchmarks such as Rayleigh-Taylor instability and demonstrates improved performance compared to baselines.\n\nWeaknesses: \n\n-- The evaluation of long-time rollout is limited in duration. If delta t = 1 s, the maximum number of time steps in the examples is atmost 50. \n\n-- The design lacks any specific treatment for flows with wall effects and is unclear how the current architecture will generalize to real-world turbulence with irregular geometries or wall-bounded flows. \n\n-- Evaluation metrics are limited to relative L2 and doesn't check if physical quantities are conserved. Since the model also doesn't enforce any hard physical constraints (e.g., conservation laws), it's unclear if conservation is met and how these models will generalize to OOD.\n\n-- The reported speedup is for a specific problem size (256^3) for a specific benchmark. It is unclear how EddyFormer scales at larger sizes and how that compares to DNS scaling. Quadratic scaling in attention costs might limit the scalability of the model at higher resolutions.", "was_revised": true, "final_justification": "The authors present results for total energy over 50 time steps and claim they preserve energy balance compared to F-FNO but the numbers go down, then up, then down so the conclusion of significant systematic dissipation is unconvincing. Scalability to domains beyond 256^3 is still an open question that's unaddressed. For these reasons, I keep my current score.", "rebuttal": "We thank the reviewer for their thoughtful feedback and for recognizing the strengths of our work, including our integration of turbulence theory, the dual-stream design inspired by LES, and our comprehensive evaluations on challenging turbulence benchmarks. We particularly appreciate the reviewer's acknowledgment of the improved performance demonstrated by EddyFormer against strong baselines.\n\nWe have performed additional energy conservation checks, as well as introduced a new benchmark on domains with irregular geometries. Regarding your concerns on our evaluation and the applicability of EddyFormer, we address each point below.\n\nQ1: **Evaluation metrics are limited to relative L2 and don't check if physical quantities are conserved. Since the model also doesn't enforce any hard physical constraints (e.g., conservation laws), it's unclear if conservation is met and how these models will generalize to OOD.**\n\nWe acknowledge the importance of evaluating conservative physical quantities. We want to highlight that we have already conducted extensive evaluations that go beyond standard error metrics (like L2 relative error) to assess whether invariant physical properties are preserved implicitly. Specifically, we have:\n\n- **Energy spectrum**. In Fig. 7, we compare the energy spectrum of the model predictions with ground truth, capturing how well the model preserves energy across different scales.\n- **Structure function**. In Fig. 3c and 5c, we evaluate the two-point third-order structure function, a key diagnostic in turbulence that reflects inter-scale energy transfer. As discussed around Eq. 18, this metric directly tests whether the model preserves correct inter-scale statistics, which are tightly linked to conservation laws in turbulent systems.\n- Furthermore, since the turbulent flow cases are in statistical equilibrium, evaluating model predictions over long temporal rollouts already implicitly tests for conservation-like invariants. The sustained match over time between predicted and reference statistics indicates that the model generalizes well and does not accumulate unphysical errors.\n\nNevertheless, we have performed additional energy conservation checks on KF4 and find that EddyFormer is able to preserve energy balance over an extended period of time. This flow problem receives energy injection from the sinusoidal forcing, and dissipates energy at the smallest scales. Specifically, the energy components are:\n\n\\begin{equation}\n  E(t) = \\langle \\frac 1 2 \\mathbf{v}(t, x)^2 \\rangle_x, \\quad\n  \\frac {dE_\\mathsf{dis}} {dt} = \\nu \\langle \\nabla \\mathbf{v}(t, x)^2 \\rangle_x, \\quad\n  \\frac {dE_\\mathsf{inj}} {dt} = \\langle \\mathbf{f}(x) \\cdot \\mathbf{v}(t, x) \\rangle_x, \n\\end{equation}\n\nTo test energy conservation, we measure all three components of the energy budget, and record the accumulated energy $E_\\mathsf{tot}$, $E_\\mathsf{tot}(t) = E(t) + \\int_0^t dE_\\mathsf{dis} - dE_\\mathsf{inj}$, over 50s on the test set. The results below show that EddyFormer is able to preserve the energy balance over an extended period, while F-FNO (the model that we find is second best for this problem, among our benchmarks) shows a significant and systematic dissipation over time.\n\n**Table 1**\n| $E_\\mathsf{tot}(t)$ | 0~5 | 5~10 | 10~15 | 15~20 | 20~25 | 25~30 | 30~35 | 35~40 | 40~45 | 45~50 |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| EddyFormer | 1.728 | 1.668 | 1.692 | 1.660 | 1.712 | 1.736 | 1.695 | 1.694 | 1.706 | 1.670 |\n| F-FNO | 1.728 | 1.668 | 1.688 | 1.588 | 1.650 | 1.623 | 1.524 | 1.518 | 1.502 | 1.511 |\n\nQ2: **The evaluation of long-time rollout is limited in duration. If delta t = 1 s, the maximum number of time steps in the examples is at most 50.**\n\nThank you for your comment. We note that our rollout of 50s in KF4 already represents a significantly longer duration than many related works. For example:\n\nFNO reports rollouts of up to 30s,\nPDE-Refiner adopts a similar temporal range of 15s,\nAPEBench [2], which specifically targets long-rollout evaluation, uses 40 steps at most.\n\nThus, our rollout length is among the longest in current literature and sufficiently captures the chaotic regime where prediction stability and invariant measures are most challenging.\n\nQ3: **The design lacks any specific treatment for flows with wall effects and is unclear how the current architecture will generalize to real-world turbulence with irregular geometries or wall-bounded flows.**\n\nThis is an important consideration. The SEM framework underlying EddyFormer inherently supports geometric flexibility, making it naturally compatible with wall-bounded flows and irregular geometries. Although our current study primarily focuses on turbulence in homogeneous domains, extending EddyFormer to incorporate explicit wall effects and complex geometries represents a promising direction for future work, leveraging this intrinsic compatibility. We will discuss this point and make it more clear in our updated manuscript.\n\nTo test this, we have also performed an additional experiment on acoustic scattering in a maze. This is another subset in The Well, featuring wave reflections in complex geometries. We also follow the training protocol described in The Well, and report the VRMSE scores below.\n\n**Table 2**\n|            | FNO    | TFNO   | U-Net  | CNextU-Net | EF (leg) |\n|------------|--------|--------|--------|------------|----------|\n| Test VRMSE | 0.5062 | 0.5057 | 0.0351 | 0.0153     | 0.01932  |\n\nDue to time constraints, we didn’t modify the model to support wall treatment using attention masking, etc. Still, EddyFormer is able to match the performance of the best baseline. The comparisons with FNOs show that the geometric flexibility of SEM is significant.\n\nQ4: **The reported speedup is for a specific problem size (256^3) for a specific benchmark. It is unclear how EddyFormer scales at larger sizes and how that compares to DNS scaling. Quadratic scaling in attention costs might limit the scalability of the model at higher resolutions.**\n\nWe acknowledge the importance of evaluating and extending EddyFormer to larger problem sizes. While our current implementation is tested on $256^3$ resolution, EddyFormer adopts standard attention mechanisms similar to those used in large language models, enabling the use of well-established techniques (such as memory-efficient attention variants and system-level optimizations) to scale to significantly larger contexts. For extreme-scale turbulence problems, additional architectural adaptations, such as localized attention windows or hierarchical representations during training, can further improve scalability. We view this as a promising and tractable direction for future work.\n\nQ5: **Data efficiency is not addressed. There is an underlying assumption that there is access to large training data for transformer models but this is not the case in turbulence.**\n\nThank you for this point, we will make the data efficiency aspect more clear, and expand on it here: we note that we use ~100k snapshots to train both KF4 and Re94. For previous works, both FNO and JAX-CFD also use ~100k solution snapshots. We highlight that Re94 is a three-dimensional problem, which exhibits more complex dynamics. Yet, EddyFormer is able to achieve high accuracy and generalization within the 100k data limit.\n\n---\n\n[1] Lippe, Phillip, et al. \"Pde-refiner: Achieving accurate long rollouts with neural pde solvers.\" Advances in Neural Information Processing Systems 36 (2023): 67398-67433.\n\n[2] Koehler, Felix, Simon Niedermayr, and Nils Thuerey. \"APEBench: A benchmark for autoregressive neural emulators of PDEs.\" Advances in Neural Information Processing Systems 37 (2024): 120252-120310."}, {"review_id": "fHXYFUS7JY", "reviewer": "Reviewer_inpY", "summary": "This paper targets deep learning in the context of turbulence simulations, motivating its approach with the well established self similarity of eddies in the turbulence cascade. An architecture is proposed that uses spectral elements in the context of Transformer models, processing inputs in terms of s subgrid-scale and and LES branch. This idea is a classic one, and well-established in turbulence learning [1]. Unfortunately, the paper does not acknowledge this, and instead tries to present this idea as a new contribution, without citing the classic method. While there are clearly differences to [1], the job of the authors would be to discuss and evaluate the differences clearly. Unfortunately, the lack of context and lack of previous work comparisons is continuing later on in the paper.\n\nIn the proposed architecture, the SGS part uses FNOs, which are combined with attention based methods for the second branch. The FNO part swaps the FFT for SEM basis functions. The advantages of the SEM approach over an FFT are claimed as a contribution, but its advantages are not made clear in a direct comparison. The FNO approach also means it inherits the negative aspects: especially the bad scaling of FNOs for 3D seems problematic here. Global basis functions for larger 3D resolutions are inherently problematic, which is, however, mitigated with the axial attention.\n\nScale separation is done via a spectral cutoff, the method uses a manual split into low and high frequencies in each. Low frequencies are provided to the high-frequency branch - in L140 the authors discuss this as surprising, but it's directly in line with the primary scattering of the turbulence cascade. That the other direction is not helpful points to limited accuracy: it's the classic \"closure problem\" of turbulence, and in the general case the small scales should play a role once the overall approach is sufficiently accurate. (I would recommend to change this paragraph accordingly.)\n\nDue to the lower dimensionality of the low \"LES\" frequencies, a Transformer with dense attention is applied. The tokens for this stage are likewise processed via the SEM basis functions.\n\nThe paper evaluates the method with a learned predictions for 3D isotropic turbulence, and a solver-in-the-loop case for 2D kolmogorov flows. Here, likewise, the authors only focus on the work by Kochkov et al., but do not provide context of their method for newer works from the turbulence area, e.g. [2,3] just to name a few. Generally, the writing and exposition of the paper is problematic - the paper tries to focus on turbulence in its outset, but then seems to turn into a paper focusing on NN architectures. \n\nThe authors have also moved related work and the discussion of limitations to the appendix. This naturally frees up room in the main paper, but the NeurIPS page limit is there for a reason. Given the requirements for a NeurIPS paper, these parts should be included in the main text. And naturally, for a paper targeting turbulence, a discussion of ML approaches in this area would be necessary in the main text (not only citing classic text books).\n\nThe different methods focus on time stepping, but the paper is unfortunately unclear on the details here. The authors provide details on unrolling (like \"N=5\"), but don't specify for how many steps the networks were evaluated. Some tests seem to use integer time steps, others like in table 8 use smaller ones. This is crucial information to evaluate the time stability of the method.\n\n[1] Rui Wang, Karthik Kashinath, Mustafa Mustafa, Adrian Albert, and Rose Yu. \nTowards physics-informed deep learning for turbulent flow prediction. SIGKDD, 2020\n[2] Bjoern List, Liwei Chen, and Nils Thuerey.\nLearned Turbulence Modelling with Differentiable Fluid Solvers. JFM, 2022\n[3] V. Shankar, D. Chakraborty, V. Viswanathan, and R. Maulik.\nDifferentiable turbulence: Closure as a partial differential equation constrained optimization. PRF, 2025", "questions": "- can the authors demonstrate that the SEM approach has benefits over an FFT by swapping out the basis functions within the full proposed architecture?\n\n- what are the time steps, and how many rollout steps evaluated? \n\n- how is the spectral cutoff chosen, and how much influence does it have. I.e., what is k_max in each case, and how much influence does it have in practice? An ablation would be important here to whether the scale separation actually has a positive influence.", "limitations": "Limitations are not discussed in the main paper.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\n\n- 3D PDEs , and turbulence are tough and interesting tasks. They're definitely worth studying, and not many papers address 3D up to now.\n\n- the paper shows some strong results: the isotropic turbulence case with 256^3 is neat to see. This is a high resolution, and the \"fair\" comparison with a high-resolved baseline makes this a strong case\n\n- the paper comes with a good set of \"generic\" baselines that, however, seem quite arbitrary: the data is structured, then why compare to \"unstructured\" methods like GNOT and Transolver? \n\n- it's good to see the architecture behaves like a convolutional network and can deal with different domain sizes\n\nWeaknesses\n\n- despite focusing on turbulence, the paper does not provide context of previous work for turbulence-focused ML approaches \n\n- the central SGS/LES split was proposed in previous work many years ago [1]; the authors do not seem to be aware of this; a direct comparison with tf-Net is missing\n\n- the contribution of an SEM approach is likewise claimed as contribution, but it is not mdke clear how much improvements SEM provides over an FFT basis (the Legendre vs Chebyshev comparison implies that the basis does not have a big influence)\n\n- the details on the time discretizations are missing. without this information, the results are difficult to interpret\n\n- similarly, details of the spectral cutoff for the SGS vs LES split are missing\n\nSo overall, I think the paper makes some promising steps, but my impression is that it needs a larger, properly checked revision before publication can be recommended.", "was_revised": true, "final_justification": "The authors have added comments and clarifications. I think especially the SEM vs FFT comparison is an important one that was missing in the paper. \n\nI think it would be important to double check the final version of the paper, to ensure that the it discusses previous work on learning with scale-separation for turbulence in an appropriate manner. I think it is important for a NeurIPS paper to properly acknowledge widely seen previous work, and this paper clearly reduces the scope of the claims. Also, the authors clearly have work to do to clearly explain their method, the submission had many unclear aspects. Nonetheless, the paper still makes interesting points, so I'm open to supporting an \"accept\". E.g., the SEM bases are interesting, and scaling up NN-based models to large models above 128^3 is a non-trivial feat.", "rebuttal": "We thank the reviewer for their detailed and constructive feedback. We appreciate the recognition of the novelty and difficulty of the 3D turbulence case, as well as the acknowledgment of our strong results—particularly the high-resolution results and the model’s ability to generalize across domain sizes.\n\nRegarding your concerns, we agree that additional context on prior works, as well as ablation studies on our model architecture, would help clarify our contributions. We have performed additional ablation studies on SEM basis and the LES cutoff frequency, as well as introduced two new baseline models to reflect prior works. We will discuss each point and address them below.\n\nQ1: **Advantages of SEM basis over FFT basis. The Legendre vs Chebyshev comparison implies that the basis does not have a big influence.**\n\nThank you for raising this important point. We answer and expand on this question here and also add an ablation experiment. The use of the SEM basis is not only an advantage, but a necessary design choice in our architecture. Specifically:\n\nThe SEM basis enables efficient implementation of compact convolutions using local spectral elements, which is essential for modeling localized interactions in complex dynamics.\nOur design of self-attention over coarse spatial meshes relies on the locality and interpolation properties of SEM—this would be infeasible using global FFT bases.\n\nThe comparison of Chebyshev and Legendre basis does not imply that the SEM basis is an arbitrary choice, but instead shows that multiple bases are compatible in the SEM framework.\n\nOn the other hand, if we use FFT basis, the model is limited to global basis, and as you mentioned, exhibits bad scaling w.r.t. the number of modes that it tries to resolve. To assess the impact of this choice, we conducted an ablation study by replacing SEMConv with a standard F-FNO layer, named EF (FFT). The results degraded significantly, highlighting that SEM is advantageous over FFT basis, serving as a core element of our architecture. The training speed results also show that SEMConv with a compact kernel ($s=2$ in this case) is more efficient than FFT on the global field.\n\nFurthermore, we also benchmarked Re94 with an ablation model which only has the LES stream. The resulting accuracy is almost identical to EF (FFT), proving that 1) SEMAttn models large-scale dynamics better than F-FNO does, and 2) SEMConv models small-scale dynamics that FFT basis fails to capture.\n\n**Table 1**\n|             | EF      | EF (FFT) | EF (LES only) | F-FNO   |\n|-------------|---------|----------|---------------|---------|\n| Test error  | 8.52%   | 12.8%    | 12.6%         | 22.5%   |\n| Train speed | 5.2s/it | 6.0s/it  | 2.9s/it       | 5.1s/it |\n\nQ2: **Why compare to \"unstructured\" methods like GNOT and Transolver?**\n\nWe appreciate the reviewer for this important point. GNOT and Transolver are two SOTA Transformers that have been successfully applied to 3D problems. While there are other “structured” models on rectilinear input meshes, e.g., Poseidon and Multiple-Physics Pretraining [1] (MPP), they were only implemented, trained, and evaluated on 2D problems.\n\nTo provide an additional comparison, we have adopted AViT [2] (used by MPP [1]) to 3D and benchmarked its accuracy on Re94. We consider AViT a strong and relevant baseline for EddyFormer due to its structured attention mechanism and demonstrated effectiveness on spatially regular data. We set the “patch” size to be $16^3$ for consistency with its 2D counterpart. The results below show that although AViT achieved better performance than other Transformer baselines, its error is still significantly higher than EddyFormer.\n\n**Table 2**\n|            | EF    | AViT  | Transolver | GNOT  |\n|------------|-------|-------|------------|-------|\n| Test error | 8.52% | 24.5% | 57.7%      | 69.7% |\n\nQ3: **A direct comparison with TF-Net is missing.**\n\nWe appreciate the reviewer for bringing our attention to this work, and we will discuss it in the paper. As you mentioned, the architecture of TF-Net is fundamentally different from our approach. Most importantly, TF-Net uses three identical conv-nets (at the same resolution) to model the mean and fluctuating components, which do not utilize different properties of these fields.\n\nIn FNO’s paper, TF-Net was shown to be outperformed by neural operators. We have also performed additional benchmarking of TF-Net on Re94 to strengthen our argument. The results below show that TF-Net is still worse than F-FNO on three-dimensional problems (F-FNO is best baseline), which is consistent with FNO’s results. In contrast, EddyFormer outperforms both of them by a large margin.\n\n**Table 3**\n|            | EF    | F-FNO | TF-Net |\n|------------|-------|-------|--------|\n| Test error | 8.52% | 22.5% | 30.4%  |\n\nWe will revise the manuscript to acknowledge TF-Net, discuss its approach to the LES/SGS split, clarify how our architecture differs, and include this direct comparison.\n\nQ4: **What are the time steps, and how many rollout steps are evaluated?**\n\nThe time discretizations for two problems are described on line 178 and line 216, respectively. Re94 uses $\\Delta t = 0.5$ and KF4 uses $\\Delta t = 1$.\n\nThe number of rollout steps are described in Tab. 3 for the training set, and on line 415 for the test set. Re94 is tested for a total trajectory length of $20s$, while KF4 is tested for $50s$. We note that the number of rollout steps is sufficiently large to reach the regime where predictions decorrelate from the ground truth, at which point invariant measures become the more meaningful and reliable indicators of model performance.\n\nQ5: **The details of the spectral cutoff for the SGS vs LES split are missing. How is the spectral cutoff chosen, and how much influence does it have?**\n\nThe spectral cutoff frequency $k_\\mathsf{max}$, as well as other architecture parameters, are summarized in Tab. 4 and 5. Re94 uses $5^3$ modes and KF4 uses $4^3$ modes in the LES stream.\n\nThe value of $k_\\mathsf{max}$ is chosen empirically by balancing efficiency and accuracy. The capacity of the SEMAttn determines the range of large-scale structures that the LES stream can effectively capture. If the spectral filter includes excessively high-frequency modes, the LES stream is unable to fully utilize them due to its limited resolution capacity—particularly under a fixed model size. In such cases, increasing the cutoff frequency adds computational cost without improving performance, as the attention module cannot effectively model these finer scales.\n\nTo quantify the impact of the spectral filter size, we performed an ablation study on Re94 by varying $k_\\mathsf{max}$ while keeping all other hyperparameters fixed. Specifically, we compared models using both smaller and larger filter sizes than our default value of $5$. The results show that model accuracy increases as $k_\\mathsf{max}$ grows up to $5$, beyond which the improvements plateau. This saturation indicates that the LES stream reaches its effective resolution limit around $k_\\mathsf{max} = 5$, and including higher-frequency modes does not contribute further—validating our choice of spectral cutoff as both sufficient and efficient.\n\n**Table 4**\n|            | EF (cheb, k=2) | EF (cheb, k=5) | EF (cheb, k=6) | EF (cheb, k=8) |\n|------------|----------|----------------------------|----------|----------|\n| Test error | 18.76%   | 8.61%                      | 8.66%    | 8.34%    |\n\n---\n\n[1] McCabe, Michael, et al. \"Multiple physics pretraining for physical surrogate models.\" arXiv preprint arXiv:2310.02994 (2023).\n\n[2] Ho, Jonathan, et al. \"Axial attention in multidimensional transformers.\" arXiv preprint arXiv:1912.12180 (2019)."}, {"review_id": "giZ8ZvBq8q", "reviewer": "Reviewer_c37T", "summary": "This work focus on studying neural operators on direct numerical simulation (DNS) dataset. The existing study of neural operators all tends to perform poorly on this challenging task due to the large-scale data (e.g. $256^3$ and $384^3$) and the complexity of the nature of turbulence.\n\nThe architecture of the proposed neural operator, EddyFormer, composes of two parts: SEMConv block and SEMAttn block. This design is inspired by the formulation of large eddy simulation (LES), where the solution is decomposed into two parts: LES stream and subgrid-scale (SGS) stream as equation (6). And the two streams are handled by the above blocks accordingly. Namely, SEMConv deals with local eddy dynamics of SGS stream, and SEMAttn handles long-range dependencies of LES stream (See Fig. 2).\n\nSuch heuristic design is proven to be highly advantageous with DNS datasets. Datasets are challenging that scale from $96^3$ to $256^3$ and compared with numerical solution on $384^3$ for homogeneous isotropic turbulence. Also, 2D turbulence model, Kolmogrov flow is considered. Finally, turbulence beyond homogeneous isotropic is benchmark in the Well dataset. In three benchmarks, EddyFormer surpasses all baselines by large gap.", "questions": "N.A.", "limitations": "N.A.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "**Strength**\n\nThe proposed method, EddyFormer, is deliberately designed with LES decomposition, which could be the reason for outperforming general neural operators, e.g., FNO and transformer-based neural operators (GNOT, Transolver).\n\nThe paper clearly presents the method, and comprehensive experiments are well deployed to show its advantage. DNS is a challenging problem of high significance, and the method is showing strong evidence (in terms of accuracy and speed) of practical applications of neural operators in CFD.", "was_revised": true, "final_justification": "While myself was not able to point out further question or issues on this paper, after reading other reviews, I did learn some good points being asked. Some are addressed, and some seem left open. For example,\n\nAddressed:\n1. Why choosing convolution-based SEM instead of FFT? The answer is better local feature extraction due to convolution than global basis of FFT. \n2. The KF4 dataset is similar to Kochkov paper but different. The scale is $256^2$ here in this paper versus $64^2$. This difference causes the prior art performed much worse on KF4.\n\nOpen:\n1. The second review from Reviewer inpY mentioned a few prior work in turbulence modeling. It seems the authors did not compare the difference to these works.\n\nOverall, the proposed method is well motivated and highly effective in both accuracy and efficiency. The experiment is conducted comprehensively both on dataset and baselines, showing great advantage. Although the idea of splitting solution as LES may be proposed before with neural networks, this work presents great combination of latest art in neural operators, both convs and transformers, and achieves superior results. Therefore, my score remains to 6.", "rebuttal": "We thank the reviewer for their feedback and for recognizing the strengths of our work, including the dual-stream design inspired by LES, and our comprehensive evaluations on turbulence benchmarks. We particularly appreciate the reviewer’s acknowledgment of EddyFormer's improved performance and acceleration on the challenging three-dimensional case."}, {"review_id": "XKNgFtYzoZ", "reviewer": "Reviewer_Qm4N", "summary": "The paper presents a novel method for learned simulation of high-res turbulent flows. It combines a SEM representation, LES-style filtering and a specialized transformer variant. It claims higher accuracy on three experimental domains with 2D and 3D turbulence.", "questions": "1. The method needs to be explained better:\n  - It sounds like the transformer operates on N^3 tokens, so what happens to the M^3 modes? Are they stacked into the feature dimensions? But also the latent width is specified as 32, so is there a latent encoding from M^3 -> 32 somewhere? What about SEMConv?The paper shouid clarify which operation act in which dimensions, and how those are transformed.\n  - If SEMConv also operates on N^3 tokens, why do we want a spectral convolution? The N^3 blocks are in real space, right?\n  - SEMConv in general needs a more detailed description (maybe in the appendix)-- padding, integration, etc.\n  - How is the model optimized? L2 loss on the output u?\n  - Which inputs and outputs are filtered, and how? (3) mentions SGS is unfiltered and LES is filtered, but then 3.1 suggests there are filters W_{in}, and W_{SGS} for SGS? \n2. Is there actually any advantage to splitting the model into a SGS, LES path? I understand that the architecture is inspired by LES solvers, but from a model perspective there's little actually enforcing these semantics. Loss is on the sum of both path, and after a layer or two information will be mixed in latents across both branches anyways. Have you tried just using one path (either with both filtered u_SGS and u_LES inputs, or just u) with more layers or higher capacity?\n3. Transformer with convolutional blocks is a bit of a weird combination. Can you just use a (bigger) standard transformer with FFNs, maybe with more blocks so that attention can actually do it's thing? Or if the SEMConv layers are very useful, do you need the transformer at all-- i.e. can you just run a FNO-style method on SEM elements?\n4. Besides the comments on strange baseline performance numbers, there also isn't any information on how the baselines are parametrized and run. Eg How many layers/width/... (ResNet and FNO can mean a lot of things). What resolution did you choose (you probably can't run a ResNet on 256^3)?  How was the field downsampled?\n5. It seems in general FNO performs only a bit worse than EF, but then it completely fails on RB/RT instabilities while EF performs very well. Why do you think that is?\n\nI think this paper really needs a careful revision, investigating baseline performance, and careful examination & ablation of the model architecture. This is probably not feasible to achieve within the rebuttal timeline. While I will increase my score on the off-chance that the authors can quickly clear up the comparisons, and add interesting findings re/ architecture, I'd actually encourage to spend a bit of time on this and resubmit a potentially much stronger & more impactful paper.", "limitations": "yes.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**: \n\n3D turbulent flow simulation is quite challenging, and the results are impressive. In general, the paper is well written, provides a nice problem overview & experimental setting. It also performs a very decent evaluations on the experimental results from a CFD perspective, studying e.g. correlation and energy spectra.\n\n**Weaknesses**:\n\nWhile the paper seems thorough on the CFD side, it falls flat with its contributions to ML. \nFirst, the method description remains vague and at a high level, and it would likely be hard to replicate with the information provided (see questions below for specifics). \n\nNext, the architecture feels quite arbitrary (again see questions for details), and more importantly, the choices aren't ablated at all. So we don't know if any performance gains are due to SEM, or how much the SEMConv layers matter, etc.\nAnd finally, I'm quite concerned about whether baseline comparisons were performed properly. For example, Transolver claims superior performance on several CFD benchmarks, including to FNO-- but here it performs significantly worse than FNO. Worse, KF4 is almost identical to the dataset used in the Kochkov paper, yet in this paper Kochkov seems to perform 10x worse than a simple Resnet-- this can't be right.\n\nIt's possible there are certain properties of the domains studied here that cause these differences, or maybe evaluation in these previous works has issues. But this needs to be properly investigated, otherwise we end up with several papers all claiming SOTA with opposing results, and we learn nothing.\n\nSimilarly, the impact of the architectural choices needs to be studied and ablated. It's unlikely that someone will reimplement _exactly_ this method as it has a lot of moving pieces--but maybe SEM could be a useful representation for other problems? Or maybe LES splitting, or adding conv layers into a transform is actually very helpful for CFD? These are questions we'd want answered in order for others to build on this work.", "was_revised": true, "final_justification": "After rebuttal and discussion, I still have some reservation about the paper, particularly surrounding baselines, evaluations and missing implementation detail.\nThe authors have performed some additional experiments and provided some explanations, which will put the paper in a bit of a better spot; I still think a major revisions, with a more thorough set of comparisons and ablations would be best. While I'm still leaning slightly on the negative side I wouldn't object to the paper being published with the additional experiments and exposition. I have hence changed my score from 'reject' to 'borderline reject'.", "rebuttal": "We thank the reviewer for their detailed feedback. We appreciate the recognition of the difficulty of the 3D turbulence, as well as the acknowledgment of our strong results.\n\nRegarding your concerns, we have performed multiple ablation studies on the LES/SGS splitting and the SEMConv block, as well as introduced two new baseline models to further strengthen our results. We address each of your points below and will incorporate the corresponding updates in our manuscript.\n\nQ1: **Clarifications on how SEMAttn is performed.**\n\nThe SEMAttn is defined by Eqn. 11, which mixes features on the same local coordinate from different elements (line 149). Therefore, the $M^3$ modes are neither stacked nor projected, but instead treated independently on every collocation point (line 150).\n\nQ2: **Clarifications on how SEMConv is implemented.**\n\nThe SEMConv is implemented by directly evaluating Eqn. 8. Since the input features are also defined on collocation points, the integral can be accurately computed as a weighted dot product using the corresponding Gaussian weights (line 131). Specifically, the kernel is evaluated at each collocation point, and a dot product is taken with the input features, weighted by the quadrature weights. Since our problems have periodic boundary conditions, we do not apply boundary treatment and instead use simple wrapping.\n\nWe will revise the manuscript to include these implementation details for clarity.\n\nQ3: **Why do we want a spectral convolution if SEMConv operates “in real space”?**\n\nWhile SEMConv operates on $M^3$ collocation points (not $N^3$ tokens as you mentioned), spectral parameterization is still preferred over standard convolutions. Spectral models such as FNO have proven highly effective for modeling spatiotemporal dynamics and are significantly more expressive than conventional conv-nets, even though they also operate on real-space inputs.\n\nTo validate this, we conducted an ablation study on Re94 by replacing SEMConv with standard convolutions (see table below). Consistent with the performance gap between ResNet and FNO, the modified model exhibits a notable drop in accuracy.\n\n**Table 1**\n| | EF w/ SEMConv | EF w/ CNN | F-FNO | ResNet |\n|---|---|---|---|---|\n| Test error | 8.52% | 25.3% | 22.5% | 87.2%  |\n\nQ4: **How is the model optimized?**\n\nThe model is optimized using relative L2 error. (Re94: line 187 and Eq. 14; KF4: line 219 and Eq. 16).\n\nQ5: **Which inputs and outputs are filtered, and how?**\n\nThe LES stream uses a spectral cutoff filter with k_max modes for both input and output (lines 115–116). We thank the reviewer for catching the typo on line 119; the correct sentence is: “The initial feature is the projection of the unfiltered input.”\n\nQ6: **The architecture feels quite arbitrary, and the choices aren't ablated at all. So we don't know if any performance gains are due to SEM, or how much the SEMConv layers matter.**\n\nThe SEM basis is not just advantageous but essential to our architecture. It enables efficient localized convolutions, and our coarse-mesh attention relies on the locality and interpolation properties of SEM—features not supported by FFT bases.\n\nTo assess this design choice, we conducted an ablation on Re94 by replacing SEMConv with an F-FNO layer. The degraded accuracy confirms SEM as a core component. The training speed also shows that SEMConv is more efficient than FFT-based layers.\n\n**Table 2**\n| | EF | EF (FFT) | EF (LES only) | F-FNO |\n|---|---|---|---|---|\n| Test error  | 8.52% | 12.8% | 12.6% | 22.5% |\n| Train speed | 5.2s/it | 6.0s/it  | 2.9s/it | 5.1s/it |\n\nWe have already performed an ablation study in Q3 to examine the difference between SEMConv and standard convolution on grid-based inputs, and the results demonstrate that the spectral parameterization in SEMConv is important to model performance.\n\nQ7: **Is there actually any advantage to splitting the model into a SGS, LES path?**\n\nOur core insight is to model large- and small-scale dynamics separately, as they exhibit distinct statistical properties and demand different modeling strategies, as discussed on lines 25–31 and 53–59. A single-stream approach is suboptimal: the LES stream cannot capture fine-scale features, while the SGS stream lacks the global expressivity for large-scale dynamics. Even with inter-scale information transfer, their distinct architectures serve different purposes and are not identical.\n\nTo investigate this, we have performed new ablation studies on Re94 using LES/SGS stream only. The results below show that the LES stream on its own is weaker without the SGS stream, and the SGS stream on its own is slightly worse than its corresponding F-FNO. As a side note, this also suggests that many global modes in FNOs might be redundant in turbulence modeling tasks.\n\n**Table 3**\n| | EF | F-FNO | EF (LES-only) | EF (SGS-only) | ResNet |\n|---|---|---|---|---|---|\n| Test error | 8.52% | 22.5% | 12.6% | 23.2% | 87.2% |\n\nTo further support the design of our design, we also included TF-Net [2] in our benchmarks (see Table 4 below). TF-Net uses the LES/SGS splitting but still models both streams using standard U-Nets. TF-Net’s original paper already shows that the LES/SGS splitting is beneficial, and here we reproduced their findings. The comparison highlights that our specialized LES and SGS stream designs offer substantial improvements, demonstrating effectiveness of our architecture.\n\nQ8: **Can you just use a (bigger) standard Transformer with more tokens?**\n\nA long-standing issue of using Transformers for spatiotemporal modeling is the unfavorable quadratic scaling of computation w.r.t. the number of tokens. Many previous works took different approaches to mitigate this issue (see discussion in Appendix, line 468-483). For Re94, the total number of modes required to resolve the flow is of $\\mathcal O(10^7)$. Therefore, using a bigger transformer is not viable.\n\nTo strengthen our claim, we conducted additional benchmarks on Re94 using the Axial Vision Transformer [1] (AViT). AViT employs axial attention on regular meshes, making it a fair and representative Transformer baseline. As shown in the table below, AViT underperforms compared to both FNO and EddyFormer, indicating that pure Transformer architectures are less effective for capturing turbulent dynamics. This result further supports our design choice to integrate Transformers with spectral-based convolution.\n\n**Table 4**\n| | EF    | F-FNO | AViT  | TF-Net |\n|---|---|---|---|---|\n| Test error | 8.52% | 22.5% | 24.5% | 30.4% |\n\nQ9: **I’m concerned about whether baseline comparisons were performed properly. For example, Transolver claims superior performance on several CFD benchmarks, including to FNO -- but here it performs significantly worse than FNO.**\n\nIn Tab. 1, we benchmarked several Transformer variants in previous works. They were primarily evaluated on problems with irregular meshes. For instance, Transolver encodes geometric features by grouping similar mesh points into slice-based tokens—a design well-suited for unstructured domains. We believe its stronger performance compared to FNO is due to FNO’s known limitations on irregular meshes.\n\nAdditionally, turbulent flows exhibit intricate fine-scale structures that differ substantially from those in prior benchmarks. For example, Transolver was evaluated on FNO’s 2D turbulence dataset, which is relatively less challenging and has already been shown to be well-handled by new models like F-FNO (Fig. 3 in [4]), which we also include as a baseline. In contrast, our 3D turbulence setting presents significantly greater complexity, making it a more demanding benchmark for evaluating model performance.\n\nQ10: **It seems in general FNO performs only a bit worse than EF, but then it completely fails on RB/RT instabilities while EF performs very well. Why do you think that is?**\n\nWe note that in our data-only benchmark (Tab. 1), the one-step errors of FNOs are almost 3x larger than EddyFormer’s. This gap is fairly large, and is very likely to be enlarged during the solution rollout (Fig. 8 shows the model prediction after 30 roll-out steps).\n\nRT is unstable w.r.t. perturbations. Error is amplified in the rollout stage, which affects both training and inference stage. We believe EddyFormer performs well due to its slower error accumulation over time, in contrast to FNO, whose errors grow rapidly and lead to instability during rollout.\n\nQ11: **Details on baseline architectures and resolution. How is the reference solution downsampled?**\n\nFor F-FNO, we use 64 modes, 4 layers with 32 channels. For FNO, we use 16 modes because of size constraint. For ResNet, we use the standard ResNet-18. For all other Transformers, we use their default architecture.\n\nWe refer the reviewer to Tab. 3 for details on dataset resolution, as well as other flow parameters. Re94 is recorded on $96^3$, while KF4 is recorded on $256^2$. The reference solution is downsampled using FFT.\n\nQ12: **KF4 is almost identical to the dataset used in the Kochkov paper, yet in this paper Kochkov seems to perform 10x worse than a simple ResNet -- this can't be right.**\n\nKF4 follows the same problem setting as Kochkov et al., but differs in both dataset and solver. As noted in Fig. 5, their results are based on correcting a $64^2$ finite volume (FVM) solver, which their own paper [3] acknowledges to be less accurate and efficient than spectral methods. We include their result (dashed green line) as a reference only.\n\nIn contrast, our results (line 213) are based on correcting a $256^2$ pseudo-spectral solver, which offers much better accuracy and efficiency. For a fair comparison, we also reproduced their setup using a ResNet trained on our high-accuracy dataset by correcting our $256^2$ solver (Fig. 5, solid green line). We will revise the manuscript to clarify this distinction for readers unfamiliar with the background.\n\n---\n\n[1] arXiv:2310.02994.\n\n[2] arXiv:1911.08655.\n\n[3] arXiv:2207.00556.\n\n[4] arXiv:2111.13802."}]

Author rebuttal:

Meta-review: This paper presents a novel neural operator architecture that combines spectral element convolutions and transformer-based attention to efficiently model large-scale three-dimensional turbulence. Key idea is model large-scale (grid-scale) and small-scale (subgrid-scale) dynamics separately. The initial reviews for this work were highly divergent, spanning the full range. There were significant initial weaknesses in its framing, methodological rigor, and connections within prior work. The discussion and author rebuttal have been pivotal in clarifying the paper's standing. Further validation on more diverse and practical turbulence scenarios would strengthen the case for broader impact.

Despite some lingering reservations from the reviewers I recommend acceptance as the flaws are not fatal. Honestly, some of it is a critique of community-wide practices, and the authors have made a good-faith effort to improve transparency.
Decision: Accept (poster)