LOGLO-FNO: Efficient Learning of Local and Global Features in Fourier Neural Operators
Authors: Marimuthu Kalimuthu
David Holzmüller
Mathias Niepert
Venue: Accepted by TMLR ()

Abstract: Modeling high-frequency information is a critical challenge in scientific machine learning. For instance, fully turbulent flow simulations of the Navier-Stokes equations at Reynolds numbers 3500 and above can generate high-frequency signals due to swirling fluid motions caused by eddies and vortices. Faithfully modeling such signals using neural networks depends on the accurate reconstruction of moderate to high frequencies. However, it has been well known that deep neural nets exhibit the so-called spectral or frequency bias towards learning low-frequency components. Meanwhile, Fourier Neural Operators (FNOs) have emerged as a popular class of data-driven models in recent years for solving Partial Differential Equations (PDEs) and for surrogate modeling in general. Although impressive results have been achieved on several PDE benchmark problems, FNOs often perform poorly in learning non-dominant frequencies characterized by local features. This limitation stems from the spectral bias inherent in neural networks and the explicit exclusion of high-frequency modes in FNOs and their variants. Therefore, to mitigate these issues and improve FNO's spectral learning capabilities to represent a broad range of frequency components, we propose two key architectural enhancements: (i) a parallel branch performing local spectral convolutions and (ii) a high-frequency propagation module. Moreover, we propose a novel frequency-sensitive loss term based on radially binned spectral errors. This introduction of a parallel branch for local convolutions reduces the number of trainable parameters by up to 50% while achieving the accuracy of the baseline FNO that relies solely on global convolutions. Moreover, our findings demonstrate that the proposed model improves the stability over longer rollouts. Experiments on six challenging PDE problems in fluid mechanics, wave propagation, and biological pattern formation, and the qualitative and spectral analysis of predictions, show the effectiveness of our method over the state-of-the-art neural operator families of baselines.

Published in Transactions on Machine Learning Research (12/2025)
LOGLO-FNO: Efficient Learning of Local and Global Features in Fourier Neural Operators
Marimuthu Kalimuthu
Universität Stuttgort, Stuttgart Center for Simalation Science, Sim Tech International Mox Planck Research School for Intelligent Systems (IMPRS-IS)
David Holzmüller
INRIA Saclay, SODA Team
Mathias Niepert
fastnoame.lsstnameeekie wttp://wwwaaaeee.e.....utuutrro.
Universilät Stullgart, SimTech, IMPRS-18
Reviewed on OpenReview: https: // openreview. net/ forum? id= d
Abstract
Modeling frge-frey information critical challenge in scientific machine learning. For instance, fully turbulent flow simulations of the Navier-Stokes equations at Reynolds numbers 3500 and above generate high-frequency signals due to swirling fluid motions caused by eddies and vortices. Faithfully modeling such signals using neural networks depends on the accurate reconstruction of moderate to high frequencies. However, it has been well known that deep neural nets exhibit the so-called spectral or frequency bias towards learning low-frequency components. Meanwhile, Fourier Neural Operators (FNOs) have emerged as popular class of data-driven models in recent years for solving Partial Differential Equations (PDEs) and for surrogate modeling in general. Although impressive results have been achieved several PDE benchmark problems, FNOs often perform poorly in learning non-dominant frequencies characterized by local features. This limitation stems from the spectral bias inherent in neural networks and the explicit exclusion of high-frequency modes in FNOs and their variants. Therefore, lo mitigate these issues and improve FNO's spectral ral learning capabilities to represent a broad range frequency components, we propose two key architectural enhancements: (i) a parallel branch performing local spectral convolutions and (ii) high-fre propagation module. Moreover, propose a novel frequency-sensitive loss term based radially binned spectral errors. This introduction of parallel branch for local convolutions reduces the number of trainable parameters by up to 50% while achieving the accuracy of the bascline FNO that relies solely on global convolutions. Moreover, our findings demonstrate that the proposed model improves the stability over longer rollouts. Experiments on six challenging PDE problems fluid mechanics, wave propagation, and biological pattern formation, and the qualitative and spectral analysis of predictions, show the effectiveness of our method over the seate-of-the-art neural operator families of baselines.
1 Introduction
Simulating real-world physical systems and problems in thermodynamics, biology, weather and climate modeling, hydrodynamicss, and astrophysics, to name a few, involves solving partial differential equations (PDEs). Numerical PDE solvers based on Finite Volume, Finite Difference, and Finite Element methods might be slow due the need for fine discrctization of the computational mesh, work only for given set of input parameters, and must be run from scratch when the settings, such as initial and boundary conditions, change. In recent years, neural networks have been used to build generalizable surrogate models

---

Published in Transactions on Machine Learning Research (12/2025)
of such dynamical systems (i.e., forward problem) and for the inverse task of the discovery of PDEs and their parameters from data (Brumton & Kutz, 2023; 2024).
Effective modeling of the entire frequency spectrum, including high frequencies, is important for tasks such as super-resolutinn and turbulence modeling (Wang et al., 2020; Fan et al., 2024). In the latter case, large eddies can give rise to small eddies, resulting in chaotic dynamics, which can further be influenced by forcing functions (Fan et al., 2024). Large Eddy Simulation (LES) and Direct Numerical Simulation (DNS) require prohibitive costs for high-resolution scenarios (e.g., 2048x2048, Tran et al. (2023); Fan et al. (2024)). Neural Operators (NOs) are class of surrogate models that can be trained at low spatial and temporal resolutions, saving cost, time, and compute, and can be used to predict solutions at resolutions unobserved during training, a technique called zero-shot super-resolution. Although the predominant neural operators exhibit, this property, their accuracy is limited on simulation tasks that exhibit fine ane-grained details and complex structures. For instance, the Navier-Stokes equation exhibits high frequencies at low viscosities (i.e., le-*) or high Reynolds numbers (c.g., Re = 5000; Li et al. (2022b)). NOs suffer from the spectral bias of neural nets and have difficully modeling high frequencies and local features. We review related literature concerning this challenge in Appendix A. Motivated by the limitations of existing NOs and inspired by multipath neural net architectural designs (Chi ct al., 2020; Chen et al., 2021; Pan et al., 2022; Li et al., 2024; Liu-Schiaffin et al., 2024), we propose LOGLO-FNO with the following key contributions: (i) local spectral convolutions, (ii) a high-freyuency propagation component, and (iii) a novel frequency-aware loss term.
2 Background on Neural Operators (NO)
Neural Operators learn an approximation of operators between unfinite-dimennsinaa Banach spaces of functions. A neural operator is a function g : A - 24 between two Banach spaces of functions A, defined on bounded domains.
Fourier Neural Operator (FNO). FNO (Li et al., 2021; Kossaifi et al., 2024b), which is one of the effective instantiations of NO, learns the operator using the Discrete Fourier Transform (DFT). In compact terms, u = G(a)= = (4o ..... o L P) (a), with the lifting and projection layers P. Q and the function composition o. A Fourier layer L can mathematically be described as the mapping in Eqn. 1, with Z E X='*,,,,,,,,, being the output of the lifting operation for the first Fourier layer and the previous
Fourier layer outputs thereafter,
[(Z) = of (ChannelMLPP (Y Yloobal) + SoftGating (2)), where Y Yolobal == (KKg(Z) +Conii) (1)
where (.) is the global kernel integral operator realizing DFT with Fast Fourier Transforms (FFT) for uniformly discrctized data, a is the GELU non-lincarity" acting point-wise, SoftGating operation applies a learnable, feature-wise afline or linear transformation to its input, Couv1D represents N₂ x Ny is the spatial resolution of the 2D spatial data, and No is the width or the number of hidden channels.
convolution,
FNO and its variants, barring incremental FNO (George et al., 2024), retain only fixed, albeit tunable, number of frequency modes corresponding to low frequencies and truncate the high-frequency ones (Kovachki et al., 2023; Helwig et al., 2023; Brandstetter et al., 2023; Gupta & Brandstetter, 2023). This modeling choice, although beneficial in managing the model parameters, results in suboptimal reconstruction quality of predictions since it pushes the model to ignore high frequencies in the data. This is inconsequential when the data in question contains mainly low-frequency structures. However, high-fidelity reconstruction is paramount. for tasks such as turbulence modeling (e.g., LES) and resolving multiple scales in multiphysics simulations. For instance, Hassan et al. (2023) benchmark FNO and its variants on the BubbleML dataset, a multiphase and multiphysics simulation of boiling scenarios. In such cases, the temperature profile can exhibit sharp jumps, resulting in discontinuities along the bubble interfaces, a typical manifostation of high frequencies. Their study has observed that FNO variants face significant difficulties in this task.
"skipped for the last Fourier layer in the network.
2

---

Published in Transactions on Machine Learning Research (12/2025)
3 Method
In this section, describe main contributions. The LOGLO-FNO model aims to improve the class of Fourier Neural Operators (Li et al., 2021; Kossaifi et al., 2024a;b; Tran et al., 2023) in boosting their capacities to effectively learn local patterns and non-dominant frequencies. Such model supports high-fidelity outputs and achieves improved stability over long rollouts (McCabe et al., 2023). Towards this end, LOGI.O-FO introduces two key modules as architectural enhancements, namely (i) local spectral convolution module and (ii) a high-frequency propagation module. Moreover, we propose a frequency-aware spectral space loss function based on radially binned spectral errors (see Appendix D.2 for the full algorithm and pseudocode), using which conduct experiments. In addition, we explore additional loss, namely, spectral patch high-frequency emphasized residual energy (see Appendix D.1), in line with the idea of patching the spatial domain to model local features. Furthermore, we employ attention- based fusion strategies in LOGLO-FNO to seamlessly integrate multi-scale features, and assess the effect of such a feature (Appendix G.3).
LOGLO-FNO. Our proposed LOGLO-FNO model modifies Eqn. to Eqn. 2: the LOGLO Fourier layer takes three input tensors. (i) Z E R N"*NNxNN,xM (ii) 2 E R="NN,,,,, N, being the outputs of the lifting operation P for the first global and local Fourier layers, respectively, and the previous LOGLO Fourier layer outputs thereafter, and (iii) Z' XNN, xN, the lifted features of the extracted high frequency structures XH (cf. Eqn. 4) for the first LOGLO Fourier layer and the previous channel MLP outputs thereafter.
global o (g(Z) Co)vID'(Z)).
Local := (<Ci(2) + ConvID'(2)).
CZZ 2, Z = ChannelMLP (Y giobbtt ) + Soft.ating (Z) +
ChannelMLP' (Yroea)) + SoftGating (2) + ChannelMLP (z')).
where the suffixes and denote global and local Fourier kernels, respectively, and N₂ x Ny the spatial resolution of a 2D patch. The notations of the input tensors naturally extend to 3D spatial data.
The LOGLO-FNO architecture, depicted in Figure 1, maintains all the essential properties of NOs, such as discretization invariance. In addition the main global branch, consists of two auxiliary branches: (i) parallel local branch that retains all Fourier modes and (ii) another parallel high-frequency feature propagation branch.
3.1 Architectural Improvements for FNO
Local Spectral Convolution Branch. We propose employing auxiliary local branches to the global Fourier layer to enable the model to learn rich local features by performing convolutions restricted to local regions (c.g., patches). Unlike the global branch, the local one operates on patches of the input signal. First, we create non-overlapping patches similar to the input of prototypical vision transformers (Dosovitskiy et al., 2021). These are then fed to the lifting layer P before being passed on to the local spectral convolution layer, which retains all Fourier modes. Akin a CNN, the local branch operating on patches performs local convolution since the spatial domain is now limited the patch size, capturing local and small-scale patterns, whereas the main branch performs global convolution, modeling high-level phenomena (e.g., overall fluid flow direction from left to right or bottom to top).
We consider a partition of the domain D into non non- non-overanpping hypercubes PI....,PP PM called patches, such that UM Pm. = D. Then, the local kernel integral operator K2(-) with a learnable & can be defined as
(z=()))z=)(====
k6læ - v) 2 (v) dy.
(3)
Pmnce
3

---

Published in Transactions on Machine Learning Research (12/2025)
L
x
Z,
i Z'
a(x)
P
LOGLO Fourier Layer
2 (2)
X
Z Lifted Features for Global Branch.
Lifted Features for Local Branch.
z' Lifted Features for HPP Branch.
Channel MLP (skip)
2
F
R
Channel MLP
1 x 1 Convolution
F
R
Channel MLP
x Convolution
Channel MLP (skip)
Z'
Channel MLP (hfp)
Figure 1: The overall archilecture of our proposed LOGLO-FNO model. X is the discretization of a(x), X the patched version, and Xμ the extracted high frequencies (cf. Eqn. 4). The full network has & repetitions of identical LOGLO Fourier layers, and the final activation function is applied on all but the last LOGLO Fourier layer. The features from the three branches and the skip connections are fused by simple summation.
where Pm(()) is the mh patch containing Unlike the global integral operator K.y+)) in Eqn. the kernel integral operator K2(-) in Eqn. 3 is restricted to local region Pm with customizable spatial extent. The suffix _ denotes the th instance of the local Fourier layer and m indexes specific patch out of the total M patches. Therefore, ggz stands for the features fed as input to the th local Fourier layer.
High-Frequence Propagation (HFP) Branch. In order to provide stronger inductive bias of high- frequency features, we propose an HFP module as the third branch, placed in parallel, that encourages the accurate reconstruction high-frequencies (Liu et al., 2020) that are otherwise subdued in the Fourier layers of the global branch due to explicit truncation of high-frequency modes. We employ one level downsampling the input signal using average pooling and an upsampling block using interpolation. The blurriness of the downsampled signal (e.g,, field variable) is directly proportional to the degree of spatial distortion, which, in the case of average pooling, is controlled by the kernel size and stride. The upsampling operation undoes this effect to reconstruct the original signal resolution. The extracted high frequencies X (cf. Figure 1) are lifted using P with shared parameters for a tight coupling of the high-pass filtered features with that of the full signal features, propagated using a channel MIP, and added to the output features the local and global Fourier layers. Overall, the HFP block serves as a high-pass filter and can be written compactly as
4

---

Published in Transactions on Machine Learning Research (12/2025)
X = X Interpolate(AAi AvgPool(X kernel size, stride); size = (..., , Ne, Na, Ny, (N:)).
(4)
3.2 Frequency-aware Loss based Radially Binned Spectral Energy Errors
Since our aim to direct the optimization process to enable the model to faithfully reconstruct non-dominant frequencies in the predictions, we propose frequency-sensitie loss and penalize the band-classifidd (mid- and high-frequency) spectral
as additional weighted term in the loss. Let Y and Y be the model prediction and target, respectively. The pointwise error in physical space is Au = Y - Y, and the spectral error Aá = F(Au) is obtained by applying the FFT on the spatial dimensions. Then, we define the radially binned, temporal and channel-wist spectral error on 2D spatial data as,
N
Lag
Ly
Che (r) =
(ks.or(kes kyy
(5)
N
Ng
No
b=
(kz s.t. Ry)::ky))=r
where the inner summation aggregates the energy of spectral errors within cach radial bin Lx and Ly are the spatial extents of the domain, whereas N₂ and Ny are the corresponding total number of observation points along those spatial axes, Subsequently, the binned errors can be band-classifted as low, mid, and high frequencics by considering suitable radial cutoff values (e.g., low=[1,4), mid=(5,12), and high=[13, M]: see Appendix K.6) and summed or averaged over the physical variables and temporal dimensions.
Spectral Energy Error Magnitude
Radially Binned Spectral Error
0
Low-freq region
0.030
Mid froq region
10
0.025
High- freq region
Energy of Spectral Error
FFI Index (ky)
20
0.025
30
0.020
0.015
0.015
50
0.010
0.010
60
10
60
20
FFT Index (kx)
Radial Distance
Figure 2: Representative illustration of radially binned spectral error. Left: the spectral energy of error magnitude with overlaid radial bins (blue low frequency, green mid frequency cutoff, the rest high frequency band); Right: the energy of the radially binned spectral error across radial distances line plot.
A detailed step-by-step procedure (Algorithm 1) and PyTorch-tyyle pscudocode to compute the frequency loss for 2D spatial data are provided in Appendix D.2, and the visualizations are placed in Appendix D.3.
3.3 Fourier Layer Parameter Budget for 2D Spatial Data
The majority of the parameter mass in FNO is concentrated on the Fourier layers (specifically, the spectral convolution modules) and their cardinality. Tt is dependent three factors: (i) the spatial dimensionality
5

---

Published in Transactions on Machine Learning Research (12/2025)
of the problem, (ii) the number of selected Fourier modes K, and (iit) the channel' dimension de- Denote the number of Fourier layers as L. Then, for 2D spatial data (s = 2), the parameter complexity of the global spectral convolutional layer is ((de. de) K* -). Note the quadratic growth in parameters as the number of chosen modes K increases. This results in a higher number of trainable parameters for inputs of reasonably high resolutions (e.g., 512x512, or above). Considering this inefficiency and the lack of local convolutions, we consider an approach to distribute the apriori allotted paramcter budget across a scries of branches, each processing the inputs at different resolutions, mimicking the multi-scale modeling paradigm (Chen et al., 2021; Rahman et al., 2023; Gupta & Brandstetter, 2023). However, we differ from the prevalent approaches in that we do not downsample the incoming spatial resolution in the local branch but decompose it into smaller domains (i.e., patches) to process each independently. Therefore, the parameter complexity of the local Fourier layers performing local spectral convolution on the patches of 2D spatial data would be ((de do) (ne (Pazze/ 2 + 1)) L). where Prize denotes the patch size, +1 is for the Nyquist frequency, and n2 the x-axis resolution of the patches. Note that we retain all Fourier modes in the local branch.
3.4 Fourier Layer Parameter Budget for 3D Spatial Data
Extending the analysis of the dependence of trainable parameters on the spectral filter size K for 2D spatial data in $3.3 to 3D spatial data, we obtain the parameter complexity of the global Fourier layer performing global spectral convolutions as O(de. de) 10---- C), while noting the cubic growth in trainable parameters
increase the number of chosen Fourier modes K for more accurate reconstruction of predictions and de indicating the hidden channel dimension. In contrast, the parameter complexity of the local Fourier layers performing local spectral convolution on the patches of 3D spatial data would be ((de de) (Na my (Paize/ /2 + 1)) L), where Psize denotes the patch size, +1 for the Nyquist frequency, and n, and ny are the x- and y-axes resolutions of the patches. Note that, as in the case of 2D spatial data, we always retain all Fourier modes in the local branch all layers.
3.5 Distributing Parameters across Local and Global Branches
Following the observations made in $3.3, we consider a scenario where the goal is to train a neural operator given a parameter budget. A baseline FNO can be trained with this budget cost to reach a certain accuracy. In contrast, we argue that we can reach the same accuracy using fewer parameters with LOGLO-FNO by making the allotted parameters less concentrated on the global branch (thereby controlling the quadratic parameter growth in terms of selected modes) and distributing them to the local branch, which has a more subdued parameter growth (for modes) depending on the patch size. If, on the other hand, we use the entire parameter budget, better accuracy than the baseline FNO model can be achieved (see Figures 3, 14, 15, 16, and 20).
4 Experiments and Results
In this section, we explain the different training setups we use and present the quantitative results of the baselines and LOGLO-FNO on the six challenging time-dependent PDE problems, ie., five 2D PDEs (Kolmogorov Flow, CNS Turb, Wave-Gauss, Compressible Euler four-quadrant Riemann problem, and Diffusion-Reacctoon) and a 3D PDE (Turbulent Radiative Mixing Layer from the Well benchmark), introduced and described in Appendix B.
Baseline Models. We consider the following neural operators as relevant and strong baselines in our work. The FNO-related baselines include base FNO (Li et al., 2021; Kossaifi et al., 2024a), U-FNO (Wen el al., 2022) that is well-suited for multiscale modeling, F-FNO (Tran el al., 2023) that factorizes the Fourier transform over the spatial dimensions, and variants of NO-LIDK (Liu-Schiaffnni et al., 2024) that achieve local convolutions. Morcover, we include a modern version of U-Net from PDEArena (Gupta & Brandstetter, 2023), LSM (Wu et al., 2023) that applics spectral methods in learned latent space, and a physics-attention based transformer, namely, Transolver (Wu et al., 2024). Further specific details of baselines are in Appendix C.
fa.ka. width or hidden channels the Neural Operator framework: gattth..ww//eeemmnnnneeeeereeree
6

---

Published in Transactions on Machine Learning Research (12/2025)
Training Objective, Procedure, and Evaluation Metrics. We train the baselines using the standard MSE loss (CMSE), whereas our frequency loss term is added on top for the LOGL.O-FOO models. Consequently, the 1-step training loss for N trajectories, each comprising T timesteps, is,
T-1 0* = arg min ΣΣ ((No(u))a=t))) n=
C = CMSE + A Creq: 0 H A S
(6)
#
where ud,, are the input-outpit pairs constructed from trajectories and C is the weighted sum of cost functions, MSE (CMSe) and our band-classffed frequency loss (Ctrea). Specifically, we minimize only the spectral errors corresponding to the mid-and high-frequency bands in our experiments. We use the Adam optimizer and halve the learning rate every 33 epochs in the case of Kolmogorov Flow and 100 epochs for Iiffusion-Reaction 2D for a fair comparison with NO-LIDK (Liu-Schiafinii et al., 2024). However, the learning rate halved every 10 epochs on the Turbulent Radiative Layer 3D datasct due to reduced training epochs. The full set of hyperparameters is listed in Appendix O.8. We evaluate metries from PDEBench (Takamolo et al., 2022), which contain spectral, physics- and sata-vie--bbsee error measures (fRMSE, cRMSE, nRMSE, MaxError). Additionally, we include metrics that measure the energy spectra deviations, viz. MELR and WLR (Wan et al., 2023). More elaborate details on the evaluation metrics can be found in Appendix G.1.
4.0.1 1-step Training and Evaluation
2D Kolmogorov Flow. We train ing 1-step loss, a.k.a. teacher-forcing -tt + -t. This training scheme mimics the functioning of classical numer- ical solvers and is commouly employed (Gupta & Brandstetter, 2023).
Table 1: 1-step and 5-step AR evaluation of LOGLO-FNO compared with SOTA baselines the lest set of 2D Kolmogorov Flow (Li et al., 2022b). REL. % DIFF indicates improvement (-) or degradation (+) with respect to FNO. LOGLO-FNO uses 40 and (16, 9) modes in the global and local branches, respectively, whereas the width is set as 65. NO-LIDK" denotes using only localized integral kernel, NO-LIDK' means only differential kernel, and NO-LIDK means employing both. Transolver' indicates a
Gaussian noise is added in order to count for the distribution mismatch be- tween this type of teacher-foor werter-fforin--eeee based training and autoregressive rollout for in- ference (Pfaff et al., 2021; Stachenfeld et. al., 2022). In a similar spirit, we in- ject adaptive Gaussian noise, which is de- pendent on the high-frequency structures extracted by the HFP module (see Ap- pendix E for full details). Further, we found it essential to employ gradient clip- ping for stabilized training when model- ing high frequencies. The LOGLÓ-FNO models have boen trained with a single local branch (patch size 8 x 8 or 16 x 16 for 2D spatial data) in addition to single global branch operating on the full spatial resolution.
longer training time of the model for 500 epochs due to convergence issues at shorter training epochs of 136.
Model
aRMSE (L)
IRMSE())
(RMSE(M)
IRMSE(H) (1)
1-step Evaluation
U-Net
13- 10-1
2.21 10-2
3.57- 10
1.39 10-2
Transolver
1.39
2.71
448 10
1.65 10
FNO
1.47
1.36 2
2.05 2
2
F-FNO
1.37 -10-1
1.49 -10-
2.15 -10-2
4.36 -10
LSM
1.36-10-1
20
4.42-10
1.61-
U-PNO
1.12 10
1.27-10 10
2.22-10
3.71 10
NO-LIDK
1.33 10-1
10-2
2.89-10"
4.55 10
NO-LIDK
1.11 10
1.65 10
2.29 2.29-10"*
3.91 10-2
NO-LIDK!
1.07 10
1.41- 10
2.46 -0
3.82
LOGLO-FNO
1.21
1.81
3.54
REL. Dur
27.21
-11.22
-11.8
-2161 %
5-step Autoregressive Evaluation
U.Nel.
2.65
6.53 10
1.04 10
9.38 10
Transolver'
3.36 10
7.11-00
1.51 10
1.18-10
FNO
2.35 -10
3.37 -10
5.50 -10
8.37 -10
1-step Evaluation. Once trained, we evaluate the models for 1-step errors. A summary of results comparing pro- posed LOGLO-FNO with range of strong baselines, including the FNO-based current. state-of-the-art NO-LIDK (Liu- Schiaffini et al., 2024), in Table
F-FNO
2.28 -10
3.60 -10
5.52 -10
8.12 -10
LSM
3.18 10-1
7.6- 16-2
1.11 1-1--0--1
1.12- 10-100
U-PNO
2,03
2.69
5.33 5.33-10'
7.35 10
NO-LIDK"
2.39 10
3.14 2
6.
8.91-00
NO-LIDK
2.01-10-1
3.71 10
5.96-10
7.6-10-2
NO-LIDKI
2.05- 10
3.31 10
-0-2
7.74 10
LOGLO-FNO
1.92 10
2,80
10
4.55 -0
6.93 10
REL. % DIPP
-18.39 96
-16.86 %
-21.02 %
-17.26 1
Since our method also performs convolution using local spectral kernels, we compare our results with their local integral kernel (NO-LIDK*)) scores, whereas the other variants (NO-T.IDK", NO-LIDK!) are provided for
7

---

Published in Transactions on Machine Learning Research (12/2025)
0.06
0.2
FNO LOGLO-FNO
FRMSE (Low)
0.04
IRMSE (Mid)
0.1
IRMSE (High)
0.08
0.08
0.06
0.06
0.02
0.04
0.04
0.02
4 12 20
32
48
64
4
12
20
32
48
64
4 12 20
32
48
64
Fourier modes (global)
Fourier modes (global)
Fourier modes (global)
Figure 3: Comparison of FNO vs. LOGLO-FNO showing 1-step IRMSE (4) on the test set of Kolmogorov Flow dataset (Re 5k) (Li et al., 2022b) for varying number of modes in the global branch and the full set of modes (i.e., (16, 9) for patch size (16 16)) in the local branch.
the sake of completeness. We observe that LOGLO-FNO oulperform the baselines all metrics. Since set out to model high-frequencies, note that the mid- and high-frequency errors are close to 12% and 25% less compared to base FNO, indicating better preservation of high-frequency details, whereas MELR down by over 75% (see Table 8). The results of a study analyzing the influence an increasing number global branch Fourier modes and full count of local branch Fourier modes (i.e., (16, 9) for patch size (16 16)) for LOGLO-FNO on the frequency and energy spectra errors are visualized in Figure 3 and Figure 16, respectively. More plots showing other metries placed in Appendix G.4.
Log Spectral Energy Error Magnitude
Radially Binned Spectral Error
0.035
0.06
FNO
10
NO-LIDK
Energy of Spectral Error
UNett-Mod
LSM
20
FFT Index (la)
Transolver
0.025
Low-freq region
30
Mid-freq region
0.03
High-treq region
40
0.015
0.02
50
0.01
60
10
30
40
60
20
40
60
80
FFT Index (lx))
Radial Distance (r)
Figure 4: Comparison of radially binned spectral errors of baselines and LOGLO-FNO the Lest set of Kolmogorov Flow (Re = 5k) (Li et al., 2022b).
Autoregressive Evaluation. In this setup, we evaluate the 1-step trained models on autoregressive rollouts for varying numbers of timesteps and compute the errors. The 5-step autoregressive rollout errors are shown in Table 1. We observe that LOGLO-FNO outperforms other models on seven out of the ten metrics and is superior to FNO on all metrics, achieving noticeable reduction in mid- and high-frequency errors.
8

---

Published in Transactions on Machine Learning Research (12/2025)
These 5-step and extended rollout results (sce Figure 5) indicate that LOGLO-FNO is more robust to the autoregressive error accumulation problem than base FNO. Further metrics are visualized in Figures 17, 18, and 19 and the results with the full set of evaluation metrics are provided in Table in Appendix G.2.1.
0.4
0.4
IRSE (Low)
FRMSE (Mid)
FRMSS (FFigh)
0.2
0.1
0.08
0.1
0.04
10
15
20
25
5
10
15
20
5
10
15
20
25
AR Rollout Timeslops
AR Rollout Timesleps
AR Rollout Timesleps
PNO (K=8)
PNO (K=16)
FNO (K=32)
FNO (K=64)
LOGLO-FNO (K=8)
LOGLO-ENO (K=16)
LOGLO-FNO (K332)
LOGLO-FNO (K=64)
Figure Comparison ENO vs. LOGLO-ENO for autoregressive rollout showing fRMSE (())) error growth over variable number of modes (K) in the global branch and varying timesteps in the trajectories on the test of Kolmogorov Flow (Re 5k) (Li al., 2022b). The local branch uses patch size of 16 x 16.
CNS Turb 2D. We extend our exper- iments to include compressible Navier- Stokos equations (see Eqn. 8 in Ap- pendix B.2 for details on the PDE), specif- ically with turbulent initial conditions and sonic regime (M0c.=1.0).
Table 2: Top: 1-step and 5-step AR evaluation of LOGLO-FNO compared with SOTA baselines the test set of CNS Turb 2D. LOGLO-FNO uses 40 and (16, 9) modes in the global and local branches, respectively, whereas the width is set 65. NO-LIDK denotes using only localized integral kernel, NO-LIDK means only differential kernel, and NO-LIDK means employing both. REL. % DIFF indicates improvement (-) or degradation (+) with respect to FNO.
The dataset consists of 1k trajectories, of which 900 for training and 100 for testing, following PDEBench (Takamoto et al., 2022) split. Input-output data pairs are constructed by slicing succes- sive frames across the timesteps, yielding 18k for training and 2k for testing. Each snapshot has two scalar fields (i.e., den- sity and pressure) and one 2D vector field (i.c., velocity: velocity-x, vclocity-y).
Model
#RMSE (J)
(RMSEL))
(RMSE(M)
(RMSE(H) (L)
CNS Turb 2D: 1-step Evaluation
U-Net
10-2
2.47 10-3
2.9 10-3
1.48 10-3
Transolver'
4.22 10
4.77 10
4.11 10
4.47 10
FNO
4.82 10 2
128 10
1.78 10
1.57
LSM
8.08 10
6.17 10
60-5
2.01 10
U-FNO
4.49 10
2.24 10
2.01 10-3
1.43 10
NO-LIDE"
10-1
1.5-10 10-3
2.14 10-3
1.98 10 10
NO-LIDK
5.54
1.35
L.8- 10
1,76
We replicate the 1-step training strategy of Kolmogorov Flow 2D problem, but in- stead use the AdamW optimizer for all models. We observe from the 1-step and 5-step AR errors listed in Table that LOGLO-FNO yields consistent improve- ments across spatial and spectral metrics and outperforms all baselines in both eval- uation settings.
NO-LIDK
5.57 10
1.37 1.37-10 10-3
L.9- 10-3
1.77 10-3
LOGLO-FNO
10
9.01 10-1
1.1 10-3
1.37 10-3
REI. % DIFF
-1.91 %
29.57%
38.1 %
-12388 %
CNS Turb 2D: 3-step Autoregressive Evaluation
U-Net
10
3.08
Transolver'
10
9.21
10
4.7.10
FNO
7.78 10-2
3.76 10-3
5.37 10
2.4-10-
LSM
10
3.4-10
10
4.16-10
U-FNO
8.42
10
2.54-10
NO-LIDK"
3.55-10
4.14
5.92-10
2.62-10
NO DI-DIKK
7.88-10"
3.82 10
5.35 10
2.44-10-3
The results on the full set of evaluation metrics are provided in Table in Ap- pendix G.2.1, and the hyperparameters are placed in Appendix O.
NO- NO-LIDKI
7.91 10
3.82 10
5.38-10 10-3
2.47-10-3
LOGLO-FNO
10-2
3.97 10-3
2.12
REL. % DIEF
-14.31 5
-26.7
-26.09 %
-11.42%

---

Published in Transactions on Machine Learning Research (12/2025)
Wave- Gauss 2D. As additional problem exhibiting different physical phenomenon of wave propagation, we in- chude the 2D wave equation, specifically Wave-Gauss, from Poseidon (Herde et al., 2024). We refer the readers to Eqn. Appendix B.3 for details on the PDE and the initial and boundary conditions. The spatially dependent propagation speed is generated as a sum of Gaussians.
Table 3: 1-step and 5-step AR evaluation of LOGLO-FNO on the test set of Wave-Gauss 2D. LOGLO-FNO uses 20 and (16, 9) modes the global and local branches, respectively, whereas the width is set to 32. NO-LIDK denotes using only localized integral kernel, NO-LIDK' means only differential kernel, and NO-LIDK' means employing both. REL. % DIFF indicates improvement or degradation (+) with respect to FNO.
Model
nRMSE (J)
(RMSEL)
(RMSE(M)
(IMSE(II) (I)
Wave-Gauss 2D: 1-step Evaluation
Transolver"
2.29 10-1
3.51 10-2
2.76 10
2.2 10-3
As with the previous problems in this sec- tion, we employ 1-step training strategy with the objective in Eqn. and evalu- ale on 1-step and 5-step rollouts. The training data contains about 150k imput- output pairs, 900 for validation, and 3.6k is reserved for testing.
FNO
4.03 10-2
7.5 10-3
5.45 10-3
6.22 10
LSM
3.08 10
4.91 10-3
3.86 10-3
8,54
U-FNO
3.49 10-2
6.62 10-3
4.52 10
7
NO-LIDK
3,93 10
6.95 10
5.1 5.1-10-1
10 1.
NO-LIDK"
3.58 10-2
6.49 10-3
4.78-10
7,56
NO-LIDK!
3.5- 10-2
6.31 10-3
4.61 10-3
7.4-10-4
LOGLO-FNO
3.11 10
6.43
3.84-10"
10
REL. % DIEP
-2.77 %
-1.
-29.54%
-44.65%
Similar to the input data formatting setup of Herde et al. (2024), we concatenate the static propagation field the time- dependent displacement field in the in-
Wave- Gauss 2D: 5-step Autoregressive Evaluation
Transolver"
3.62 10
7.73 10-2
3.47 10
2.93 10
FNO
6.75 10
166 10
8.35 10
8.57 10
LSM
4.34 10-2
8.64 10-3
5.68 10-3
1.1 10-3
U-FNO
6.26 10-2
1.6-10" 10-2
7.64- 10-3
9.27 10
put and let the model learn to perform the identity mapping (i.e., copying) of the propagation field in the output.
NO-LIDK
5.96 10-2
1.34 10-2
7 10-3
NO-LIDK
1.21 10-1
2.43- 10-2
1.12- 10-2
-
NO-LIDK'
1.06 10-1
2.1- 10-2
1.02-10 10-2
- 10-3
LOGLO-PNO
4.81 10
1.29 10
5.43 10
4.57
As we observe from Table 3, although L.OLLO-FNO achieves the lowest. mid and high frequency errors, the LSM baseline model outperforms all other models on the spatial and low froquency error metrics, and this trend is maintained on the complete trajectory rollouts (sce Table 10).
REL. % DIFF
28.76%
19.52%
34.91%
46.6%
The results on the full set of evaluation metries and extended full trajectory rollout results are provided in Table 10 in the Appendix G.2.1, and the hyperparameters for all the models are in Appendix O.
Compressible Euler four-quadrant Riemann problem 2D. A challenging problem for neural operators is modeling shock waves and contact discontinuities. Towards this end, we conduct experiments on the Compressible Euler four-quadrant Riemann problem 2D from Poseidon (Herde et al., 2024). This PDE has four variables, viz. density, horizontal and vertical velocity fields, and pressure. See Appendix B.4 for further details. As with other problems this section, we optimize using 1-step loss (cf. Eqn. 6), where the models are trained to predict all four field variables for future timestep. We then evaluate the performance of LOGLO-PNO and compare against Base FNO and NO-LIDK variants. The quantitative results (i.e., 1-step and 5-step rollouts) are shown in Table 4, whereas the qualitative visualization of predictions along with the errors for the density, velocity-x, and pressure fields are shown in Figures 6, 7, and 8, respectively.
Ground Truth
Base FNO Pred.
Base FNO Error
LOGLO CIGLO-FNO Pred.
LOGLO-FNO Error
2.88
2.88
2,88
2.22
2.22
0.14
2.22
0.14
Denaity
0.91
0.91
0.14
0.91
0.14
0.25
Figure 6: Qualitative results comparing the 1-step density field predictions of Base FNO and LOGLO-ENO on a random sample from the test set of compressible Fuler four-quadrant Riemann problem 2D.
10

---

Published in Transactions on Machine Learning Research (12/2025)
Table 4: 1-step and 5-step autoregressive rollout evaluations of LOGLO-FNO compared against SOTA basclines on the test of Compressible Euler four-quadrant Riemann problem 2D (Herde al., 2024). LOGLO-FNO uses 40 and (16, 9) modes in the global and local branches, respectively, whereas the width is set to 65. NO-LIDK" denotes using only the localized integral kernel with radius cutoff value of 0.0078125, and NO-LIDK means only the differential kernel is enabled.
Model
nRMSE (1)
(L
(RMSE(M)
(RMSE(H) (L)
1-step Evaluation
FNO
2.08 10-2
9.42 10-1
1.03 10-3
1.41 10-3
NO-LIDK"
3.58 10
1.5 10-3
2.35 10
2,47 10-3
NO-LIDK
1.66 10
1.05 10
8.54 10 4
1.11 10
LOGLO-FNO
1.5- 10
5.79 10
5.09 10
9.67 10
REL. % DIFF
-2..9%
-38.56%
-50.11%
-31 31.58%
5-step Autoregressive Evaluation
FNO
3.43 10-2
2.38 10-3
2.7 10
2.35 10-3
NO-LIDK"
4.02 10
3.19 10
3.58 10 3
2.47 10 3
NO-LIDK
3.13 10
2.5 10
2.38
10
2.12 10
LOGLO-FNO
2.34 10
1.64 10
1.35
10
1.56 10
REL. DIFF
-31.75%
-367%%
-50%
-3.4 %
We observe from Table 4 that LOGL.O-FNO achieves better results than the baselines on all metries, while reducing the mid- and high-frequency errors by over 50% and 30%, respectively, The full set of evaluation metrics and extended full trajectory rollouts are placed in Table 11 in the Appendix G.2.1, and the hyperparameters for all models studied are in Appendix O.
Ground Truth
Base FNO Pred.
Base FNO Error
LOGLO-FNO Pred.
LOGLO-FNO Error
0.42
0.42
0.06
0.42
0.06
0.12
0.12
Velueily-x
0.18
0.18
0.00
0.18
0.00
.0.79
20.79
-0.06
-0.79
0.06
Figure 7: Qualitative results comparing 1-step horizontal velocity field predictions of Base FNO and LOGLO- FNO on a random sample from the test set of compressible Euler four four-quadrant Riemann problem 2D.
Ground Truth
Base FNO Pred.
Base FNO Error
LOGLO-FNO Pred.
LOGLO-PNO Error
0.30
0.38
0.04
0.38
0.04
0.29
0.29
0.29
Pressure
0.20
0.20
0.00
0.20
0.00
0.11
4.00
0.01
0.01
-0.04
0.01
-0.04
Figure 8: Qualitative results comparing 1-step pressure field predictions of Base FNO and LOGLO-FNO models on a random sample from the test set of compressible Euler four-quadrant Riemann problem 2D.
The above qualitative visualizations, specifically the error maps, reveal that LOGLO-FNO mitigates the errors to considerable extent in the regions of high gradients. However, the model still struggles to climinate them entirely, signifying the challenge of modeling discontinuities and sharp gradients that arise from shocks.
11

---

Published in Transactions on Machine Learning Research (12/2025)
Turbulent Radiative Layer 3D (TRL3D). Noting the absence of sup- port for local convolutions in FNO for 3D spatial data, we extend the LOGLO- FNO implementation to 3D and conduct experiments on a challenging version of a time-dependent turbulence simulation, viz., Turbulent Radiative Layer 3D (Field- ing et al., 2020; Ohana et al., 2024) from the Well benchmark.
Table 5: 1-step and autoregressive evaluation of LOGLO-FNO compared with statc-of-the-art basclines on the test set of Tur- bulent Radiative Mixing Layer 3D dataset (Fielding et al., 2020; Ohana et al., 2024). REL. % DIFF indicates an improvement (-) or degradation (+) with respect to base FNO, which uses a spectral filter size of 12 and 48 hidden channels. LOGLO-FNO uses 12 and (16, 16, 17) modes in the global and local branches, respectively, whereas the width is set to 52. (*This is because the Turbulent Radiative Layer 3D dataset has 2x more sampling points on the z-axis relative to the x and y axes resolutions (128 128 x 256).) Base FNO means our implementation of FNO (sce Figure 1). [t:t+A[] symbolizes the inclusion of the timesteps at both ends of the interval when computing time-averaged vRMSE.
Similar to the Kolmogorov Flow 2D setup; the models are trained using 1-step train- ing loss (cf. Eqn. 6), however, with the AdamW optimizer and only for 50 epochs by halving the learning rate every 10 epochs, maintaining the proportion of learning rate decays as with other prob- lems. We chose this setup since we ob- served overfitting when training for longer epochs, such as 136 or even further. We attribute this behavior to the limited amount of available training data. Fol- lowing Ohana et al. (2024), we consider the full data covering the entire range of fcool parameters, where fcool E (0.03, 0.06, 0.1, 0.18, 0.32, 0.56, 1.00, 1.78, 3.163. For each looal value, ten trajectories, each spanning 101 timesteps and of spatial res-
1-step Eval.
Autoregressive Evaluation
Model
1-step vRMSE (L)
[6:1]] Time-Avg. vRMSE (1)
[13:30] Time-Avg. vRMSE (U)
FNO
5.28 10
8.1 111-10
9.4 10
TFNO
1.19-10"
>10
>10
U-Net
3.73. 10-1
9.5- 10-1
1.09 10"
CNextU-Net
3.67 10-1
7.7 10-1
8.6 10-1
LSM
2.74-10-1
7.67 10-1
1.02 10
NO-LIDK
4.05 10
6.0- 10
6.52 109
Base FNO
3.18-
7.49 10-1
8.58 10-1
LOGLO-FNO
2.76 20-
7.09 7009-10-1
7.72 10-1
REL. % DIFF
-13.22%
-5.3%
-9.98%
Model
nRMSE (1)
IRMSE(L)
(RMSE(M)
(RMSE(H) (U)
1-step Evaluation
olution 128 x 128 x 256, are provided.
LSM
8.39 10
2.4- 10-
9.7-10"
3.43 10
The datasel also comes with an explicit train (8), validation (1), and test (1) splits. We then construct 7200, 900, and 900 1- step data pairs for training, validation, and testing, respectively.
NO-LIDK
1.34-10'
1.22-10
1.23 10
Base FNO
3.06 10-
188-10-2
4 10-2
2.89-10-
LOGLO-FNO
2.65 10
4.45 10
3.68-10-2
2
REL. % DIFF
-1..27%
-8.61%
-19.57%
-14.0%%
Following recent investigations for conditioning the neural operators to generalize to unseen PDE parame- ters (Takamoto et al., 2023), we append the fcool coefficients as additional channels. The goal of the so-called forward problem for this PDE is then to predict the scalar (i.e., density and pressure) and vector (i.e., given the solution at the previous timestep
velocity-x, velocity-y, and vclocity-z) physical quantities of w²1((.) (.) (.)). Further details on the PDE are provided in Appendix B.6.
Note that we have improved the 1-step vRMSE score (on the TRL3D test set) of the baseline FNO model by 41.4% compared to results reported by Ohana et al. (2024) (cf. Table 2: Model Performance Comparison) and, hence, our version of FNO is strong baseline. We further note that the local integral kernel (DISCO cony layers) of NO-LIDK (Liu-Schiaffnni et al., 2024) is not implemented for 3D spatial data and, hence, is not compared against in our experimental sctup. We observe that LOGLO-FNO achieves the best results on all metrics and outperforms UNextUNNet, the best results of Ohana et al. (2024) on the Turbulent Radiative Layer 3D test dataset, on the vRMSE metric by 24.5%. The qualitative visualizations of predictions are provided in Appendix H.0.2 and the table with the full set of metrics is in Table 12 in the Appendix.
4.0.2 Autoregressive Training and Evaluation
We evaluate LOGLO-FNO on a fully autoregressive training sctup on the Diffusion-Reaction 2D PDE from Takamoto et al. (2022). The model is trained with an initial context of 10 frames (i.e., At 10), predicting one future state at a time until reaching the fully evolved state: u(t -., u(t,)). In other
12

---

Published in Transactions on Machine Learning Research (12/2025)
words, since each simulation trajectory consists of 101 timesteps, AR rollout is performed for 91 timesteps both during training and evaluation to be consistent and for a fair comparison with the training setup of NO-LIDK (Liu-Schiafinni et al., 2024).
2D Diffusion-Reaction. We observe from Table 6 that LOGLO-FNO achieves signil- icant reduction in frequency errors across all bands (55,, 43% 21%, resp.) compared to baseline FNO and superior to NO-LIDK on low- and mid-frequencies (30%, 30.4%, resp.). The tabulation of results with all metries provided in Table 13 in the Appendix G.2.2.
Table 6: Fully autoregressive evaluation of LOGLO-FNO compared with SOTA baselines the test set of challenging 2D Diffusion-Reactioo coupled problem from PDEBench. We also report the REL. % DIFF to indicate the error improvement (-) with respect to bascline FNO.
Model
nRMSE (J) [LMSEL)
IRMSE/M)
RMMS HI) (a)
U.Net
8.4 8.4-10-
1.7 1.7-10-2
8.2 -10-4
5.7 6.7-10-2
Thansolver
2.63 10-1
2.99 2.99-10-3
2.03 10-3
5.51-10-4
U-FNO
26--10
3.4 a
1.6-10
2.6
5 Ablation Studies
FNO
8.3 8.3-10-2
6.2 6.2-10-
5.6 -10-1
24 -10
F-FNO
7.0-10-2
9.6-10-4
4.7
10
LSM
4.47-10-1
7.17-10-3
24-10--
3.67-10-4
In order to investigate the effect of each of the proposed modules in LOGLO-ENO or the ra- dially binned frequency loss, we conduct an ab- lation study on the Turbulent Radiative Layer 3D dataset, isolating the components. The LOGLO-FNO model (cf. Fig. 1) by default uses the local spectral convolution and the HFP branches, and is trained with the radially binned spectral loss. This configuration corresponds to the result on the second row in Table 7 and achieves the best result, emphasizing the need to retain all the proposed contributions.
NO-LIDK fle. inl)
6.
4.0-10
4.6
1.5 -10
LOGLO-FNO
6.1-10-2
2.8-10-1
3.2- 10-4
1.9 1.9-10-1
REL. % DIFF
-22.75
-54.84 %
42.86 %
-20.33 %
Aflerwards, to obtain the result in the next row, we disable the frequency loss in the training objective Equ. 6 while retaining all other hy- perparameters as before, quantifying the effect of the absence of radially binned frequency loss.
Table 7: 1-step results of LOGLO-FNO on the test set of TRL3D (Fielding et al., 2020; Ohana et al., 2024). Base FNO uses a spectral filter size of 18 and 48 hidden channels. LOGLO-FNO uses 16 and (16, 16, 17)" Fourier modes in the global and local branches, respectively, whereas the width is 52. ("Turbulent Radiative Layer 3D dataset has 2x more sampling points on the z-axis relative to the x- and y-axes resolutions (128 x 128 x 256).)
In the next setup, additionally disable the HFP branch to yield configuration of train- ing LOGLO-FNO with only the local branch and MSE loss. The result for this is on the penultimate row of Table 7.
Model
nRMSE (L) (RMSE/L) (RMSE(M) (RMSE(H) vRMSE (L)
1-etep Evaluation
To isolate the effect of removing the HFP branch, we disable it and train LOGLO-FNO with the training objective in Eqn. 6. The re- sult of this model configuration is presented in the last row of Table 7.
Base FNO
2.97 10 5.17 10-2 -.4
2.76
10
3.00 10-1
LOGLO-FNO
(+Hrg lose, +HFP)
2558 11--10-1 10
10-2-10-2
2.45
10 2.68
LOOLLO-NNO
(-freg loss, + HFP)
2.76
1.-10-22
4.00-10
263-10"2
288 18-1
LOGLO-FNO
(freg loss, -HFP)
207
4.
1.13-10-2
200--0-2
To summarize, although just using any one of the proposed contributions is sufficient to
L.GGLO-FN
(1 froq loss, -HIP)) 2.66 10-1 4.53
10-2
3.76-10"
2.48 10-2
2.76 100
obtain an improved result over baseline FNO, both local spectral convoluution and HFP branches, and the radially binned frequency loss are necessary to achieve the best result.
6 Conclusions, Limitations, and Future Work
In this paper, we have proposed an enhancement to the FNO architecture (Li et al., 2021; Kossaifi et al., 2024b;a) through auxiliary parallel branches for local spectral convolution and high-frequency feature propagation. Through our experiments on six relevant and highly challenging PDE problems, namely, Kolmogorov Flow 2D, CNS Turb 2D, Wavo-Gauss 2D, Compressible Euler four-quadrant Riemann problem 2D Turbulent Radiative Layer 3D, and lineally coupled rise to turbulence high frequencies, we have demonstrated the promise to significantly reduce both the
2D problems that give
13

---

Published in Transactions on Machine Learning Research (12/2025)
spatial and spectral errors, showing the importance of local convolutions in mitigating the spectral bias to considerable extent. In addition, our proposed radially binned spectral loss has also been found to be useful in this aspect, As a result of incorporating a local spectral convolutional branch in the architecture, our model has paved a way for reducing the number of trainable paramcters to reach the baseline model accuracy. The current work, however, in order to allow a fair comparison to Liu-Schiafini al. (2024), has not considered any sophisticated parameter-effiient spectral convolution methods (e.g. tensor factorization techniques introduced in Kossaifi et al. (2024b)), which we see as limitation. Future work could look into improving the scalability of LOGLO-FNO, provide theoretical analysis of its benefits, explore optimized implementation to accelerate the training and inference speed.
Broader Impact Statement
Accelerated and efficient Neural PDE Solvers help reduce the cost of running cost-prohibitive simulations such weather forecasting, cyclone predictions, and extreme weather events. As consequence, disaster preparedness, the design, and manufacturing timelines can be accelerated, resulting in life and cost savings well as decreased CO2 emissions. As negative side effect, we cannot rule out the possibility of misuse by bad actors, as computational fluid dynamics and turbulent flow simulations are also used to design military equipment within the military-industrial complex.
Acknowledgmentt
The authors are thankful for the suggestions and questions of the anonymous reviewers OpenReview, which have been helpful in improving the manuscript. We thank Miguel Liu-Schiaffini for the correspondence on NO-LIDK and Julius Berner for the initial discussions on local convolutions. Furthermore, we thank Artur Petrov Toshev for the insightful discussions on operator learning and brainstorming ideas in general. Moreover, we thank Julius Herb for the discussions and feedback on the initial draft. In addition, we thank Makoto Takamoto for the fruitful discussions during the rebuttal phase. Marimuthu Kalimuthu and Mathias Niepert funded by Deutsche Forschungseemennscaaft (DFG, German Research Foundation) under Germany's Excellence Strategy EXC 2075 390740016. We acknowledge the support of the Stuttgart Center for Simulation Science (Sim Toch). The authors thank the International Max Planck Rescarch School for Intelligent Systems (IMPRS-IS) for supporting Marimuthu Kalimuthu and Mathias Niepert. Last but not least, the authors gratefully acknowledge the computing time provided to them at the NHR. Center NHRACES at RWTH Aachen University (project number p0021158). This funded by the Federal Ministry of Education and Research, and the state governments participating on the basis of the resolutions of the GWK for national bigh-peef computing at universities (htip:// (http://www.n
http://www.nh-veeein.. www.nnhrr--eeeee.c uuseeee-paaatti mnsere-partner).
14

---

Published in Transactions on Machine Learning Research (12/2025)
References
Jason Ansel, Edward Z. Yang, Horace He, Natalia Gimelshein, Animesh Jain, Michael Voznesensky, Bin Bao, Peter Bell, David Berard, Ergeni Burovski, Geeta Chauhan, Anjali Chourdia, Will Constable, Alban Desmaison, Zachary DeVito, Elias Ellison, Will Feng, Jiong Gong, Michael Gschwind, Brian Hirsh, Sherlock Huang, Kshiteej Kalambarkar, Laurent Kirsch, Michael Lazos, Mario Lezcano, Yanbo Liang, Jason Liang, Yinghai Lu, C. K. Luk, Bert. Maher, Yunjie Pan, Christian Puhrsch, Matthias Reso, Mark Saroufim, Marcos Yukio Siraichi, Helen Suk, Shunting Zhang, Michael Suo, Phil Tillet, Xu Zhao, Eikan Wang, Keren Zhou, Richard Zou, Xiaodong Wang, Ajit Mathews, William Wen, Gregory Chanan, Peng Wu, and Soumith Chintala. Pytorch 2: Faster machine learning through dynamic python bytecode transformation and graph compilation. In Rajiv Gupta, Nael B. Abu-Ghazaleh, Madan Musuvathi, and Dan Tsafrir (eds.), Proceedings of the 29th ACM International Conference Architectural Support for Programming Languages and Operating Systems, Volume 2, ASPLOS 2024, La Jolla, CA, USA, 27 April 2024- 1 May 2024, pp. 929-999 947. ACM, 2024. doi: 10.1145 10.1145/3620665... 002.6600066663663360
URL https: / //doi.//oioo/oo http://doo.oog/////// 10. 1145/3620663 2ww.//3336666666666666.6. 3640366.
Ronen Basri, David W. Jacobs, Yoni Kasten, and Shira Kritchman. The convergence rate of neural networks for learned functions of different frequencies. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' d'Alché Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Newral Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Conada, pp. 7763-4772, 2019.
Johannes Brandstetter, Rianne van den Berg, Max Welling, and Jayesh K. Gupta. Clifford neural layers for PDE modeling. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.nel. 2023. URL https: http:///paareeeeeeeeeeee/e/tttaa 2404 c4x84.
Steven L. Brunton and J. Nathan Kutz. Machine learning for partial differential equations. CoRR, abs/2303.1700 2023. doi:
http//AAXXV....3.11.. URL
2303. 17078.
Steven L. Brunton and J. Nathan Kutz. Promising directions of machine learning for partial differential oquations. Nat. Comput. Sci., 4(7):483-49) 2024. doi: 10.1038/S43588 (24-00063--2-------2 URL https: //tw:./or/////ee org/10. 1038/003//400 143588-0024-0643--
Yuan Cao, Zhiying Fang, Yue Wu, Ding-Xuan Zhou, and Quanquan Gu. Towards understanding the spectral bias of deep learning. In Zhi-Hua Zhou (ed.), Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021, pp. ijeai.org, 2021. doi: 10.24963 10.24963/TJCAT..0 IJCAT.201 /3344 URL
(49633/jjcal 2021/304.
Chun-Fu (Richard) Chen, Quanfu Fan, and Rameswar Panda. Crossvit: Cross-attention multi-scale vision transformer for image classification. In 2021 IEE//CVF Internalional Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pp. 347 356. IEEE, 2021. doi: 10.1109/ ICCV48922.2021..0000 100041. URL https://doi o0g/10.100 1tw.11119/////990 IIC//8922VV22222002 00041.
Lu Chi, Borui Jiang, and Yadong Mu. Fast Fourier Convolution. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (ods.), Advances in Neural Information. Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virlual, 2020.
James W Cooley and John W Tukey, An algorithm for the machine calculation of complex fourier series. Mathemalics of compulation, 19(90):297 301, 1965.
Yimian Dai, Fabian Gieseke, Stefan Oehmcke, Yiquan Wu, and Kobus Barnard. Attentional feature fusion. In IEEE Winter Conference on Applications of Computer Vision, WACV 2021, Waskoloo, HT, USA, January 3-8, 2021, pp. 3559-3568. IEEE, 2021. doi: 10.1109/WACV 4BACCV88630.200 2020..022..0000 0006000600 http:/00.1109/AA44450000 2021 00360.
URL
15

---

Published in Transactions on Machine Learning Research (12/2025)
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 9-7, 2021. OpenReview.neg .nt, 2021. URL https:://ppereeteee http////wwevieeeeeeettoormmeeeee aeet/forumnideeicbaaarre
Xiantao Fan, Deepak Akhare, and Jian-Xun Wang. Neural differentiable modeling with diffusion- based super-resolution for two-dinensional spatiotemporal turbulence. arXiv preprint arXim:206.20000 2024.
Drummond B Fielding, Eve C Ostriker, Greg I, Bryan, and Adam S Jermyn. Multiphase gas and the fractal nature of radiative turbulent mixing layers. The Astrophysical Journal Letters, (94(2), 2020.
Cong Fu, Jacob Helwig, and Shuiwang Ji. Semi-supervised learning for high-fidelity fluid flow reconstruction. In Soledad Villar and Benjamin Chamberlain (eds.), Learning on Graphs Conference, 27-30 November 2023, Virtual Event, volume 231 of Proceedings of Machine Learning Research, pp. 36. PMLR, 2023. URL https: attpss//prroceedingg. mmrtinng...rr....mm.m.m...... wwt.eeereee/v/eeeeeeeee html.
Robert Joseph George, Jiawei Zhao, Jean Kossaifi, Zongyi Li, and Anima Anandkumar. Incremental spatial and spectral learning of neural operators for solving large-scale PDEs. Transactions on Machine Learning Research, 2024. URL
Jayesh K. Gupta and Johannes Brandstetter. Towards multi-spatiooeempomnssalll generalized PDE modeling. Trans. Mach. Learn. Res., 2023, 2023. URL http://ppeeeereeee..eeeer/rraaa/
Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Scbastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernández del Río, Mark Wiebe, Pearu Peterson, Pierre
Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. Array programming with NumPy. Nature, 585(7825): 357 362, 2020. doi: 10.1038/ 10.11008/41555 -20-6602000--99-22222
URL https: //tw///oiiii/ org/ 10. 1038//41586---/ /401/44010020229---222
Sheikh Md Shakeel Hassan, Arthur Focncy, Akash Dhruv, Jihoon Kim, Youngjoon Suh, Jaiyoung Ryu, Yoonjin Bubbleml: A multiphase multiphysics dataset and benchmarks for machine learning. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Newral Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NewrIPS 2023, New Orleans, LA, USA, December 10- 16, 2023, 2023.
Won, and Aparna Chandramowlisswwaree
Jacob Helwig, Xuan Zhang, Cong Fu, Jerry Kurtin, Stephan Wojtowytsch, and Shuiwang Ji. Group equivariant fourier neural operators for partial differential equations. In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Reseorch, pp. 12907-12930 PMLR, 2023. URL
. mlr, peress/v20//iee w202/hewwwg23a. html.
Maximilian Herde, Bogdan Raonic, Tobias Rohner, Roger Käppeli, Roberto Molinaro, Emmamuel de Bézenac, and Siddhartha Mishra. Poseidon: Efficient foundation models for pdes. CoRR, /2///22051190 2024. doi: D0.48550/ARXTV..2 24.. wwt.:/AAAXXV..4.....
URL attps://ddl org/10. 48550/arXiv.. 2405 19101.
Qingkai Kong, Caifeng Zou, Youngsoo Choi, Eric M. Matzel, Kamyar Azizzaennsheli, Zachary E. Ross, Arthur J. Rodgers, and Robert W. Clayton. Reducing frequency bias of fourier neural operators in 3d seismic wavefield simulations through multi-stage training, 2025. URL https: //ar/iv///xv...//// .rggabss/2503
Jean Kossaifi, Nikola Kovachki, Zongyi Li, Davit Pitt, Miguel Liu-Schiaffini, Robert Joseph George, Boris Bonev, Kamyar Azizzadenesheli, Julius Berner, and Anima Anandkumar. A library for learning neural operators. arXiv preprint arxi::24 10354, 2024a.
Joan Kossaifi, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, and Anima Anandkumar. Multi-grid tensorized fourier neural operator for high- resolution PDEs. Transactions on Machine Learning Research, 2024b. ISSN 2835-8856. URL http://pppneeeeeeeeeeeefoeeeeeee
16

---

Published in Transactions on Machine Learning Research (12/2025)
Nikola B. Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew M. Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces with applications
to pdes. J. Mach. Learn. Res., 24:89:1 19:97,, 2023. URL http::///j.....ogg/ppprrr//////s o html.
Samuel Lanthaler, Zongyi Li, and Andrew M. Stuart. Nonlocality and nonlincarity implics universality in operator learning. CoRR, abs/ 23s/2334.13221, 2024. doi: 10.48550/ ARXIV. 13201. URL https:
///oi..or//0/ wtt..o///.....5//rrr......
48550/arXiv 2304.. 13221.
Hongyu Li, Ximeng Ye, Peng Jiang, Guoliang Qin, and Tiejun Wang. Local neural operator for solving transient partial differential equations varied domains. Computer Methods in Applied Mechanics and Engineering, 427:117062, 2024.
Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew M. Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential oquations. In Ith International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.nel, 2021. URL
Zongyi Li, Miguel Liu-Schiaffni, Nikola Kovachki, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhat- tacharya, Andrew Stuart, and Anima Anandkumar. Learning Dissipative Dynamics in Chaotic Systems (Datasets), December 2022a. URL https://doi. wtg/ 00t.://22/2eeoo74888888088 7495555.
Zongyi Li, Miguel Liu-Schiaffini, Nikola B. Kovachki, Kamyar Axizzadeneshelii Burigede Liu, Kaushik Bhattacharya, Andrew M. Stuart, and Anima Anandkumar. Learning chaotic dynamics in dissipative systems. In Sammi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NewrIPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022b.
Phillip Lippe, Bas Veeling, Paris Perdikaris, Richard E. Turner, and Johannes Brandstetter. Pde-refiner: Achicving accurate long rollouts with neural PDE solvers. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Newral Information Processing Systems 36: Annual Conference on Newral Information Processing Systems 2023, NewrIPS 2023, New Orleans, LA, USA, December 10- 16, 2023, 2023.
Hong Liu, Zhisheng Lu, Wei Shi, and Juanhui Tu. fast and accurate super-resolution network using progressive residual learning. In ICASSP 2020 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1818 1822 2020. doi: 10.1109/ICAA SALLLL//CCAASPP77772220000033300
Xinliang Liu, Bo Xu, Shuhao Cao, and Lei Zhang. Mitigating spectral bias for the multiscale operator learning. J. Comput. Phys., 506: 106:112944, 2024. doi: 10. 10.1016/U.JCPP.222.1122 URL https://doi.oon 10. 1016100111100 jcp. 2024. 2222222222..... 112944.
Miguel Lih-Schiafiin, Julius Berner, Boris Bonev, Thorsten Kurth, Kamyar Azizzadenesheli, and Anima Anandkumar. Neural operators with localized integral and differential kernels. In Forty-first International Conference on Machine Leurning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.nef 2024. URL https:// htt.:/awwttttttoorrreeeeeeeeosse
Michael McCabe, Peter Harrington, Shashank Subramanian, and Jed Brown. Towards stability of autoregres- sive neural operators. Trans. Mach. Learn. Res., 2023, 2023. URL RFFUUtXYOG.
htwp:p/wwrrreeieoeeooeoermraaaaa fet//orummddd
Juan Molina, Mireea Petrache, Francisco Sahli Costabal, and Matias Courdurier, Understanding the dynamics of the frequency bias in neural networks. CoRR, abs/ 2ts/22055.1955 2024. doi: 10.48550 ARt.://4XXX...2...4402 URL https://doi.org 18..0.55//a...i 1405..19557
Jakin Ng, Yongji Wang, and Ching-Yan Yao Lai. Spectrum-infommed multistage neural network: Multiscale
function approximator of machine precision. In ICML 2024 AI for Science Workshop, 2024. URL
https://openrevier. htt::forrr/iiiii/hhhhdppp.ppppp
17

---

Published in Transactions on Machine Learning Research (12/2025)
Ruben Ohana, Michael McCabe, Lucas Meyer, Rudy Morel, Fruzsina J. Agocs, Mignel Beneitez, Marsha Berger, Blakesley Burkhart, Stuart B. Dalziel, Drummond B. Fielding, Daniel Fortunato, Jared A. Goldberg, Keiya Hirashima, Yan-Fei Jiang, Rich R. Kenswell, Suryanarayana Maddu, Jonah Miller, Payel Mukhopadhyay, Stefan S. Nixon, Jeff Shen, Romain Watteaux, Bruno Régaldo-Saint Blancard, François Rozet, Liam Holden Parker, Miles D. Cranmer, and Shirley Ho. The well: a large-scale collection of diverse physics simulations for machine learning. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Voncouver, BC, Canada, December 10 15, 2024, 2024. URL Abbstract-Dataane
nips. cc/paper files/ http:pppppp////aaa/e Sedd__Bnchmark____
html.
Vivek Oommen, Aniruddha Bora, Zhen Zhang, and George Em Karniadakis. Integrating Neural Operators with Diffusson Models Improves Spectral Representation in Turbulence Modeling, arXiv e-prints, art. arXiv:2409.08477, September 2024.
Zizheng Pan, Jianfei Cai, and Bohan Zhuang. Fast vision transformers with hilo attention. In Sammi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Newral Information Processing Systems 2022, NewrIPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022.
Tobias Pfaff, Meire Fortunato, Alvaro Gonzaez-Gonzale,, and Peter W. Battaglia. Learning mesh-based simulation with graph networks. In 9th International Conference Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net net, 2021. URL id=roNqYLo_xX __P
Michael Poli, Stefano Massaroli, Federico Berto, Jinkyoo Park, Tri Dao, Christopher Ré, and Stefano Ermon. Transform once: Efficient operator learning in frequency domain. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NewrIPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022.
Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred A. Hamprecht, Yoshua Bengio, and Aaron C. Courville. On the spectral bias of neural networks. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, Californía, USA, pp. 5301-5310 PMLR, 2019. URL http://proceeeding: mlr press/v97/rahamanl /at///aaaaaaaaaaaaa html. html....
Md Ashiqur Rahman, Zachary E. Ross, and Kamyar Avizzadenesheli. U-NO: u-shaped neural operators. Trans. Mach. Learn. Res., 2023, 2023. URL https: http://paarrrreeeeeeeeerrrrrrraa
Katja Schwarz, Yiyi Liao, and Andreas Geiger. On the frequency bias of generative models. In Marc'Auree Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NewrIPS 2021, December 6-14, 2021, viriual, pp. 18126-18136, 2021. URL https: Abstract html.
//proceedings. wwweodiinnn.nuuuurrne heuripss.o
Kim Stachenfeld, Drummond Buschman Fielding, Dmitrii Kochkov Miles Cranmer, Tobias Pfaff, Jonathan Godwin, Can Cui, Shirley Ho, Peter Battaglia, and Alvaro Sanchez- -Gozzazzz Learned simulators for turbulence. In International Conference on Learning Representations, 2022. URL https://openrevier fet//forummiddmmRBBo]]3
James M Stone, Kengo Tomida, Christopher J White, and Kyle G Felker. The athena++ adaptive mesh refinement framework: Design and magnetohydrodynnami solvers. The Astrophysical Journal Supplement Series, (1)(1): 2020.
Chuanhao Sun, Zhihang Yuan, Kai Xu, Luo Mai, N. Siddharth, Shuo Chen, and Mahesh K. Marina, Learning high-frequency functions made easy with sinusoidal positional encoding. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview net, 2024a. URL https://openreevier http:rrrw..eeeoomm/ooooooo.o.o..
18

---

Published in Transactions on Machine Learning Research (12/2025)
Y Qiang Sun, Ashesh K. Chattopadhyay, and Pedram Hassanzadeh. Hallucinations of Al-based models: the good, the bad, and the unstable. In 24th Conference on Almospheric & Oceanic. Fluid Dynamiccs & 22nd Conference on Middle Atmosphere. AMS, 2024b.
Makoto Takamoto, Timothy Praditia, Raphael Leiteritz, Daniel MacKinlay, Francesco Alesiani, Dirk Pftüger, and Mathias Niepert. Pdebench: An extensive benchmark for scientific machine learning. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Newral Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NewrIPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022.
Makoto Takamoto, Francesco Alesiani, and Mathias Niepert.
Learning neural PDE solvers with parameter- guided channel attention. In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. PMLR, 2023. URL https:// https://ppoceeddiie mariiinngsmrr....mm mlr. wtwpsprwsssee//e/eeet takamoto23a.aaa.mmmmaaaaa htt..mtooooaaa..........mm
Alasdair Tran, Alexander Patrick Mathews, Lexing Xie, and Cheng Soon Ong. Factorized fourier neural operators. In The Eleventh International Conference Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https://openreviiee met:forrmm/dd-mmIIIP1111
Zhong Yi Wan, Ricardo Baptista, Anudhyan Boral, Yi-Fan Chen, John Anderson, Fei Sha, and Leonardo Repeda-Núñes' Debias coarsely, sample conditionally: Statistical downscaling through optimal transport and probabilistic diffusion models. In Advances in Newral Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, New Orleans, LA, USA, December, 2023.
Rui Wang, Karthik Kashinath, Mustafa Mustafa, Adrian Albert, and Rose Yu. Towards physics-informee deep learning for turbulent flow prediction. In Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash (eds.), KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 28-27, 2020, pp. 1457-1466. ACM, 2020. doi: 10.1145/33344886340033 URL https://doi.oom org/
Yongji Wang and Ching- Yao Lai. Multi-stage neural networks: Function approximator of machine precision. Journal of Computational Physics, 504:112865, 2024.
Gege Wen, Zongyi Li, Kamyar Arizzadeneshelii, Anima Anandkumar, and Sally M. Benson. U-FNO an enhanced fourier neural operator- based deep-learning model for multiphase flow. Advances in Water Resources, 163:104180, 2022. ISSN 0309-1708. doi: http:://bbb.org/11..0///...aa www.://se//..dwwtrrsss...m 2012.111000...1111 URL surrogates. 2025. URL httpa//cc.c....aa..cccccccooa//a
https://ww..cceeneddreee
Daniel E. Worrall, Miles Cranmer, J. Nathan Kutz, and Peter Battaglia. Spectral shaping for neural PDE
Haixu WU, Tengge Hu, Huakun Luo, Jianmin Wang, and Mingsheng Long. Solving hign-dimensional pdes with latent spectral models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning, 1CML 2023, 23-29 July 2029, Honoluhu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. 37417 37438. PMLR, 2023. URL https: wwt..mml..eer.ee wwtpp///0////a//e//eee html.
mll.
Haixu Wu, Huakun Luo, Haowen Wang, Jianmin Wang, and Mingsheng Long. Transolver: a fast transformer solver for pdes on general geometries. In Proceedings of the 41st International Conference on Machine Learning, ICML/24. JMr...org, 2024.
Zhi-Que thi-Qin John Xu, Yaoyu Zhang, Tao Luo, Yanyang Xiao, and Zheng Ma. Frequency principle: Fourier analysis sheds light on deep neural networks. ColR, abs/ 1901.06523, 2019. URL /ttp://arxiv.ort org/abs/ 1901.06523.
Annan Yu, Dongwei Lyu, Soon Hoe Lim, Michael W Mahoney, and N Benjamin Erichson. Tuning frequency bias of state space models. arXiv preprint arXiv: 10. 10.00035, 2024.
19

---

Published in Transactions on Machine Learning Research (12/2025)
LOGLO-FNO: EFFICIENT LEARNING OF LOCAL AND GLOBAL FEATURES IN FOURIER NEURAL OPERATORS
SUPPLEMENTARY MATERIAL To THE MAIN PAPER
PROJECT PAGE: HTTPS: / /ONS:////M/OO3.... GITHUB IO/LOGLO-NN/
A Related Work
22
B Benchmark PDEs and Datasets
23
B.1 2D Incompressible Navier-Stokes (LNS)
23
B.2 2D Compressible Navier-Stokes (CNS, PDEBench)
23
B.3 2D Wave Equation (Wave-Gauss, Poseidon).
24
B.4 Compressible Euler four-quadrant Riemann problem 2D (CE-RP, Poscidon)
................................
24
B.5 2D Diffusion-Reaction (PDEBench)......
25
B.6 3D Turbulent Radiative Mixing Layers (TRL3D, The Well)
25
C Baseline Neural Operator Models
26
C.1 Modern U-Net
26
C.2 FNO: Fourier Neural Operators
................................
26
C.3 F-FNO: Factorized Fourier Neural Operators
................................
C.4 U-FNO: An Enhanced FNO for Multiphase Flow
................................
27
C.5 LSM: Latent Spectral Models
27
C.6 Transolver: Fast Transformer Solver for PDEs on General Geometries
27
C.7 NO-LIDK: Neural Operators with Localized Integral and Differential Kernels
................................
27
D Spectral Space Loss Functions
28
D.1 Spectral Patch High-frequency Emphasized Residual Energy (SPHERE) Loss
................................
D.2 Radially Binned Spectral Energy Errors as Frequency-aware Loss
29
D.3 Visualizing Radially Binned Energy of Spectral Errors of Baselines and LOGL.O-FNO
31
E High-Frequency Feature Adaptive Gaussian Noise
34
F Complexity Analysis of Local and Global Spectral Convolutions
35
F.1 2D Spectral Convolutions
35
F.2 3D Spectral Convolutions
36
Detailed Quantitative Results and Analysis
37
G.1 Evaluation Motrics
G.2 Detailed Quantitative Results with Full Set of Metrics
42
20

---

Published in Transactions on Machine Learning Research (12/2025)
G.3 Further Improvements with Attentional Feature Fusion, Multi-scale Channel Attention, and SPHERE Loss
47
G.4 Significance of Adding More Modes in the Fourier Layers of Global Branch
48
H Qualitative Results and Analysis
53
Energy Spectra Visualization and Analysis
57
J. Analysis of Correlation with Ground Truth on AR Rollout
61
J.1 Pearson's Correlation of Sample Trajectories with Ground Truth
61
Ablation Studies
64
K.1 LOGLO-FNO Devoid of Radially Binned Frequency Loss (Crr
64
K.2 LOGLO-FNO Devoid of HFP Module and Radially Binned Frequency Loss (HFP, Eprial
64
K.3 LOGLO-FNO Devoid of High Frequency Propagation Module (HFP)
64
K.4 Patch-Size Sensitivity of LOGLO-FNO
65
K.5 Influence of High-Frequency Adaptive Gaussian Noise on Rollout Stability LOGLO-ENO
K.6 Choice of Radial Bin Cutoff Values for the Radially Binned Frequency Loss in LOGLO-ENO
67
L Zero-Shot Super-Resolution (ZSSR) Capability and Performance of LOGLO-FNO
68
M Training and Inference Time Comparison of LOGLO-FNO
70
M.1 Comparison with Numerical Solver
70
M.2 Training & Inference Time and GPU Memory Usage Comparison with Base FNO and NO-LIDK
70
N Confidence Analysis of LOGLO-FNO and Basclines
72
O Model Hyperparameters
74
0.1 Modern U-Net Hyperparameters
74
0.2 Base FNO Hyperparameters.
0.3 F-FNO Hyperparameters
74
0.4 LSM Hyperparameters
74
0.5 Transolver Hyperparameters..
75
O.6 U-FNO Hyperparameters
76
0.7 NO-LIDK Hyperparameters
0.8 LOGLO-FNO Implementation and Hyperparamettrs
76
P Table of Notations and Mathematical Symbols
78
21

---

Published in Transactions on Machine Learning Research (12/2025)
A Related Work
It has been a well-ostablished fact that deep neural networks, trained with first-order gradient descent optimization methods, have a spectral bias for learning functions with low frequencies before picking up high-frequency details, explaining their superior generalization capabilities on downstream tasks (Xu et al., 2019; Rahaman et al., 2019; Cao et al., 2021; Schwarz et al., 2021; Molina et al., 2024; Yu et al., 2024). This means that deep networks require longer training times (Basri et al., 2019), necessitate multi-stage residuc- based training (Wang & Lai, 2024; Ng et al., 2024; Kong et. al., 2025), or experiences difficulty altogether, in learning complex functions with high-frequency components. This phenomenon is apropos of the neural PDE solving community (Li et al., 2022b; Fu et al., 2023; Liu et al., 2024) where the high-frequencies are more pronounced in multiscale, multiphysics, time-dependent PDEs and can evolve in time. This limitation of deep neural nets affects not only an accurate reconstruction of the spectral components but also rollouts (Worrall et al., 2025) and causes hallucinations (Sun et al., 2024b).
In the field of computer graphics and vision, several approaches have been proposed to solve this problem. Sun et al. (2024a) explore learning high-frequencies for generative tasks in computer graphics and propose sinusoidal positional encoding method to learn high-frequency features effectively. Pan et al. (2022) propose faster variant of VIT with HiLo atlention by splitting the attention heads to learn high and low frequencies in a disentangled fashion.
In the research area of neural PDE solving and SciML applications, Liu et al. (2024) study the limitations of existing neural operators for multiscale PDFs and propose a hierarchical attention method and H loss to direct the emulator to learn high-frequencies. Other approaches in improving the capability of neural operators to learn high frequencics include (i) an adaptive selection of Fourier modes and resolution (George et al., 2024) since using suboptimal choice for the number of modes may be detrimental on many PDE problems (Lanthaler et al., 2024), (ii) a two-staged approach of first predicting the future states, given some historical temporal context, using neural operator and feeding that as an input (i.e., conditioning factor) to a diffusion model in stage 2 (Oommen et al., 2024). Similarly, Fan et al. (2024) employ a two-stage pipeline of first generating low-resolution Kolmogorov flow turbulence simulations using a hybrid DNS neural solver on coarse grids and leveraging a conditional diffusion model to super-resolve for high frequencies and fine-grained structures in the second stage. Poli el al. (2022) study frequency-domain models for PDE solving with an aim to accelerate the training process. They propose method to learn solution operators directly in the frequency domain, which is made possible with a variance-preservingg weight initialization scheme. Closest to our work in terms of achieving local convolution and modeling high-frequency features in FNO is NO-LIDK (Liu-Schiaffini et al., 2024), which designs two different. local layers for learning the differential and integral operators that capture local and high- frequency features. Hence, we consider this as the relevant and competitive bascline. Although LOGLO-FNO has high-level similarity to NO-LIDK by introducing architectural modifications to FNO with additional parallel branch for local convolutions, it differs from NO-LIDK in the manner it achieves local convolutions. It does so by decomposing the input domain into sub-domains and then individually applying spectral convolutions on these sub-domains, thereby effectively capturing localized features. Moreover, an additional parallel branch specifically designed for modeling and providing an inductive bias of high-frequency features is introduced.
Chen et al. (2021) develop a multi-scale vision transformer for image classification tasks and explore different fusion methods aimed an effective fusion of multi-scale features from the respective branches. Dai et al. (2021) explore attention mechanisms for aggregating the local and global context. features and demonstrate their effectiveness image classification and object localization tasks in computer vision. Specifically, they propose a multi-scale channel atlention module and attentional feature fusion blocks for fusing multi-scale features in deep neural networks.
22

---

Published in Transactions on Machine Learning Research (12/2025)
B Benchmark PDEs and Datasets
We experiment with 2D and 3D transient PDEs that exhibit high frequencies, turbulence behaviour, and rich local patterns due to their inherent nature (e.g,, coupled variables), controllable parameters (i.e., PDE coefficients, Reynolds number, forcing terms), chaotic dynamics, shocks, discontinuities, porosity of the medium, and higher-order derivatives.
B.1 2D Incompressible Navier-Stokes (INS)
Kolmogorov Flow. Following prior literature (Li et al., 2022b; George et al., 2024; Liu-Schiaffini et al., 2024; Lanthaler et al., 2024), we adopt 2D Kolmogorov Flow, which is a form of Navier-Stokes PDE, as one of the test cases.
Ou Vu
+ u
Au = Vp + sin(ny)
t 0,
on [0, 2x,2 2 x (0, oc)
(7)
Øt
Re
Reynolds number
forcing term: (n=4)
We use the dataset configuration of Li al. (2022a) for Re5000 that exhibits high-frequency structures due to the flow being fully burbulent. It consists of 100 samples, each with spatial and temporal resolution of 128x 128 and 500 snapshots, respectively. As with George et al. (2024), we exclude the first 100 timesteps to let the trajectories reach the attractor, obtaining total of 40,000 pairs of samples, out of which we use 36K input-output pairs for training and 4K for testing.
B.2 2D Compressible Navier-Storn Stokes (CNS, PDEBench)
In order to study problem exhibiting turbulence and also capable of describing shock wave formation and propagation, we experiment with the 2D CNS equations (Eq. 8a to 8c) from PDEBench (Takamoto et al., 2022). More specifically, we use the sonic regime dataset (Mach=1.0) where the initial conditions comprise turbulent velocity with uniform mass density and pressure, and the boundary conditions are periodic.
density (mass per unit volume)
shear viscosity
8.p+ V (p vpv)=00 =0,
(8a)
n VV(V v),
0(0.+ Vv) -p+ (C
(8b)
bulk viscosity
pressure
2v
+ 2
0,(E
+ [[ +
= 0.
(8c)
internal energy
viscous stress tensor
The dataset (Mach-1.0) contains 1000 trajectories, each comprising 21 timesteps. Following the data split of the PDEBench" reposilory, we use 900 samples for training and the rest for testing, To facilitate the 1-step training and testing tasks, we construct the input-output pairs by slicing the solution fields (i.e., density, pressure, velocity-x, and velocity-y) at successive timesteps along the length of the trajectory, yielding 18K samples for training and 2K for testing. Due to the high spatial resolution of the data of 512 x 512, we subsample it 256 256 throughout our experiments. However, we use the full spatial resolution evaluate spatial ZSSR capabilities.
23

---

Published in Transactions on Machine Learning Research (12/2025)
B.3 2D Wave Equation (Wave-Gauss Poseidon)
As an additional challenging PDE problem modeling different physical phenomenon of the propagation of acoustic waves through spatially varying medium, we consider the time-dependent 2D Wave Equa- tion, specifically the so-called Wavo-Gauss, from the PDEgym benchmark of Poscidon (Herde et al.,
2024).
Øu(r, 2, 012
( c(a,y) )² (Pu(a,y,t) v.t) Ox2
θ Pu(a,y,1) 0,
in D (0,7)
99)
Oy²
spatially varying propagation field
The initial condition v (x, y) of the displacement u, and the propagation field c(x, y) are both generated using a sum of several Gaussians, and hence the name.
uo( y) Σ gi(,,,), k, E (0, 1)
(Fop - x)2 (Ve. y)² )²²
gr u) = exp
2; E (0
2s
where 8; the standard deviation of the 7th Gaussian in the propagation field c.
cxr.v) v) = +
vz, x. E (0, 1)
i=1
(_(x; (x; + dx; x)² + (ys + day - y)² v²), x, E (0,
fi(x,1) U; cxp
20
where and are the amplitude and standard deviation of the 8th Gaussian, respectively.
The dataset has a total of 10,512 trajectories of 16 timesteps each, with a spatial resolution of 128 x 128, of which we use 10,212 trajectories for training, 60 trajectories for validation, and 240 trajectories for testing, following the data split provided by the authors. Since we perform 1-step training and evaluation, we construct input-output pairs by sampling subsequent timesteps in the length of the trajectory, yielding 153.18K training, 900 validation, and 3.6K testing sample pairs. The autoregressive rollout is evaluated against the full trajectory length by predicting one future timestep at time.
B.4 Compressible Euler four-quadrant Riemann problem 2D (CE-RP, Poseidon)
In order to test the capabilities of LOGLO-FNO in modeling shock waves and discontinuities, consider the highly challenging Compressible Fuler four-quadrant Riemann problem 2D from Poseidon (Herde el al., 2024). The four-quadrant Riemann problem is a generalization of the Sod shock tube to 2D space.
Drj (z.y) T
TT
i-1 p
s x
-
j j-1 P
Sy
< 2
where 2 is the 2D torus, and considering unit square D [0, 1)2 it can be divided into 2 x 2 square sub-domains when 2. The underlying solution operator S(t. Po. vo Po) [p(t), ((),p(() solving the
thattps:/ccmml ethz. wttp///eiioo///iindeeeeeeeeeeeee
24

---

Published in Transactions on Machine Learning Research (12/2025)
compressible Euler equations with periodic boundary conditions, has four fields, viz. density, pressure, and velocity.
The dataset consists of 10k trajectories, with each sample containing 21 snapshots at a spatial resolution of 128 x 128. Following the pre-defined train/val/tess split of Herde et al. (2024), we use the initial 9640 trajectories for training, the next. 120 for validation, and the ultimate 240 for testing. As with other PDE problems we consider, slice successive timesteps as input-output pairs starting from the initial condition of trajectory. This yields us 192.8k training, 2.4k validation, and 4.8k testing data pairs.
B.5 2D Diffusion-Reaction (PDEBench)
2D Diffusion- (Takamoto al., 2022) is a challenging SciML benchmark problem, modeling biological pattern formation. It is challenging because of the nonlinear coupling of the solution variables, viz. activator u(t, and inhibitor v(z,y,t) from the time-dependent PDE,
Ou
du D2O Dr00y + (u, (-1,1)² (0. (10)
Du D + R,, (u, Ōt
where Du and Dy are diffusion coefficients for the activator and inhibitor, respectively, and the reaction functions R, and Rg in Eqn. 10 are defined by the Sttzhugh-Naguma equations as,
R, u) u k v, (u, v) v:
(11)
The simulation data is generated by setting D, = 1 10-3, D. = 5 10-3, and = 10 10 with no-flow
Neumann boundary condition. The training and test sets consist of 900 and 100 trajectories, respectivcly, where each simulation is of shape (128 128 101 2), indicating the solutions for 101 timesteps and channel each for the activator (u) and inhibitor (v).
B.6 3D Turbulent Radiative Mixing Layers (TRL3D, The Well)
In three-dimensional astrophysical environments, dense cold gas clouds (the "cold phase") move through a hotter, diffuse medium (the "hot phase"), generating turbulent mixing their interfaces. This 3D turbulent. mixing produces multiscale enhanced cooling efficiency at intermediate temperatures. As energy is lost via photon emission, the mixed gas condenses and accretes onto the cold phase.
layer, where gas rapidly cools radiatively due to the
Growth us. Dissolution of Cold Clumps
When radiative cooling dominates turbulent mixing (troul tmix), the cold phase grows mixed gas rapidly cools and accretes onto cold clouds.
When mixing dominates
(Pcool > Imix), the cold phase evaporates into the hot medium.
Cooling-Man Mass Transfer Relation: The volume-integrated cooling rate Broot and mass (ransfer rate M from hot to cold phases scale as
3/4 3/444-11 Éccol Ad Uzal t-1/4
where Urat is the 3D relotive velocity between phases, and tcool is the cooling timescale.
These 3D simulations explicitly model the competition between turbulent mixing (driven by shear, Kelvin- Helmholtz Instabilities (KHT), and turbulence) and radiative cooling, providing a more complete picture of multiphase gas dynamics in astrophysical environments (e.g., galactic halos, stellar winds, or the interstellar medium) (Fielding et al,, 2020). The equations underlying these simulations are given by Ohana et al. (2024),
25

---

Published in Transactions on Machine Learning Research (12/2025)
density
Pp 16
+ V 0
velocity field
momentum flux tensor
Opv
0t + V. pv
+ P = 3
pressure tensor
OE v (E + Plv)
E
0t
cool
cooling timescale
5
E = p - 1)
where
3
adiabatic index
monatomic gas)
C Baseline Neural Operator Models
We choose the following list of seven diverse, competitive, and recent state-of-the-art baselines.
C.1 Modern U-Net
We benchmark the modern version of U-Net (https://github cwtp///deereeeaadaarreereeeeeaea pdearena/moddlle)) from PDEArena (Gupta & Brandstetter, 2023). Considering that we do not focus on testing the generalization capabilities of neural operators on a diverse set of PDE parameters, the pa- rameter conditioning is skipped. The U-Net architecture models multi-scale spatio-temporal phenomena through a series of downsampling and an equal number of upsampling blocks, with the skip-connections passing information from the downsampling to the upsampling pass. We employ four levels of downsampling and set the channel multipliers to (1, 2, 2, 3, 4) to maintain parameter parity with other baselines. Following prior studics (Lippe et al., 2023), the network directly predicts the residual to the next step instcad of the actual solution, and then we down weight this output with the factor TO Note that this setting improves the results of the modern U-Net model on Kolmogorov Flow significantly (1-step nRMSE REL. % DIFF of 22%) compared to the reported results of NO-LIDK (Liu-Schiaffini al., 2024) and hence is a strong baseline. The hyperparameters are listed in Appendix O.1.
3
C.2 FNO: Fourier Neural Operators
As the original FNO architecture (Li et al,, 2021) has been improved and optimized over the years, utilize the current stateofftthe-ars implementation of FNO htttps://gituut com from Kossaifi et al. (2024a) for Kolmogorov Flow 2D, CNS Turb 2D, Wave-Gauss 2D, and Compressible Euler four-quadrant Riemann problem 2D PDEs. However, we omitied tensor lactorization techniques in our experiments so as to be able to directly compare our results with NO-LIDK (Liu-Schiaffini et al., 2024), which was also devoid of any such data compression strategics. We use the implementation of Takamoto Diffusion-Reaction al. (2022) for the Diffusion- Reaction 2D problem following Lin-Schiaffii (2024) for fair comparison. The hyperparameters provided in Appendix 0.2.
26

---

Published in Transactions on Machine Learning Research (12/2025)
C.3 F-FNO: Factorized Fourier Neural Operators
Tran et al. (2023) propose to improve the original FNO architecture of Li et al. (2021) by factorizing the FFT computation for data with spatial dimensions of two or greater, among other contributions such as skip connections, Markov assumption, Gaussian noise, cosine learning rate schodule, and hence the model Factorized FNO. The factorization leads to a massive drop in the number of trainable parameters, resulting in a wight and efficient model. Therefore, we use this as one of the baselines. Having borrowed their open-source implementation bestps://githn httgg/ittb..cmomaaaeeeeeeeee htt.a//aaaarrrrooorieeeeeeeeeeee problems, provide the hyperparameters we use in Appendix O.3.
and adapted to our PDE
C.4 U-FNO: An Enhanced FNO for Multiphase Flow
Wen et al. (2022) propose U-FNO as an improved version of FNO specifically largeted for modeling multiphase flows. The network consists of six layers in total, three of which are standard Fourier layers from Li et al. (2021) and three UNet layers added to the Fourier layers. We adapt the open-source implementation (https://gtt /tte:///gttbb.... com/ hemm//ggwee//ffoo provided by Wen et al. (2022) for 3D spatial data to our experiments on 2D spatial data. We tune the model hyperparameters, viz., learning rate, modes, and channel dimension, to approximately match the number trainable parameters as the other baselines while also reaching a competitive accuracy. The full set of hyperparamoters is provided in Appendix O.6.
C.5 LSM: Latent Spectral Models
Wu et al. (2023) introduce a multi-scale neural operator architocture with the use of hicrarchical projection network mapping the high-dimensional input space into latent. dimensions and propose to model the dynamics in this low-dimensional latent space. The essential component of LSM is the so-called Neural Spectral Block, which consists of an pncode-process-deoode style of processing. Considering the robustness of MSE to LSM's hyperparameters (cf. Appendix G in Wu et al. (2023)), such as scales, number of latent tokens, and mumber of basis operators, we use 5 scales, 4 latent, tokens, and 12 basis operators in the hierarchical projection network. This yields 64, 128, 256, 512, and 512 channels for the five scales in question, starting with the full spatial resolution and initial channel dimensions of 64 for the 2D Kolmogorov Flow, CNS Turb 2D, and Wave- 2D problems. As for the -eacttion 2D PDE, we use the initial channel dimensions of 32. Therefore, the channel dimensions for the five scales starting with the full incoming spatial resolution would be 32, 64, 128, 256, and 256, respectively. The other hyperparameterss are provided in Appendix O.4.
C.6 Transolver: Fast Transformer Solver for PDEs on General Geometries
Wu et al. (2024) design a transformer-basee neural operator model, namely Transolver, to solve PDEs on both regular and irregular domains. The idea stems from the observation that self-attention over individual mesh points is unnecessary, expensive, and not scalable because it is not uncommon to have real-world data spanning tens of millions of points or highor, resulting from the discretization of the continuous physical fields. Towards this end, Transolver introduces so-called *Physics Atlention' mechanism where the mesh points belonging to similar physical states are grouped into slices and the self-attention is computed over these slices. The resultant model achieves near-linear complexity and scales to reasonably sized mesh points (e.g., 16,384 - 128 x 128). Therefore, to include transformer-basee benchmark this model on the 2D Kolmogorov Flow, CNS Turb 2D, Wave-Gauss 2D, and DiffutionnRReaction 2D PDEs by adapting the implementation provided in https: /ttp//ggttttttt/ee/ thub. com homm/thml//////mmmmmmmmmm/m httpt/ttmmmmmmn///oo wwwee///aaaannenneeeaaaaa ttree/ Saain/DDB-SSllvin-Sttatee StwndatdBennnnaarrr....
model in the baselines, we
However, we note that Transolver needs an extended training time of 500 epochs for convergence on the challenging Kolmogorov Flow problem, while the other baselines converge to an optimal loss value in just 136 epochs. The model architecture and training hyperparamotors are listed in Appendix 0.5.
C.7 NO-LIDK: Neural Operators with Localized Integral and Differential Kernels
Noticing the deficiencies of FNO in learning local and finer scale features due to the absence of support for local receptive fields, Tiu-Schiaffini et al. (2024) propose two additional discretizatiooninvaaiin convolutional
27

---

Published in Transactions on Machine Learning Research (12/2025)
layers (one for the differential operator and another for the local integral kernel operator) that can be placed in parallel to the (global) Fourier layers of (spherical) FNO. These additional pathways enrich the architecture with capabilities for local convolutions, which are realized through localized integral and differential kernels. The authors test the efficacy of these layers by conducting experiments on a wide range of two-dimensional PDE problems, such as Darey Flow, turbulent Navier-Stokes, and Diflusion-Reaction on the 2D planar domain, and the Shallow Water Equations on the spherical computational domain. Their results significantly improve over the baseline FNO, ranging from 34% to 87% in terms of nRMSE, demonstrating the effectiveness and importance of these local operations for operator learning. Since our work also achieves local convolutions, albeit by decomposing the input spatial resolution into sub-domains (e.g., patches), we consider the NO-LIDK model as closely related and competitive baseline. Considering that we report additional evaluation metrics, we train three variants of NO-LIDK (namely, (i) NO-LIDK" denoling only the presence of local integral kernel layers, (ii) NO-LIDK representing only the existence of differential kernel layers, and (iii) NO-LIDK' indicating the use of both local integral kernel layers and differential kernel layers, in addition the global FNO branch) on the INS Kolmogorov Flow 2D dataset and reproduce the reported results. Following the authors' suggestion,
radius cutoff of 0.05m for the local integral kernel layers and set the domain length to [2T,, 2x] (ef. Equ 7). We borrow the other hyperparameters, such the number of Fourier modes, hidden channels, epochs, learning rate, scheduler, and scheduler steps, for each of the model variants from their paper. The specific values used for each of these are provided in Appendix O.7, which also lists the training and model configuration settings, including the radius cutoff values for the CNS Turb 2D, Wave-Gauss 2D, and Compressible Euler four-quadrant Riemann problem 2D PDEs..
Spectral Space Loss Functions
In this section, we focus on two loss functions that operate entirely in the frequency domain.
D.1 Spectral Patch frehherre Emphasized Residual Energy (SPHERE) Loss
In congruence with the local branch of LOGLO-FNO, we propose a spectral loss localized to each of the sub-domains Pm of the domain D ($3.1).
Let Upred and Ugt be the predictions and ground truths of non-overlapping patches of the solution in the domain, respectively. First, compute the residual in the physical space: Au Upred lgt- Second, these residuulls of patches are transformed into the frequency domain using 2D DFT realized through FFT: Au Kiz ky) F(Au).
The SPHERE loss is then computed as the weighted squared magnitude of Fourier coefficients as
SPHERE = W(kiz, kzy) @ A ky)l
(12)
where the weight function W(ke, ky) emphasizing high-ffequeeccis is defined as,
P
FM
W(kin, ky) = +
FM = V 12 + hb
(13)
(FM
[F
whereby a and p control the emphasis on high- figh--frqqueciis. For our experiments, we use = 2.
The loss finally mean or sum reduced over the channel dimensions and the number of patches, and added to the MSE loss during training.
28

---

Published in Transactions on Machine Learning Research (12/2025)
def oonpueeeetruueenng..iessssssaaap
P. paize):
kx = torch afrc (pizze, d-1.0) (1s 1, 1, psize) ky torch fft. fftfreq (paize, d-1.0) reshape 1, paize, 1) freg_nagnitude torch sqrth ky**2)
Normalize and define frequency veighting function freg_nagn /- (freq. return alpha (freq_nagnitude p)
1e-8)
10 def extract_patchea (x, paize):
11
Reshape to (ubent*ne, 1, nx, ny) to apply unfold over spatial dims nb, nx, ny, nt, ne x. shape x_reshp permmtue 3, 4, 1, 2) reshape(nb at nc, 1, nx, ny)
12
13
14
15
# Extract patchea using unfold patches F. (x_
18
seneel_izeepssiee stride=psize)
# Reshape back (nb, nt, num_patches, patch_aize, patch_size) num_ptch = patches. [-1] patches patches. (nb, at, num_ptch, psize, psize)
21
22
patches
def SPHERELosS (preds, target, alpha, P. size, reduction="menn)
25
extract non-overlapping patches
26
pred_patches extract_patches patch_size)
27
target_patches extract_patches (target, patch_size)
26
20
Compute difference (i.e., error) real space once diff _atfc_e pred_patches target_patches
30
31
32
# Use vmap to apply FFT over temporal and channel dins def fft_and,_Righht weight(diff))
33
30
diff_fft torch. fft. fft2ffft norm-"ortho") seight
mampute_frequnncr pooquency_weightade (alpha, (izch_size)) return weight * (diff_fft.real ** diff_fft. imag ** 2)
Apply vmap along both (nt, nc) dimensions
fft_loss torch, vnap(
torch. vnap and__aand_eellle in_dins=1), in_dim==2 (diff_patcher
43
43
44
Aggregate spectral loss over all patches, time, and channels spect_loss ift fft_loss.. mean (din-(1, 2, 3))
45
46
47
if reduction "mean":
48
return apect_loss... mlossmmean mean ()
49
else: return spect_ loss sun
50
Unlike the function compute. frequency weights(...) shown here for expediency, in practice, we compute the weight only once per patch size and cache it for efficiency reasons.
D.2 Radially Binned Spectral Energy Errors as Frequency-aware Loss
We elaborate on the frequency-awar loss term based on radially binning of energy spectra errors introduced in $3.2 by considering 2D spatiotemporal domain.
The detailed procedure to compute the frequency loss for 2D spatial data is shown in the algorithm below.
29

---

Published in Transactions on Machine Learning Research (12/2025)
Algorithm 1 Frequency-awarre Loss based on Radially Binned Spectral Errors
1: Input: Prediction Y E NNN XN,,NN XN, and Target E N X NN,N,, N₂ and N, the spatial resolutions, Nr the batch size, Nr the total timesteps, N₂ the number of physical fields, Ly and Ly the length of the spatial domain along the respective axes of the 2D domain, and x and y be the 2D mesh grid of frequency indices. Let iL and iH be the radial bin cutoff indices, ie., iL is the count of low-frequeney bins, and iH is the count of wow-middfrqqqeene
bins.
// kxx ky below are frequency domain indices (e.g., h E [0, N - 1], ky E N,/2 11) Ep c, kiz ky: t) F(Yb,e, -Yooe)tYYoc))0eae (k,, kyy I2
{Energy Spectra Error (ESE) }
R(kxs ky) - x2
(Binned radial distances] {Max radius}
M + maxk2.ky (R(ke,))y))
5: Sraalb, Σ ky) R()
(p kn () for r ..
(Sum ESE into bin T, per b,c,
) s.t.
Srad(c,r,t) means(Srad(b,,a)
{Average summed ESE per bin over batch}
1, t v S Srad r,,)
1z
Lu Nv
[Normalize 1D radial error spectrum}
N:
8: enrl (c, 1) + mean, E (r(crr,)) err (c,t) mean F(C,T, high
fLov-frequency band errors} {Mid-frequency band errors} (High-frequency band errors}
10: erry (c,t) t mean Er((,r,t)
The concrete steps are listed in the PyTorch-style pseudocode below. Since we use forch.fft.fft in the 'backward' normalization mode, normalization term is included. Let preds and target be the predictions and ground truth, respectively, and the 2D spatial data has single channel and timestep. The computation for multi-channel and multi-timester data is straightforward since they are obtained independently for each physical variable and timestep in the input.
def BndiaalBaneeeeeeeellllsss (prede, target): input data shape and params nb, nc, nx, ny, nt size iLow, 1High 4, 12 Lx, Ly 1.0, 1.0
# Compute error in Fourier space err_phys = preds target err_fft torch. forc orr_fft_sq torch. ab (er orr_fft_sq_h ort_fft..s(Ellipiia :nx//2, :ny//2, :]
dim=[2, 3))
Create radial indices
14
arange
15
y torch
16
x, torch. forchhmoshgrid 3, indexing="ij")
17
radii torch sorch (X++2 (X+22
Y*+2) floor O to (torchh.iit) # Radial dist.
18
max_radiue
(torch. max (radii))
19
20
flatten radii for binary mask radii_flat =
21
(nx//2 ny//2)
22
25
# Spatially flatten Fourier space error; (nd, ne, ax//2 + ny//2, nt)
err err fft_ffft__lla err_fft_aq.h contiguous r).sshaaee (nb, -1, nt)
# initialize output tensor to hold the Fourier error
# for each radial bin at distance r from the origin orr_F_vect_flll torch.zeron (nb,
max_radius
nt)
Apply Lindez_add_ for all radii and accumulate the valid radis valid_r radii_flat max_radius binary
31
32
30

---

Published in Transactions on Machine Learning Research (12/2025)
Sum for valid radial indices
34
=re___vect__uu full index_add_ (2,
35
radii flat [valid_rr)
36
err err_fft.eq_flatt. [:, ;, valid_r]
37
38
30
Normalize & conpute mean over batch; => (nc, (nx/12/2 ny ny//2), at) nrm = (Lx/nx) (Lz/ny)
40
41.
_err_F torch. sqrt (torch, mean (ear_eor___ F__ect__eelll dim=0)) nrm
42.
43
Classify Fourier space error into three bands err_F torch. zeros 3, nt])
44
45
err_F(:, 0] += torch. nean[_er__[;,, :ilow], dim=1) # low freqs
err_Fl;, 1] += torch. (_err_F[;, iLow: iBigh] dim=1) err_F[:, 2] +- torch. (_err_F[c iHigh:], dim-1) high freqe
# mid freqs
mean
channels and time dimensions
if reduction "mean":
freq. torch, mean (err_F, dim-(0, -1])
elif reduction "sum"s
53
freq_lose torch. (err_F, dim-(o, -1])
D.3 Visualizing Radially Binned Energy of Spectral Errors of Baselines and LOGLO-FNO
In this section, we provide a visualization of the proposed radially binned energy of the spectral errors for 2D spatial data. In the interest of providing an uncluttered representation, we only show every 2nd radial bin in the Figures 9, 10, 11, and 12.
Log Spectral Energy Error Magnitude
Radially Binned Spectral Error
0.14
Low-freq region
0.175
Mid freq region
High-freq region
0.150
Energy of Spectral Error
FFI Index (ky)
20
0,125
0.08
30
0,100
0.06
0.075
0.04
50
0.025
60
0.00
0
10
30
50
60
20
40
60
80
PPT Index (kx)
Radial Distance (r)
Figure 9: Visualization of the radially binned energy of spectral errors of baseline FNO predictions on a random trajectory and timestep from the test set of Kolmogorov Flow (Re = 5k) (Li et al., 2022b). The left plot depicts the radial bins and the location of the boundaries of mid and high-frequency groups. Note that show only every other radial bin. The right plot visualizes the radially binned spectral error as a line plot over the radial distance from the DC component. The dashed blue and green vertical lines indicate the starting locations of the mid and high- frehefreyyency regions, respectively.
31

---

Published in Transactions on Machine Learning Research (12/2025)
Log Spectral Energy Error Magnitude
Radially Binned Spectral Error
Low freq
0.10
0.07
Mid-free region
10
High freg region
Energy of Spectral Error
0.06
20
0.08
Index (ky)
30
0.06
0.04
0.03
40
0.04
0.02
50
0.02
0.01
0.00
0
10
20
30
40
50
60
20
40
60
80
FFT Index (lx))
Radial Distance (r)
Figure 10: Visualization of the radially binned energy of spectral errors of LOGLO-FNO predictions on the same random trajectory and timestep (as that of the one for baseline FNO) from the test set of Kolmogorov Flow (Re = 5k) (Li et al., 2022b). The left plot dopicts the radial bins and the location the boundaries of mid and high-frequency groups. Note that we show only every other radial bin. The right plot visualizes the radially binned spectral error as line plot over the radial distance from the DC component. The dashed blue and green vertical lines indicate the starting locations of the mid and high-frequency regions, respectively.
Log Spectral Energy Error Magnitude
Radially Binned Spectral Error
0.14
Low-freq region
0.08
Mid-fres frlefre region
0.12
High freg region
Energy of Spectral Error
FFT Index (ky)
20
0.10
30
0.08
0.06
0.04
40
50
0.02
0.02
0
10
20
30
40
50
60
20
40
60
80
FFT Index (lor)
Radial Distance (r)
Figure 11: Visualization of the radially binned energy of spectral errors of NO-LIDK predictions on the same random trajectory and timestep (as that of the one for baseline FNO) from the test set of Kolmogorov Flow (Re = 5k) (Li et al., 2022b). The left. plot depicts the radial bins and the location of the boundaries of mid and high-frequue high-freeuency groups. Note that we show only every other radial bin. The right plot visualizes the radially binnod spectral error as line plot over the radial distance from the DC component. The dashod blue and green vertical lines indicate the starting locations of the mid and high-freeuency regions, respectively.
32

---

Published in Transactions on Machine Learning Research (12/2025)
Log Spectral Energy Error Magnitude
Radially Binned Spectral Error
0.25
0.200
Low-freq region
Mid-freq region
10
0.20
High frog region
Energy of Spectral Error
20
0.150
FFT Index (ky)
0.15
30
0,100
0.10
40
50
0.050
0.025
0
10
20
30
40
50
60
20
40
60
80
FFT Index (kx)
Radial Distance (r)
Figure 12: Visualization of the radially binned energy of spectral errors of modern U-Net predictions on the same random trajectory and timestep (as that of the one for baseline FNO) from the test set of Kolmogorov Flow (Re = 5k) (Li et al., 2022b). The left plot depicts the radial bins and the location the boundaries of mid and high-frequency groups. Note that we show only every other radial bin. The right plot visualizes the radially binned spectral error as line plot over the radial distance from the DC component. The dashed blue and green vertical lines indicate the starting locations of the mid and high-frequency regions, respectively.
Log Spectral Energy Error Magnitude
Radially Binned Spectral Error
0.35
Low-freq region
0.20
Mid-fres region
0.30
High freg region
Energy of Spectral Error
20
0.25
0.15
FFT Index (ky)
30
0.20
0.10
0.15
40
0.10
0.05
50
0.05
60
0.00
0
10
20
30
40
50
60
20
40
60
80
FFT Index (kx))
Radial Distance (r)
Figure 13: Visualization of the radially binned energy of spectral errors of LSM predictions on the same random trajectory and timestep (as that of the one for baseline FNO) from the test set of Kolmogorov Flow (Re = 5k) (Li et al., 2022b). The left plot depicts the radial bins and the location of the boundaries of mid and high-frequency groups. Note that we show only every other radial bin. The right plot visualizes the radially binned spectral error as line plot over the radial distance from the DC component. The dashod blue and green vertical lines indicate the starting locations of the mid and high-frequency regions, respectively.
33

---

Published in Transactions on Machine Learning Research (12/2025)
E High-Frequency Feature Adaptive Gaussian Noise
In many machine learning and signal processing tasks, high-frequency (HF) features often capture fine-grained details or noise-like patterns in the data. Introducing controlled noise based on these features during training can improve model robustness, prevent overfitting, and enhance generalization. It has been shown in prior surrogate modeling investigations that adding a small amount of Gaussian noise helps with long rollouts (Pfaff et al., 2021; Stachenfeld al., 2022) and stabilized training (Tran al., 2023). However, traditional Gaussian noise, which is static and independent of the input. data, may not adequately capture the variability inherent. in turbulent PDEs and physics simulation data rich in high frequencies. To address this shortcoming, we propose novel method to generate dynamic Gaussian noise that adapts to the statistical properties of the imput HF features. Specifically, we obtain the free-freeuency feature adaptive Gaussian noise by scaling (a) the standard Gaussian noise (0, 1) based on the mean (μ) and standard deviation (o) of the HF features.
N dynamic = N,
N ~~
(14)
High-Froquency Feature Adaptive Gaussian Noise
Let X IRNxXNxXN,xXN.N.. denote the input tensor of HF features, where:
No is the batch size,
N₂ and Ny are the resolutions of the spatial dimensions, and
N₂ N₂ represents the combined temporal and channel dimensions.
The figgynrry feature adaptive Gaussian noise Ndynamic is then computed as follows:
Compute (per sample) Mean (px) and Standard Deviation (at) of Sigg--Freyyncy Features
N
N.N. Nv
hf bij.k
No N2 No No
k=1
N. N.N.
ΣΣΣ (X K.s.k 22) +
No Ny N N
where is small constant added for numerical stability, and μ and are obtained by stacking the per sample statistics along the batch dimension.
2. Generate Standard Gaussion Noise
N ~ N(0,1
Scale Noise Dynamicalll
N dynamic = -
where is small value such 0.025 and Ndynamic has the same shape the input X
Nâmamic can now be added to the batch of inputs to the global and local branches during the training phase of LOGLO-FNO.
34

---

Published in Transactions on Machine Learning Research (12/2025)
F Complexity Analysis of Local and Global Spectral Convolutions
F.1 2D Spectral Convolutions
Global branch. FLOPs for FFT and IFFT computations (Cooley & Tukey, 1965) on the full 2D spatial resolution.
Global Branch FLOPs Calculation for FFT & IFFT in Global Spectral Convolution 2D
Let X E NNM denote the input tensor for the 2D global spectral convolution module, where:
No is the batch size,
N₂ and Ng are the resolutions of the spatial dimensions,
N₂ represents the total number of hidden channels.
FLOPSEFT 5 No Cin NN N2 logg(N22 N2 FLOPSIFFT = 5 No Cout N2 N₂ log2(N, Ny
where Gn and Cout are typically of the same dimension and are referred to as either width or embed- ding/hidden channel dimension.
Therefore, FFT computation on the global branch with the full 2D spatial resolution has complexity of (N2 Ny logz(N2 Nyy) per channel, making it expensive for large Nu and N, such as 2048 2048 higher spatial resolutions.
Local branch. FLOPs for FFT and IFFT computations (Cooley Tukey, 1963) on the domain decomposed 2D spatial resolution, for instance, with non-overlapping patches.
Local Branch FLOPs Calculation for FFT & IFFT in Local Spectral Convolution 2D
Let X E where:
denote the input tensor for the 2D local spectral convolution module,
Np is the number of patches obtained by No
N₂ and Ny are the resolutions of the spatial dimensions,
Psize x Psize is the patch size (e.g., x 16
N₂ represents the width or the number of hidden channels.
FLOPsPFT = = No Om Np Psize Psise logz Paize)
= No Cin N₂ Ny -logo (Pgz(Paize Paize)
FLOPSIFFT = 5. No Cout. Np Paize Psize logz( Paize)
= No Cout Ng - Ny logz(Psise' Psize)
where and Cout are typically set to be of the same dimension (e.g., 128).
Thus, the FFT and IFFT computations (Cooley & Tukey, 1965) on the local branch operating on the patches has a computational complexity of O N2 Ny (ogg(Pstan Paize) ) per channel. Since O N, Ny log, (P (Paize
35

---

Published in Transactions on Machine Learning Research (12/2025)
Paize) < ONN.. No 1og2(N, Nv)) in practice the FFT computation is significantly cheaper in the local branch. Moreover, is highly parallelizable when computing FFTs since each patch be processed independently, leveraging modern accelerators such GPUs and TPUs.
F.2 3D Spectral Convolutions
Global branch. FLOPs for FFT and IFFT computations (Cooley & Tukey, 1965) on the full 3D spatial resolution.
Global Branch FLOPs Calculation for FFT & IFFT in Global Spectral Convolution 3D
Let X E NNN.N,NN, N,, xN, denote the input tensor for the 3D global spectral convolution module, where:
Nb is the batch size,
Nz, Ny, and N, are the resolutions of the spatial dimensions,
Ne represents the total number of hidden channels.
FLOPSFPT = 5 N₂ Cin Nv Ny N₂ log2(N2 Ny N2) FLOPSIFFT = 5 N₁ Cout N2 . Ny N₂ logg(N, N,, N₂)
where Oin and Cout are typically of the same dimension and are referred either width or embed-
ding/hiddes hhideen channel dimension.
Therefore, FFT computation on the global branch with the full 3D spatial resolution has a complexity of ONN, Ny N₂ loga(N; (Na Ny -Na) per channel, making it highly expensive for large values of Nz, Ny, and N₂ such as 512 x 512 x 512 or further higher spatial resolutions.
Local branch. FLOPs for FFT and IFFT computations (Cooley Tukey, 1965) on the domain decomposod 3D spatial resolution, for instance, with non-overlapping patches.
36

---

Published in Transactions on Machine Learning Research (12/2025)
Local Branch FLOPs Calculation for FFT & IFFT in Local Spectral Convolution 3D
Let X E IRNNxNN,, Pasce Pasee Paize denote the input tensor for the 3D local spectral convolution
module, where:
N is the number of patches obtained by No
P
Ng, Ny: and N₂ are the resolutions of the spatial dimensions,
Pasze x Paize x Poice is the patch size (e.g., 16 x 16 x 16, 32 x 32 x 32, etc.),
Ne represents the width number of hidden channels.
FLOPSFFT =5- No- Cin Np Psize Psize Paize logg Psize Paize)
=6-No Cin No Ny N2 loog (Peize Psize Psize)
FIFFT 5- No Cout Np. Paize Psize Pasze (ogz(Psize Psize Psize)
5. No Cout Ng Ny N₂ log2(Prize Psize Psize)
where Gin and Cout are typically set to be of the same dimension (e.g., 64, 128, etc.).
Thus, the FFT and IFFT computations (Cooley & Tukey, 1965) on the local branch operating on 3D patches has computational complexity of O (N Ny N₂ loga Psize Paize) per channel. Since ONN, N, N logg (Pssse Prize Paise) < O N₂ N2 . -oog N2)N in practice, the FFT computation is significantly cheaper in the local branch. Moreover, it is highly parallelizable when computing FFTs since each patch can be processed independently, leveraging modern accelerators such as GPUs and TPUs.
G Detailed Quantitative Results and Analysis
G.1 Evaluation Metrics
Following prior research efforts (Takamoto et al., 2022; Wan al., 2023), we evaluate our models on a wide range of metrics, viz. RMSE, nRMSE, fRMSE (low, mid, high), MaxError, MELR, and WLR. In addition, introduce REL. % DIFF (RPD) to report relative performance improvement w.r.t baseline. For the sake of completeness, explain the computation of the metrics briefly.
G.1.1 Normalized Root Mean Square Error (nRMSE) a.k.a. Normalized L2 a.k.a. Relative L2 Error
The normalized L2 norm of vector (e1, .2, en) )x is given by:
||e||2 ||||a
Normalized L2
E
yi
where y (V1, (w,, B,)"" is the reference vector.
The normalized RMSE is defined as:
1 Dn
VE
Normalized RMSE
/ E M
VΣ
37

---

Published in Transactions on Machine Learning Research (12/2025)
The factor vn
cancels out in the numerator and denominator, leading to:
Normalized RMSE = Normalized L2.
Given a prodiction tensor Y E R N No N and a target tensor E RMNNMNNNNN, where b= 1=..,,,, c=
Nas == NN, = l,..,, Ny: and l,..., Nr. The nRMSE is computed as
follows:
1. Mean Squared Error over the spatial dimensions:
MSE
Y Y.o.m. Yb.oza1)
No Nu ==
2. Root Mean Squared Error over the spatial dimensions:
N₂
RMSE,,c.t
Y Deant Yo....
N₂ Ny 2= y=1
3. Normalization Factor:
N N N N a== y=
Normalization Factor
b.g.a.u.
4. Normalized RMSE (nRMSE):
RMSE nRMSE Normalization Factorb,e.1
5. Mean nRMSE over the batch, channel, and temporal dimensions:
1 ΣΣΣ RRSS
nRMSF= No No N₂
G.1.2 Max Error
Considering 2D spatial data, we can deduce that Takamoto al. (2022) define the maximum or the worst error in the entire test set as,
N₂
MaxEnroг
max 6,4,9
Y Y......y. Yeccmmid
No N₂
c=1
38

---

Published in Transactions on Machine Learning Research (12/2025)
G.1.3 RMSE at the Boundaries of the Spatial Domain (bRMSE)
Again, taking 2D spatial data as an exemplary case, we write the formula for bRMSE from Takamoto et al. (2022) as,
1. Compute Sum of Squared Errors (SSE) on the Boundaries of the Spatial Domain:
left vertical boundary
right vertical boundary
Ny Y Yos
N YYC.II1- - Yo,.2.1111
N= (Yo.o..NN.d Yo.c.s.N...)
% bottom horizontal boundary % top horizontal boundary
2. Compute Per-Sample Boundary RMSE:
SSEb,c,t 2.N + 2N,
bRMSE
V
3. Compute Final bRMSE:
bRMSE
1 No No No
ΣΣΣ
bRMSE, b..t
b=
Or more succinctly put,
No N N
N
bRMSE N.N.N,
ΣΣΣ
A...... + +bb.NNo.v..) +
( D. + D D.c.N
2N + 2N,
y=1
where Ab......1 (Yoc,,e.uu
Y Yb.cz...11
G.1.4 RMSE of Conserved Physical Quantities (cRMSE)
We again consider 2D spatial data for illustrative purposes here since computing the same metric for 3D spatial data straightforward. Takamoto et al. (2022) define cRMSE follows.
1. Compute the Error of Spatial Sum (As):
For each sample b, channel and timestep t, calculate the difference between the spatial sum of the predicted field and the spatial sum of the target field.
As.b.c.t
ΣΣ 2=1 y=
Y YY,e,z.p.! Yo.e
39

---

Published in Transactions on Machine Learning Research (12/2025)
2. Compute Normalized cRMSE:
Na (AS,,.c.)
cRMSE
N₂ Ny
No
3. Compute Final cRMSE:
N
cRMSE
cRMSE
No N c=1
Alternatively, as one-line expression:
2
cRMSE
Y Yorcam.e))
No N₁ N₂ Nv
Nb
y=1
G.1.5 Variance-scaled Root Mean Square Error (vRMSE)
Ohana et al. (2024) introduce vRMSE, which merely RMSE normalizod by the variance of the ground truth. Considering 3D spatial data,
N₂
MSE. No Ny N
ΣΣΣ
Y b.c.m.e... Y).
#=1
N N No Ny N2
RMSE,
Y.c.yyy Yo..
N. N₂ N.
Mcanp.e,t
ΣΣ>
Y V.c.a.y...
N₂ Ny No
N N N2
Varb.c,
Y.C.z.g.2.f Mean),e..))
N₂ Ny N
1=
No N.
N
RMSE
vRMSE
N₁ N N
Varbictt
b=1
1
No
N.
MSE
vRMSE
N N
N
Vara,e,
t=1
where the mean and variance are computed over the spatial dimensions in Meanse,, and Var respectively. In other words, the mean and variance are computed for each sample, timestep, and channel independently. The vRMSE score finally computed by taking mean over the batch, channel, and temporal dimensions, yielding single scalar value.
40

---

Published in Transactions on Machine Learning Research (12/2025)
G.1.6 Mean Energy Log Ratio (MELR) and Weighted Log Ratio (WLR)
Let K the set of wavenumbers (frequency bins) being evaluated, |K] the cardinality of K, and Erea (k) and Exe((k) the energy of the predicted and reference (i.e., ground truth) spectrum al wavenumber Wan et al. (2023) defines the MELR metric as,
Ek)ea(k) Eve((k) (
MELR
Σ KEK
wk log
(15)
with the energy spectrum being defined as,
E(k)
[a(k))
Σ b
Σ
u(x;) exp(-j2nk= 2z/
(16)
where L is the length of the physical domain, u is the snapshot of the system state, ((k)) is the Fourier coefficient at wave-vector k, k is the magnitude of the wave-vector k (i.e., for 2D spatial data, k = |
k2 + kg)
Treating each wavenumber's log ratio equally and assuming N₁ samples the test set, we arrive the formula for the unweighted MELR or simply MELR,
N
Fyley (R))
MELR
Σ REK
log
(17)
No
K
(b) (A)
WLR is then the weighted version of MELR with the weights wn computed using the reference energy spectra,
No No
E (1c)
L (k:) E (k) ))
WLR
log
(18)
Ejex Eloi (i)
b=1 WEK
G.1.7 Relative Percentage Difference (Rel. % Diff)
We report REL. % DIFF, which indicates the relative improvement (negative value) or degradation (positive value) of errors over the baseline. Denote the proposed model's score of chosen evaluation metrie (e.g., IRMSE) as á and the baseline model's score as b. Then, the REL. % DIFF defined as,
(0- 8)
REL. % DIPF
100
(19)
41

---

Published in Transactions on Machine Learning Research (12/2025)
G.2 Detailed Quantitative Results with Full Set of Metrics
G.2.1 1-step Training and Evaluation
To expand on the abridged results presented in the main paper, we present the results with the full set of evaluation metrics for the setting of 1-step training of baselines and our proposed LOGLO-FNO model on four 2D PDE problems and one 3D PDE in the sections that follow.
Kolmogorov Flow 2D. Supplementing the results in the main paper, Table 8 shows the full set of evaluation metries for 1-step and 5-step AR rollouts on Kolmogorov Flow test dataset.
Table 8: 1-step and 5-step AR evaluation of LOGLO-FNO compared with SOTA baselines the test set of 2D Kolmogorov Flow (Li et al., 2022b). REL. % DIFF indicates improvement. (-) or degradation (+) with respect to FNO. LOGLO-ENO uses 40 and (16, 9) modes in the global and local branches, respectively, whereas the width is set as 65. NO-LIDK* denotes using only localized integral kernel, NO-LIDK mcans only differential kernel, and NO-LIDK' means employing both. Transolver' indicates a longer training time of the model for 500 epochs due convergence issues at shorter training epochs of 136.
Model
RMSE (1)
DRMSE
bRMSE
cRMSE FRMSEL))
(MMS(()
HRMSEL)) MaxError
MELR (L)
WLR (L)
1-step Evaluation
U-Net
7.17 10
13-10
1.47 10
1.74 10
2.24 10
3.57
10
4.39 10
2,01 10"
1.61 10
1.48 10
Transolver'
7.69
10-1
109
4.48
10'
FNO
8.08
1.47 -10
7.91
1.1
1.36
100
2.05
-10
1.7 -17
1.46 -10
5.2 -10
2.83
F-FNO
7.53 -10-1
1.37 -10
7.41 7.41-10- -10-
1.5 115-
1-19 10
2.15 -10-2
4.36 -10
1.42 -10
4.74 -10
2:28 -10-
7.49
1.36
1.47 10P
10
10-
10
U-FNO
6.13 10
1.12-10
1.0- 108
7.09-10
1.27 10
2.22
3.71 10
1.64-10 10"
1,38 16
1.09
NO-LIDK"
7.25 10
1.33 10
1.12 10
9.05 10-3
1.45
10
10 2.60
4.55 10-2
1.65 10a
1.85
10-1
1.54
10
NO-LIDK
6.13
1.110 10
NO-LIDK
5.86
10-1
1.07-100
10-1
1.05-10 10-2
1.44 10
2.46
10-2
3.82
10-2
1.47 10'
7.11
10-2
1.0-10-2
LOGLO-ENO
5.89
10
1.07
1.07-107 10
6.74 10
7.23
10
1.21
10
1.81
10
3.54
10
1.33 10'
1.29
10
1.06 10
REL. DIFF
-2.11 %
-2.21
-15.12 %
34.31
-11.22
-1188
21.64
-8.99
-75.12
-62.63
Autoregrossive
Evaluation
U-Net
1.51 10"
2.65 10-1
2.31-107 10
5.39-10 10-
6.53-10" 10"2
1.04-10-1
9.38 10-
1.81-00 10'
2.13 1.13-100 10
3.52 10-2
Transolver
1.91 10"
3.36
10-1
2.59 10P
6.12
10-1
7.14 10-2
1.51 10-1
1.18 10-1
2.38 10a
2.45 10-1
4.92
FNO
1.33 -10"
2.35 -1
-00
1.46
-0
3.37 -10-
1.80-10"*
8.37 -10-
.10"
6.18 6.18-10-1
108-1002
F-FNO
1.29 100
2.28
10-1
1.27 -10
2.28
10-2
3.60 100
5.52 10
8.12 8.12-10-2
1,50 10
5.37 -1
3.90
-10
LSM
1.9
3.18
2.76 10
3.87 10
2.-10'
1.44
1.12
2,08 10*
2.04-19
4.39
U.PNO
1.15 10"
2.03 10
1.45 10P
6.3-10-a
2.69-10 10-2
5.33 10-2
7.35 100
1.56- 10'
2.11 2.0-16--1
2.51
10-2
NO-LIDK'
1.36 10P
2.39 10
1.8 109
1.1-10-2
3.14 1114-10
6.86
8.91 101-10"
1.59- 10'
2.23 10
2.93
NO-LIDK
1.17-10 10"
2.04 10
1.12
1.92
3.71
5.96-10" 10-2
10
1.61 10
1,42 10
2.12
NO-LIDK
1.17-10P
10-2
1.56-10
LOGLO-FNO
1.09 10"
1.92 10
1.12 10P
8.00 10-3
2,80 10
4.55 10-2
6.93 10-2
1.26
1,67 10-1
2.07 10
REL. DIFF
-1826
-18.39
16.12 96
38.21
-16.86
-2162
-1.22
-2131
-73.04 %
-5800
42

---

Published in Transactions on Machine Learning Research (12/2025)
CNS Turb 2D. Table 9 presents the full set of evaluation metrics for 1-step and 5-step autoregressive rollouts on the test split of CNS Turb 2D dataset.
Table 9: 1-step and 5-step AR evaluation of LOGLO-FNO compared with SOTA basclines the test set of CNS Turb 2D (Takamoto et al., 2022). REL. % DIFF indicates improvement (-) or degradation (+) with respect to FNO. LOGLO-FNO uses 40 and (16, 9) modes in the global and local branches, respectively, whereas the width is set as 65. NO-LIDK* denotes using only localized integral kernel, NO-LIDK means only differential kernel, and NO-LIDK' means employing both.
RMSE (L)
bRMSE
cRMSE
(L
IRMSB(M;
IRMSE(H)
MaxError (1)
(J)
1-step Evaluation
U-Net
3.73-10
19 10
9.27 10
3.06
2.17
2.9
1.48-
2.1
1.87-
7.39
Transolver
3.13-10-1
4.22-10 10
2.08-10"
10
4.77-10"
4.11 10-2
- 10-3
3.66 10"
6.01-10-1
3.33-10
ENO
3.69
1.8 10-
3.61 10
1.42-00
1.28 10
1.78 10
1.57 10
1.96 10"
2.78 10
7.8 7.8-10"
LSM
5.08-10-2
1.67-10
3.53
6.17-107
6.56 10-3
2.01-10
2.43-12
2.7-00-
U-ENO
3.43-10-2
1.49-10-2
1.92-10
3.42 10-3
222--0--
2.01-10-1
1.43-10-3
2.0-10P
1.32-10-1
9.5-10-a
NO-LIDK"
4.81-1022
10-2
4.77 10
1.72
10-3
1-5-10-1
2.14 2.14-10-3
1.98 10-3
2.35 10
2.98 2.98-10-1
1.31-10
NO-LID
10-
1.35
1.76-10
NO-LIDK
4.2-10
10
5.57-10-2
10
4.16
10
1.56
10
1.37
10-3
1,89 1:0-10-3
1.77-107 10-3
2.26-10
2.08-10"
9.54 10-1
LOGLO-ENO
3.23
10
4.24 10
3.34
10
1.24
10
9.01 10
1.1 10
1.37 10
1.86 10
1.71-10
5.5
ReL.
-8.3%
12.64
-20.7%%
12.38
4.79
-38.30%
S-step Autoregressive
Evaluation
U-Net
1.15-10-1
1.44 10-1
1.98 10
7.13 10-3
1.42 1.42-10-2
1.3- 10-2
3.18-100 10-3
2,87 10P
1.37- 10-1
2.13 10-2
Transobvr*
5.29
6.69- 10
5.3
2.7
9.21-10
4.7 4.7-10-3
4.23 10
3.54
1.25 10P
FNO
6.3
7.78-10-2
8.21
2.45-10 10-1
3.76 10-3
5,37 10-4
2.4- 10
2.45 10"
10
9.36 10
LSM
2.12-10-1
2.67 10
3.74-10" 10
1.91 10-2
34-10-"
2.5 10
4.16 10-3
3.16-10' 10"
2.68-10"
-.34-10-'
6.84-10-2
10
2.54- 10-3
NO-LIDK"
6.95-10
8.56-10 10
6.86 10
2.83
10
4.14-10 10
6.92-10
2.62-10 102-10-
283-10'
2.06-10
1.13-10
NO-LIDK
6.41 10
7.88
6.32
10
2.84
10-3
3.82
10
5.15 10-3
2.44 10-3
2,49 10P
1.3-10-1
9.57
NO-LIDK
10-1
2.63
3.82
5.38 10-3
1.01 10-
LOGLO-FNO
5.43-10
6.67
5.48
10
2.1-10
2.76
3.97
2.12-10 112-100
2.25-10"
1.29
6.67 10
REL. DIPF
-14.30%
-14.31 76
-1.88%
-14.23 76
-26.7
-26.00 %
-11.42%
-6.00%
-1.8%
-28 70%
43

---

Published in Transactions on Machine Learning Research (12/2025)
Wave- Gauss 2D. In Table 10, present results with the full set of evaluation metrics the test set of Wave-Gaus: 2D dataset.
Table 1-step, 5-slep, and 15-step AR evaluation of LOGLO-FNO compared against. SOTA baselines on the test set of Wave-Gause 2D (Herde et al., 2024). REL. % DIFF indicates improvement (-) or degradation (+) with respect to FNO. LOGLO-FNO uses 20 and (16, 9) modes in the global and local branches, respectively, whereas the width is set to 32. NO-LIDK' denotes using only localized integral kernel, NO-LIDK means only differential kernel, and NO-LIDK means employing both.
Model
RMSE (1)
#RMSE
BRMSE
cRMSE
IRMSB())
IRMSE(M) IRMSE(H) MaxError (1)
MELR (J)
WLR (4)
1-step Evaluation
Transobes
2.01-10-1
2.29- 10-1
7:29-10-1
1.7
10-2
3.51
10"2
2.76 10-
2.2 10-3
2.94 10
1.29 10"
1.1 10
FNO
3.76
1.03
10
2.82
7.5
5.45
6:22
1.31
9.01
LSM
2.83 2.83-10-2
3.08- 10-2
3.04 10
2.60
10-3
10
3.86-10" 10-3
8.54 10"
10-1
10"
6.2 10-
U-ENO
3.23-10-2
3.49-10-
10-
2.09-
3.89 10-100-
6.62
1.52 10
T.21- 10
1.2 10P
1.11 10"
6.82 6--2-10-
NO-LIDK"
3.02-10-2
3.93
10-2
2.94
6.5
10
1.27
8.28 10-2
NO-LIDK"
3.33-10
3.58-10
1.68-10
2.57
6.49-10
4.78-10 10-3
7.56-101
1.3-10P
1.31 108
7.32-10
NO-LIDK
3.25-10
10-
1.68 10
2.7 10-3
6.31 10
-.81-10-3
7.4 10-
1.19 10"
1,28 10"
7.04-10-1
LOGLO-FNO
2.9-00
3.11
1.17
2.75
6.43
3.84 10
3.44-10
1.12-10
9.29
6.82 1022-000
ReL.
DIPF
22.90%
-22.79
-3.60%
2.5%%
-14.27%
-0.54%
-44.8%%
-14.99%
-3.77%
-24.30%
Sestep Autoregressive
Evaluation
Trazoheer
3.45 10
3.82 10-1
1.31-10-1
6.09 10-2
7.73 10--
3.47 10-2
2.93 10-3
3.68 10"
2.07 10"
1.79 10-1
FNO
6.91100 10
6.75-10" 10-
2.06- 10-2
6.5 10-a
1.6-10""
8.35 10-3
8.57 10-1
2.23-
1.5-10
10
LSM
4.33 4.33-10-2
4.34
10-2
3.63 10
5.48
10-3
8.64 10
5.68 10-3
1.1-10->
2.12 10'
1.72 10"
1.24-10-*
U-PNO
6.37 10-2
6.26- 10
2.55-10-2
144
1.6
7.64 10-3
9:27 10
2.54-
2.0
1.82 10
NO-LIDK"
6.05-10-2
5.96-10-2
1.91-10-2
10-3 7.76
1.34-10-2
7.92 10-3
1.19 10-3
2.65-00
1.99 10"
1.81-10 10
NO-LIDK
1.17-10
1.21-10
2.3
10
1.14
2.43-00
1.12 10
4.07 10
2.96-10
2.42 108
4.02
NO-LIDK
1.02-
1.06-00 10
2,16
10
133
2.1-10
1.02
3.85
2,55 10P
2.41 10
3,02
LOGLO-FNO
61-100-
181--0022
1.12
10
6:35
10-3
1.20 10
5.43-10 10
4.57 10-4
1.62-10"
1.48 10F
1.30 10-2
REL. % DIFF
26.2%
-28.70%
-5.43%
-.11%
-19.2%
-3191%
36.6%
-2.116%
-1.71%
-27.85%
15-step Autoregresiive Evaluation
Transober'
1.3 10-1
1.96
10
1.15 10-1
3.61
10-
10
3.81- 10"
1.73
2.36 10-1
FNO
1.21- 10-1
1.34
10
3.86 10-2
1.76. 10-2
3.13 10-2
1.32 10
1.30 10
2.42 10"
1.16 10"
3.53 10-2
LSM
7.66-10-2
8.49
5,72 10
1.29
-0
1.79
10
9.33 10
1.61-10
1.98-0P
1.29
2.63
C-FNO
1.15-10-1
1.27
10
40-2
3.08
10
3.14-10-1
1.24 1.24-10-2
1.17-10-2
2.67-
1.54- 10"
3.32 10
NO-LIDK"
9.98-10
1.13
10
3.14
10
1.81
10:
261 161-10
1.21-10
1,38 10
3,09 10P
1.57 10"
3,22 10
NO-LIDK
10
10
3.2
NO-LIDK
7.89
10"
3.1-10
1.77 10
1.75
6.00-
2.93
-
3.00 10
2.06
LOGLO-FNO
0.19 10
1.02 10
2.25
10 2
158 10
261
10
9.24 10
7,38
1.86 10"
1.04 10
2.54
REL.
DIFF
233.83%
-24.5%
-1..61%
-10.45%
-16.8%
.30.195
-16.99%
-2.46%%
-2.00%
44

---

Published in Transactions on Machine Learning Research (12/2025)
Compressible Euler four-quadrant Riemann problem 2D. In Table 11, present the preliminary 1-step and varying timesteps of rollout evaluation results from the experiments we conducted on the highly challenging Compressible Euler four-quadrant Riemann problem 2D problem, which exhibits shock waves and discontinuitics.
Table 11:
1-step and varying timesteps (5, 10, 15, 20) autoregressive rollout evaluations of LOGLO-PNO compared against SOTA baselines the test set of Compressible Euler four-quadrant Riemann problem 2D (Herde et al., 2024). REL. % DIFF indicates improvement (-) or degradation (+) with respect to FNO. LOGLO-FNO uses 40 and (16, 9) modes in the global and local branches, respectively, whereas the width is set to 65. NO-LIDK* denotes using only the localized integral kernel with a radius cutoff value of 0.0078125, and NO-LIDK' means only the differential kernel.
Model
RMSE (1)
bRMSE
cRMSE
FMMBEL)) RMMSEM;
(MME(II)
(J)
1-step Evaluation
FNO
2.02
208 10-2
233
846-
9.42 142-10-1
103-10 10-a
141 10-
1.58-
8.8. 3.8-10"2
3.09 10"3
NO-LIDK"
3-56-10-2
3.58-10" 10
3.94 10
1.14
10-3
1--10--
2005-10-3
2.47-10" 10-3
2.35 10'
7.26-10
5.67. 10-1
NO-LIDK
1.59
1.06 10
188
10
147-
10
1.05-10
8.54 10
1.11-10 111-101
144100
4.34-10
2.54-00
LOGLO-PNO
1.48
10-2
10-2
6.36
10-4
10-1
5.09
9.67 10-4
1.31-
5.42-10"
1.75 1075
REL. % DIFF
-2..6%%
-2.9%
-1.29%
-2182%
-3850%
-50.71%
-3.58%
-16.9%
-38.15%
13.200
6-step Autoregreasive Evaluation
FNO
3.39 10
3.43
10-2
5.62 10-2
2.69 10
2.58 10-
2.7 20-10--
2.35 10-
2/49
1.18 10"
NO-LIDK'
3.91 10-2
4.02 10
6.5 10-2
2.79 10
3.19 10-1
3.58 10-3
2.47 10-3
2.7 10P
100--001
9.75 10-1
NO-LIDK
3.08-10 10-2
3.13
10-2
2.66 10-3
25-10
2.38-10- 10-3
2.12- 10-3
2.1--0'
8.17-10-2
7.52 10-3
LOGLO-FNO
10-2
10-2
10-3
10-3
REL.
DIFF
-31 82%
-31.7%
-2.611%
27.35%
-3677%
-50%
33.1
-20.26 %
-31-11%
-11 58%
10-step Antorogresive Evaluation
FNO
5.06 10
4.76 10
6.33 10-2
4.14 10
1.5
4.79. 10-0
3.22 10
3.48
1.11 10-
1.04
NO-LIDK'
6.15-10
5.87 10
7.68- 10
163
5.88 10
6.35 10
3.59-10 10
3.67 10P
9.65
1.35
2
NO-LIDK
1.72110
4.52- 10-2
1.6-10-
4.39 10-3
4.55 10
4.37-10"*
3.02 10-3
3.55-10 10"
7.95- 10-2
9.75 10
LOGLO-ENO
10-2
10-3
-
2
10-4
-.67-10P
7
10-1
REL. % DIFF
-32 33%
-31.7%
-2.77%%
23.71%
36.71%
-12.7%
-31.9%%
-23.2%
-32 32.65%
-39 -3%
15-step Antoregresive Evaluation
10
6.37
10
6.91 10
10
4.11 10
10
1.31
NO-LIDK'
8.
7.92-10
9.08 10
6.68
9.01 10
9.36
1.61-00 -10
1,09 10
9.57-10 2
1.79
NO-LIDK
6.81 10
8.49 10-2
6.91 1001-002 10-2
6.51 10-3
7.78
10-1
7.24 10-3
4.08 10-3
4.4 108
8.97-10-1
1.35 10-2
LOGLO-ENO
10
10
4.53
10
3.49-10"
7.4-10-2
8.02
REL. % DIPF
-0.73%
12%12%
20.86%
19.17%
-3.04%
-36.5%
-29.0%%
-1.33%
-0.94%
-38.74%
20-step (fill trajectory) Antoregressive Evaluation
10
7.86
10
8.94 10
10-1
4.8 10
10
NO-LIDK"
1.03 10
9.76
10
1.06 10
8.5-
1.18 10
1.18
5.36 10
4.41 10
1.01
2.16
NO-LIDK
8.79 10
8.48
10
8.67 10
8.8
10
1.14 10
9.97
4.95
10
4.99 10P
9.92 9
1.74
LOGLO-FNO
6.02
10
10
5.76
6.12-10
6.25
3.95-10"
9.85 7.66-10-2
REL. % DIPF
28.4%
-27.7%%
-2%%
-1.77%
-3153%
-32.37%
12%
-11.3%
-28.92%
-35.93%
We observe that LOGLO-FNO significantly and cousistently outperforms Base FNO and the state-of-the-art NO-LIDK basclines on both 1-step and autoregrossive rollout evaluations, up to the full trajectory length of 20 timesteps.
45

---

Published in Transactions on Machine Learning Research (12/2025)
Turbulent Radiative Layer 3D (TRL3D). To complement the results in the main paper, we present the full set of evaluation metrics for the 1-step evaluation on the test set of Turbulent, Radiative Layer 3D problem in Table 12.
Table 12: 1-step evaluation of LOGLO-PNO compared with state-of-the-ard baselines on the test set of Turbulent Radiative Mixing Layer 3D dataset (Fielding et al., 2020; Ohana et al., 2024). REL. % DIFF indicates an improvement, (-) or degradation (+) with respect to base FNO, which uses spectral filter size of 12 and 48 hidden channels. LOGLO-FNO uses 12 and (16, 16, 17)* modes the global and local branches, respectively, whereas the width set. to 52. (*This is because the Turbulent Radiative Layer 3D dataset has 2x more sampling points on the z-axis relative the and y axes resolutions (128 128 256).) Base FNO means implementation of FNO (sce Figure 1).
Model
RMSE (1)
#RMSE
bRMSE
cRMSE (RMSE())
(RMSE(M)
(RMS((H) MaxError (L)
1-step Evaluation
LSM
5.67 10-1
10
1,25
1.41
9.7 10-2
3.43 10-2
NO-LIDK
1.04 10"
4.34
4.59
2.17
6.87 10-2
1.22 10-1
1.23 10-1
Base FNO
2.84 10
3.06 10
7.05 10
1004-100
4.87 10
4.58 10-2
2.80
1.12 10a
LOCLO-PNO
2.45-10-
2.65 10-1
6.33 10-1
1.76. 10
4.45 10
3.68 10
2.48-10 10-2
1.06 10a
REL. % DIFF
-1355%%
-13.27%
-10.2%
-1388%
-8.61%
-1..67%
-14.01%
-5.17%
G.2.2 Autoregressive Training and Evaluation
In Table 13, we present the evaluation results with the full set evaluation metrics for the fully autoregressive training on the Diffusion-Reactinn 2D problem.
Table 13: Fully autoregressive evaluation of LOGLO-FNO compared with SOTA baselines on the test set of challenging 2D Diffusion-Reactinn coupled problem from PDEBench (Takamoto et al., 2022). We also report. the REL. % DIFF indicate the error improvement (-) or degradation (+) w.r.t FNO.
Model
RMSE (.)
nRMSE
bRMSE cRMSE
(RMSE(
(RMSE(M)
(RMSE(H) MaxError
MELR (1)
WLR
U-Net
6.1-10-2
8.4
844-10-
7.810
399
30--
1.7 17-10--
812-10
5.7 47-10-
1.910
X
X
Transolver
174--0-2
2.63-10-1
3.05 10
3.37- 10
2.99- 2099-10-
2.03-10-1
5.51 1.51-10-4
1.11- 10
7.19 10-1
1.15
10-1
U-FNO
14.10
2.6110
1
2.0
4.310
3.4-10 3
16-10
2.6
7.8 2
45-10
7.9
-10
2
FNO
5.2
833-10
15.10
1.2
244 24-10
7.3 7.3-10
2.96
F-FNO
4.3-1--
7.0-10-2
7.9-10
-1-4
9.6-10
117-10-4
1.3-10
5.3-10
10-1001
1.3 -10
LSM
2.81-10
4.4-
3.45
5.92
10
7.17-10
2.4-10
3.67
1.32 102--10
3.43 10
2.08
NO-LIDK (loe. int)
3.6-
6.3
1.0
4.8
10
4.0-10
4.6-10
1.5
5.0
X
X
LOGLO-FNO
3.89 100
6.1
10-
5.2 5.2-10
10
46 10
2.8 10
32-10-1
1.9
2.2 10
1.6 10
7.9 10
REL. % DIFF
-25.19 %
-22.75 %
-65.13
-61.67 %
-54.84 %
-42.86 %
-20.83 %
-69.97 %
-44.09
-36.98
46

---

Published in Transactions on Machine Learning Research (12/2025)
G.3 Further Improvements with Attentional Feature Fusion, sulti-scale Channel Attention, and SPHHERE Loss
2D Kolmogorov Flow. In this section, we explore ways to further improve the results by considering advanced feature fusion strategies. The global and local branches in LOGLO-FNO models input features on different scales, whereas the HFP module learns different latent. features that are representalive of high frequencies. Therefore, we propose to employ sophisticated feature fusion modules targeted towards a seamless aggregation of these multi-scale and diverse features, as opposed to a simple summation (Figure 1). Towards this end, we adopt the multi-scale channel attention and atlentional feature fusion modules roduced in Dai et al. (2021) to fuse the features of the local and global branches and the HFP branch. The results of an experiment with this fusion in LOGLO-FNO, as well as penalizing the SPHERE loss the Kolmogorov Flow 2D dataset, are presented in Table 14. We term this model as LOGLO-FNO+ Comparing the results in Table and Table 14, we observe that these improvements lead to an additional 12% in 1-step nRMSE and (6%, 11%) for the mid-and high-frequency errors, respectively. The MELR and WLR errors are also down by 8% and 7%, respectively. In a similar manner, we see consistent improvements in 5-step uRMSE (8%), fRMSE-mid (2%), iRMSE-high (8%), MELR and WLR (3% each) over the LOGLO- model not using feature fusion and the SPHERE loss.
Table 14: 1-step and 5-step AR evaluation of LOGLO-FNO+ employing multi-scale feature fusion and patch-based energy spectra loss (SPHERE) compared with SOTA baselines on the test set of 2D Kolmogorov Flow (Li ct al., 2022b). We also report the REL. % DIFF to indicate the improvement (-) or the degradation (+) with respect to FNO. The number of modes used in the global branch is 40, and the local branch uses a patch size of 16 x 16. The mumber of hidden channels has been set as 65. NO-LIDK indicates the usage of both differential kernel and localized integral kernel layers in its architecture, NO-LIDK* denotes the model employing only the local integral kernel layer, and NO-LIDK stands for the use of only the differential layer as an additional parallel layer to the global spectral convolution layer of base FNO (see Table 4 in Liu-Schiaffi et al. (2024)). Transolver" indicates a longer training time of the model for 500 epochs due to convergence issues at shorter training cpochs of 136. We boldface the best result and underline the second-best result.
Eval Type
Model
RMSE (1)
aRMSE
BRMSE
cRMSE
IRMSE(L) (RMSE(M) (MSE( MaxError (L) MELR (L) WLR (L)
1-step Evaluntion
U-Net
7.17-10"1
1.3- 10
1.47
1.74 10
2.24 10
3.57 10-2
4.30- 10-2
2.01-107
1.64 10
1.48 10--
Transolver
7.69-10-3
1.30 10-1
1.42 10"
7.65
10-3
2.74
10
4.48
10
4.65 10-2
1.52 10¹
1.43 10
1.83
10-2
FNO
8.08 -10
-1
1.36
2.05
:10
5.2-10 12-10
2.83 -10
F-FNO
7.53
1.57 -10
7.11
-10
1.5
-10
1.49 -10
2.15 -10
4.36 -10
1.42 -10'
1.71
2:28 -10
LSM
7.49-10-1
1,36 10
1.77
10"
1 10
2.59-10-2
142-10
4.64- 10-2
2.05 10'
1.40-10"1
L.6- 10
1-stop
U-FNO
6.13-10-2
1.1. 10
1.0
10"
7.09- -0-3
1.27-10 10-2
2-22-10-2
3.71- 17-2
1.61-101
1.38-10
1.09- 11-2
NO-LIDK"
7:25-10-1
1 10-1
1,12-
10"
10-3
10-2
260-10--2
10-6-10-2
1.65-104
1.85-10 10-
1.54-10-2
NO-LIDK
6.13-10
1,11
10
5.95
10
1.65
10-2
222
10
3.91
10
1.5- 10
9.57
10
1.1
NO-LIDK
5.96-10-2
L07
10
5.84
10
105
10-2
1.44
10
2.46-10-2
3.82
10
L47-10"
7.11 10
1.0 10--10-2
ONO
5.89-10" 10
1.7
10
6.74
1774-10
7,23-
1
1.21
10
181-10-2
3.51-10'2 10
L.3--00
1.29-10")
1.06-10-2
LOGLO-FNO
4.88
10
8.87
10
5.57
10
8.78
10-3
LA1
10-2
1.68-10" 10-2
3.04 10
133 10'
7.50 10-
8.76 10-3
REL. DIFF
Tusion
27.
27.21
-51.31
11 11.22%
X
21.1
%
775122
-62.63 %
REI.. 19 Durr
my fusson
38.6 1g
39.78 %
20.36%
3,87 %
18.19%
36.39 %
7.91 %
$0.54 96
.09.00%
S-step Autoregreesive Evaluation
1.51 10
2.65 10
2.31
5.39
6.53 10
1.04 10
9.38- 10-9
1.81
2.13 10
3.52 10
Transolver
191- 106
3.36
10-1
2.59
10P
6.12
10-3
7.44
10
1.51
10-1
1.18
10-1
2.36-10 10'
2.45 10
4.92 10-
FNO
133--10
-10"
1.46-105
-10-2
3.37
-0
5.8 -1-
6.18
-10
F-ENO
1.28 -10"
2:28 -10
1:27
-10P
2.28
-10-
3.60
110
5.52
110
8.12 -1
1.50 -10
5.37 -10
3.99 -10
1SM
1.81-10 10
3.18
10
2.76
10"
3.87-0022 10
7.6
10
1.44 10
1.12-1001
2.08 10'
2.04 10
4.30- 10-
S-step AR
U-FNO
1.15 10"
2.03
10
1.45
10"
10-3 6.3
10-2 2.69
5.33 10
7.35 10-2
1.56 10a
2.01 10
2.51
10
"IDKI
L.36-10 10"
2.39-
10
L8- 10"
11-100-2
3.14-10-2
6
8.91-10-2
1.59-10'
1-23-10-1
2.53 1000-10-2
NO-LIDK
1.17 10
2.04-
10
1.12-10"
1.92-10-9
3.71-10
5.96-10"*
7.60- 10-
1.61-10'
1.42-10"
2.12 10"
LO-LIDK
1.17 10
2.05-
10
11"
1-23-10-2
3.91-10
5.91-10-2
7.71-10-2
1.56-10'
1.03-107 10-1
2
LOGLO-FNO
1.09- 10"
1,92-
10
1.12-10 10"
3.99-10-1
2.8 10-2
1.55-10
6.98- 10-2
1.28 104
1.67-10-1
2.07 10--2
LOGLO-PNO+
9.81-11 10
1.72
10
1.02.
16"
1.05- 10
2.9 10-
10"3
6.27 10
131 10!
1.17. 10-1
1.90- 10
REL. DIFF
Eusson
18.26 06
16.12
38.21
10.30
21 02.6223 36
17.26 18
21.11
73.00
56.00
REL. %% DIPF
fusson
26.37
26.63 9
25.43 95
27.99
11.05
23.58 96
25.06 95
15.02
77.70 %
61.43
Note that the terminology REL. % DIFF w/ fusion in the Table 14 above indicates the use of SPHERE loss in addition to feature fusion. This detail has been omitted in the row for an uncluttered presentation.
47

---

Published in Transactions on Machine Learning Research (12/2025)
G.4 Significance of Adding More Modes in the Fourier Layers of Global Branch
G.4.1 Kolmogorov Flow 2D
In this study, we analyze the influence of adding more modes in the global branch while keeping the number of modes constant in the local branch, which done by using a single local branch with a patch size of 16 x 16. This sctting yiclds the local spectral convolution branch with 9 modes, including the Nyquist frequency for the y-axis of the 2D spatial daía. Further, we also fix the embedding dimension to 48. Therefore, the number of modes in the global branch is the only varying factor. The experiments conducted on the turbulent 2D Kolmogorov Flow dataset (Li et al., 2022b).
1-step Evaluation. Here, we evaluate the baseline FNO and LOGLO-FNO models 1-step errors. The LOGLO-FNO models have been trained with single local branch (patch size 16 16) in addition to the global branch. Figures 14, 3, 15, and 16 plot xRMSE, MaxError, and the energy spectra (MELR and WLR) metries.
24
FNO LOGLO-FNO
0.4
22
0.3
20
MaxError
ARMSE
18
0.2
16
14
0.1
0.09
12
20
32
48
64
4
12
20
32
48
64
Fourier modes (global)
Fourier modes (global)
Figure 14: Comparison of FNO vs. LOGLO-FNO showing 1-step nRMSE and MaxError (1) on the test set of Kolmogorov Flow (Re = 5k) (Li et al., 2022b) for varying number of modes in global branch and full of modes (i.e., (16, 9) for patch size 16 16) in local branch.
In Figure 14, observe that as increase the modes in the global branch, it generally leads to reduction in nRMSE and MaxError for both models, except for slight increase in MaxError when utilizing the full set of modes, potentially due to overfitting. Also, note that LOGLO-FNO outperforms base FNO significantly, particularly when the modes used in the global branch are fewer (i.e., sparse modes). We hypothesize that the local branch compensates this case when there a lack of expressivity. Even when using the full of available modes, LOGLO-FNO yiclds improved crror. This behavior shows that local convolutions always supplement global convolutions.
Figure 3 in the main paper visualizes the low, mid, and freeuferyynncy band-classified errors for varying numbers of modes in the global branch. A similar trend as obscrved for nRMSE also holds here. Additionally, we observe that the error improvement for LOGLO-FNO over FNO for the high-frequency more pronounced than for low and mid-free fiddefrequencis. Note the logarithmic scale on the y-axis. Further, FNO tends to overfit when employing the full set of modes (IRMSE (Low) & fRMSE (Mid)), suggesting potential overfitting and corroborating the findings of Lanthaler al. (2024). In contrast, LOGLO-FNO does not exhibit this behavior and achieves the lowest error when utilizing the full set of modes. Notably, I.OGLO-FNO with
48

---

Published in Transactions on Machine Learning Research (12/2025)
32 modes attains lower frequency errors than the base FNO with 48 modes, highlighting its more efficient representation of complex features while maintaining robustness against overfitting.
FNO LOGLO-FNO
2
RMSE
BRMSE
1
0.8
0.8
0.6
0.6
4
12
20
32
48
64
4
12
20
32
48
64
Fourier modes (global)
Fourier modes (global)
Figure 15: Comparison of FNO vs. LOGLO-FNO showing 1-step RMSE and bRMSE (L) on the test of Kolmogorov Flow (Re 5k) (Li al., 2022b) for varying number of modes in the global branch and full set of modes (i.c., (16, for patch size 16 16) in the local branch.
0.8
0.2
FNO LOGLO-FNO
0.6
0.4
0.08
0.06
MELR
WLR
0.04
0.2
0.02
0.01
0.1
0.008
4
12
20
32
48
64
12
20
32
48
64
Fourier modes (global)
Fourier modes (global)
Figure 16: Comparison of FNO vs. LOGLO-FNO showing 1-step MELR and WLR (L) on the test set of Kolmogorov Flow (Re 5k) (Li al., 2022b) for a varying number of modes in the global branch and full set of modes (i.e., (16, 9) for patch size 16 16) in the local branch.
In Figure 16, we plot the energy spectra scalar measures, viz. MELR and WLR, which quantify the spectral discrepancy between the prediction and reference simulations, Here, we observe that LOGLO-FNO is significantly better than baseline FNO.
49

---

Published in Transactions on Machine Learning Research (12/2025)
Extended Autoregressive Evaluation. In this inference setup, evaluate the 1-step trained models autoregressive rollouts for varying numbers of timesteps and evaluate the rollout errors. We roll out the trajectories until reaching 25 timesteps to understand the behavior of LOGLO-FNO compared to base FNO in terms of autoregrossive error accumulation.
In Figures 17, 18, 5, and we plot the autoregressive errors of metrics, such as nRMSE, MaxError, bRMSE, FRMSE, MELR, and WLR. We observe that, overall, LOGLO-FNO consistently yields lower errors compared to base FNO and maintains the same behavior throughout the rollout extent. Although the autoregressive error accumulation is not completely mitigated, problem than the bascline FNO model across all evaluated metrics.
conclude that LOGLO-FNO suffers less from this
0.8
0.7
30
0.6
0.5
MaxError
nRMSE
0.4
20
0.3
0.2
5
10
15
20
25
10
15
20
25
AR Rollout Timesteps
AR Rollout Timesteps
FNO (K=8)
LOGLO (K=16)
FNO (K=32)
FNO (K=64)
LOCLO-FNO (K=8)
-
FNO (K=20)
LOCLO-FNO (K=32)
LOGLO-FNO (K=64)
FNO (K=16)
LOGLO-FNO (K=20)
Figure 17: Comparison of FNO vs. LOGLO-FNO for autoregressive rollout showing nRMSE and MaxError (L) growth over varying number of modes (K) in global branch and timesteps in the trajectorics the test set of Kolmogorov Flow (Re = 5k) (Li al., 2022b). The local branch uses patch size of 16 x 16.
In Figure 19, we visualize the energy spectra deviations through MELR and WLR metrics. We observe that LOGLO-FNO outperforms base FNO by a significant margin, indicating that the deviation or gap between the energy spectra the ground truth and the predictions of LOGLO-FNO is narrower than that of the baseline FNO model.
50

---

Published in Transactions on Machine Learning Research (12/2025)
4
RMSE
bRMSE
1
5
10
15
20
25
10
15
20
25
AR Rollout Timesteps
AR Rollout Timesteps
FNO (K=8)
FNO (K=16)
FNO (K=32)
FNO (K=64)
LOCLO-FNO (K=8)
LOCLO-FNO (K-16)
LOGLO-FNO (K=32)
LOGLO-FNO (K-64)
Figure 18:
Comparison of FNO vs. LOGLO-FNO for autoregressive rollout showing RMSE and bRMSE (4)
error growth over varying modes (K) in global branch and number of timesteps in the trajectories on the test set of Kolmogorov Flow (Re = 5k) (Li et al., 2022b). The local branch uses a patch size x 16.
0.6
0.08
MELR
0.4
WLR
0.06
0.04
0.2
0.02
10
15
20
25
10
15
20
25
AR Rollout Timesteps
Rollout Timesteps
FNO (K=12)
LOGLO-FNO (K=16)
FNO (K=48)
FNO (K=64)
LOGLO-FNO (K=12)
FNO (K=24)
LOGLO-FNO (K=48)
LOGLO-FNO (K=64)
ENO (K=16)
LOGLO-FNO (K=24)
Figure 19: Comparison of FNO vs. LOGLO-FNO for autoregressive rollout showing MELR and WLR ())
error growth over varying modes (K) in global branch and number of timesteps in the trajectories on the
of Kolmogorov Flow (Re 5k) (Li et al., 2022b). The local branch patch size of 16 x 16.
51

---

Published in Transactions on Machine Learning Research (12/2025)
G.4.2
Turbulent Radiative Layer 3D
Spatial Domain Errors
Frequency Component Errors (fRMSE)
3.4 10
Base LOGLO-ENO
10
Modes
12 Modes
16 Modes
3.2 10
BRANS SE Value
x 10
FRMSE Value
x 10-2
2.8
3 10
x 10-1
20-1
IRMSE-low
IRMSE-mid
ERMSE-high
Figure 20: Comparison of base FNO vs. LOGLO-FNO showing 1-step spatial domain xRMSE (left) and frequency space fRMSE (right) over varying Fourier modes (K) in the global branch and the local branch utilizing a patch size of 16 16 x 32 and the resulting full set of modes (16, 16, 17) on the Lest set of the Turbulent Radiative Layer 3D dataset (Ohana et al., 2024).
Time Window: t = [6, 12]
Time Window: t [13, 30]
Base FND
LOGLO-FNO
0.9019
0,8718
lime-Aroraged vRMSE
0. 0.8578
0.8426
83445
0.85
0.80
0.7613
0.7694
0. 77722
0. 0.7486
0.7424
7363
0.75
0. 7089
0.70
Modes
12 Modes
16 Modes
8 Modes
12 Modes
16 Modes
Figure 21: Comparison of base FNO vs. LOGLO- showing vRMSE values for the autoregressive rollout over the time windows [6:12] (left) and [13:30] (right, both ends inclusive), for variable Fourier modes (K) in the global branch while the local branch employing patch size of 16 x 16 32 and the resulting full set of modes (16, 16, 17) on the test set of the Turbulent Radiative Layer 3D dataset (Ohana et al., 2024).
52

---

Published in Transactions on Machine Learning Research (12/2025)
H Qualitative Results and Analysis
H.0.1 Kolmogorov Flow 2D
In this section, we provide qualitative visualizations the predictions of a few random trajectories from the test set of Kolmogorov Flow 2D dataset (Li et al., 2022b). The predictions are obtained from the models trained with 32 modes the global branch and patch size of 16 in the local branch for LOGLO-FNO. Both models use a constant width of 48.
Figure 22 provides qualitative visualization comparing the ground truth, predictions, and the corresponding absolute errors of two consecutive timesteps of two random trajectories. We observe that LOGLO-FNO retains sharp details, resulting in a ictable reduction of errors, whereas the base FNO yields smoothed- out prodictions. Note that the absolute crrors of both predictions are placed on the same scale for a fair comparison.
Ground
LoGio-FNO
Abs. Error (LoGlo-FNO)
Abs. Error (FNO)
18
10
10
Traj: 3, T-1
10
10
-10
10
10
10
10
Traj; T=2
20
-20
16
-20
16
Traj: 1, T=1
12
90
10
-10
-00
20.
-20
20
175
194
150
125
125
Traj: 1,
-0
102
-20
Figure 22: Qualitative comparison of predictions, ground truths, and absolute errors of FNO and LOGLO- FNO for random trajectories from the test set of Kolmogorov Flow (Re 5k) (Li et al,, 2022b).
Similar to Figure 22, we visualize the errors of two different trajectories from the test set in Figure 23. However, we additionally localize the regions the predictions corresponding regions in the absolute error that exceed 10% of the maximum error in the whole domain. This could result in multiple such regions. Therefore, limit ourselves to single contiguous region with the highest intensity in the interest.
53

---

Published in Transactions on Machine Learning Research (12/2025)
of providing unchuttered visualization. The displayed regions inside the red bounding boxes the ones with the highest error intensity. We observe that the regions overlap for both trajectories at timestep 1. This implies that both models struggle to get the details correct in this region. However, note that the bounding box regions are smaller for LOGLO-FNO compared to base FNO. Hence, we can deduce that LOGLO-FNO is effective in mitigaling errors in regions where the base FNO strugyles. In scenarios when the bounding boxes do not overlap, as the case in both model predictions for timestep 2, is generally desirable have bounding boxes with smaller areas, as this indicates the errors are localized small region and not spread out over large area.
Ground Truth
LOGLO-NNO
FNO
Abs. Eror (LOGLO-FNO)
Abs. Error (FNO)
-10
-20
-20
-20
Traj: T-2
20
20
Traj:
-10
-10 --1
20
20
Tral: TH2
10
-10
Figure 23: Qualitative comparison of FNO vs. LOGLO-FNO on random samples from the test set of Kolmogorov Flow (Re= 5k). The red bounding boxes highlight a single spatially contiguous region where the absolute error exceeds 10% of the maximum error in the domain.
H.0.2 Turbulent Radiative Layer 3D
In this section, we provide qualitative visualizations of predictions of a few random trajectories from the test set of the Turbulent Radiative Layer 3D dataset (Ohana et al., 2024). The LOGLO-FNO and base FNO predictions are obtained using 1-step evaluation. In the following plots, we show the test set trajectory corresponding the fcool parameter 0.03. The x, y, and z axes show the spatial extents (Lz, Ly, and Lg) of the simulation. The ground truth solution is repeated in the left column for ease of comparison and clarity in the representation. Note the deviation in the range of density and velocity values predicted by base ENO and LÖGLO-FNO with respect to the ground truth. Further, observe from Figure 24 that base ENO predicts unphysical values (i.c., negative density), whereas the density values predicted by LOGLO-FNO are non-negative.
54

---

Published in Transactions on Machine Learning Research (12/2025)
timestep 5, tcool 0.03
timestep tcool 0.03
1.00
1.00
0.75
0.75
0.50
0.50
0.25
0.00 Z
0.00 Z
-0.25
-0.25
-0,50
-0.50
-0.75
-0.75
-1.00
-1.00
zo
Co
20
20
20-
density
density
0.98
11.18
21.34
31.52
41.70
$1.87
05
72.23
41
92 59
0.03
11.57
23.12
1.666
46.21
57.75
69.30
00.85
92.39
94
62
82
34
103
Ground Truth
(b) LOGLO-FNO
timestep 5. Icool 0.03
timestep 5. tcool 0.03
1.00
1.00
0.50
0.50
0.25
0.25
-0.25
-0.25
-0.50
-0.50
-0 75
-0 75
1.00
1.00
20
20
20-
zo-
density
density
0.98
11.16
21.34
31.52
41.70
51.87
62.05
72.23
82.41
92 59
-12.93
0.00
12.83
25,66
38.50
51.33
64.16
76.99
89.02
102 66
Ground Truth
(d) Base FNO
Figure 24: cross-sectional (XY) slice at the middle of the Z-axis, embedded in the XY, YZ, and XZ planar view, showing the density channel from the test set trajectory (for tcool 0.03) at timestep comparing the ground truth (left), LOGLO-FNO and base FNO predictions (right).
55

---

Published in Transactions on Machine Learning Research (12/2025)
timestep 5, tcool 0.03
timestep tcool 0.03
1.00
1.00
0.75
0.75
0.50
0.50
0.25
0.00 Z
0.00 Z
-0.25
-0.25
-0.50
-0.50
-0.75
-0.75
11.00
-1.00
ro
2o
2o
20-
z--
velocity-z
velocity-z
1.
8.89
5.74
1.15
0.00
1.14
48
-5.67
4,86
4.05
-3.24
-2.33
-1.22
40.81
40.00
0.81
(a) Ground Truth
(b) LOGLO-FNO
timestep 5. tcool 0.03
timestep 5. tcool 0.03
1.00
1.00
0.75
0.0
0.50
0.25
0,25
-0.25
-0.25
-0.50
-0.50
-0 75
00 75
1.00
1.00
20
20
2--
zo-
velocity-z
velocity-2
..18
-8.03
6.89
5.74
4.59
3.45
2.30
1.15
.0.00
1.14
.4.9
4.21
.351
2,80
2.30
11.0
0.70
0.00
0.70
1.1
(c) Ground Truth
(d) Base FNO
Figure 25: cross-sectional (XY) slice at the middle of the Z-axis, embedded in the XY, YZ, and XZ planar
view, showing the velocity-2 channel from the test trajectory (for tcool 0.03) at timestep comparing the ground truth (left), LOGLO-FNO and base FNO predictions (right).
56

---

Published in Transactions on Machine Learning Research (12/2025)
Energy Spectra Visualization and Analysis
Kolmogorov Flow. In this section, we provide comparative analysis of the energy spectra of predictions of state-of-the-art neural operator baselines versus the reference solution on the Kolmogorov Flow 2D dataset. The predictions are obtained by autoregressive rollout of the 1-step trained models for the entire duration the trajectory, In Figures 26, 27, 28, 29, 30, 31, 32, and 33, we plot the energy spectra for two consocutive timesteps 50 timestep intervals.
Timestep 0
Timestep 1
10a
10a
10a
10a
107
Energy
105
10c
104
10a
10a
10a
FNO
U-FNO
104
FNO LOGLO (aurs)
U-FNO UNet
(ours)
10
NO-LIDK
Reference u(.)
10a
NO-LIDK
Reference ut.)
10
20
30
40
50
60
10
20
30
40
50
60
Wavenumber
Wavenumber
Figure 26: Energy spectra comparison of predictions of state-of-the-art neural operator baselines and our LOGLO-FNO ground truth at two consecutive timesteps random sample from the test set of Kolmogorov Flow 2D (Re 5k) (Li et al., 2022a). NO-LIDK" denotes the use of both local integral and differential kernels for local convolutions.
Timestep 50
Timestep 51
109
10a
10a
10a
Energy
10c
10a
10a
10
104
104
FNO
U-ENO
FNO
U-FNO
LOGLO-ENO (ours)
UNet-Mod
LOGLO-FNO (ours)
UNet-Mod
10a
NO LIDK!
Reference u(.)
NO LIDK!
Reference ut.)
10
20
30
40
50
60
10
20
30
40
50
60
Wavenumber
Wavenumber
Figure 27: Energy spectra comparison of predictions of state-of-the-art neural operator basclines and our LOGLO-FNO vs. ground truth at two consecutive timesteps on a random sample from the test set. of Kolmogorov Flow 2D = = 5k) (Li et al., 2022a). NO-LIDK! indicates the use of both local integral and differential kernels for local convolutions.
57

---

Published in Transactions on Machine Learning Research (12/2025)
Timestep 100
Timestep 101
10a
10a
10a
10a
Energy
10
10c
10a
10a
104
FNO
U-FNO
10a
FNO
U-FNO
10a
LOGLO-ENO (ours)
UNet-Mod
LOGLO-FNO (ours)
UNet-Mod
NO-LIDK
Reference u(.)
NO-LIDK
Reference ut.)
50
10
50
Wavenumber
Wavenumber
Figure 28: Energy spectra comparison predictions of state-of-the-art neural operator baselines and our
LOGLO-FNO vs. ground truth two consecutive timesteps on a random sample from the test set of Kolmogorov Flow 2D (Re 5k) (Li et al., 2022a). NO-LIDK* denotes the use of both local integral and differential kernels for local convolutions.
Timestep 150
Timestep 151
10a
10a
10
10a
10
10a
Energy
10a
10a
10a
FNO
U-ENO
FNO
U-ENO
LOGLO-ENO (ours)
UNet-Mod
LOGLO-FNO (ours)
UNet-Mod
NO-LIDK
Reference v(.)
10a
LIDLIDK!
Reference ut.)
Wavenumber
Wavenumber
Figure 29: Energy spectra comparison of predictions of stale-of-the-arl neural operator baselines and our
LOGLO-FNO vs. ground truth two consecutive timesteps on random sample from the test set of Kolmogorov Flow 2D (Re = 5k) (Li al., 2022a). NO-LIDK! indicates the of both local integral and differential kernels for local convolutions.
58

---

Published in Transactions on Machine Learning Research (12/2025)
Timestep 200
Timestep 201
1010
10a
10a
107
Energy
10c
10a
10a
10a
FNO
U-FNO
FNO
U-FNO
LOGLO-FNO (ours)
UNet-Mod
10a
LOGLO-FNO (ours)
UNet-Mod
NO-LIDK
Relerence u())
NO-LIDK
Reference ut.)
10
20
60
40
50
Wavenumber
Wavenumber
Figure 30: Energy spectra comparison predictions of state-of-the-art neural operator baselines and our
LOGLO-FNO vs. ground truth two consecutive timesteps on a random sample from the test set of Kolmogorov Flow 2D (Re 5k) (Li al., 2022a). NO-LIDK represents the use of both local integral and differential kernels for local convolutions.
Timestep 250
Timestep 251
10a
10a
10a
10a
Energy
10c
10
10a
10a
104
104
10a
FNO
U-ENO
FNO
U-ENO
10a
LOGLO-ENO (ours)
UNet-Mod
102
LOGLO-FNO (ours)
UNet-Mod
NO-LIDK
Reference v(.)
NIDLIDK!
Refersnce ut.)
10
Wavenumber
Wavenumber
Figure 31: Energy spectra comparison of predictions of stale-of-the-arl neural operator baselines and our
LOGLO-FNO vs. ground truth two consecutive timesteps on random sample from the test set of Kolmogorov Flow 2D (Re = 5k) (Li al., 2022a). NO-LIDK! indicates the of both local integral and differential kernels for local convolutions.
59

---

Published in Transactions on Machine Learning Research (12/2025)
Timestep 300
Timestep 301
10a
10a
10a
107
Energy
10c
10a
10
10a
104
10a
FNO
U-FNO
FNO
U-FNO
10a
LOGLO-ENO (ours)
UNet-Mod
10a
LOGLO-FNO (ours)
UNet-Mod
102
NO-LIDK
Reference ut.)
NO-LIDK!
Reference ut.)
50
10
50
Wavenumber
Wavenumber
Figure 32: Energy spectra comparison predictions of state-of-the-art neural operator baselines and our
LOGLO-FNO vs. ground truth two consecutive timesteps on a random sample from the test set of Kolmogorov Flow 2D (Re 5k) (Li et al., 2022a). NO-LIDK* denotes the use of both local integral and differential kernels for local convolutions.
Timestep 350
Timestep 351
10a
10a
Energy
10a
10c
10a
10a
104
FNO
U-ENO
FNO
U-ENO
104
LOGLO-FNO (ours)
UNet-Mod
10a
LOGLO-FNO (ours)
UNet-Mod
LI-LI"K
Reference v(.)
NO-LIDK!
Refersnce ut.)
10
Wavenumber
Wavenumber
Figure 33: Energy spectra comparison of predictions of stale-of-the-arl neural operator baselines and our LOGLO-FNO vs. ground truth two consecutive timesteps on random sample from the test set of Kolmogorov Flow 2D (Re = 5k) (Li al., 2022a). NO-LIDK! indicates the of both local integral and differential kernels for local convolutions.
60

---

Published in Transactions on Machine Learning Research (12/2025)
J Analysis of Correlation with Ground Truth on AR Rollout
J.1 Pearson's Correlation of Sample Trajectories with Ground Truth
In this section, we first describe the steps for Pearson's correlation coefficient computation and then provide visualizations of Pearson's correlation of the baselines and our proposed LOGLO-FNO model predictions with respect to the reference solution randomly sampled trajectories from the test set of Kolmogorov Flow 2D (Re 5k) (Li et al., 2022a).
Let X E NM,,N,N,,,N,, and Y E &MM, represent the predictions from a model and the ground truth, respectively, where:
Ng is the batch size and N, is the sequence length the number of time frames,
N₂ and Ny are the resolutions of spatial dimensions (c.g., width and height for 2D data),
We compute the Pearson correlation coefficient between X and Y in vectorized manner follows.
The tensors X and Y are reshaped by flattening the spatial dimensions N, and Ny into single dimension S = N₂ x Ny.
Step 1: Reshape Tensors.
X X reshape E R N
Y
Y reshape E R N
Step 2: Compute Mean. dimension S are computed as,
The mean values of X (predictions) and Y (ground truth) along the spatial
px
Y. Y
where:
μx E N.NNN and ba E NNN.N are the mean tensors for and Y, respectively,
The colon notation ; indicates that the operation is performed over the spatial dimension S for each sample in the batch N₂ and for each time step in N,-
Compute Standard Deviation. The standard deviations of (predictions) and Y (ground truth) along the spatial dimension S are computed as,
Step 3:
1 X
1
σx
5
mnd =xx)2
ay
5
mnd 2x)
where ox NoX. and ay E RNN.N are the standard deviations for X and Y, respectively.
Step 4: Compute Covariance. The covariance between X (predictions) and Y (ground truth) can then be computed as,
Cov (ov(X, Y)
Σ X px (Y... px).
S
where cov(X, Y) NoN. is the covariance tensor and denotes element-wise multiplication.
61

---

Published in Transactions on Machine Learning Research (12/2025)
Step 5: Compute Pearson's Correlation Coefficient. sample in the batch N₂ and time step N, is computed using,
The Pearson's correlation coefficient for each
corr X, Y) cov (X, Y) ox OY
To ensure numerical stability, the denominator is clamped to a small positive value (for instance, set forch.finfo(torch. float.3)...t.
cov (X, Y)
corr(X,Y) m O oY,
where corr(X, Y) RNNxN, is the output tensor containing the Pearson's correlation coefficients for each sample in the batch and time step.
Interpretation. Pearson's correlation coefficient measures the linear relationship between the model predictions X and the ground truth solution Y. Therefore, a value
indicates perfect positive linear relationship,
indicates a perfect negative linear relationship, and
( indicates no linear relationship.
1.0
0.8
Pearson's correlation confficiont
FNO
U-FNO
LOGLO-FNO
UNet-Mod Reference ut.)
NO-LIDK
10
25
30
40
45
60
75
80
95
100
Timesteps
Figure 34: Comparison of Pearson's correlation of predictions of state-of-the-art neural operator baselines and our LOGLO-ENO vs. ground truth on a random sample from the test set of Kolmogorov Flow 2D (Re 5k) (Li al., 2022a). NO-LIDKt indicates the use of both local integral and differential kernels for local convolutions.
62

---

Published in Transactions on Machine Learning Research (12/2025)
0.8
Pearson's correlation coefficient
0.6
0.4
0.2
PNO
U-FNO
LOGLO
NO-LIDK
Reference uC.)
5
10
15
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95
100
Timestops
Figure 35: Comparison of Pcarson's correlation of predictions of stato-of-thh-art neural operator basclines and our LOGLO-FNO ground truth different random sample from the test set of Kolmogorov Flow 2D (Re 5k) (Li et al., 2022a). NO-LIDKI indicates the use of both local integral and differential kernels for local convolutions.
63

---

Published in Transactions on Machine Learning Research (12/2025)
K
Ablation Studies
Through the following ablation studies, we attempt to ascertain the impact and importance of each of the introduced components (i.e., local spectral convolution branch and high-frequency propagation module) as well the patch size sensitivity, high-frequency adaptive Gaussian noise, and radially binned frequency-aware loss term.
K.1 LOGLO-FNO Devoid of Radially Binned Frequency Loss (Crraa)
We conduct an ablation study to assess the relative importance of the proposed radially binned spectral energy of errors as a frequency-aware loss term compared to the architectural modifications by the introduction of layers for local spectral convolution and high-frequency propagation. Therefore, we train LOGLO-FNO by retaining only the local branch and high-frequency propagation module parallel to the global branch. By setting Cfreq 0, the cost function C for model training then reduces to the plain MSE loss.
T-1 Σ
0- = arg min
Cass(No(u')), (
The results of this study on the Turbulent Radiative Layer 3D dataset provided in Table 7 on row 3.
K.2 LOGLO-FNO Devoid of HFP Module and Radially Binned Frequency Loss (HFP, Cproa)
In this ablation setup, we remove the high-frequency propagation module and keep only the local branch. The radially binned frequency loss term is also disabled by setting Cfreg = O. Therefore, the LOGLO-FNO architecture comprises only the local and global branches in the Fourier layers, and is trained with plain MSE.
The results of this ablation choice on Turbulent Radiative Layer 3D are provided in Table on row
K.3 LOGLO-FNO Devoid of High Frequency Propagation Module (HFP)
This ablation setup is similar to the previous study in $K.2, except that we now enable the radially binned frequency loss in the training objective to isolate the effect of removing the HFP module. The 1-step evaluation results for this model setup on Turbulent Radiative Layer 3D are placed in the last row of Table 7.
Table 15: 1-step evaluation of LOGLO-FNO on the test set of Turbulent Radiative Mixing Layer 3D dataset (Fielding et al., 2020; Ohana et al., 2024). Base FNO uses a spectral filter size of 18 and 48 hidden channels. LOGLO-FNO uses 16 and (16, 16, 17)" Fourier modes in the global and local branches, respectively, whereas the width is set to 52. ("This is bocause the Turbulent Radiative Layer 3D dataset has 2x more sampling points on the z-axis relative to the and y-axes resolutions (128 128 256).)
Model
RMSE (1) mRMSE
bRMSE
cRMSE IRMSE(L) IRMSB(M) IRMSE(H)
vRMSE (1) MaxError (L)
1-step Evaluation
Base FNO
2.76 10
297 10
6.3
10
23-10
5.17 10
4.45 10
276 10
3.09 10
1.11 10¹
LOGLO-FNO
(+freg lose, +HFP)
2.12 10
2.58 10
6.19 10
1.75-10-2
11-10-2
3.62 10-2
2.45 10-2
2.68 10
1.06-10' 10'
LOGLOOFNO
(-freq loss, +HFP)
2.57
10
2.75-10 10
6.1 10
108-
44- 10-2
4.09 10-2
2.63-10-2
2.86 10-1
1.09
LOGLL-FNÓ
(-freg loss, -HFP)
258
10-1
2.76 10-1
6.55 10-1
1.83-10-
44- 10-2
4.13 10-2
2.64 2064
2.87 10-1
1.1 10
LOGLO-FNO
(+freq loss -HFP
2.47- 2,06 10-1
6332
10-1
10-2
10-2
3.76
10-2
2.48-10
2.76 10-1
1.07-
As we observe from Table 7, our model achieves the lowest errors across all metries when the local branch is combined with both the radially binned frequency loss and the HFP module.
64

---

Published in Transactions on Machine Learning Research (12/2025)
K.4 Patch-Size Sensitivity of LOGLO-FNO
In this section, we first provide the results of an ablation study investigating the sensitivity of LOGL,OFNO on the patch sizes for the local branch through an experiment on the Turbulent Radiative Layer 3D PDE problem. Across the lested patch size variations (Psize 8,,8, 16)), only the patch size values are altered while all other hyperparameters, including the learning rate and random seed, remain the same during training.
Time: Averaged vRMSE
0.86
0.84
nRMSE VRMSS
0.295
0.040
11)
0.82
0.290
0.80
0.285
0.035
IRMSE
0.78
0,280
0.030
0.76
0.275
8
16
8
16
Time Window
nRMSE VRMSE
RRSSE())
[UMSE(H)
Time-Averaged VRMSE (6:12)
Time-Averaged vRMSE [13:30]
(RMSE(M)
Figure 36: Effect of patch sizes the Time- Averagee vRMSE and 1-step spatial and spectral evaluation metrics on the Turbulent Radiative Layer 3D dataset.
In Figure 36, we visualize the Time-Averager vRMSE and 1-step xRMSE evaluation scores for patch sizes 4, 8, and 16. While we observe trend of minor improvement in 1-step errors with the increase of patch sizes, the Averaere autoregressive errors do not exhibit this clear trend. We note from Figure 36 that an increase in patch size directly results in an increased number of trainable parameters (and by virtue of it, increascs model expressivity) since we use the full set of Fouricr modes the local branch. Therefore, as a trade-off between errors and training time memory usage, we use patch size of 16.
As an additional case study patch size sensitivity analysis, conduct experiments on the Compressible Euler four-quadrant Riemann problem 2D from Poscidon (Herde et al., 2024), and provide the 1-step and autoregressive error evolution of overall (nRMSE) and Trequency-decomppseed (IRMSE) evaluations over an increasing number of timesteps for four different patch size configurations of LOGLO-FNO in Figures 37 and 38, respectively.
0.019
0,0010
ARMSE & VRMSE
IRMSE (L. HD
0.018
ARMSE YRMSE
(RMSEL
0.0008
0.017
(RMSB(H)
0.016
0.0006
0.015
2a
23
24
25
22
23
24
2
Figure 37: Effect of patch sizes on the 1-step spatial (left: nRMSE and vRMSE) and spectral (right: FRMSE) evaluation metrics on the Compressible Euler Tour-quadrant Riemann problem 2D (Herde el al., 2024).
Shifting our attention the rollout evaluations, we note from Figure a trend of error improvements as we increase the patch sizes from to 16. However, a further increase patch size 32 does not yield an improvement, but rather detrimental, long rollout scenarios. Therefore, was previously found in the
65

---

Published in Transactions on Machine Learning Research (12/2025)
case of the Turbulent Radiative Layer 3D problem, come to the conclusion that patch size of 16 is sulficiently good choice.
x10
x10
2
6.0
23
Pow
5.5
Pale
5.0
IRMSE (Low)
ARMSE
4.5
4.0
3.5
2.5
x10-1
x10-3
7
3.5
6
IRMSE (Mid)
5
3.0
IRMSE (High)
2.5
2.0
2
1.5
10
15
20
10
15
20
AR Rollout Timesteps
Rollout Timesteps
Figure 38: Error accumulation and patch size sensitivity during autoregressive rollouts on the Compressible Euler four-quadrant Riemann problem 2D (Herde et al., 2024).
66

---

Published in Transactions on Machine Learning Research (12/2025)
K.5 Influence of High-Frequene Adaptive Gaussian Noise Rollout Stability in LOGLO-FNO
In this section, we conduct ablation study to investigate the influence of high-frequency adaplive Gaussian noise (HF-AGN) in LOGLO-FNO on the stability of trajectory rollouts for the Kolmogorov Flow 2D (Rc5000) datasct. We report the mean and standard deviation values of errors at timestep 400 over threel runs in Table 16. The results suggest that HF-AGN is helpful for the stability of longer rollout scenarios.
Table 16: Rollout errors of LOGLO-FNO,, with and without adaptive Gaussian noise (HF-AGN) injected during 1-step training, at timestop 400 on the Kolmogorov Flow 2D (Rc5000) datasct. The autoregressive evaluation over three runs.
Group
RMSE (1)
aRMSE
bRASE
cRMSE
(RMSE(L)
(RMSE(M)
(RMSE(H)
MaxError (J)
MELR (J)
WLR (J)
LOGLLO-FNO
(-HF-AGN)
7,8891
1.4522
7,5518
0.0472
1.4525
0.8524
0.1726
66.2814
0.4811
0.2797
(#12118)
(202219)
(L1O178)
(30.000)
(20.222))
(40.458)
(40.227)
(113.3936)
020.002)
(40.089))
LOGLÖ-FNO
(+HF-AGN)
7.1982
1.3227
7.0794
0.0113
1.2965
0.7362
0.1667
58.9953
0.4061
0.2228
(+0.1837)
(+0.0887)
(-0.2203)
(+0.0050)
(+0.0512)
(+0.0161)
(+0.0012)
(#1.7618)
(+0.0796)
(=0.0070)
K.6 Choice of Radial Bin Cutoff Values for the Radially Binned Frequency Loss in LOGLO-FNO
In order to assess the sensitivity of the cutoff boundaries when demarcating the low, mid, and high frequency regions for the radially binned spectral loss, we conduct an ablation study on the 2D Kolmogorov Flow dataset and provide the 5-step AR errors in Table 17.
We note that the choice iow== and iHigh=10 achieves the lowest spatial (nRMSE) and low- low- frequency errors and ties with the iLow=4 and iHigh=12 setup on the high-frequency error. However, the latter might be an optimal choice since it exhibits the lowest variance on the spatial (nRMSE), mid-, and high-frequengr errors and is more stable over long rollouts.
Table 17: Sensitivity analysis of radial bin cutoffs for the radially binned frequency loss applied for training LOGLO-FNO, The experiments are on the Kolmogorov Flow 2D (Re5000) dataset, and the evaluations are over five independent runs with unique random seeds.
Cutoff Boundary
Unpenalized Region
Penalized Region
RRMSE (J)
FRMSE(L)
FRMSE(M)
FRMSE(H) (L)
0.1931 0.1917
0.0262
0.0466
0.0694
iLow= 2, illigh-10 10
r <<
r
(1 0.000)
(1 0.000s)
(1 0.000))
(1 0.001
0.0285
0.0462
0.0694
iLow=4, 1High=12
r <<4
r
(+ 0.002))
(+ 0.002))
(+ 0.0020)
(+ 0.000)
0.1942
0.0281
0.0485
0.0702
iLow=6, 15
r <.K
r 6
(+ 0.005a)
(+ (.0000))
(+ 0.0oz)
(= 0.0013)
IOF the total four conducted, we exclude rmn from the mean computation due blow-up errors for the model without HF-AGN:
67

---

Published in Transactions on Machine Learning Research (12/2025)
L Tero-Shot Super- Super-Resolution (ZSSR) Capability and Performance of LOGLO-FNO
In this section, we evaluate the capability and performance of LOGLO-FNO on spatial ZSSR tasks and compare it with the results of NO-LIDK (Liu-Schiaffini et al., 2024) and the baseline FNO model.
Table 18: ZSSR evaluation of LOGL.O-FNO the test set of Kolmogorov Flow 2D dataset (Li et al., 2022b). Base FNO and NO-LIDK" use spectral filter size of 12 and 20 hidden channels. NO-LIDK* denotes using only localized integral kernel and NO-LIDK" means employing both disco and diff layers. LOGLO-FNO uses 12 and (16, 9) Fourier modes in the global and local branches, respectively, whereas the width is set to 20. Training spatial resolution (64 x 64) and denoted as 1x, whereas the ZSSR test resolution is (128 x 128) and denoted as 2x. We boldface the lowest errors and underline the value when the 2nd best model is LOGLO-FNO.
Model
Resolution
nRMSE (1)
FRMSE(L)
(RMSE(M)
IRMSE(I)
MELR (.)
WLR (L)
Trai:: 1x
1-step Evaluation
1x
2.99- 10
4.71 10-2
833-10 2
1.59 10"
6.41 110
1.2 10
Base FNO
2x
2.93 10-1
3.14
10-2
1.16 10-1
9.61 10-1
1.43 10-1
1x
2.79-
1.65 10
10-2
141. 10-1
5.89-10-1
1.04 104-100
NO-LIDK'
3.71
5.16 10-2
1-8
1.12-10-1
9121-10-1
10-1
1x
2.56-10 10
161-10-2
7.77-10-2
1228 128-10-1
10
5,89 5.89- 10-2
NO-LIDK
4.35
1.11-10"1
1.75-10")
1.39-10-1
2.6-10
6.6-10-2
2.32 10-1
4.57-10-2
7.35 10
1.22
2.12-10
4.68 10-2
LOGLO
2x
2.9 10
3.19-10 10-
6.76 10
1.05 10
2.32 10-
6.34 10
5-step Autoregressive Evaluation
1x
1.54-10"
1.05 10-1
1,96 10-1
244-10--1
7.03 10-1
1.68 10-1
Base FNO
2x
4.4
9.17-10 117-10-
1.65- 10
1.55 10
1.13 10
1.95 10
NO-LIDK
1x
4.32
1.05 10
2.03 10
2.27 10
8.58 8.58-10 10
1.86 10
2x
9.89 10-2
1.97 1097
1558 10-1
1.28 10
2.59 10-1
1x
3.85 10
9.15 10
1.09-0
2.16 10
3.75
1.06 10
NO-LIKK
2x
7.58 10-1
2.59 10-1
5.06 10-1
2.11
4.38 10-1
1.78 10-1
LOGLO-FNO
1x
3.78 10
8.23 10-4"
1.55 10-1
2.02 10-1
3.56 10-
9.46 =0
2x
4.29
10
7.95 10
1.47 10
1.51 10
4.39 10
1.4 10
We provide the spatial ZSSR results of baseline FNO, NO-LIDK variants, and our proposed LOGLO-FNO on the test set of the 2D Kolmogorov Flow dataset (Li et al., 2022a) in Table 18. The original spatial resolution of the datasct 128 128. Therefore, we perform an even rate 2x subsampling to obtain training data at resolution of 64 64, denoted 1x in the table. Naturally, the ZSSR task then evaluate the capability of the models for the next higher resolution of 128 128, for which we have the ground truth. We observe that baseline FNO exhibits superior spatial ZSSR capabilities, except on the spectral metrics (MELR & WLR), on which it incurs degradation. LOGLO-FNO, on the other hand, incurs degradation in spatial and spectral errors. However, it exhibits superior improvements on the frequency domain errors, outperforming base FNO on all metrics, but fRMSE (Low). Most importantly, we find that LOGLO-FNO is better than the NO-LIDK variants on all metrics. Subsequently, we perform ZSSR evaluation on the 5-step autoregressive rollouts, and find a similar trend.
68

---

Published in Transactions on Machine Learning Research (12/2025)
Table 19: Spatial ZSSR evaluation of LOGLO-FNO on the test set CNS Turb 2D dataset (Takamoto et al., 2022). Base FNO uses a spectral filter size of 40 and 65 hidden channels. LOGLO-FNO uses 40 and (16, 9) Fourier modes in the global and local branches, respectively, whereas the width set to 65. NO-LIDK" uses 40 modes and 65 hidden channels and radius cutoff value of 0.00390625. NO-LIDK* denotes using only localized integral kernel and NO-LIDK means employing both disco and diff layers. Training spatial resolution is (256 256), being noted as 1x, whereas the ZSSR test resolution is (512 x 512) and denoted as 2x. We boldface the lowest errors and underline the value when the 2nd best model is LOGI.O-FNO.
Model
Resolution
#RMSE (1)
IRMSE(L)
(RMSE(M)
IRMSE())
MELR (.)
WLR (L)
Train: 1x Test: 2x
1-step Evaluation
1x
1.28 10-3
1.78 10-3
1.57 10-3
2.78 10-1
7.8 10-3
Base FNO
2x
4.81-10
1.26 10
1.77 10
9.24- 10
5.5 10
7.85 10
1x
6.4 10"2
1.5
2.14 10--10-3
1.98 10
2.98
1.31 10
NO-LIDK
2x
7.61 10-2
7.73 10-3
2.92 2.92-10- 10-
10-3
5.29- 10-1
2.45 10-2
1x
5.57 10-2
1.37 10
1.89 10-3
1.77 10-3
2.08
9.54-10
NO-LIDK
2x
8.66 10-2
4.8 10-3
2.67 10-5
1.93 10-3
9.0 10-1
1.21 10-2
LOGLO
1x
4.34 10
9.74 10
1.17 10
1.41 10-3
1.83
5.93 10-100
2x
4.54
1.08 10
1.27 107-10" 10
8.91 10
3.56
7.75 10
5-step Autoregressive Evaluation
1x
7.78-10
3.76 10-2
5.37 10-3
2.4-10
2.22 10
9.36 10
Base FNO
2x
7.06 10
3,31 10
4.77 4.77 10
1.37 10
5.12 10
8.1 10
1x
8.55 8.55
4.14-10
5.92 10-3
2.62- 10-3
2.06
1.15 10
NO-LIDK
2x
1.35 10
10-1
11 11-10- 10-2
2.11- 10-3
4.25- 10
3.48 10
1x
7,91 10-2
3,82 10-3
5.38 10-3
2/7 10-3
1445-10
1.01 10-2
NO
2x
3.9
4.17 10-1
1041-0-11
7.21 10-2
2.36 10
5.25 10-1
LOGLO FNO
1x
6.72
2.8-10
4.0 10
2.14 10-3
1.31 10
6.78 10
2x
7,31 10-2
3.2 10-3
4.86 10-3
1.43 10-3
3.18 10
8.36 10
20-step (entire trajectory) Autoregressive Evaluation
1x
1.81 1811-10" 10
1.25-10 10
1.44 10-2
3.26 10-3
288 288-10 10-1
3.22 10-2
Base FNO
2x
1.82 10
1.26 1.26-10- 10-2
1.45 1455-0-2
1.78 10-3
5.52 10-1
3,24 10-2
1x
1.97-10-1
144 10
16-
3.36 10-3
2.95 10
3.94 10
NO-LIDK
2x
3.72 10-1
4.63 10-2
3.16 10-2
2.39 10-3
3.66 10-1
1.09 10-1
NO-LIDKI
1x
1.87
1.35 10-2
1.48 1.48-002 10--10-2
3.37 10-3
2.1 10-10-
3.7 10-2
2x
1.04 10
1.18 10
5.64 100
1.77 10"
1.33 10a
9.99
LOGLO-FNE FNO
1x
1.62 10
1.07 10
1.26 10
3.09 10
1.87 10
2.7 10
2x
1.72
1.16 10
1.34 10-
1.79 10-3
$.49
3.02
69

---

Published in Transactions on Machine Learning Research (12/2025)
M Training and Inference Time Comparison of LOGLO-FNO
M.1 Comparison with Numerical Solver
The inference time of LOGLO-FNO to generate predictions on the Turbulent Radiative Layer 3D dataset is provided in Table 20 and compared against the CPU-only numerical solver Athena+ used to generate the original simulations of the TRL3D dataset. However, in order to establish fair comparison with the numerical solver, the training time of the neural surrogate (LOGLO-FNO), which is about 12 hours for 50 epochs on TRL3D, should also be considered.
Table 20: Comparison of runtimes to generate all trajectories at the full resolution for the Turbulent Radiative Layer 3D dataset.
Solver Type
Hardware
Runtime
Numerical Solver: Athena++
CPU (128 cores)
34,560 CPU-h
(Stone et al., 2020; Ohana et al., 2024)
Neural Surrogate: LOGLO-FNO
one H100 GPU
38.143767 mins
Table 21: Scaling factor of runtimes to generate all trajectories at training and higher (2x) spatial resolutions for the Turbulent Radiative Mixing Layer 3D dataset.
Data Split
Spatial Resolution (64x64x128)
Spatial Resolution (128x 128x2666
Scaling Factor
Metric
Total Grid Points
x
524,288
4,194,304
8x
31,430.94 1 0.80 ms
228,862.66 60 1 0.81 ms (-3.8 min)
Runtime (9 Trajectories)
Test
-7.28x
(-31.4 sec)
314,309.4 + 8.0 ms
2,288,626.0 + 8.1 ms
Runtime (90 Trajectories)
Full
-7.28x
(-5.2 min)
(-38.1 min)
We measure the runtimes by generating the trajectory predictions only for the test set, containing 9 trajectories, each of which has 101 timesteps (row 2 in Table 21). The runtime consumed by the numerical solver +thena+ is taken from Ohana et al. (2024) The runtime for the full dataset containing 90 trajectories (row in Table 21) obtained by naive scaling (i.c., multiplying by 10) of the GPU runtime for the trajectories. During inference, we use batch size of 1 and report the mean runtime and standard deviation over 10 iterations for both spatial resolutions listed in Table 21.
M.2 Training Inference Time and GPU Memory Usage Comparison with Base FNO and NO-LIDK
In this part, we provide the training and inference times well as the GPU memory usage of LOGLO-FNO and compare them with FNO and NO-LIDK variants of baseline models since these are the most related model architectures for LOGLO-FNO. We note that all experiments, configured through the Slurm workload manager, are conducted on single H100 NVIDIA graphics card with 96 GB of memory. The benchmark problem we consider is the 2D Kolmogorov Flow (Li et al., 2022a) and 1-step training outlined in $4.0.1.
In order to establish a complete picture of the trade-off among these models terms of memory usage and total epoch times, we list the training times and GPU memory usage for the forward and backward passes Table 22. We observe that baseline FNO wins all other models, both in terms of GPU memory consumed and time needed for the completion of the full training cycle of 136 epochs. NO-LIDK 2, employing a differential kernel as parallel branch, occupies the second place. LOGLO-FNOY comes third in training time, while it requires slightly more memory than NO-LIDK*, which uses local integral kernel, takes fourth
hnctp:://polymathiccatt http:hhhe.ell//aaaats///utttee radiatiieeaaeieeeeeaaaaeaa 1ayyer_r5
layer_3D
70

---

Published in Transactions on Machine Learning Research (12/2025)
place when comparing training time. In last place is the NO-LIDK variant, which utilizes both kernels parallel branches in addition to the global Fourier branch.
Table 22:
Training times and GPU memory usage for the forward and backward passes during the 1-step training for total of 136 epochs the Kolmogorov Flow 2D training dataset. The reported values are the average over 4 training runs on the training set. NO-LIDK", NO-LIDK", and NO-LIDK' represent the variants trained with the local integral (DISCO), differential kernel, and both layers as additional parallel branches, in that corresponding order.
Model
Training Time
Slowdown wrt. FNO
GPU
Mem. Fact.
Batch Size
Spatial Res.
Fourier
Hidden Channel
Paize
Radius Cutoff
Mem. (GB)
va. FNO
Modes
Base FNO
1h 36m 17s + 8.35s
ref.
33.90 i 0.81
ref.
256
128x128
40
65
X
LOGLO-FNO'S
5h 19m 40s + 16.28s
3.30x
76.02 + 2.10
2.24x
256
128x128
40
65
16 x
16
X
NO-LIDK'
7h 23m 30s + 16.50s
4.58x
69.33 + 2.02
2.05x
256
128x128
20
129
X
0.05m
NO-LIDK"
2h 62m 51s + 7.55e
1.79x
35.63
= 1.12
1.06x
256
128x128
40
65
x
x
NO-LIDK!
7h 31m 11s + 13.54s
1.66x
78.75 + 4.63
2.32x
256
128x118
20
127
x
0.05m
Subsequently, we compute the inference times and GPU memory usage for the 1-step predictions and list them in Table 23, with the provided values averaged over 100 repetitions.
Table 23: Inference times and GPU memory usage for the forward pass during 1-step evaluation the Kolmogorov Flow 2D test dataset. The reported values are the average over 100 iterations on the test set. NO-LIDK", NO-LIDK", and NO-LIDK represent the variants trained with the local integral (DISCO), differential kernel, and both layers as additional parallel branches, in that corresponding order.
Model
Inference Time (ms)
Slowdown wrt. FNO
Mem. (GB)
GPU
Batch Size
Spatial Res.
Fourier Modes
Hidden Channel
Paize
Radius Cutoff
Base FNO
1471.41 0.08
ref.
10.74 0.0
256
128x128
40
65
X
X
LOGLO-FNOT
4899.53 0.15
3.33x
18.31
256
128x128
40
65
16
X
NO-LIDK'
7601.22 7.17
5.17x
19.63
256
128x128
20
129
0.05m
NO-LIDK
2477.68 0.14
10.19 + 0.0
128x128
40
65
NO-LIDK
7096.81 + 4.34
4.82x
21.43 = 0.0
256
128x128
20
127
X
0.05m
We observe a similar trend with baseline FNO being the fastest, however, consuming slightly higher GPU memory than NO-LIDK", which about 1.7x slower. Tn the third place is our LOGLO--FO model, incurring a 3.3x slowdown factor. Pertinent to our model NO-LIDK*, and it is about 1.6x slower while also incurring slightly higher memory consumption than LOGLO-FNO under the considered hyperparameter configurations, following Liu-Schiaffini et al. (2024). Finally, the NO-LIDK variant, consisting of both branches, incurs the highest GPU memory, although it is slightly faster than the NO-LIDK* variant with only the local integral kernel branch as additional pathway.
71

---

Published in Transactions on Machine Learning Research (12/2025)
N Confidence Analysis of LOGLO-FNO and Baselines
In this section, we report the mean and standard deviation values for the base FNO and NO-LIDK variants, comparing them with our proposed LOGLO-FNO model. We repeat the 1-step training setup discussed in $4.0.1 on the 2D Kolmogorov Flow dataset four times with different random seeds, and evaluate the 1-step as well as the 5-step autoregressive rollouts. The plot in Figure 39 shows the mean and 95% confidence interval values of fRMSE and WLR errors, with the hatched and solid bars visualizing the 1-step and 5-stop autoregressive rollout errors in that order.
Timestep
1-Step
S-Step
LOGLO-PNO
NO-LIDK *
NO-LIDK
0.08
Spectral Eeve Value
0.06
0.04
FRMSB(H)
FRMSE(
(RMSELL
WIR
Figure 39: Confidence analysis of baseline FNO, NO-LIDK (Liu-Schiafinni et al., 2024), and proposed LOGLO-FNO for the 1-step and 5-step autoregressive evaluation of (RMSE and WLR (Wan et al., 2023) metrics on the test set of 2D Kolmogorov Flow (Li et al., 2022b). NO-LIDK* and NO-LIDK represent the variants trained with the local integral (DISCO) and differential kernel layers, respectively.
We observe that LOGLO-FNO consistently outperforms baseline FNO and significantly better than NO-LIDK*, utilizing the local integral kernel, across all metrics. Particnlarly noteworthy are the improvements we observe in the rollout scenarios.
In a similar vein, in Figure 40, we visualize the mean and 95% confidence interval values of nRMSE and MELR errors, whereas Figure 41 establishes a comparison of the plain and boundary RMSE values.
We find that while LOGLO-FNO the best model on the nRMSE metric, NO-LIDK employing a differential kernel layer outperforms all models, reinforcing the results found in Table However, we note that the results NO-LIDK are provided only for the sake of completeness, and LOGLO-FNO should be compared with NO-LIDK* when assessing its effectiveness.
Liu-Schiaffini et al. (2024) find the differential kernel layer to be effective in reducing the boundary error for non-periodic boundary conditions. Additionally, in our study on Kolmogorov Flow 2D, it has been found to effective for periodic boundary conditions as well, enforced through padding. However, this advantage is not maintained rollouts (see Figure 41).
72

---

Published in Transactions on Machine Learning Research (12/2025)
Timestep
Models
0.7
1-Step
Base FNO
5-Step
LOGLO -NO
NO-LIDK
0.6
NO-LIDK
Spectral Value
0.4
0.3
0.2
#RMSE
MELR
Figure 40: Confidence analysis of baseline FNO, NO-LIDK (Liu-Schiaffini et al., 2024), and proposed LOGLO-FNO for the 1-step and 5-step autoregressive evaluation of nRMSE and MELR (Wan et al., 2023) metrics the test set of 2D Kolmogorov Flow (Li et al., 2022b). NO-LIDK' and NO-LIDK represent the variants trained with the local integral (DISCO) and differential kernel layers, respectively.
Timestep
Base ENO
1-Step
LOGLO-FNO
5-Step
NO-LIDK *
1.50
NO-LIDK
1.25
XRMSE Value
1.00
0.75
0.25
RMSE
bRMSE
Figure 41: Confidence analysis of bascline FNO, NO-LIDK (Liu-Schiaffni et al., 2024), and proposed LOGLO-FNO for the 1-step and 5-step autoregressive evaluation of xRMSE (plain and boundary) metrics the test set of 2D Kolmogorov Flow (I. et al., 2022b). NO-LIDK* and NO-LIDK represent the variants trained with the local integral (DISCO) and differential kernel layers, respectively.
73

---

Published in Transactions on Machine Learning Research (12/2025)
O Model Hyperparameters
O.1 Modern U-Net Hyperparameters
The Table 24 below presents the hyperparameters for the modern version of U-Net. baseline we use from the PDEArena (Gupta & Brandstetter, 2023) benchmark codebase.
PDE Problem
Channel Multiplier
Learning
Sched. Steps
Sched. Gamma
Train Type
Train Loss
Total Epochs
Rate (LR)
Sched.
21) Kolmogorov Flow 2, 2, 4)
1.0 1.0e -e
StepLR
33
0.5
1-step
MSE
136
2D CNS Turb
2,
1.0 e
StepLR
33
0.5
1-step
MSE
136
Table 24: Model hyperparameters for the modern U-Net (Gupta & Brandstetter, 2023) baseline.
O.2
Base FNO Hyperparameters
The Table 25 below lists the hyperparameters for training the FNO baseline model on the two-dimensional and diree-dimensional PDE problems considered in our experiments. Note that we borrow the hyperparameters from Liu-Schiaffini et al. (2024) for the Kolmogorov Flow case and Takamoto et al. (2022) for the Diffusion- Reaction 2D PDE.
PDE Problem
Fourier Modes
Channel Dimens.
Learning
LR Scheduler
Sched. Steps
Sched. Gamma
Train
Train
Total Epochs
(LRR)
Type
2D Diffuslon-Readte
12
20
1.0 1-0---
StepLR
100
0.5
AR
MSE
500
2D Kolmoguruv Flow
40
65
1.0
StepLR
33
0.5
sstep
MSE
136
21) Wave-Ganss
1.0
Stepl.R
33
0.5
1-step
MSE
136
CNS Turb
10 <
StepLR
0.5
1-step
MSE
136
2D Comp. Euler RP
1.0
StepLR
0.5
1-step
MSE
136
Torbulent Rediative Layer 31
1.0
-5
StepLR
0.5
1-step
MSE
50
Table 25: Model hyperparameters for FNO baseline.
O.3 F-FNO Hyperparameters
Table 26 provides the hyperparameters for the F-FNO baseline model from Tran et al. (2023). As with the baseline FNO model, we use four Fourier layers.
PDE Problem
Fourier Modes
Channel Dimens.
Learning
LR Scheduler
Warmup Steps
Train Type
Train Loss
Total Epochs
Rate (LR)
Diffusion-
Reaction Diffusion-Reaction
16
32
6.0
Cosine Warmup
200
AR
MSF
500
2D Kolttogorvv Flow
40
65
1.0 c
Cosine Warmup
60
1-step
MSE
136
Table 26: Model hyperparameters for F-FNO (Tran et al., 2023) baseline.
0.4 LSM Hyperparameters
Tables 27 provides the hyperparameters used for training the Latent Spectral Model (Wu et al., 2023) bascline on the Kolmogorov Flow 2D, CNS Turb 2D, Wave-Gause 2D, and Diffusion-Reaaciion 2D datasets, whereas Table 28 lists the model hyperparameters used for the LSM architecture configuration.
We found the model be very sensitive to the learning rate in the full autoregressive training setup and, hence, we use low learning rate of 5.0 -5 in addition employing gradient clipping with max norm of 5.0.
74

---

Published in Transactions on Machine Learning Research (12/2025)
PDE Problem
Learning Rate (LR)
Sched. Steps
Sched. Gamma
Train
Train
Total Epochs
Scheduler
Type
Loss
2D Cillusion-Reaction
5.0 -
StepLR
100
0.5
AR
MSE
500
2D) Kolmogorov Flow
1.0 -0
StepR
33
0.5
1step
MSE
136
2D CNS Turb
1.0
StepLR
33
0.5
1-step
MSE
136
2D Wave- Gauss
1.0 v--
StepLR
33
0.5
1-step
MSE
136
3D Turbulent Radiative Layer
StepLR
10
0.5
1-step
MSE
50
Table 27: Training settings for the LSM (Wu et al., 2023) bascline model.
PDE Problem
Num. Tokens
Num. Basis
Channel Dimens. (init.)
Downsample Ratio
Patch
Interpol.
Scales
Size
Type
2D Diffusion-Reactionn Reaction
32
5/10
(4.4)
bilinear
2D Kohmngoror Flow
16
64
5/10
(4.4)
bilinear
2D CNS Turb
16
64
5/10
(4.4)
bilinear
2D Wave-Gauss
32
5/10
(4.4)
bilinear
3D Turbulent Radiative Layer
64
5/10
(4,4,8)
trilinear
Table 28: Further Model hyperparameters for LSM (Wu et al., 2023) baseline.
Channel dimensions (init.) represent the number of channels used for the incoming full spatial resolution, including which the remaining four spatial scales and their respective channols follow.
0.5 Transolver Hyperparameters
Tables 29 provides the hyperparameters used for training the Transolver Model (Wu et al., 2024) bascline on the Kolmogorov Flow 2D, CNS Turb 2D, Wave-Gauss 2D, and Diffusion-Reactioo 2D datasets, whereas Table 30 lists the model hyperparameters used for the Transolver model architecture configuration.
PDE
Optimizer
Learning Rate (LR)
LR Scheduler
Weight Decay
Train Type
Train Loss
Total Epochs
Problem
Diffusion-Reaction Reaction
AdamW
4.0 400e-1
OueCycleLR
10 e0
AR
MSE
500
2D Kolmogorov Flow
AdamW
-
OneCycleL.R
10 10
1-step
MSE
500
2D CNS Turb
AdamW
6.0 -3
OneCycleLR
10 e-
1-step
MSE
2D Wave Gauss
AdamW
6.0 .25
OneCyeleL.R
1.0 --
1-step
MSE
500
Table 29: Training settings for the Transolver (Wu et al., 2024) baseline model.
PDE Problem
Num. Layers
Num. Heads
Num. Slices
Hidden Channels
Activation Function
MLP Ratio
Apply
Dropout
2D Diffusion-Reaction
32
64
GeLU
False
2D Kolmogorov Flow
64
128
GeLU
False
CNS Turb
9
8
64
128
GeLU
False
2D Wave-Gauss
9
8
64
32
GeLU
False
Table 30: Model architecture hyperparameters for Trranslvvrr (Wu et al., 2024) baseline.
75

---

Published in Transactions on Machine Learning Research (12/2025)
O.6 U-FNO Hyperparameters
Table 31 below lists the hyperparameters for the U-FNO baseline model from Wen et al. (2022).
PDE Problem
Fourier Modes
Channel Dimens.
Learning
LR Scheduler
Sched. Steps
Sched. Camma
Train
Train
Total
Rate (LR)
Type
Loss
Epochs
2D DifltsioonReactionn
10
19
5.0 --!
StepLR
10
0.9
AR
MSE
500
2D Kolmogorow Flow
36
48
1.0
StepI.R
0.9
1step
MSE
136
2D CNS Turb
36
48
StepLR
10
0.9
1-step
MSE
136
2D Wave-Gauss
16
32
5.0 --
StepLR
10
1-step
MSE
136
Table 31: Model hyperparameters for U-FNO (Wen et al., 2022) baseline.
0.7 NO-LIDK Hyperparameters
Table 32 lists the hyperparameters for different configurations of the Neural Operators with Localized Integral and Differential Kernels (NO-LIDK) model from Liu-Schiaffini et al. (2024). For the Kolmogorov Flow 2D data, following the authors' training setup (see Table 4 in Liu-Schiaffini et al. (2024)), we use 20 modes and a hidden channel dimension of 127 for the model employing both local integral and differential kernel layers (NO-LIDK*), 20 modes and a hidden channel dimension 129 for the model using only the local intogral kernel layer (NO-LIDK*), and 40 modes and a hidden channel dimension of 65 for the model with just the differential kernel layer (NO-LIDK") as parallel layers in addition to the global spectral convolution layer of base FNO. All these variants use 4 Fourier layers in total.
Integral Kernel
Differential Kernel
Learning Rate (LR)
Sched. Steps
Sched. Gamma
Problem
Sched.
Type
Loss
Epochs
2D Kolmogorov Flow
1.0 10
StepLR
0.5
1-seep
MSE
136
2D Wave-Gaus
10
StepLk
33
0.5
1-step
MSE
136
Comp. Euler RP
StepLR
0.5
1-step
MSE
136
2D CNS Tub
-
StepLR
0.5
1-step
MSE
136
Turbulent Radistive Layer 3D
6.0 -3
OneCyeleLR
10
1-step
MSE
50
Table 32: Model hyperparameters for NO-LIDK (Liu-Schiaffnni et al., 2024) baseline.
The local integral kernel for the disco convolution employs radius cutoff value of 0.0078125, 0.0078125, and 0.00390625 for the Wave-Gause 2D, Compressible Euler four-quadrant Riemann problem 2D, and CNS Turb 2D problems, respectively, following the hcuristic suggested by Liu-Schiaffini et al. (2024). As in the case of the baseline FNO model, the number of spectral filters and hidden channels is set as 40 and 65, resp., for all three configurations of the trained models (i.e., disco layers - NO-LIDK*, differential layers - NO-LIDK", and diffe--diflerentiall layers NO-LIDK!) for the CNS Turb 2D dataset, whereas we use 20 modes and 32 hidden channels for the Waav-Gaaus 2D problem. We use the hyperparameter settings applied for 2D Kolmogorov Flow for the Compressible Fuler four-quadrant Riemann problem 2D PDF.
For the Turbulent Radiative Layer 3D data, use 16, 16, and 20 modes along the x, y, and z axes. The number of hidden channels is set as 96, and the model consists of 4 layers, as in the case of all 2D experiments.
O.8 LOGLO-FNO Implementation and Hyperparameters
Implementation. Our implementation uses PyTorch v2 (Ansel et al., 2024), NumPy (Harris et al., 2020), and FNO base implementation adapted from the neural operator library (Kossaifi al., 2024a).
Hyperparametors. The Kolmogorov Flow problem uses a batch size of 256 and a patch size of 16 x 16 (in the local branch) for the 1-step training strategy, whereas it is set 8 for Diffusion-Reaction 2D due memory linitations with fully autoregressive training on long trajectories of 91 timesteps. An experiment.
76

---

Published in Transactions on Machine Learning Research (12/2025)
employing data-paralle, training using a patch size of 16 has not resulted in improved performance. Therefore, we set the patch size to 8 when extracting non-overlapping patches on the domain for the Diflusion-Reaction problem and report the results for this setting.
PDE Problem
Fourler Modes (Global)
Channel Dimens.
Learning
LR Scheduler
Schod.
Sched. Gamma
Traio
Train
Total
Rate (LR)
Loss
Epochs
Doffesion-Roaction
80
StepLR
MSE (usdkkligh) Preq
500
2D Kolmogotov Flow
SueaRR
1step
(usdkkaagh) Rice.
2D Wave Guras
SupLR
33
0.5
1step MSE + (midkhhigh) Preq
136
2D Comp. Eake RP
dn
65
StepLR
30
0.5
1step
MSE + (milkhhigh) Preq.
136
2D CNS Turb
StepLR
0.5
MSE
(midkchigh) Preq
136
Turbaket Radlictive Layer
4.
SteplLR
(midk&higgh) Frog. 50
Table 33: Model hyperparameterss for our proposed LOGLO-FNO in the main paper with simple summation (+) as the fusion mothod (Fig. 1) for the features from the HFP, global, and local branches.
All models use a batch size of 64, 256, and 256 for the CNS Turb 2D, Compressible Euler four-quadrant Riemann problem 2D, and Wave-Gauss 2D problems, respectively. LOGLO-FNO uses patch size of 16 for the Compressible Euler four-quadrant Riemann problem 2D and Wave- Gauss 2D from Poseidon (Herde et al., 2024) and CNS Turb 2D datasct, which is provided by PDEBench (Takamoto et al., 2022).
The kernel size and stride have been set to 4 in the high-frequency extraction and propagation modules for all PDE problems. The local branch (e.g., with a patch size of 8 x 8 or 16 x 16) uses all available frequency modes. This would yield 16 and 9 Fourier modes for the spatial dimensions and y, respectively, when a patch size of 16x16 is considered. Analogously, assuming an equal number of sampling points along all three spatial axes of 3D spatial data, the local branch configured with a patch size of 16 16 16 would use all the 16, 16, and 9 Fourier modes for the spatial dimensions x, y, and z, respectively. Since Turbulent Radiative Layer 3D (Ohana al., 2024) has 2x more observation points on the z-axis (i.e., 128 128 256), we proportionate patch size of 16 x 16 x 32, resulting in 16, 16, and 17 Fourier modes along the x, y, and z axes, respectively.
77

---

Published in Transactions on Machine Learning Research (12/2025)
P Table of Notations and Mathematical Symbols
To serve as a handy guide, the following tabulation describes the symbols used within this manuscript.
R
Set of positive real numbers. Specifically, t E (0, T
V
Gradient or vector derivative operator (
0 a de By ...)
A
Laplacian (V2)
A,u
Banach spaces of functions
Partial derivative with respect to
Partial derivative with respect to
Og
Partial derivative with respect to
t(x,t)
Solution of PDE time for given spatial coordinate Velocity vector field (x,, ...)
p
PDE paramcter values, cither scalar or vector
Ng
Neural network with learnable parameters
P,Q
Lifting and projection layers, respectively
KG
Local and global kernel integral operator, respectively Number of layers in the network
L
c
Cost function
(x)
Tnitial data of the PDE at time
Ω
Domain of the PDE under consideration
00
Boundary of the domain under consideration
N₂
Total number of timesteps in the simulation
N
Total number of spatial points along the x-axis of the simulation
Ny
Total number of spatial points along the y-axis of the simulation
N₂
Total number spatial points along the w-axis of the simulation Total number of channels or fields in the simulation
Ne
de
channel dimension a.k.a. width of the network
psise
Patch size or sub-domain size
Viscosity coefficient of Navier-Stokes equations
Re
Reynolds Number
p
Mass densily in Navier-Stokes equations
Shear viscosity in Compressible Navier-Stokes equations
<
Bulk viscosity in Compressible Navier-Stokes equations
Internal energy in Compressible Navier-tookes equations
Du. D.
Diffusion cocfficients for the activator and inhibitor in the Diffusion- Reciiion equation, respectively
F
Fourier transform of signal
78


Reviews:
[{"review_id": "cETSLa91dS", "reviewer": "Reviewer_143c", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper attempts to preserve high frequency content of the data and the solution in a Fourier Neural Operator. The authors introduce (i) a local convolution operation on smaller subdomains which helps capturing the local features (small scale or high frequency information), and (ii) a high-frequency propagator, i.e., a blurring and reconstruction process and then calculating the error introduced in that operation. In addition, the loss function is enhanced to capture the errors introduced by the truncation step of FNOs. Finally, the authors test their method on 2D Kolmogorov flow, 3D turbulent radiative mixing layer and a 2D diffusion-reaction problem. The methods produces a marginally lower error on these problems with respect to the ground truth.", "claims_and_evidence_yn": "Yes", "claims_explanation": "* The paper is well-written, and is a good attempt at solving a genuine issue with FNOs\n* The authors use two simple ideas from the literature and apply them to improve FNOs\n* The ideas are discussed in detail\n* Numerical results are presented supporting the claims.\n* The paper is tightly focused toward a limited set of equations, but does a good job delivering on that.", "audience_yn": "Yes", "audience_explanation": "The paper deals with retaining the high frequency contents of the data and the solution in a neural operator. The authors apply a local convolution operator and an artificial high-frequency error propagator to improve the high frequency errors in the final prediction. This is of great importance in the scientific machine learning research.", "requested_changes": "Weaknesses:\n\n* The applications are limited to a chosen few examples. However, I do not necessarily think this is a weakness.\n* A pictorial representation of the high-frequence propagator would be helpful.\n* Many of the metrics are non-standard. The authors have provided reference for all the metrics, and also listed them in Appendix G. However, not all metrics are listed. A complete list of formulas would be appreciated.\n* The improvements in errors (as per Table 1, 2 and 3) are only marginal.\n* How do the results compare with traditional numerical results? Is it possible to illustrate that with a standard problem?\n\nQuestions:\n\n* I imagine that the local convolutions introduce a \"zooming in\" effect. Different convolution applied to subdomains rather than the full domain, helps in capturing the local features. Do the authors agree with this interpretation?\n* Did the authors consider or try problems with shock features?\n* I commend the authors on this method. Still, what is special in this method that warrants its application, especially considering that the gains are only marginal?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "GYa6NNZlj1", "reviewer": "Reviewer_UKSR", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors introduce LOGLO-FNO, an enhanced Fourier Neural Operator designed to overcome the common neural network issue of \"spectral bias,\" where models struggle to capture high-frequency details crucial for complex physical simulations like turbulence. The authors' solution integrates three key architectural enhancements: a parallel branch for **local spectral convolutions** to learn fine-grained features from data patches, a **high-frequency propagation module** to ensure critical details aren't lost, and a novel **frequency-aware loss function** that specifically penalizes errors in non-dominant frequencies during training. However, these enhancements result in a model that is not only more accurate and stable over long-term predictions but is also significantly more parameter-efficient, achieving the accuracy of baseline models with up to 50% fewer parameters. The primary limitation noted by the authors is the intentional exclusion of advanced parameter-efficiency techniques for the sake of a fair comparison, suggesting a clear direction for future improvements. Overall its cool work but I have some concerns which I will talk about it more in the following sections.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Personally for me the authors have done a solid job supporting their claims with convincing evidence. The core assertion that FNOs struggle with high-frequency features due to spectral bias is well-established, and their proposed LOGLO-FNO architecture provides a reasonable solution. What I find particularly compelling is how they've structured their evaluation - they don't just show overall improvements, but specifically target frequency-domain metrics that directly relate to their main claims. The 11-25% improvements in high-frequency error reconstruction across three different PDE problems (Kolmogorov flow, diffusion-reaction, and turbulent radiative mixing) demonstrate consistency that goes beyond cherry-picking results. I appreciate that they've included comprehensive baselines including recent state-of-the-art methods like NO-LIDK, and the parameter efficiency gains (achieving better accuracy with 50% fewer parameters) make the contribution practically valuable. The experimental design shows good scientific rigor with proper ablation studies, energy spectra visualizations that provide intuitive understanding of the improvements, and thorough analysis of autoregressive stability. The mathematical framework appears sound, and I think the frequency-aware loss function is a clever addition that directly addresses the spectral bias problem. Thanks for the detailed ablations and studies in the Appendix!\n\nHowever, there are some notable limitations that weaken the overall contribution. The evaluation scope is quite narrow - only three PDE problems, all from similar domains (fluid dynamics and related physics). While the authors claim generalizability, testing on a broader range of PDEs, especially those from different scientific domains like electromagnetics, quantum mechanics, or materials science, would strengthen their claims significantly. The choice of patch sizes and architectural decisions also seems somewhat ad-hoc - for instance, using 8×8 patches for diffusion-reaction but 16×16 for Kolmogorov flow due to \"memory limitations\" suggests the method may not scale as elegantly as claimed. Additionally, the comparison with some baselines isn't entirely fair since they disable tensor factorization techniques in FNO \"for fair comparison with NO-LIDK,\" which artificially handicaps what could be stronger baselines.\n\nThe theoretical analysis could also be deeper - while they provide parameter complexity analysis, there's limited theoretical insight into why their local-global decomposition should work better than alternatives, or under what conditions we might expect it to fail. The frequency loss design, while intuitive, lacks principled justification for the specific binning strategy and weighting choices. Some experimental choices also raise questions - the different training epochs across datasets (50 for TRL3D vs 136 for Kolmogorov) and the model-specific hyperparameter tuning suggest the method may require careful dataset-specific optimization rather than being a robust drop-in replacement for standard FNOs.", "audience_yn": "Yes", "audience_explanation": "I definitely think TMLR's audience would find this paper quite interesting and valuable. The neural operator community has been growing rapidly, especially with applications in scientific computing where traditional numerical methods are becoming computationally prohibitive for high-resolution simulations. The spectral bias problem that the authors address is a well-known issue that many researchers working with FNOs have encountered - anyone who has tried to model turbulent flows or high-frequency phenomena with neural operators has probably struggled with this exact limitation.", "requested_changes": "Again some changes and thoughts that I had:\n\n• Expand experimental evaluation beyond 3 PDEs - The current evaluation is too narrow with only Kolmogorov flow, diffusion-reaction, and turbulent radiative mixing. Include at least 2-3 additional PDE types from different domains (e.g., electromagnetics, wave equations, or heat conduction) to demonstrate true generalizability of the approach.\n\n• Address inconsistent experimental settings - Different training epochs across datasets (50 for TRL3D, 136 for Kolmogorov, 500 for diffusion-reaction) and varying patch sizes (8×8 vs 16×16) due to \"memory limitations\" suggest the method may not be as robust as claimed. Provide systematic study of these choices or standardize the experimental protocol.\n\nSome thoughts on what would strengthen this work:\n\n• Improve frequency band justification - The radial binning strategy (low=[1,4], mid=[5,12], high=[13,M]) in Algorithm 1 appears arbitrary without theoretical or empirical justification for these specific cutoffs.\n\n• Add statistical significance analysis - Results show consistent improvements but lack confidence intervals or significance tests to demonstrate improvements are statistically meaningful beyond empirical observation. This is fine if you cannot do in the limited time, but good for the full paper: )!\n\n• Add failure case discussion - Paper only shows successful cases. Including examples where method doesn't improve over baselines and analyzing why would provide balanced perspective. Like i am curious where your method did indeed fail in your opinion as well as how well does your method scale?\n\nOverall good job!", "broader_impact_concerns": "N/A", "additional_comments": "N/A", "extra_scores": {}, "extra_text": {}}, {"review_id": "H95UkBjlvK", "reviewer": "Reviewer_CTmE", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces a variant of the Fourier Neural Operator (FNO) designed to improve its ability to capture high-frequency and localized features. The proposed modifications include:\n- A patch-wise FNO module running in parallel with the standard FNO.\n- An additional input head restricted to high-frequency components.\n- A frequency-aware loss leveraging radial binning.\n\nThe method is conceptually sound and shows consistent empirical gains over strong baselines. However, the added complexity raises questions about generalization and computational cost.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The method is compared on standard benchmarks with competitive baselines, but some contributions are not properly assessed.", "audience_yn": "Yes", "audience_explanation": "The paper does not constitute a major conceptual breakthrough, but the approach is technically sound and empirically convincing. Its ease of use and good performance may make it appealing in applied settings.", "requested_changes": "## Mathematical clarity\nWhile the paper is technically solid, the presentation suffers from unnecessary mathematical overhead that makes reading uneasy. I strongly recommend improving the notation and offloading details to the appendix. Below are some recommendations that would (in my opinion), improve the paper\n\n1) Avoid using uncommon symbols, unless all other characters have already been used. For instance, $\\Upsilon$, $\\mathbf{W}$square and $\\varkappa$ are not standard and could be readily replaced with $Y, W_i$ and $p_\\text{size}$, making the equation easier to read.\n\n2) Many symbols appear only once and add little to no value. I suggest removing them from the main paper to improve clarity and delaying all mathematical details in the appendix. Few examples:\n\n\t- Bounded domains: $\\Omega_{d_a}$ and $\\Omega_{d_u}$,\n\t- All matrix parameters in Equations (1) and (2). Equation (1) could be simplified as:\n\n$Y = \\sigma\\big(K(Z) + \\text{Conv1D}^0(Z))$\n\n$\\mathcal{L}(Z) = \\text{ChannelMLP}^0(Y) + \\text{SoftGating}^0(Z)$\n\nwhich aligns with typical FNO implementation, is easy to read, and does not introduce unnecessary variables. The same remark applies to Equation (2). \n\n3) There are several ambiguous or inconsistent definitions:\n\t- **Equation (2)**: I suppose $K_g$ and $\\Upsilon_{\\text{global}}$ are aliases for $\\Upsilon$ and $\\mathcal{K}$ from equation (1) ? If so, why changing notations? $\\hat Z$ and $Z'$ are also not clearly introduced.\n\t- **Equation (3)** What does $a$ refer to ? It is used as global input to the network  $u = \\mathcal{G}(a)$ and in Equation (3). Why not using $\\hat Z$ ? What are $z_t^{(m)}$, $t$ and $m$ ? If they represent time and patch indices, state so explicitly.\n\t- **Next unlabeled equation**: Please add a label. Add a label. Symbols like $\\mathcal{H}$, $X_S$, and $X_U$ are inconsistent with previous notation and absent from Equation (2) and Figure 1. Why changing the notation again?\n\t- **Algorithm 1**: The introduction of new symbols $\\hat P$ and $\\hat T$ ? and conflicting naming ($X$ and $Y$ to denote the spatial axis and $x$ and $y$ for the frequency grid) introduce confusion.\n\nIn particular, I strongly encourage the author to improve the readability and clarity of Equation (2), as it is one of the main contributions of the paper.\n\n## Reorganize the paper\n\nThe paper, especially with its appendix, is overly long and would benefit greatly from a reorganization, since many important results are in the appendix. Here are my main suggestions:\n\n**Move important content to the main paper:**\n\n- **Figures 4–6** are critical to understanding the HF loss and should be integrated into the main text, ideally with a consolidated version for readability.\n- **Baseline discussion** should be summarized early in the main paper, along with key distinctions from LOGLO-FNO.\n- **Table 7**, showing performance breakdowns, is important enough to warrant inclusion in the main body.\n\nAlgorithm (1) and Equation (4) are difficult to parse and can be relegated to the appendix with a clearer summary or diagram in the main text. However, Figure 4-6 are critical for understanding the loss. A figure inspired from these plots should be integrated into the main paper.\n\nTable 1 can be condensed to highlight the most informative metrics (e.g., RMSE, and fRMSE for L, M, H). The full table can stay in the appendix. Figure 4, 5 and 6 (right) could also be merged and displayed in the main paper for rapid evaluation of the difference between LOGLO-FNO and the baselines.  While extensively detailed in the appendix, the baselines and metrics should be quickly presented in the main paper, especially by stating the difference with LOGLO-FNO. \n\nThe related work section is complete to the best of my knowledge, however I would recommend improving the 3rd paragraph to not only list competitors but also motivate how LOGLO-FNO differs from existing methods.\n\nFinally, the appendix can be cleaned and shorten by avoiding unnecessary white spaces, repeating figures from the main paper, and repeating table 5, 6, and 7 which have redundant lines. I also find Table 7 quite important, and would recommend moving it to the main paper. Section G.2 feels disconnected.", "broader_impact_concerns": "- **Training performance**: The authors discussed extensively the impact of their method on the number of parameters, but this should be completed by a training and inference time comparison, as I would expect that the additional terms and the frequency loss impacts greatly the efficiency. This is important to assess the balance between the added computational cost and the performance gain.  The impact on GPU memory usage could also be very relevant.\n- **Resolution invariance**: The authors propose to slice the input domain into patches to handle local features, akin to ViTs.  Yet, this might impact the generalizability of the method to higher or lower resolutions, as we now ViTs does not necessarily generalize to resolution outside their training set. Moreover, zero-shot multi-resolution is the core motivation behind FNO, hence assessing that LOGLO-FNO maintains this feature is detrimental. Such evaluation is missing from the paper.\n- **Missing ablations**: Several architectural components are introduced but not validated. Missing ablations include:\n\t- Patch-size sensitivity, especially given spectral resolution trade-offs (decreasing patch-size might improve the model on local features, but also decreases the number of modes per patches)\n\t- Adaptive Gaussian noise\n\t- High-frequency component in the model. It is unclear for me how this affect the model performances, since the additional filtered input remains high frequency, and thus difficult to learn with a neural network even if the low frequency component has been removed.\n- **Performances**: It is unclear why the author also supervise mid-range frequencies while the paper focuses on high-frequencies. In that case, why not supervising all frequency ranges ? Moreover, according to Table (1), the gain in high frequency RMSE does not transfer well when increasing the prediction horizon.\n\nTo summarize, the paper shares an interesting solution, yet not groundbreaking. The method is sound, addresses a known limitation of FNOs, and seems to perform reasonably well. My main concerns are on the writing and organisation of the paper, which deserves a refactoring effort. As it stands, it does not align with the standard of rigor for a TMLR paper. Yet this can be quickly addressed during revision.\n\nI also have several scientific concerns, especially about the trade-off between performances and impact on training time and memory usage. Few ablations are also missing to confirm that LOGLO-FNO maintains the important properties of FNO (resolution invariance), and assessing all introduced contributions properly.", "additional_comments": "Based on the implementation of the FNO block [1], I'm not sure that equation (1) is correct. Using notation from equation (1), the implementation seems to be:\n\n$Z' = W_f*Z + b_f$\n\n$Y = \\sigma\\big(K(Z') + Z'\\big)$\n\nCould the authors double-check this equation and ensure that it corresponds to the one implemented?\n\n[1] https://github.com/neuraloperator/neuraloperator/blob/main/neuralop/layers/fno_block.py", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)