# LOGLO-FNO: Efficient Learning of Local and Global Features in Fourier Neural Operators

Marimuthu Kalimuthu, David Holzmüller, Mathias Niepert

► To cite this version:

Marimuthu Kalimuthu, David Holzmüller, Mathias Niepert. LOGLO-FNO: Efficient Learning of Local and Global Features in Fourier Neural Operators. Transactions on Machine Learning Research Journal, 2025, 10.48550/arXiv.2504.04260. [https://doi.org/10.51369/53v2](https://doi.org/10.51369/53v2)

HAL Id: hal-05136953

https://hal.science/hal-05136953v2

Submitted on 17 Dec 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# LOGLO-FNO: Efficient Learning of Local and Global Features in Fourier Neural Operators

**Marimuthu Kalimuthu**

Universität Stuttgart, Stuttgart Center for Simulation Science, SimTech
International Max Planck Research School for Intelligent Systems (IMPRS-IS)

firstname.lastname@ki.uni-stuttgart.de

**David Holzmüller**

INRIA Saclay, SODA Team

firstname.holzmuller@inria.fr

**Mathias Niepert**

Universität Stuttgart, SimTech, IMPRS-IS

firstname.lastname@ki.uni-stuttgart.de

Reviewed on OpenReview: https://openreview.net/forum?id=MQ1dRdHTpi

## Abstract

Modeling high-frequency information is a critical challenge in scientific machine learning. For instance, fully turbulent flow simulations of the Navier-Stokes equations at Reynolds numbers 3500 and above can generate high-frequency signals due to swirling fluid motions caused by eddies and vortices. Faithfully modeling such signals using neural networks depends on the accurate reconstruction of moderate to high frequencies. However, it has been well known that deep neural nets exhibit the so-called spectral or frequency bias towards learning low-frequency components. Meanwhile, Fourier Neural Operators (FNOs) have emerged as a popular class of data-driven models in recent years for solving Partial Differential Equations (PDEs) and for surrogate modeling in general. Although impressive results have been achieved on several PDE benchmark problems, FNOs often perform poorly in learning non-dominant frequencies characterized by local features. This limitation stems from the spectral bias inherent in neural networks and the explicit exclusion of high-frequency modes in FNOs and their variants. Therefore, to mitigate these issues and improve FNO's spectral learning capabilities to represent a broad range of frequency components, we propose two key architectural enhancements: (i) a parallel branch performing local spectral convolutions and (ii) a high-frequency propagation module. Moreover, we propose a novel frequency-sensitive loss term based on radially binned spectral errors. This introduction of a parallel branch for local convolutions reduces the number of trainable parameters by up to 50% while achieving the accuracy of the baseline FNO that relies solely on global convolutions. Moreover, our findings demonstrate that the proposed model improves the stability over longer rollouts. Experiments on six challenging PDE problems in fluid mechanics, wave propagation, and biological pattern formation, and the qualitative and spectral analysis of predictions, show the effectiveness of our method over the state-of-the-art neural operator families of baselines.

## 1 Introduction

Simulating real-world physical systems and problems in thermodynamics, biology, weather and climate modeling, hydrodynamics, and astrophysics, to name a few, involves solving partial differential equations (PDEs). Numerical PDE solvers based on Finite Volume, Finite Difference, and Finite Element methods might be slow due to the need for fine discretization of the computational mesh, work only for a given set of input parameters, and must be run from scratch when the settings, such as initial and boundary conditions, change. In recent years, neural networks have been used to build generalizable surrogate models
of such dynamical systems (i.e., forward problem) and for the inverse task of the discovery of PDEs and their parameters from data (Brunton & Kutz, 2023; 2024).

Effective modeling of the entire frequency spectrum, including high frequencies, is important for tasks such as super-resolution and turbulence modeling (Wang et al., 2020; Fan et al., 2024). In the latter case, large eddies can give rise to small eddies, resulting in chaotic dynamics, which can further be influenced by forcing functions (Fan et al., 2024). Large Eddy Simulation (LES) and Direct Numerical Simulation (DNS) require prohibitive costs for high-resolution scenarios (e.g., 2048×2048; Tran et al. (2023); Fan et al. (2024)). Neural Operators (NOs) are a class of surrogate models that can be trained at low spatial and temporal resolutions, saving cost, time, and compute, and can be used to predict solutions at resolutions unobserved during training, a technique called zero-shot super-resolution. Although the predominant neural operators exhibit this property, their accuracy is limited on simulation tasks that exhibit fine-grained details and complex structures. For instance, the Navier-Stokes equation exhibits high frequencies at low viscosities (i.e., $\nu = 1e^{-4}$) or high Reynolds numbers (e.g., Re = 5000; Li et al. (2022b)). NOs suffer from the spectral bias of neural nets and have difficulty modeling high frequencies and local features. We review related literature concerning this challenge in Appendix A. Motivated by the limitations of existing NOs and inspired by multipath neural net architectural designs (Chi et al., 2020; Chen et al., 2021; Pan et al., 2022; Li et al., 2024; Liu-Schiaffini et al., 2024), we propose LOGLO-FNO with the following key contributions: (i) local spectral convolutions, (ii) a high-frequency propagation component, and (iii) a novel frequency-aware loss term.

## 2 Background on Neural Operators (NO)

Neural Operators learn an approximation of operators between infinite-dimensional Banach spaces of functions. A neural operator is a function $\mathcal{G}: \mathcal{A} \to \mathcal{U}$ between two Banach spaces of functions $\mathcal{A}, \mathcal{U}$ defined on bounded domains.

**Fourier Neural Operator (FNO).** FNO (Li et al., 2021; Kossaifi et al., 2024b), which is one of the effective instantiations of a NO, learns the operator $\mathcal{G}$ using the Discrete Fourier Transform (DFT). In compact terms, $u = \mathcal{G}(a) = (Q \circ \mathcal{L}^L \circ \dots \circ \mathcal{L}^1 \circ \mathcal{P})(a)$, with the lifting and projection layers $\mathcal{P}, Q$ and the function composition $\circ$. A Fourier layer $\mathcal{L}$ can mathematically be described as the mapping in Eqn. 1, with $\mathbf{Z} \in \mathbb{R}^{\dots \times N_c \times N_x \times N_y}$ being the output of the lifting operation for the first Fourier layer and the previous Fourier layer outputs thereafter,

$$ \mathcal{L}(\mathbf{Z}) := \sigma(\text{ChannelMLP}^1(\mathbf{Y}_{\text{global}}) + \text{SoftGating}^1(\mathbf{Z})), \quad \text{where } \mathbf{Y}_{\text{global}} := \sigma(\mathcal{K}_g(\mathbf{Z}) + \text{Conv1D}^1(\mathbf{Z})) \quad (1) $$

where $\mathcal{K}_g(\cdot)$ is the global kernel integral operator realizing DFT with Fast Fourier Transforms (FFT) for uniformly discretized data, $\sigma$ is the GELU non-linearity* acting point-wise, SoftGating operation applies a learnable, feature-wise affine or linear transformation to its input, Conv1D represents 1 × 1 convolution, $N_x \times N_y$ is the spatial resolution of the 2D spatial data, and $N_c$ is the width or the number of hidden channels.

FNO and its variants, barring incremental FNO (George et al., 2024), retain only a fixed, albeit tunable, number of frequency modes corresponding to low frequencies and truncate the high-frequency ones (Kovachki et al., 2023; Helwig et al., 2023; Brandstetter et al., 2023; Gupta & Brandstetter, 2023). This modeling choice, although beneficial in managing the model parameters, results in suboptimal reconstruction quality of predictions since it pushes the model to ignore high frequencies in the data. This is inconsequential when the data in question contains mainly low-frequency structures. However, high-fidelity reconstruction is paramount for tasks such as turbulence modeling (e.g., LES) and resolving multiple scales in multiphysics simulations. For instance, Hassan et al. (2023) benchmark FNO and its variants on the BubbleML dataset, a multiphase and multiphysics simulation of boiling scenarios. In such cases, the temperature profile can exhibit sharp jumps, resulting in discontinuities along the bubble interfaces, a typical manifestation of high frequencies. Their study has observed that FNO variants face significant difficulties in this task.

<footnote>*skipped for the last Fourier layer in the network.</footnote>
## 3 Method

In this section, we describe our main contributions. The LOGLO-FNO model aims to improve the class of Fourier Neural Operators (Li et al., 2021; Kossaifi et al., 2024a; Tran et al., 2023) in boosting their capacities to effectively learn local patterns and non-dominant frequencies. Such a model supports high-fidelity outputs and achieves improved stability over long rollouts (McCabe et al., 2023). Towards this end, LOGLO-FNO introduces two key modules as architectural enhancements, namely (i) a local spectral convolution module and (ii) a high-frequency propagation module. Moreover, we propose a frequency-aware spectral space loss function based on radially binned spectral errors (see Appendix D.2 for the full algorithm and pseudocode), using which we conduct our experiments. In addition, we explore an additional loss, namely, spectral patch high-frequency emphasized residual energy (see Appendix D.1), in line with the idea of patching the spatial domain to model local features. Furthermore, we employ attention-based fusion strategies in LOGLO-FNO to seamlessly integrate multi-scale features, and assess the effect of such a feature (Appendix G.3).

**LOGLO-FNO.** Our proposed LOGLO-FNO model modifies Eqn. 1 to Eqn. 2: the LOGLO Fourier layer takes three input tensors. (i) $\mathbf{Z} \in \mathbb{R}^{\cdots \times N_c \times N_x \times N_y}$, (ii) $\hat{\mathbf{Z}} \in \mathbb{R}^{\cdots \times N_c \times \hat{N}_x \times \hat{N}_y}$ being the outputs of the *lifting* operation $\mathcal{P}$ for the first global and local Fourier layers, respectively, and the previous LOGLO Fourier layer outputs thereafter, and (iii) $\mathbf{Z}' \in \mathbb{R}^{\cdots \times N_c \times N_x \times N_y}$ is the *lifted* features of the extracted high frequency structures $\mathbf{X}_H$ (cf. Eqn. 4) for the first LOGLO Fourier layer and the previous channel MLP outputs thereafter.

$$ 
\begin{aligned}
\mathbf{Y}_{\text{global}} &:= \sigma(\mathcal{K}_g(\mathbf{Z}) + \text{Conv1D}^1(\mathbf{Z})), & \mathbf{Y}_{\text{local}} &:= \sigma(\mathcal{K}_l(\hat{\mathbf{Z}}) + \text{Conv1D}^1(\hat{\mathbf{Z}})), \\
\mathcal{L}(\mathbf{Z}, \hat{\mathbf{Z}}, \mathbf{Z}') &:= \sigma(\text{ChannelMLP}^1(\mathbf{Y}_{\text{global}}) + \text{SoftGating}^1(\mathbf{Z}) + \\
& \qquad \text{ChannelMLP}^1(\mathbf{Y}_{\text{local}}) + \text{SoftGating}^1(\hat{\mathbf{Z}}) + \text{ChannelMLP}^1(\mathbf{Z}'))
\end{aligned}
\quad (2) $$

where the suffixes $g$ and $l$ denote global and local Fourier kernels, respectively, and $\hat{N}_x \times \hat{N}_y$ the spatial resolution of a 2D patch. The notations of the input tensors naturally extend to 3D spatial data.

The LOGLO-FNO architecture, depicted in Figure 1, maintains all the essential properties of NOs, such as discretization invariance. In addition to the main *global* branch, it consists of two auxiliary branches: (i) one parallel *local* branch that retains all Fourier modes and (ii) another parallel high-frequency feature propagation branch.

### 3.1 Architectural Improvements for FNO

**Local Spectral Convolution Branch.** We propose employing auxiliary local branches to the global Fourier layer to enable the model to learn rich local features by performing convolutions restricted to local regions (e.g., patches). Unlike the global branch, the local one operates on patches of the input signal. First, we create non-overlapping patches similar to the input of prototypical vision transformers (Dosovitskiy et al., 2021). These are then fed to the *lifting* layer $\mathcal{P}$ before being passed on to the local spectral convolution layer, which retains all Fourier modes. Akin to a CNN, the local branch operating on patches performs local convolution since the spatial domain is now limited to the patch size, capturing local and small-scale patterns, whereas the main branch performs global convolution, modeling high-level phenomena (e.g., overall fluid flow direction from left to right or bottom to top).

We consider a partition of the domain $D$ into non-overlapping hypercubes $P_1, \dots, P_M$ called patches, such that $\bigcup_{m=1}^M P_m = D$. Then, the local kernel integral operator $\mathcal{K}_l(\cdot)$ with a learnable $\phi$ can be defined as

$$ (\mathcal{K}_l(\phi)\hat{Z}_{\hat{\mathcal{L}}^n})(x) = \int_{P_{m(x)}} \kappa_{\phi}(x-y)\hat{Z}_{\hat{\mathcal{L}}^n}(y) dy, \quad (3) $$
<figure><img src="image_2.png" /><figcaption>Figure 1: The overall architecture of our proposed LOGLO-FNO model. **X** is the discretization of $a(x)$, $\hat{X}$ is the patched version, and $X_H$ the extracted high frequencies (cf. Eqn. 4). The full network has **L** repetitions of identical LOGLO Fourier layers, and the final activation function is applied on all but the last LOGLO Fourier layer. The features from the three branches and the skip connections are fused by a simple summation.</figcaption></figure>

where $P_{m(x)}$ is the $m^{th}$ patch containing $x$. Unlike the global integral operator $K_q(\cdot)$ in Eqn. 1, the kernel integral operator $K_l(\cdot)$ in Eqn. 3 is restricted to a local region $P_m$ with a customizable spatial extent. The suffix $\hat{\mathcal{L}}^n$ denotes the $n^{th}$ instance of the local Fourier layer and $m$ indexes a specific patch out of the total $M$ patches. Therefore, $\hat{Z}_{\mathcal{L}^n}$ stands for the features fed as input to the $n^{th}$ local Fourier layer.

**High-Frequency Propagation (HFP) Branch.** In order to provide a stronger inductive bias of high-frequency features, we propose an HFP module as the third branch, placed in parallel, that encourages the accurate reconstruction of high-frequencies (Liu et al., 2020) that are otherwise subdued in the Fourier layers of the global branch due to explicit truncation of high-frequency modes. We employ one level of downsampling to the input signal using average pooling and an upsampling block using interpolation. The blurriness of the downsampled signal (e.g., a field variable) is directly proportional to the degree of spatial distortion, which, in the case of average pooling, is controlled by the kernel size and stride. The upsampling operation undoes this effect to reconstruct the original signal resolution. The extracted high frequencies $X_H$ (cf. Figure 1) are *lifted* using $\mathcal{P}$ with shared parameters for a tight coupling of the high-pass filtered features with that of the full signal features, propagated using a channel MLP, and added to the output features of the local and global Fourier layers. Overall, the HFP block serves as a high-pass filter and can be written compactly as
$$ \mathbf{X}_H = \mathbf{X} - \text{Interpolate}(\text{AvgPool}(\mathbf{X}; \text{kernel\_size}, \text{stride})); \text{size} = (\dots, N_c, N_x, N_y, [N_z]). \quad (4) $$

## 3.2 Frequency-aware Loss based on Radially Binned Spectral Energy Errors

Since our aim is to direct the optimization process to enable the model to faithfully reconstruct non-dominant frequencies in the predictions, we propose a frequency-sensitive loss and penalize the band-classified (mid- and high-frequency) spectral errors as an additional weighted term in the loss. Let $\tilde{\mathbf{Y}}$ and $\mathbf{Y}$ be the model prediction and target, respectively. The pointwise error in physical space is $\Delta u = \tilde{\mathbf{Y}} - \mathbf{Y}$, and the spectral error $\Delta \hat{u} = \mathcal{F}(\Delta u)$ is obtained by applying the FFT on the spatial dimensions. Then, we define the radially binned, temporal and channel-wise spectral error on 2D spatial data as,

$$ C_{\text{freq}}^{t,c}(r) = \frac{L_x}{N_x} \cdot \frac{L_y}{N_y} \sqrt{\frac{1}{N_b} \sum_{b=1}^{N_b} \left( \sum_{\substack{(k_x, k_y) \text{ s.t.} \\ \mathcal{R}(k_x, k_y)=r}} |\Delta \hat{u}_{b,c,t}(k_x, k_y)|^2 \right)}, \quad (5) $$

where the inner summation aggregates the energy of spectral errors within each radial bin $r$, $L_x$ and $L_y$ are the spatial extents of the domain, whereas $N_x$ and $N_y$ are the corresponding total number of observation points along those spatial axes. Subsequently, the binned errors can be band-classified as low, mid, and high frequencies by considering suitable radial cutoff values (e.g., low=[1,4], mid=[5,12], and high=[13, $\mathcal{M}$]; see Appendix K.6) and summed or averaged over the physical variables and temporal dimensions.

<figure><img src="image_3.png" /><figcaption>Figure 2: Representative illustration of radially binned spectral error. Left: the spectral energy of error magnitude with overlaid radial bins (blue – low frequency, green – mid frequency cutoff, the rest – high frequency band); Right: the energy of the radially binned spectral error across radial distances as a line plot.</figcaption></figure>

A detailed step-by-step procedure (Algorithm 1) and a PyTorch-style pseudocode to compute the frequency loss for 2D spatial data are provided in Appendix D.2, and the visualizations are placed in Appendix D.3.

## 3.3 Fourier Layer Parameter Budget for 2D Spatial Data

The majority of the parameter mass in FNO is concentrated on the Fourier layers (specifically, the spectral convolution modules) and their cardinality. It is dependent on three factors: (i) the spatial dimensionality $s$
of the problem, (ii) the number of selected Fourier modes $K$, and (iii) the channel$^†$ dimension $d_c$. Denote the number of Fourier layers as $\mathcal{L}$. Then, for 2D spatial data ($s = 2$), the parameter complexity of the global spectral convolutional layer is $O((d_c \cdot d_c) \cdot K^s \cdot \mathcal{L})$. Note the quadratic growth in parameters as the number of chosen modes $K$ increases. This results in a higher number of trainable parameters for inputs of reasonably high resolutions (e.g., 512×512, or above). Considering this inefficiency and the lack of local convolutions, we consider an approach to distribute the apriori allotted parameter budget across a series of branches, each processing the inputs at different resolutions, mimicking the multi-scale modeling paradigm (Chen et al., 2021; Rahman et al., 2023; Gupta & Brandstetter, 2023). However, we differ from the prevalent approaches in that we do not downsample the incoming spatial resolution in the local branch but decompose it into smaller domains (i.e., patches) to process each independently. Therefore, the parameter complexity of the local Fourier layers performing local spectral convolution on the patches of 2D spatial data would be $O((d_c \cdot d_c) \cdot (n_x \cdot (p_{size}/2 + 1)) \cdot \mathcal{L})$, where $p_{size}$ denotes the patch size, +1 is for the Nyquist frequency, and $n_x$ is the x-axis resolution of the patches. Note that we retain all Fourier modes in the local branch.

## 3.4 Fourier Layer Parameter Budget for 3D Spatial Data

Extending the analysis of the dependence of trainable parameters on the spectral filter size $K$ for 2D spatial data in §3.3 to 3D spatial data, we obtain the parameter complexity of the global Fourier layer performing global spectral convolutions as $O((d_c \cdot d_c) \cdot K^3 \cdot \mathcal{L})$, while noting the cubic growth in trainable parameters as we increase the number of chosen Fourier modes $K$ for more accurate reconstruction of predictions and $d_c$ indicating the hidden channel dimension. In contrast, the parameter complexity of the local Fourier layers performing local spectral convolution on the patches of 3D spatial data would be $O((d_c \cdot d_c) \cdot (n_x \cdot n_y \cdot (p_{size}/2 + 1)) \cdot \mathcal{L})$, where $p_{size}$ denotes the patch size, +1 for the Nyquist frequency, and $n_x$ and $n_y$ are the x- and y-axes resolutions of the patches. Note that, as in the case of 2D spatial data, we always retain all Fourier modes in the local branch in all layers.

## 3.5 Distributing Parameters across Local and Global Branches

Following the observations made in §3.3, we consider a scenario where the goal is to train a neural operator given a parameter budget. A baseline FNO can be trained with this budget cost to reach a certain accuracy. In contrast, we argue that we can reach the same accuracy using fewer parameters with LOGLO-FNO by making the allotted parameters less concentrated on the global branch (thereby controlling the quadratic parameter growth in terms of selected modes) and distributing them to the local branch, which has a more subdued parameter growth (for modes) depending on the patch size. If, on the other hand, we use the entire parameter budget, better accuracy than the baseline FNO model can be achieved (see Figures 3, 14, 15, 16, and 20).

# 4 Experiments and Results

In this section, we explain the different training setups we use and present the quantitative results of the baselines and LOGLO-FNO on the six challenging time-dependent PDE problems, i.e., five 2D PDEs (Kolmogorov Flow, CNS Turb, Wave-Gauss, Compressible Euler four-quadrant Riemann problem, and Diffusion-Reaction) and a 3D PDE (Turbulent Radiative Mixing Layer from the Well benchmark), introduced and described in Appendix B.

**Baseline Models.** We consider the following neural operators as relevant and strong baselines in our work. The FNO-related baselines include base FNO (Li et al., 2021; Kossaifi et al., 2024a), U-FNO (Wen et al., 2022) that is well-suited for multiscale modeling, F-FNO (Tran et al., 2023) that factorizes the Fourier transform over the spatial dimensions, and variants of NO-LIDK (Liu-Schiaffini et al., 2024) that achieve local convolutions. Moreover, we include a modern version of U-Net from PDEarena (Gupta & Brandstetter, 2023), LSM (Wu et al., 2023) that applies spectral methods in learned latent space, and a physics-attention based transformer, namely, Transolver (Wu et al., 2024). Further specific details of baselines are in Appendix C.

<footnote>†a.k.a. width or hidden channels in the Neural Operator framework: github.com/neuraloperator/neuraloperator</footnote>
**Training Objective, Procedure, and Evaluation Metrics.** We train the baselines using the standard MSE loss ($C_{MSE}$), whereas our frequency loss term is added on top for the LOGLO-FNO models. Consequently, the 1-step training loss for N trajectories, each comprising T timesteps, is,

$$ \theta^* = \arg \min_{\theta} \sum_{n=1}^{N} \sum_{t=1}^{T-1} \mathcal{C}(\mathcal{N}_{\theta}(u^t), u^{t+1}), \quad \mathcal{C} = C_{\text{MSE}} + \lambda \cdot C_{\text{freq}}, \quad 0 \le \lambda \le 1 \quad (6) $$

where $(u^t, u^{t+1})$ are the input-output pairs constructed from trajectories and $\mathcal{C}$ is the weighted sum of cost functions, MSE ($C_{MSE}$) and our band-classified frequency loss ($C_{freq}$). Specifically, we minimize only the spectral errors corresponding to the mid-and high-frequency bands in our experiments. We use the Adam optimizer and halve the learning rate every 33 epochs in the case of Kolmogorov Flow and 100 epochs for Diffusion-Reaction 2D for a fair comparison with NO-LIDK (Liu-Schiaffini et al., 2024). However, the learning rate is halved every 10 epochs on the Turbulent Radiative Layer 3D dataset due to reduced training epochs. The full set of hyperparameters is listed in Appendix O.8. We evaluate metrics from PDEBench (Takamoto et al., 2022), which contain spectral, physics- and data-view-based error measures (fRMSE, cRMSE, nRMSE, MaxError). Additionally, we include metrics that measure the energy spectra deviations, viz. MELR and WLR (Wan et al., 2023). More elaborate details on the evaluation metrics can be found in Appendix G.1.

### 4.0.1 1-step Training and Evaluation

**2D Kolmogorov Flow.** We train using 1-step loss, a.k.a. teacher-forcing: $u(t-\Delta t, \cdot) \rightarrow u(t, \cdot)$. This training scheme mimics the functioning of classical numerical solvers and is commonly employed (Gupta & Brandstetter, 2023).

Gaussian noise is added in order to account for the distribution mismatch between this type of teacher-forcing-based training and autoregressive rollout for inference (Pfaff et al., 2021; Stachenfeld et al., 2022). In a similar spirit, we inject *adaptive* Gaussian noise, which is dependent on the high-frequency structures extracted by the HFP module (see Appendix E for full details). Further, we found it essential to employ gradient clipping for stabilized training when modeling high frequencies. The LOGLO-FNO models have been trained with a single local branch (patch size 8 × 8 or 16 × 16 for 2D spatial data) in addition to a single global branch operating on the full spatial resolution.

**1-step Evaluation.** Once trained, we evaluate the models for 1-step errors. A summary of results comparing proposed LOGLO-FNO with a range of strong baselines, including the FNO-based current state-of-the-art NO-LIDK (Liu-Schiaffini et al., 2024), is in Table 1.

<table><caption>Table 1: 1-step and 5-step AR evaluation of LOGLO-FNO compared with SOTA baselines on the test set of 2D Kolmogorov Flow (Li et al., 2022b). REL. % DIFF indicates improvement (-) or degradation (+) with respect to FNO. LOGLO-FNO uses 40 and (16, 9) modes in the global and local branches, respectively, whereas the width is set as 65. NO-LIDK\* denotes using only localized integral kernel, NO-LIDK\* means only differential kernel, and NO-LIDK\*\dagger means employing both. Transolver\* indicates a longer training time of the model for 500 epochs due to convergence issues at shorter training epochs of 136.</caption><thead><tr><th>Model</th><th>nRMSE (↓)</th><th>fRMSE(L)</th><th>fRMSE(M)</th><th>fRMSE(H) (↓)</th></tr></thead><tbody><tr><td colspan="5" style="text-align: center;">1-step Evaluation</td></tr><tr><td>U-Net</td><td>1.3 &middot; 10<sup>-1</sup></td><td>2.24 &middot; 10<sup>-2</sup></td><td>3.57 &middot; 10<sup>-2</sup></td><td>4.39 &middot; 10<sup>-2</sup></td></tr><tr><td>Transolver<sup>*</sup></td><td>1.39 &middot; 10<sup>-1</sup></td><td>2.74 &middot; 10<sup>-2</sup></td><td>4.48 &middot; 10<sup>-2</sup></td><td>4.65 &middot; 10<sup>-2</sup></td></tr><tr><td>FNO</td><td>1.47 &middot; 10<sup>-1</sup></td><td>1.36 &middot; 10<sup>-2</sup></td><td>2.05 &middot; 10<sup>-2</sup></td><td>4.7 &middot; 10<sup>-2</sup></td></tr><tr><td>F-FNO</td><td>1.37 &middot; 10<sup>-1</sup></td><td>1.49 &middot; 10<sup>-2</sup></td><td>2.15 &middot; 10<sup>-2</sup></td><td>4.36 &middot; 10<sup>-2</sup></td></tr><tr><td>LSM</td><td>1.36 &middot; 10<sup>-1</sup></td><td>2.59 &middot; 10<sup>-2</sup></td><td>4.42 &middot; 10<sup>-2</sup></td><td>4.64 &middot; 10<sup>-2</sup></td></tr><tr><td>U-FNO</td><td>1.12 &middot; 10<sup>-1</sup></td><td>1.27 &middot; 10<sup>-2</sup></td><td>2.22 &middot; 10<sup>-2</sup></td><td>3.71 &middot; 10<sup>-2</sup></td></tr><tr><td>NO-LIDK<sup>*</sup></td><td>1.33 &middot; 10<sup>-1</sup></td><td>1.45 &middot; 10<sup>-2</sup></td><td>2.69 &middot; 10<sup>-2</sup></td><td>4.55 &middot; 10<sup>-2</sup></td></tr><tr><td>NO-LIDK<sup>o</sup></td><td>1.11 &middot; 10<sup>-1</sup></td><td>1.65 &middot; 10<sup>-2</sup></td><td>2.29 &middot; 10<sup>-2</sup></td><td>3.91 &middot; 10<sup>-2</sup></td></tr><tr><td>NO-LIDK<sup>†</sup></td><td>1.07 &middot; 10<sup>-1</sup></td><td>1.44 &middot; 10<sup>-2</sup></td><td>2.46 &middot; 10<sup>-2</sup></td><td>3.82 &middot; 10<sup>-2</sup></td></tr><tr><td>LOGLO-FNO</td><td><b>1.07 &middot; 10<sup>-1</sup></b></td><td><b>1.21 &middot; 10<sup>-2</sup></b></td><td><b>1.81 &middot; 10<sup>-2</sup></b></td><td><b>3.54 &middot; 10<sup>-2</sup></b></td></tr><tr><td>REL. % DIFF</td><td>-27.21 %</td><td>-11.22 %</td><td>-11.88 %</td><td>-24.64 %</td></tr><tr><td colspan="5" style="text-align: center;">5-step Autoregressive Evaluation</td></tr><tr><td>U-Net</td><td>2.65 &middot; 10<sup>-1</sup></td><td>6.53 &middot; 10<sup>-2</sup></td><td>1.04 &middot; 10<sup>-1</sup></td><td>9.38 &middot; 10<sup>-2</sup></td></tr><tr><td>Transolver<sup>*</sup></td><td>3.36 &middot; 10<sup>-1</sup></td><td>7.44 &middot; 10<sup>-2</sup></td><td>1.51 &middot; 10<sup>-1</sup></td><td>1.18 &middot; 10<sup>-1</sup></td></tr><tr><td>FNO</td><td>2.35 &middot; 10<sup>-1</sup></td><td>3.37 &middot; 10<sup>-2</sup></td><td>5.80 &middot; 10<sup>-2</sup></td><td>8.37 &middot; 10<sup>-2</sup></td></tr><tr><td>F-FNO</td><td>2.28 &middot; 10<sup>-1</sup></td><td>3.60 &middot; 10<sup>-2</sup></td><td>5.52 &middot; 10<sup>-2</sup></td><td>8.12 &middot; 10<sup>-2</sup></td></tr><tr><td>LSM</td><td>3.18 &middot; 10<sup>-1</sup></td><td>7.6 &middot; 10<sup>-2</sup></td><td>1.44 &middot; 10<sup>-1</sup></td><td>1.12 &middot; 10<sup>-1</sup></td></tr><tr><td>U-FNO</td><td>2.03 &middot; 10<sup>-1</sup></td><td><b>2.69 &middot; 10<sup>-2</sup></b></td><td>5.33 &middot; 10<sup>-2</sup></td><td>7.35 &middot; 10<sup>-2</sup></td></tr><tr><td>NO-LIDK<sup>*</sup></td><td>2.39 &middot; 10<sup>-1</sup></td><td>3.14 &middot; 10<sup>-2</sup></td><td>6.86 &middot; 10<sup>-2</sup></td><td>8.91 &middot; 10<sup>-2</sup></td></tr><tr><td>NO-LIDK<sup>o</sup></td><td>2.04 &middot; 10<sup>-1</sup></td><td>3.71 &middot; 10<sup>-2</sup></td><td>5.96 &middot; 10<sup>-2</sup></td><td>7.6 &middot; 10<sup>-2</sup></td></tr><tr><td>NO-LIDK<sup>†</sup></td><td>2.05 &middot; 10<sup>-1</sup></td><td>3.31 &middot; 10<sup>-2</sup></td><td>5.94 &middot; 10<sup>-2</sup></td><td>7.74 &middot; 10<sup>-2</sup></td></tr><tr><td>LOGLO-FNO</td><td><b>1.92 &middot; 10<sup>-1</sup></b></td><td><b>2.80 &middot; 10<sup>-2</sup></b></td><td><b>4.55 &middot; 10<sup>-2</sup></b></td><td><b>6.93 &middot; 10<sup>-2</sup></b></td></tr><tr><td>REL. % DIFF</td><td>-18.39 %</td><td>-16.86 %</td><td>-21.62 %</td><td>-17.26 %</td></tr></tbody></table>

Since our method also performs convolution using local spectral kernels, we compare our results with their local integral kernel (NO-LIDK\*) scores, whereas the other two variants (NO-LIDK\*, NO-LIDK\*)\dagger are provided for
<figure><img src="image_4.png" /><figcaption>Figure 3: Comparison of FNO vs. LOGLO-FNO showing 1-step fRMSE (↓) on the test set of Kolmogorov Flow dataset ($Re = 5k$) (Li et al., 2022b) for a varying number of modes in the global branch and the full set of modes (i.e., (16, 9) for patch size (16 × 16)) in the local branch.</figcaption></figure>

the sake of completeness. We observe that LOGLO-FNO outperforms the baselines on all metrics. Since we set out to model high-frequencies, we note that the mid- and high-frequency errors are close to 12% and 25% less compared to base FNO, indicating better preservation of high-frequency details, whereas MELR is down by over 75% (see Table 8). The results of a study analyzing the influence of an increasing number of global branch Fourier modes and full count of local branch Fourier modes (i.e., (16, 9) for patch size (16 × 16)) for LOGLO-FNO on the frequency and energy spectra errors are visualized in Figure 3 and Figure 16, respectively. More plots showing other error metrics are placed in Appendix G.4.

<figure><img src="image_5.png" /><figcaption>Figure 4: Comparison of radially binned spectral errors of baselines and LOGLO-FNO on the test set of Kolmogorov Flow ($Re = 5k$) (Li et al., 2022b).</figcaption></figure>

**Autoregressive Evaluation.** In this setup, we evaluate the 1-step trained models on autoregressive rollouts for varying numbers of timesteps and compute the errors. The 5-step autoregressive rollout errors are shown in Table 1. We observe that LOGLO-FNO outperforms other models on seven out of the ten metrics and is superior to FNO on all metrics, achieving a noticeable reduction in mid- and high-frequency errors.
These 5-step and extended rollout results (see Figure 5) indicate that LOGLO-FNO is more robust to the autoregressive error accumulation problem than base FNO. Further metrics are visualized in Figures 17, 18, and 19 and the results with the full set of evaluation metrics are provided in Table 8 in Appendix G.2.1.

<figure><img src="image_6.png" /><figcaption>Figure 5: Comparison of FNO vs. LOGLO-FNO for autoregressive rollout showing fRMSE (↓) error growth over variable number of modes (K) in the global branch and varying timesteps in the trajectories on the test set of Kolmogorov Flow (Re = 5k) (Li et al., 2022b). The local branch uses a patch size of 16 × 16.</figcaption></figure>

**CNS Turb 2D.** We extend our experiments to include compressible Navier-Stokes equations (see Eqn. 8 in Appendix B.2 for details on the PDE), specifically with turbulent initial conditions and sonic regime (Mach=1.0).

The dataset consists of 1k trajectories, of which we use 900 for training and 100 for testing, following PDEBench (Takamoto et al., 2022) split. Input-output data pairs are constructed by slicing successive frames across the timesteps, yielding 18k for training and 2k for testing. Each snapshot has two scalar fields (i.e., density and pressure) and one 2D vector field (i.e., velocity: velocity-x, velocity-y).

We replicate the 1-step training strategy of Kolmogorov Flow 2D problem, but instead use the AdamW optimizer for all models. We observe from the 1-step and 5-step AR errors listed in Table 2, that LOGLO-FNO yields consistent improvements across spatial and spectral metrics and outperforms all baselines in both evaluation settings.

The results on the full set of evaluation metrics are provided in Table 9 in Appendix G.2.1, and the hyperparameters are placed in Appendix O.

<table><caption>Table 2: **Top:** 1-step and 5-step AR evaluation of LOGLO-FNO compared with SOTA baselines on the test set of CNS Turb 2D. LOGLO-FNO uses 40 and (16, 9) modes in the global and local branches, respectively, whereas the width is set as 65. NO-LIDK* denotes using only localized integral kernel, NO-LIDK◊ means only differential kernel, and NO-LIDK† means employing both. REL. % DIFF indicates improvement (-) or degradation (+) with respect to FNO.</caption><thead><tr><th>Model</th><th>nRMSE (↓)</th><th>fRMSE(L)</th><th>fRMSE(M)</th><th>fRMSE(H) (↓)</th></tr></thead><tbody><tr><td colspan="5" style="text-align: center;">CNS Turb 2D: 1-step Evaluation</td></tr><tr><td>U-Net</td><td>4.9 &middot; 10<sup>-2</sup></td><td>2.47 &middot; 10<sup>-3</sup></td><td>2.9 &middot; 10<sup>-3</sup></td><td>1.48 &middot; 10<sup>-3</sup></td></tr><tr><td>Transolver*</td><td>4.22 &middot; 10<sup>-1</sup></td><td>4.77 &middot; 10<sup>-2</sup></td><td>4.11 &middot; 10<sup>-2</sup></td><td>4.47 &middot; 10<sup>-3</sup></td></tr><tr><td>FNO</td><td>4.82 &middot; 10<sup>-2</sup></td><td>1.28 &middot; 10<sup>-3</sup></td><td>1.78 &middot; 10<sup>-3</sup></td><td>1.57 &middot; 10<sup>-3</sup></td></tr><tr><td>LSM</td><td>8.08 &middot; 10<sup>-2</sup></td><td>6.17 &middot; 10<sup>-3</sup></td><td>6.55 &middot; 10<sup>-3</sup></td><td>2.01 &middot; 10<sup>-3</sup></td></tr><tr><td>U-FNO</td><td>4.49 &middot; 10<sup>-2</sup></td><td>2.24 &middot; 10<sup>-3</sup></td><td>2.01 &middot; 10<sup>-3</sup></td><td>1.43 &middot; 10<sup>-3</sup></td></tr><tr><td>NO-LIDK*</td><td>6.4 &middot; 10<sup>-2</sup></td><td>1.5 &middot; 10<sup>-3</sup></td><td>2.14 &middot; 10<sup>-3</sup></td><td>1.98 &middot; 10<sup>-3</sup></td></tr><tr><td>NO-LIDK◊</td><td>5.54 &middot; 10<sup>-2</sup></td><td>1.35 &middot; 10<sup>-3</sup></td><td>1.88 &middot; 10<sup>-3</sup></td><td>1.76 &middot; 10<sup>-3</sup></td></tr><tr><td>NO-LIDK†</td><td>5.57 &middot; 10<sup>-2</sup></td><td>1.37 &middot; 10<sup>-3</sup></td><td>1.89 &middot; 10<sup>-3</sup></td><td>1.77 &middot; 10<sup>-3</sup></td></tr><tr><td>LOGLO-FNO</td><td>4.24 &middot; 10<sup>-2</sup></td><td>9.01 &middot; 10<sup>-4</sup></td><td>1.1 &middot; 10<sup>-3</sup></td><td>1.37 &middot; 10<sup>-3</sup></td></tr><tr><td>REL. % DIFF</td><td>-11.91 %</td><td>-29.57%</td><td>-38.1 %</td><td>-12.38 %</td></tr></tbody></table>

<table><thead><tr><th colspan="5" style="text-align: center;">CNS Turb 2D: 5-step Autoregressive Evaluation</th></tr></thead><tbody><tr><td>U-Net</td><td>1.44 &middot; 10<sup>-1</sup></td><td>1.42 &middot; 10<sup>-2</sup></td><td>1.3 &middot; 10<sup>-2</sup></td><td>3.08 &middot; 10<sup>-3</sup></td></tr><tr><td>Transolver*</td><td>6.69 &middot; 10<sup>-1</sup></td><td>9.21 &middot; 10<sup>-2</sup></td><td>6.0 &middot; 10<sup>-2</sup></td><td>4.7 &middot; 10<sup>-3</sup></td></tr><tr><td>FNO</td><td>7.78 &middot; 10<sup>-2</sup></td><td>3.76 &middot; 10<sup>-3</sup></td><td>5.37 &middot; 10<sup>-3</sup></td><td>2.4 &middot; 10<sup>-3</sup></td></tr><tr><td>LSM</td><td>2.67 &middot; 10<sup>-1</sup></td><td>3.4 &middot; 10<sup>-2</sup></td><td>2.5 &middot; 10<sup>-2</sup></td><td>4.16 &middot; 10<sup>-3</sup></td></tr><tr><td>U-FNO</td><td>8.42 &middot; 10<sup>-2</sup></td><td>5.38 &middot; 10<sup>-3</sup></td><td>5.9 &middot; 10<sup>-3</sup></td><td>2.54 &middot; 10<sup>-3</sup></td></tr><tr><td>NO-LIDK*</td><td>8.55 &middot; 10<sup>-2</sup></td><td>4.14 &middot; 10<sup>-3</sup></td><td>5.92 &middot; 10<sup>-3</sup></td><td>2.62 &middot; 10<sup>-3</sup></td></tr><tr><td>NO-LIDK◊</td><td>7.88 &middot; 10<sup>-2</sup></td><td>3.82 &middot; 10<sup>-3</sup></td><td>5.35 &middot; 10<sup>-3</sup></td><td>2.44 &middot; 10<sup>-3</sup></td></tr><tr><td>NO-LIDK†</td><td>7.91 &middot; 10<sup>-2</sup></td><td>3.82 &middot; 10<sup>-3</sup></td><td>5.38 &middot; 10<sup>-3</sup></td><td>2.47 &middot; 10<sup>-3</sup></td></tr><tr><td>LOGLO-FNO</td><td>6.67 &middot; 10<sup>-2</sup></td><td>2.76 &middot; 10<sup>-3</sup></td><td>3.97 &middot; 10<sup>-3</sup></td><td>2.12 &middot; 10<sup>-3</sup></td></tr><tr><td>REL. % DIFF</td><td>-14.31 %</td><td>-26.7 %</td><td>-26.09 %</td><td>-11.42 %</td></tr></tbody></table>
**Wave-Gauss 2D.** As an additional problem exhibiting a different physical phenomenon of wave propagation, we include the 2D wave equation, specifically Wave-Gauss, from Poseidon (Herde et al., 2024). We refer the readers to Eqn. 9 in Appendix B.3 for details on the PDE and the initial and boundary conditions. The spatially dependent propagation speed is generated as a sum of Gaussians.

As with the previous problems in this section, we employ 1-step training strategy with the objective in Eqn. 6 and evaluate on 1-step and 5-step rollouts. The training data contains about 150k input-output pairs, 900 for validation, and 3.6k is reserved for testing.

Similar to the input data formatting setup of Herde et al. (2024), we concatenate the static propagation field *c* to the time-dependent displacement field *u* in the input and let the model learn to perform the identity mapping (i.e., copying) of the propagation field in the output.

As we observe from Table 3, although LOGLO-FNO achieves the lowest mid

and high frequency errors, the LSM baseline model outperforms all other models on the spatial and low frequency error metrics, and this trend is maintained on the complete trajectory rollouts (see Table 10).

The results on the full set of evaluation metrics and extended full trajectory rollout results are provided in Table 10 in the Appendix G.2.1, and the hyperparameters for all the models are in Appendix O.

**Compressible Euler four-quadrant Riemann problem 2D.** A challenging problem for neural operators is modeling shock waves and contact discontinuities. Towards this end, we conduct experiments on the Compressible Euler four-quadrant Riemann problem 2D from Poseidon (Herde et al., 2024). This PDE has four variables, viz. density, horizontal and vertical velocity fields, and pressure. See Appendix B.4 for further details. As with other problems in this section, we optimize using 1-step loss (cf. Eqn. 6), where the models are trained to predict all four field variables for a future timestep. We then evaluate the performance of LOGLO-FNO and compare against Base FNO and NO-LIDK variants. The quantitative results (i.e., 1-step and 5-step rollouts) are shown in Table 4, whereas the qualitative visualization of predictions along with the errors for the density, velocity-x, and pressure fields are shown in Figures 6, 7, and 8, respectively.

<figure><img src="image_7.png" /><figcaption>Figure 6: Qualitative results comparing the 1-step density field predictions of Base FNO and LOGLO-FNO on a random sample from the test set of compressible Euler four-quadrant Riemann problem 2D.</figcaption></figure>

<table><caption>Table 3: 1-step and 5-step AR evaluation of LOGLO-FNO on the test set of Wave-Gauss 2D. LOGLO-FNO uses 20 and (16, 9) modes in the global and local branches, respectively, whereas the width is set to 32. NO-LIDK\* denotes using only localized integral kernel, NO-LIDK\diamond means only differential kernel, and NO-LIDK\dagger means employing both. REL. % DIFF indicates improvement (-) or degradation (+) with respect to FNO.</caption><thead><tr><th>Model</th><th>nRMSE (↓)</th><th>fRMSE(L)</th><th>fRMSE(M)</th><th>fRMSE(H) (↓)</th></tr></thead><tbody><tr><td colspan="5" style="text-align: center; font-weight: bold;">Wave-Gauss 2D: 1-step Evaluation</td></tr><tr><td>Transolver*</td><td>2.29 · 10<sup>-1</sup></td><td>3.51 · 10<sup>-2</sup></td><td>2.76 · 10<sup>-2</sup></td><td>2.2 · 10<sup>-3</sup></td></tr><tr><td>FNO</td><td>4.03 · 10<sup>-2</sup></td><td>7.5 · 10<sup>-3</sup></td><td>5.45 · 10<sup>-3</sup></td><td>6.22 · 10<sup>-4</sup></td></tr><tr><td>LSM</td><td><b>3.08</b> · 10<sup>-2</sup></td><td><b>4.91</b> · 10<sup>-3</sup></td><td>3.86 · 10<sup>-3</sup></td><td>8.54 · 10<sup>-4</sup></td></tr><tr><td>U-FNO</td><td>3.49 · 10<sup>-2</sup></td><td>6.62 · 10<sup>-3</sup></td><td>4.52 · 10<sup>-3</sup></td><td>7.24 · 10<sup>-4</sup></td></tr><tr><td>NO-LIDK*</td><td>3.93 · 10<sup>-2</sup></td><td>6.95 · 10<sup>-3</sup></td><td>5.1 · 10<sup>-3</sup></td><td>8.22 · 10<sup>-4</sup></td></tr><tr><td>NO-LIDK\diamond</td><td>3.58 · 10<sup>-2</sup></td><td>6.49 · 10<sup>-3</sup></td><td>4.78 · 10<sup>-3</sup></td><td>7.56 · 10<sup>-4</sup></td></tr><tr><td>NO-LIDK\dagger</td><td>3.5 · 10<sup>-2</sup></td><td>6.31 · 10<sup>-3</sup></td><td>4.61 · 10<sup>-3</sup></td><td>7.4 · 10<sup>-4</sup></td></tr><tr><td><b>LOGLO-FNO</b></td><td><b>3.11</b> · 10<sup>-2</sup></td><td><b>6.43</b> · 10<sup>-3</sup></td><td><b>3.84</b> · 10<sup>-3</sup></td><td><b>3.44</b> · 10<sup>-4</sup></td></tr><tr><td>REL. % DIFF</td><td>-22.79 %</td><td>-14.27%</td><td>-29.54%</td><td>-44.65%</td></tr><tr><td colspan="5" style="text-align: center; font-weight: bold;">Wave-Gauss 2D: 5-step Autoregressive Evaluation</td></tr><tr><td>Transolver*</td><td>3.62 · 10<sup>-1</sup></td><td>7.73 · 10<sup>-2</sup></td><td>3.47 · 10<sup>-2</sup></td><td>2.93 · 10<sup>-3</sup></td></tr><tr><td>FNO</td><td>6.75 · 10<sup>-2</sup></td><td>1.6 · 10<sup>-2</sup></td><td>8.35 · 10<sup>-3</sup></td><td>8.57 · 10<sup>-4</sup></td></tr><tr><td>LSM</td><td><b>4.34</b> · 10<sup>-2</sup></td><td><b>8.64</b> · 10<sup>-3</sup></td><td>5.68 · 10<sup>-3</sup></td><td>1.1 · 10<sup>-3</sup></td></tr><tr><td>U-FNO</td><td>6.26 · 10<sup>-2</sup></td><td>1.6 · 10<sup>-2</sup></td><td>7.64 · 10<sup>-3</sup></td><td>9.27 · 10<sup>-4</sup></td></tr><tr><td>NO-LIDK*</td><td>5.96 · 10<sup>-2</sup></td><td>1.34 · 10<sup>-2</sup></td><td>7.92 · 10<sup>-3</sup></td><td>1.19 · 10<sup>-3</sup></td></tr><tr><td>NO-LIDK\diamond</td><td>1.21 · 10<sup>-1</sup></td><td>2.43 · 10<sup>-2</sup></td><td>1.12 · 10<sup>-2</sup></td><td>4.07 · 10<sup>-3</sup></td></tr><tr><td>NO-LIDK\dagger</td><td>1.06 · 10<sup>-1</sup></td><td>2.1 · 10<sup>-2</sup></td><td>1.02 · 10<sup>-2</sup></td><td>3.85 · 10<sup>-3</sup></td></tr><tr><td><b>LOGLO-FNO</b></td><td><b>4.81</b> · 10<sup>-2</sup></td><td><b>1.29</b> · 10<sup>-2</sup></td><td><b>5.43</b> · 10<sup>-3</sup></td><td><b>4.57</b> · 10<sup>-4</sup></td></tr><tr><td>REL. % DIFF</td><td>-28.76%</td><td>-19.52%</td><td>-34.91%</td><td>-46.6%</td></tr></tbody></table>
<table><caption>Table 4: 1-step and 5-step autoregressive rollout evaluations of LOGLO-FNO compared against SOTA baselines on the test set of Compressible Euler four-quadrant Riemann problem 2D (Herde et al., 2024). LOGLO-FNO uses 40 and (16, 9) modes in the global and local branches, respectively, whereas the width is set to 65. NO-LIDK\* denotes using only the localized integral kernel with a radius cutoff value of 0.0078125, and NO-LIDK\circ means only the differential kernel is enabled.</caption><thead><tr><th>Model</th><th>nRMSE (↓)</th><th>fRMSE(L)</th><th>fRMSE(M)</th><th>fRMSE(H) (↓)</th></tr></thead><tbody><tr><td colspan="5" style="text-align: center; font-weight: bold;">1-step Evaluation</td></tr><tr><td>FNO</td><td>2.08 · 10<sup>-2</sup></td><td>9.42 · 10<sup>-4</sup></td><td>1.03 · 10<sup>-3</sup></td><td>1.41 · 10<sup>-3</sup></td></tr><tr><td>NO-LIDK\*</td><td>3.58 · 10<sup>-2</sup></td><td>1.5 · 10<sup>-3</sup></td><td>2.35 · 10<sup>-3</sup></td><td>2.47 · 10<sup>-3</sup></td></tr><tr><td>NO-LIDK\circ</td><td>1.66 · 10<sup>-2</sup></td><td>1.05 · 10<sup>-3</sup></td><td>8.54 · 10<sup>-4</sup></td><td>1.11 · 10<sup>-3</sup></td></tr><tr><td>LOGLO-FNO</td><td><strong>1.5 · 10<sup>-2</sup></strong></td><td><strong>5.79 · 10<sup>-4</sup></strong></td><td><strong>5.09 · 10<sup>-4</sup></strong></td><td><strong>9.67 · 10<sup>-4</sup></strong></td></tr><tr><td>REL. % DIFF</td><td>-27.9%</td><td>-38.56%</td><td>-50.71%</td><td>-31.58%</td></tr><tr><td colspan="5" style="text-align: center; font-weight: bold;">5-step Autoregressive Evaluation</td></tr><tr><td>FNO</td><td>3.43 · 10<sup>-2</sup></td><td>2.58 · 10<sup>-3</sup></td><td>2.7 · 10<sup>-3</sup></td><td>2.35 · 10<sup>-3</sup></td></tr><tr><td>NO-LIDK\*</td><td>4.02 · 10<sup>-2</sup></td><td>3.19 · 10<sup>-3</sup></td><td>3.58 · 10<sup>-3</sup></td><td>2.47 · 10<sup>-3</sup></td></tr><tr><td>NO-LIDK\circ</td><td>3.13 · 10<sup>-2</sup></td><td>2.5 · 10<sup>-3</sup></td><td>2.38 · 10<sup>-3</sup></td><td>2.12 · 10<sup>-3</sup></td></tr><tr><td>LOGLO-FNO</td><td><strong>2.34 · 10<sup>-2</sup></strong></td><td><strong>1.64 · 10<sup>-3</sup></strong></td><td><strong>1.35 · 10<sup>-3</sup></strong></td><td><strong>1.56 · 10<sup>-3</sup></strong></td></tr><tr><td>REL. % DIFF</td><td>-31.75%</td><td>-36.7%</td><td>-50%</td><td>-33.4%</td></tr></tbody></table>

We observe from Table 4 that LOGLO-FNO achieves better results than the baselines on all metrics, while reducing the mid- and high-frequency errors by over 50% and 30%, respectively. The full set of evaluation metrics and extended full trajectory rollouts are placed in Table 11 in the Appendix G.2.1, and the hyperparameters for all models studied are in Appendix O.

<figure><img src="image_8.png" /><figcaption>Figure 7: Qualitative results comparing 1-step horizontal velocity field predictions of Base FNO and LOGLO-FNO on a random sample from the test set of compressible Euler four-quadrant Riemann problem 2D.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Figure 8: Qualitative results comparing 1-step pressure field predictions of Base FNO and LOGLO-FNO models on a random sample from the test set of compressible Euler four-quadrant Riemann problem 2D.</figcaption></figure>

The above qualitative visualizations, specifically the error maps, reveal that LOGLO-FNO mitigates the errors to a considerable extent in the regions of high gradients. However, the model still struggles to eliminate them entirely, signifying the challenge of modeling discontinuities and sharp gradients that arise from shocks.
**Turbulent Radiative Layer 3D (TRL3D).** Noting the absence of support for local convolutions in FNO for 3D spatial data, we extend the LOGLO-FNO implementation to 3D and conduct experiments on a challenging version of a time-dependent turbulence simulation, viz., Turbulent Radiative Layer 3D (Fielding et al., 2020; Ohana et al., 2024) from the Well benchmark.

Similar to the Kolmogorov Flow 2D setup, the models are trained using 1-step training loss (cf. Eqn. 6), however, with the AdamW optimizer and only for 50 epochs by halving the learning rate every 10 epochs, maintaining the proportion of learning rate decays as with other problems. We chose this setup since we observed overfitting when training for longer epochs, such as 136 or even further. We attribute this behavior to the limited amount of available training data. Following Ohana et al. (2024), we consider the full data covering the entire range of $t_{\text{cool}}$ parameters, where $t_{\text{cool}} \in \{0.03, 0.06, 0.1, 0.18, 0.32, 0.56, 1.00, 1.78, 3.16\}$. For each $t_{\text{cool}}$ value, ten trajectories, each spanning 101 timesteps and of spatial resolution 128 × 128 × 256, are provided. The dataset also comes with an explicit train (8), validation (1), and test (1) splits. We then construct 7200, 900, and 900 1-step data pairs for training, validation, and testing, respectively.

Table 5: 1-step and autoregressive evaluation of LOGLO-FNO compared with state-of-the-art baselines on the test set of Turbulent Radiative Mixing Layer 3D dataset (Fielding et al., 2020; Ohana et al., 2024). REL. % DIFF indicates an improvement (-) or degradation (+) with respect to base FNO, which uses a spectral filter size of 12 and 48 hidden channels. LOGLO-FNO uses 12 and (16, 16, 17)* modes in the global and local branches, respectively, whereas the width is set to 52. (*This is because the Turbulent Radiative Layer 3D dataset has 2× more sampling points on the z-axis relative to the x and y axes resolutions – (128 × 128 × 256).) Base FNO† means our implementation of FNO (see Figure 1). $[t:t+\Delta t]$ symbolizes the inclusion of the timesteps at both ends of the interval when computing time-averaged vRMSE.

<table><thead><tr><th rowspan="2">Model</th><th>1-step Eval.</th><th colspan="2">Autoregressive Evaluation</th></tr><tr><th>1-step<br>vRMSE (↓)</th><th>[6:12] Time-Avg.<br>vRMSE (↓)</th><th>[13:30] Time-Avg.<br>vRMSE (↓)</th></tr></thead><tbody><tr><td>FNO</td><td>5.28 · 10<sup>-1</sup></td><td>8.1 · 10<sup>-1</sup></td><td>9.4 · 10<sup>-1</sup></td></tr><tr><td>TFNO</td><td>5.19 · 10<sup>-1</sup></td><td>>10</td><td>>10</td></tr><tr><td>U-Net</td><td>3.73 · 10<sup>-1</sup></td><td>9.5 · 10<sup>-1</sup></td><td>1.09 · 10<sup>0</sup></td></tr><tr><td>CNextU-Net</td><td>3.67 · 10<sup>-1</sup></td><td>7.7 · 10<sup>-1</sup></td><td>8.6 · 10<sup>-1</sup></td></tr><tr><td>LSM</td><td><strong>2.74 · 10<sup>-1</sup></strong></td><td>7.67 · 10<sup>-1</sup></td><td>1.02 · 10<sup>0</sup></td></tr><tr><td>NO-LIDK<sup>o</sup></td><td>4.05 · 10<sup>0</sup></td><td>6.0 · 10<sup>0</sup></td><td>6.52 · 10<sup>0</sup></td></tr><tr><td>Base FNO<sup>‡</sup></td><td>3.18 · 10<sup>-1</sup></td><td>7.49 · 10<sup>-1</sup></td><td>8.58 · 10<sup>-1</sup></td></tr><tr><td><strong>LOGLO-FNO</strong></td><td><strong>2.76 · 10<sup>-1</sup></strong></td><td><strong>7.09 · 10<sup>-1</sup></strong></td><td><strong>7.72 · 10<sup>-1</sup></strong></td></tr><tr><td>REL. % DIFF</td><td>-13.22%</td><td>-5.3%</td><td>-9.98%</td></tr></tbody></table><table><thead><tr><th rowspan="2">Model</th><th colspan="4">1-step Evaluation</th></tr><tr><th>nRMSE (↓)</th><th>fRMSE(L)</th><th>fRMSE(M)</th><th>fRMSE(H) (↓)</th></tr></thead><tbody><tr><td>LSM</td><td>8.39 · 10<sup>-1</sup></td><td>2.4 · 10<sup>-1</sup></td><td>9.7 · 10<sup>-2</sup></td><td>3.43 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK<sup>o</sup></td><td>4.34 · 10<sup>-1</sup></td><td>6.87 · 10<sup>-2</sup></td><td>1.22 · 10<sup>-1</sup></td><td>1.23 · 10<sup>-1</sup></td></tr><tr><td>Base FNO<sup>‡</sup></td><td>3.06 · 10<sup>-1</sup></td><td>4.87 · 10<sup>-2</sup></td><td>4.58 · 10<sup>-2</sup></td><td>2.89 · 10<sup>-2</sup></td></tr><tr><td><strong>LOGLO-FNO</strong></td><td><strong>2.65 · 10<sup>-1</sup></strong></td><td><strong>4.45 · 10<sup>-2</sup></strong></td><td><strong>3.68 · 10<sup>-2</sup></strong></td><td><strong>2.48 · 10<sup>-2</sup></strong></td></tr><tr><td>REL. % DIFF</td><td>-13.27%</td><td>-8.61%</td><td>-19.57%</td><td>-14.01%</td></tr></tbody></table>

Following recent investigations for conditioning the neural operators to generalize to unseen PDE parameters (Takamoto et al., 2023), we append the $t_{\text{cool}}$ coefficients as additional channels. The goal of the so-called *forward problem* for this PDE is then to predict the scalar (i.e., density and pressure) and vector (i.e., velocity-x, velocity-y, and velocity-z) physical quantities of $u^{t+1}(\cdot)$ given the solution at the previous timestep $u^t(\cdot)$. Further details on the PDE are provided in Appendix B.6.

Note that we have improved the 1-step vRMSE score (on the TRL3D test set) of the baseline FNO model by 41.4% compared to results reported by Ohana et al. (2024) (cf. Table 2: Model Performance Comparison) and, hence, our version of FNO is a strong baseline. We further note that the local integral kernel (DISCO conv layers) of NO-LIDK (Liu-Schiaffini et al., 2024) is not implemented for 3D spatial data and, hence, is not compared against in our experimental setup. We observe that LOGLO-FNO achieves the best results on all metrics and outperforms CNextU-Net, the best results of Ohana et al. (2024) on the Turbulent Radiative Layer 3D test dataset, on the vRMSE metric by 24.5%. The qualitative visualizations of predictions are provided in Appendix H.0.2 and the table with the full set of metrics is in Table 12 in the Appendix.

### 4.0.2 Autoregressive Training and Evaluation

We evaluate LOGLO-FNO on a fully autoregressive training setup on the Diffusion-Reaction 2D PDE from Takamoto et al. (2022). The model is trained with an initial context of 10 frames (i.e., $\Delta t = 10$), predicting one future state at a time until reaching the fully evolved state: $u(t - \Delta t, \cdot) \to u(t, \cdot)$. In other
words, since each simulation trajectory consists of 101 timesteps, AR rollout is performed for 91 timesteps both during training and evaluation to be consistent and for a fair comparison with the training setup of NO-LIDK (Liu-Schiaffini et al., 2024).

**2D Diffusion-Reaction.** We observe from Table 6 that LOGLO-FNO achieves a significant reduction in frequency errors across all bands (55%, 43%, 21%, resp.) compared to baseline FNO and superior to NO-LIDK on low- and mid-frequencies (30%, 30.4%, resp.). The tabulation of results with all metrics is provided in Table 13 in the Appendix G.2.2.

## 5 Ablation Studies

In order to investigate the effect of each of the proposed modules in LOGLO-FNO or the radially binned frequency loss, we conduct an ablation study on the Turbulent Radiative Layer 3D dataset, isolating the components. The

LOGLO-FNO model (cf. Fig. 1) by default uses the local spectral convolution and the HFP branches, and is trained with the radially binned spectral loss. This configuration corresponds to the result on the second row in Table 7 and achieves the best result, emphasizing the need to retain all the proposed contributions.

Afterwards, to obtain the result in the next row, we disable the frequency loss in the training objective of Eqn. 6 while retaining all other hyperparameters as before, quantifying the effect of the absence of radially binned frequency loss.

In the next setup, we additionally disable the HFP branch to yield a configuration of training LOGLO-FNO with only the local branch and MSE loss. The result for this is on the penultimate row of Table 7.

To isolate the effect of removing the HFP branch, we disable it and train LOGLO-FNO with the training objective in Eqn. 6. The result of this model configuration is presented in the last row of Table 7.

To summarize, although just using any one of the proposed contributions is sufficient to

obtain an improved result over baseline FNO, both local spectral convolution and HFP branches, and the radially binned frequency loss are necessary to achieve the best result.

## 6 Conclusions, Limitations, and Future Work

In this paper, we have proposed an enhancement to the FNO architecture (Li et al., 2021; Kossaifi et al., 2024b;a) through auxiliary parallel branches for local spectral convolution and high-frequency feature propagation. Through our experiments on six relevant and highly challenging PDE problems, namely, Kolmogorov Flow 2D, CNS Turb 2D, Wave-Gauss 2D, Compressible Euler four-quadrant Riemann problem 2D, Turbulent Radiative Layer 3D, and non-linearly coupled Diffusion-Reaction 2D problems that give rise to turbulence or high frequencies, we have demonstrated the promise to significantly reduce both the

Table 6: Fully autoregressive evaluation of LOGLO-FNO compared with SOTA baselines on the test set of challenging 2D Diffusion-Reaction coupled problem from PDEBench. We also report the REL. % DIFF to indicate the error improvement (-) with respect to baseline FNO.

<table><thead><tr><th>Model</th><th>nRMSE (↓)</th><th>fRMSE(L)</th><th>fRMSE(M)</th><th>fRMSE(H) (↓)</th></tr></thead><tbody><tr><td>U-Net</td><td>8.4 · 10<sup>-1</sup></td><td>1.7 · 10<sup>-2</sup></td><td>8.2 · 10<sup>-4</sup></td><td>5.7 · 10<sup>-2</sup></td></tr><tr><td>Transolver</td><td>2.63 · 10<sup>-1</sup></td><td>2.99 · 10<sup>-3</sup></td><td>2.03 · 10<sup>-3</sup></td><td>5.51 · 10<sup>-4</sup></td></tr><tr><td>U-FNO</td><td>2.6 · 10<sup>-1</sup></td><td>3.4 · 10<sup>-3</sup></td><td>1.6 · 10<sup>-3</sup></td><td>2.6 · 10<sup>-4</sup></td></tr><tr><td>FNO</td><td>8.3 · 10<sup>-2</sup></td><td>6.2 · 10<sup>-4</sup></td><td>5.6 · 10<sup>-4</sup></td><td>2.4 · 10<sup>-4</sup></td></tr><tr><td>F-FNO</td><td>7.0 · 10<sup>-2</sup></td><td>9.6 · 10<sup>-4</sup></td><td>4.7 · 10<sup>-4</sup></td><td><b>1.3 · 10<sup>-4</sup></b></td></tr><tr><td>LSM</td><td>4.47 · 10<sup>-1</sup></td><td>7.17 · 10<sup>-3</sup></td><td>2.4 · 10<sup>-3</sup></td><td>3.67 · 10<sup>-4</sup></td></tr><tr><td>NO-LIDK (loc. int)</td><td><b>6.3 · 10<sup>-2</sup></b></td><td>4.0 · 10<sup>-4</sup></td><td>4.6 · 10<sup>-4</sup></td><td>1.5 · 10<sup>-4</sup></td></tr><tr><td><b>LOGLO-FNO</b></td><td><b>6.4 · 10<sup>-2</sup></b></td><td><b>2.8 · 10<sup>-4</sup></b></td><td><b>3.2 · 10<sup>-4</sup></b></td><td><b>1.9 · 10<sup>-4</sup></b></td></tr><tr><td>REL. % DIFF</td><td>-22.75 %</td><td>-54.84 %</td><td>-42.86 %</td><td>-20.83 %</td></tr></tbody></table>

Table 7: 1-step results of LOGLO-FNO on the test set of TRL3D (Fielding et al., 2020; Ohana et al., 2024). Base FNO uses a spectral filter size of 18 and 48 hidden channels. LOGLO-FNO uses 16 and (16, 16, 17)* Fourier modes in the global and local branches, respectively, whereas the width is 52. (*Turbulent Radiative Layer 3D dataset has 2× more sampling points on the z-axis relative to the x- and y-axes resolutions – (128 × 128 × 256).)

<table><thead><tr><th>Model</th><th>nRMSE (↓)</th><th>fRMSE(L)</th><th>fRMSE(M)</th><th>fRMSE(H)</th><th>vRMSE (↓)</th></tr></thead><tbody><tr><td colspan="6" style="text-align: center;">1-step Evaluation</td></tr><tr><td>Base FNO</td><td>2.97 · 10<sup>-1</sup></td><td>5.17 · 10<sup>-2</sup></td><td>4.45 · 10<sup>-2</sup></td><td>2.76 · 10<sup>-2</sup></td><td>3.09 · 10<sup>-1</sup></td></tr><tr><td>LOGLO-FNO<br>(+freq loss, +HFP)</td><td><b>2.58 · 10<sup>-1</sup></b></td><td><b>4.1 · 10<sup>-2</sup></b></td><td><b>3.62 · 10<sup>-2</sup></b></td><td><b>2.45 · 10<sup>-2</sup></b></td><td><b>2.68 · 10<sup>-1</sup></b></td></tr><tr><td>(-freq loss, +HFP)</td><td>2.75 · 10<sup>-1</sup></td><td>4.4 · 10<sup>-2</sup></td><td>4.09 · 10<sup>-2</sup></td><td>2.63 · 10<sup>-2</sup></td><td>2.86 · 10<sup>-1</sup></td></tr><tr><td>LOGLO-FNO<br>(-freq loss, -HFP)</td><td>2.76 · 10<sup>-1</sup></td><td>4.4 · 10<sup>-2</sup></td><td>4.13 · 10<sup>-2</sup></td><td>2.64 · 10<sup>-2</sup></td><td>2.87 · 10<sup>-1</sup></td></tr><tr><td>(+freq loss, -HFP)</td><td>2.66 · 10<sup>-1</sup></td><td>4.53 · 10<sup>-2</sup></td><td>3.76 · 10<sup>-2</sup></td><td>2.48 · 10<sup>-2</sup></td><td>2.76 · 10<sup>-1</sup></td></tr></tbody></table>
spatial and spectral errors, showing the importance of local convolutions in mitigating the spectral bias to a considerable extent. In addition, our proposed radially binned spectral loss has also been found to be useful in this aspect. As a result of incorporating a local spectral convolutional branch in the architecture, our model has paved a way for reducing the number of trainable parameters to reach the baseline model accuracy. The current work, however, in order to allow a fair comparison to Liu-Schiaffini et al. (2024), has not considered any sophisticated parameter-efficient spectral convolution methods (e.g., tensor factorization techniques introduced in Kossaifi et al. (2024b)), which we see as a limitation. Future work could look into improving the scalability of LOGLO-FNO, provide a theoretical analysis of its benefits, or explore optimized implementation to accelerate the training and inference speed.

## Broader Impact Statement

Accelerated and efficient Neural PDE Solvers help reduce the cost of running cost-prohibitive simulations such as weather forecasting, cyclone predictions, and extreme weather events. As a consequence, disaster preparedness, the design, and manufacturing timelines can be accelerated, resulting in life and cost savings as well as decreased CO₂ emissions. As a negative side effect, we cannot rule out the possibility of misuse by bad actors, as computational fluid dynamics and turbulent flow simulations are also used to design military equipment within the military-industrial complex.

## Acknowledgments

The authors are thankful for the suggestions and questions of the anonymous reviewers on OpenReview, which have been helpful in improving the manuscript. We thank Miguel Liu-Schiaffini for the correspondence on NO-LIDK and Julius Berner for the initial discussions on local convolutions. Furthermore, we thank Artur Petrov Toshev for the insightful discussions on operator learning and brainstorming ideas in general. Moreover, we thank Julius Herb for the discussions and feedback on the initial draft. In addition, we thank Makoto Takamoto for the fruitful discussions during the rebuttal phase. Marimuthu Kalimuthu and Mathias Niepert are funded by Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy - EXC 2075 - 390740016. We acknowledge the support of the Stuttgart Center for Simulation Science (SimTech). The authors thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for supporting Marimuthu Kalimuthu and Mathias Niepert. Last but not least, the authors gratefully acknowledge the computing time provided to them at the NHR Center NHR4CES at RWTH Aachen University (project number p0021158). This is funded by the Federal Ministry of Education and Research, and the state governments participating on the basis of the resolutions of the GWK for national high-performance computing at universities (http://www.nhr-verein.de/unsere-partner).
References

Jason Ansel, Edward Z. Yang, Horace He, Natalia Gimelshein, Animesh Jain, Michael Voznesensky, Bin Bao, Peter Bell, David Berard, Evgeni Burovski, Geeta Chauhan, Anjali Chourdia, Will Constable, Alban Desmaison, Zachary DeVito, Elias Ellison, Will Feng, Jiong Gong, Michael Gschwind, Brian Hirsh, Sherlock Huang, Kshiteej Kalambarkar, Laurent Kirsch, Michael Lazos, Mario Lezcano, Yanbo Liang, Jason Liang, Yinghai Lu, C. K. Luk, Bert Maher, Yunjie Pan, Christian Puhrsch, Matthias Reso, Mark Saroufim, Marcos Yukio Siraichi, Helen Suk, Shunting Zhang, Michael Suo, Phil Tillet, Xu Zhao, Eikan Wang, Keren Zhou, Richard Zou, Xiaodong Wang, Ajit Mathews, William Wen, Gregory Chanan, Peng Wu, and Soumith Chintala. Pytorch 2: Faster machine learning through dynamic python bytecode transformation and graph compilation. In Rajiv Gupta, Nael B. Abu-Ghazaleh, Madan Musuvathi, and Dan Tsafrir (eds.), *Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2, ASPLOS 2024, La Jolla, CA, USA, 27 April 2024-1 May 2024*, pp. 929-947. ACM, 2024. doi: 10.1145/3620665.3640366. URL https://doi.org/10.1145/3620665.3640366.

Ronen Basri, David W. Jacobs, Yoni Kasten, and Shira Kritchman. The convergence rate of neural networks for learned functions of different frequencies. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d'Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), *Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019*, December 8-14, 2019, Vancouver, BC, Canada, pp. 4763-4772, 2019.

Johannes Brandstetter, Rianne van den Berg, Max Welling, and Jayesh K. Gupta. Clifford neural layers for PDE modeling. In *The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023*. OpenReview.net, 2023. URL https://openreview.net/forum?id=okwxL_c4x84.

Steven L. Brunton and J. Nathan Kutz. Machine learning for partial differential equations. *CoRR*, abs/2303.17078, 2023. doi: 10.48550/ARXIV.2303.17078. URL https://doi.org/10.48550/arXiv.2303.17078.

Steven L. Brunton and J. Nathan Kutz. Promising directions of machine learning for partial differential equations. *Nat. Comput. Sci.*, 4(7):483-494, 2024. doi: 10.1038/s43588-024-00643-2. URL https://doi.org/10.1038/s43588-024-00643-2.

Yuan Cao, Zhiying Fang, Yue Wu, Ding-Xuan Zhou, and Quanquan Gu. Towards understanding the spectral bias of deep learning. In Zhi-Hua Zhou (ed.), *Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021*, pp. 2205-2211. ijcai.org, 2021. doi: 10.24963/IJCAI.2021/304. URL https://doi.org/10.24963/ijcai.2021/304.

Chun-Fu (Richard) Chen, Quanfu Fan, and Rameswar Panda. Crossvit: Cross-attention multi-scale vision transformer for image classification. In *2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021*, pp. 347-356. IEEE, 2021. doi: 10.1109/ICCV48922.2021.00041. URL https://doi.org/10.1109/ICCV48922.2021.00041.

Lu Chi, Borui Jiang, and Yadong Mu. Fast Fourier Convolution. In Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), *Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual*, 2020.

James W Cooley and John W Tukey. An algorithm for the machine calculation of complex fourier series. *Mathematics of computation*, 19(90):297-301, 1965.

Yimian Dai, Fabian Gieseke, Stefan Oehmcke, Yiquan Wu, and Kobus Barnard. Attentional feature fusion. In *IEEE Winter Conference on Applications of Computer Vision, WACV 2021, Waikoloa, HI, USA, January 3-8, 2021*, pp. 3559-3568. IEEE, 2021. doi: 10.1109/WACV48630.2021.00360. URL https://doi.org/10.1109/WACV48630.2021.00360.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In *9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021*. OpenReview.net, 2021. URL https://openreview.net/forum?id=YicbFdNTTy.

Xiantao Fan, Deepak Akhare, and Jian-Xun Wang. Neural differentiable modeling with diffusion-based super-resolution for two-dimensional spatiotemporal turbulence. *arXiv preprint arXiv:2406.20047*, 2024.

Drummond B Fielding, Eve C Ostriker, Greg L Bryan, and Adam S Jermyn. Multiphase gas and the fractal nature of radiative turbulent mixing layers. *The Astrophysical Journal Letters*, 894(2):L24, 2020.

Cong Fu, Jacob Helwig, and Shuiwang Ji. Semi-supervised learning for high-fidelity fluid flow reconstruction. In *Soledad Villar and Benjamin Chamberlain (eds.), Learning on Graphs Conference, 27-30 November 2023, Virtual Event*, volume 231 of *Proceedings of Machine Learning Research*, pp. 36. PMLR, 2023. URL https://proceedings.mlr.press/v231/fu24b.html.

Robert Joseph George, Jiawei Zhao, Jean Kossaifi, Zongyi Li, and Anima Anandkumar. Incremental spatial and spectral learning of neural operators for solving large-scale PDEs. *Transactions on Machine Learning Research*, 2024. URL https://openreview.net/forum?id=xI6cPQObp0.

Jayesh K. Gupta and Johannes Brandstetter. Towards multi-spatiotemporal-scale generalized PDE modeling. *Trans. Mach. Learn. Res.*, 2023, 2023. URL https://openreview.net/forum?id=dPSTDbGtBY.

Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernández del Río, Mark Wiebe, Pearu Peterson, Pierre Gérard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. Array programming with NumPy. *Nature*, 585(7825): 357–362, 2020. doi: 10.1038/s41586-020-2649-2. URL https://doi.org/10.1038/s41586-020-2649-2.

Sheikh Md Shakeel Hassan, Arthur Feeney, Akash Dhruv, Jihoon Kim, Youngjoon Suh, Jaiyoung Ryu, Yoonjin Won, and Aparna Chandramowlišwaran. Bubbleml: A multiphase multiphysics dataset and benchmarks for machine learning. In *Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023*, 2023.

Jacob Helwig, Xuan Zhang, Cong Fu, Jerry Kurtin, Stephan Wojtowytsch, and Shuiwang Ji. Group equivariant fourier neural operators for partial differential equations. In *International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA*, volume 202 of *Proceedings of Machine Learning Research*, pp. 12907–12930. PMLR, 2023. URL https://proceedings.mlr.press/v202/helwig23a.html.

Maximilian Herde, Bogdan Raonic, Tobias Rohner, Roger Käppeli, Roberto Molinaro, Emmanuel de Bézenac, and Siddhartha Mishra. Poseidon: Efficient foundation models for pdes. *CoRR*, abs/2405.19101, 2024. doi: 10.48550/ARXIV.2405.19101. URL https://doi.org/10.48550/arXiv.2405.19101.

Qingkai Kong, Caifeng Zou, Youngsoo Choi, Eric M. Matzel, Kamyar Azizzadenesheli, Zachary E. Ross, Arthur J. Rodgers, and Robert W. Clayton. Reducing frequency bias of fourier neural operators in 3d seismic wavefield simulations through multi-stage training, 2025. URL https://arxiv.org/abs/2503.02023.

Jean Kossaifi, Nikola Kovachki, Zongyi Li, Davit Pitt, Miguel Liu-Schiaffini, Robert Joseph George, Boris Bonev, Kamyar Azizzadenesheli, Julius Berner, and Anima Anandkumar. A library for learning neural operators. *arXiv preprint arXiv:2412.10354*, 2024a.

Jean Kossaifi, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, and Anima Anandkumar. Multi-grid tensorized fourier neural operator for high-resolution PDEs. *Transactions on Machine Learning Research*, 2024b. ISSN 2835-8856. URL https://openreview.net/forum?id=AWiD1O63bH.
Nikola B. Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew M. Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces with applications to pdes. *J. Mach. Learn. Res.*, 24:89:1–89:97, 2023. URL https://jmlr.org/papers/v24/21-1524.html.

Samuel Lanthaler, Zongyi Li, and Andrew M. Stuart. Nonlocality and nonlinearity implies universality in operator learning. *CoRR*, abs/2304.13221, 2024. doi: 10.48550/ARXIV.2304.13221. URL https://doi.org/10.48550/arXiv.2304.13221.

Hongyu Li, Ximeng Ye, Peng Jiang, Guoliang Qin, and Tiejun Wang. Local neural operator for solving transient partial differential equations on varied domains. *Computer Methods in Applied Mechanics and Engineering*, 427:117062, 2024.

Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew M. Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. In *9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021*. OpenReview.net, 2021. URL https://openreview.net/forum?id=c8P9NQVtmn0.

Zongyi Li, Miguel Liu-Schiaffini, Nikola Kovachki, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Learning Dissipative Dynamics in Chaotic Systems (Datasets), December 2022a. URL https://doi.org/10.5281/zenodo.7495555.

Zongyi Li, Miguel Liu-Schiaffini, Nikola B. Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew M. Stuart, and Anima Anandkumar. Learning chaotic dynamics in dissipative systems. In *Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022*, 2022b.

Phillip Lippe, Bas Veeling, Paris Perdikaris, Richard E. Turner, and Johannes Brandstetter. Pde-refiner: Achieving accurate long rollouts with neural PDE solvers. In *Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023*, 2023.

Hong Liu, Zhisheng Lu, Wei Shi, and Juanhui Tu. A fast and accurate super-resolution network using progressive residual learning. In *ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*, pp. 1818–1822, 2020. doi: 10.1109/ICASSP40776.2020.9053890.

Xinliang Liu, Bo Xu, Shuhao Cao, and Lei Zhang. Mitigating spectral bias for the multiscale operator learning. *J. Comput. Phys.*, 506:112944, 2024. doi: 10.1016/J.JCP.2024.112944. URL https://doi.org/10.1016/j.jcp.2024.112944.

Miguel Liu-Schiaffini, Julius Berner, Boris Bonev, Thorsten Kurth, Kamyar Azizzadenesheli, and Anima Anandkumar. Neural operators with localized integral and differential kernels. In *Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024*. OpenReview.net, 2024. URL https://openreview.net/forum?id=v19GB3fbht.

Michael McCabe, Peter Harrington, Shashank Subramanian, and Jed Brown. Towards stability of autoregressive neural operators. *Trans. Mach. Learn. Res.*, 2023, 2023. URL https://openreview.net/forum?id=RFfUUtKYOG.

Juan Molina, Mircea Petrache, Francisco Sahli Costabal, and Matias Courdurier. Understanding the dynamics of the frequency bias in neural networks. *CoRR*, abs/2405.14957, 2024. doi: 10.48550/ARXIV.2405.14957. URL https://doi.org/10.48550/arXiv.2405.14957.

Jakin Ng, Yongji Wang, and Ching-Yao Lai. Spectrum-informed multistage neural network: Multiscale function approximator of machine precision. In *ICML 2024 AI for Science Workshop*, 2024. URL https://openreview.net/forum?id=59hdCpJyHx.
Ruben Ohana, Michael McCabe, Lucas Meyer, Rudy Morel, Fruzsina J. Agocs, Miguel Beneitez, Marsha Berger, Blakesley Burkhart, Stuart B. Dalziel, Drummond B. Fielding, Daniel Fortunato, Jared A. Goldberg, Keiya Hirashima, Yan-Fei Jiang, Rich R. Kerswell, Suryanarayana Maddu, Jonah Miller, Payel Mukhopadhyay, Stefan S. Nixon, Jeff Shen, Romain Watteaux, Bruno Régaldo-Saint Blancard, François Rozet, Liam Holden Parker, Miles D. Cranmer, and Shirley Ho. The well: a large-scale collection of diverse physics simulations for machine learning. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang (eds.), *Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024*, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024, 2024. URL http://papers.nips.cc/paper_files/paper/2024/hash/4f9a5acd91ac76569f2fe291b1f4772b-Abstract-Datasets_and_Benchmarks_Track.html.

Vivek Oommen, Aniruddha Bora, Zhen Zhang, and George Em Karniadakis. Integrating Neural Operators with Diffusion Models Improves Spectral Representation in Turbulence Modeling. arXiv e-prints, art. arXiv:2409.08477, September 2024.

Zizheng Pan, Jianfei Cai, and Bohan Zhuang. Fast vision transformers with hilo attention. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), *Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022*, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, 2022.

Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter W. Battaglia. Learning mesh-based simulation with graph networks. In *9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021*. OpenReview.net, 2021. URL https://openreview.net/forum?id=roNqYLO_XP.

Michael Poli, Stefano Massaroli, Federico Berto, Jinkyoo Park, Tri Dao, Christopher Ré, and Stefano Ermon. Transform once: Efficient operator learning in frequency domain. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), *Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022*, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, 2022.

Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred A. Hamprecht, Yoshua Bengio, and Aaron C. Courville. On the spectral bias of neural networks. In *Proceedings of the 36th International Conference on Machine Learning*, ICML 2019, 9-15 June 2019, Long Beach, California, USA, pp. 5301-5310. PMLR, 2019. URL http://proceedings.mlr.press/v97/rahaman19a.html.

Md Ashiqur Rahman, Zachary E. Ross, and Kamyar Azizzadenesheli. U-NO: u-shaped neural operators. *Trans. Mach. Learn. Res.*, 2023, 2023. URL https://openreview.net/forum?id=j3oQF9coJd.

Katja Schwarz, Yiyi Liao, and Andreas Geiger. On the frequency bias of generative models. In Marc'Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), *Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021*, NeurIPS 2021, December 6-14, 2021, virtual, pp. 18126-18136, 2021. URL https://proceedings.neurips.cc/paper/2021/hash/96bf57c6ff19504ff145e2a32991ea96-Abstract.html.

Kim Stachenfeld, Drummond Buschman Fielding, Dmitrii Kochkov, Miles Cranmer, Tobias Pfaff, Jonathan Godwin, Can Cui, Shirley Ho, Peter Battaglia, and Alvaro Sanchez-Gonzalez. Learned simulators for turbulence. In *International Conference on Learning Representations*, 2022. URL https://openreview.net/forum?id=msRBojTz-Nh.

James M Stone, Kengo Tomida, Christopher J White, and Kyle G Felker. The athena++ adaptive mesh refinement framework: Design and magnetohydrodynamic solvers. *The Astrophysical Journal Supplement Series*, 249(1):4, 2020.

Chuanhao Sun, Zhihang Yuan, Kai Xu, Luo Mai, N. Siddharth, Shuo Chen, and Mahesh K. Marina. Learning high-frequency functions made easy with sinusoidal positional encoding. In *Forty-first International Conference on Machine Learning*, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024a. URL https://openreview.net/forum?id=qqPLODkcrI.
Y Qiang Sun, Ashesh K Chattopadhyay, and Pedram Hassanzadeh. Hallucinations of AI-based models: the good, the bad, and the unstable. In *24th Conference on Atmospheric & Oceanic Fluid Dynamics & 22nd Conference on Middle Atmosphere*. AMS, 2024b.

Makoto Takamoto, Timothy Praditia, Raphael Leiteritz, Daniel MacKinlay, Francesco Alesiani, Dirk Pflüger, and Mathias Niepert. Pdebench: An extensive benchmark for scientific machine learning. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), *Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022*, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, 2022.

Makoto Takamoto, Francesco Alesiani, and Mathias Niepert. Learning neural PDE solvers with parameter-guided channel attention. In *International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA*, volume 202 of *Proceedings of Machine Learning Research*, pp. 33448–33467. PMLR, 2023. URL https://proceedings.mlr.press/v202/takamoto23a.html.

Alasdair Tran, Alexander Patrick Mathews, Lexing Xie, and Cheng Soon Ong. Factorized fourier neural operators. In *The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023*. OpenReview.net, 2023. URL https://openreview.net/forum?id=tmIiMP14IPa.

Zhong Yi Wan, Ricardo Baptista, Anudhyan Boral, Yi-Fan Chen, John Anderson, Fei Sha, and Leonardo Zepeda-Núñez. Debias coarsely, sample conditionally: Statistical downscaling through optimal transport and probabilistic diffusion models. In *Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, New Orleans, LA, USA, December, 2023*.

Rui Wang, Karthik Kashinath, Mustafa Mustafa, Adrian Albert, and Rose Yu. Towards physics-informed deep learning for turbulent flow prediction. In Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash (eds.), *KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020*, pp. 1457–1466. ACM, 2020. doi: 10.1145/3394486.3403198. URL https://doi.org/10.1145/3394486.3403198.

Yongji Wang and Ching-Yao Lai. Multi-stage neural networks: Function approximator of machine precision. *Journal of Computational Physics*, 504:112865, 2024.

Gege Wen, Zongyi Li, Kamyar Azizzadenesheli, Anima Anandkumar, and Sally M. Benson. U-FNO—an enhanced fourier neural operator-based deep-learning model for multiphase flow. *Advances in Water Resources*, 163:104180, 2022. ISSN 0309-1708. doi: https://doi.org/10.1016/j.advwatres.2022.104180. URL https://www.sciencedirect.com/science/article/pii/S0309170822000562.

Daniel E. Worrall, Miles Cranmer, J. Nathan Kutz, and Peter Battaglia. Spectral shaping for neural PDE surrogates. 2025. URL https://openreview.net/forum?id=mmDkgLtYNI.

Haixu Wu, Tengge Hu, Huakun Luo, Jianmin Wang, and Mingsheng Long. Solving high-dimensional pdes with latent spectral models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), *International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA*, volume 202 of *Proceedings of Machine Learning Research*, pp. 37417–37438. PMLR, 2023. URL https://proceedings.mlr.press/v202/wu23f.html.

Haixu Wu, Huakun Luo, Haowen Wang, Jianmin Wang, and Mingsheng Long. Transolver: a fast transformer solver for pdes on general geometries. In *Proceedings of the 41st International Conference on Machine Learning*, ICML'24. JMLR.org, 2024.

Zhi-Qin John Xu, Yaoyu Zhang, Tao Luo, Yanyang Xiao, and Zheng Ma. Frequency principle: Fourier analysis sheds light on deep neural networks. *CoRR*, abs/1901.06523, 2019. URL http://arxiv.org/abs/1901.06523.

Annan Yu, Dongwei Lyu, Soon Hoe Lim, Michael W Mahoney, and N Benjamin Erichson. Tuning frequency bias of state space models. arXiv preprint arXiv:2410.02035, 2024.
# LOGLO-FNO: EFFICIENT LEARNING OF LOCAL AND
GLOBAL FEATURES IN FOURIER NEURAL OPERATORS

SUPPLEMENTARY MATERIAL TO THE MAIN PAPER

PROJECT PAGE: HTTPS://KMARIO23.GITHUB.IO/LOGLO-FNO/

<table><tr><td>A Related Work</td><td>22</td></tr><tr><td>B Benchmark PDEs and Datasets</td><td>23</td></tr><tr><td>B.1 2D Incompressible Navier-Stokes (INS)</td><td>23</td></tr><tr><td>B.2 2D Compressible Navier-Stokes (CNS, PDEBench)</td><td>23</td></tr><tr><td>B.3 2D Wave Equation (Wave-Gauss, Poseidon)</td><td>24</td></tr><tr><td>B.4 Compressible Euler four-quadrant Riemann problem 2D (CE-RP, Poseidon)</td><td>24</td></tr><tr><td>B.5 2D Diffusion-Reaction (PDEBench)</td><td>25</td></tr><tr><td>B.6 3D Turbulent Radiative Mixing Layers (TRL3D, The Well)</td><td>25</td></tr><tr><td>C Baseline Neural Operator Models</td><td>26</td></tr><tr><td>C.1 Modern U-Net</td><td>26</td></tr><tr><td>C.2 FNO: Fourier Neural Operators</td><td>26</td></tr><tr><td>C.3 F-FNO: Factorized Fourier Neural Operators</td><td>27</td></tr><tr><td>C.4 U-FNO: An Enhanced FNO for Multiphase Flow</td><td>27</td></tr><tr><td>C.5 LSM: Latent Spectral Models</td><td>27</td></tr><tr><td>C.6 Transolver: Fast Transformer Solver for PDEs on General Geometries</td><td>27</td></tr><tr><td>C.7 NO-LIDK: Neural Operators with Localized Integral and Differential Kernels</td><td>27</td></tr><tr><td>D Spectral Space Loss Functions</td><td>28</td></tr><tr><td>D.1 Spectral Patch High-frequency Emphasized Residual Energy (SPHERE) Loss</td><td>28</td></tr><tr><td>D.2 Radially Binned Spectral Energy Errors as a Frequency-aware Loss</td><td>29</td></tr><tr><td>D.3 Visualizing Radially Binned Energy of Spectral Errors of Baselines and LOGLO-FNO</td><td>31</td></tr><tr><td>E High-Frequency Feature Adaptive Gaussian Noise</td><td>34</td></tr><tr><td>F Complexity Analysis of Local and Global Spectral Convolutions</td><td>35</td></tr><tr><td>F.1 2D Spectral Convolutions</td><td>35</td></tr><tr><td>F.2 3D Spectral Convolutions</td><td>36</td></tr><tr><td>G Detailed Quantitative Results and Analysis</td><td>37</td></tr><tr><td>G.1 Evaluation Metrics</td><td>37</td></tr><tr><td>G.2 Detailed Quantitative Results with Full Set of Metrics</td><td>42</td></tr></table>
<table>
  <tbody>
    <tr>
      <td>G.3</td>
      <td>Further Improvements with Attentional Feature Fusion, Multi-scale Channel Attention, and SPHERE Loss</td>
      <td>47</td>
    </tr>
    <tr>
      <td>G.4</td>
      <td>Significance of Adding More Modes in the Fourier Layers of Global Branch</td>
      <td>48</td>
    </tr>
    <tr>
      <td>H</td>
      <td>Qualitative Results and Analysis</td>
      <td>53</td>
    </tr>
    <tr>
      <td>I</td>
      <td>Energy Spectra Visualization and Analysis</td>
      <td>57</td>
    </tr>
    <tr>
      <td>J</td>
      <td>Analysis of Correlation with Ground Truth on AR Rollout</td>
      <td>61</td>
    </tr>
    <tr>
      <td>J.1</td>
      <td>Pearson's Correlation of Sample Trajectories with Ground Truth</td>
      <td>61</td>
    </tr>
    <tr>
      <td>K</td>
      <td>Ablation Studies</td>
      <td>64</td>
    </tr>
    <tr>
      <td>K.1</td>
      <td>LOGLO-FNO Devoid of Radially Binned Frequency Loss (C<sub>freq</sub>)</td>
      <td>64</td>
    </tr>
    <tr>
      <td>K.2</td>
      <td>LOGLO-FNO Devoid of HFP Module and Radially Binned Frequency Loss (HFP, C<sub>freq</sub>)</td>
      <td>64</td>
    </tr>
    <tr>
      <td>K.3</td>
      <td>LOGLO-FNO Devoid of High Frequency Propagation Module (HFP)</td>
      <td>64</td>
    </tr>
    <tr>
      <td>K.4</td>
      <td>Patch-Size Sensitivity of LOGLO-FNO</td>
      <td>65</td>
    </tr>
    <tr>
      <td>K.5</td>
      <td>Influence of High-Frequency Adaptive Gaussian Noise on Rollout Stability in LOGLO-FNO</td>
      <td>67</td>
    </tr>
    <tr>
      <td>K.6</td>
      <td>Choice of Radial Bin Cutoff Values for the Radially Binned Frequency Loss in LOGLO-FNO</td>
      <td>67</td>
    </tr>
    <tr>
      <td>L</td>
      <td>Zero-Shot Super-Resolution (ZSSR) Capability and Performance of LOGLO-FNO</td>
      <td>68</td>
    </tr>
    <tr>
      <td>M</td>
      <td>Training and Inference Time Comparison of LOGLO-FNO</td>
      <td>70</td>
    </tr>
    <tr>
      <td>M.1</td>
      <td>Comparison with Numerical Solver</td>
      <td>70</td>
    </tr>
    <tr>
      <td>M.2</td>
      <td>Training &amp; Inference Time and GPU Memory Usage Comparison with Base FNO and NO-LIDK</td>
      <td>70</td>
    </tr>
    <tr>
      <td>N</td>
      <td>Confidence Analysis of LOGLO-FNO and Baselines</td>
      <td>72</td>
    </tr>
    <tr>
      <td>O</td>
      <td>Model Hyperparameters</td>
      <td>74</td>
    </tr>
    <tr>
      <td>O.1</td>
      <td>Modern U-Net Hyperparameters</td>
      <td>74</td>
    </tr>
    <tr>
      <td>O.2</td>
      <td>Base FNO Hyperparameters</td>
      <td>74</td>
    </tr>
    <tr>
      <td>O.3</td>
      <td>F-FNO Hyperparameters</td>
      <td>74</td>
    </tr>
    <tr>
      <td>O.4</td>
      <td>LSM Hyperparameters</td>
      <td>74</td>
    </tr>
    <tr>
      <td>O.5</td>
      <td>Transolver Hyperparameters</td>
      <td>75</td>
    </tr>
    <tr>
      <td>O.6</td>
      <td>U-FNO Hyperparameters</td>
      <td>76</td>
    </tr>
    <tr>
      <td>O.7</td>
      <td>NO-LIDK Hyperparameters</td>
      <td>76</td>
    </tr>
    <tr>
      <td>O.8</td>
      <td>LOGLO-FNO Implementation and Hyperparameters</td>
      <td>76</td>
    </tr>
    <tr>
      <td>P</td>
      <td>Table of Notations and Mathematical Symbols</td>
      <td>78</td>
    </tr>
  </tbody>
</table>
A Related Work

It has been a well-established fact that deep neural networks, trained with first-order gradient descent optimization methods, have a spectral bias for learning functions with low frequencies before picking up high-frequency details, explaining their superior generalization capabilities on downstream tasks (Xu et al., 2019; Rahaman et al., 2019; Cao et al., 2021; Schwarz et al., 2021; Molina et al., 2024; Yu et al., 2024). This means that deep networks require longer training times (Basri et al., 2019), necessitate multi-stage residue-based training (Wang & Lai, 2024; Ng et al., 2024; Kong et al., 2025), or experiences difficulty altogether, in learning complex functions with high-frequency components. This phenomenon is apropos of the neural PDE solving community (Li et al., 2022b; Fu et al., 2023; Liu et al., 2024) where the high-frequencies are more pronounced in multiscale, multiphysics, time-dependent PDEs and can evolve in time. This limitation of deep neural nets affects not only an accurate reconstruction of the spectral components but also rollouts (Worrall et al., 2025) and causes hallucinations (Sun et al., 2024b).

In the field of computer graphics and vision, several approaches have been proposed to solve this problem. Sun et al. (2024a) explore learning high-frequencies for generative tasks in computer graphics and propose a sinusoidal positional encoding method to learn high-frequency features effectively. Pan et al. (2022) propose a faster variant of ViT with HiLo attention by splitting the attention heads to learn high and low frequencies in a disentangled fashion.

In the research area of neural PDE solving and SciML applications, Liu et al. (2024) study the limitations of existing neural operators for multiscale PDEs and propose a hierarchical attention method and $H^1$ loss to direct the emulator to learn high-frequencies. Other approaches in improving the capability of neural operators to learn high frequencies include (i) an adaptive selection of Fourier modes and resolution (George et al., 2024) since using a suboptimal choice for the number of modes may be detrimental on many PDE problems (Lanthaler et al., 2024), (ii) a two-staged approach of first predicting the future states, given some historical temporal context, using a neural operator and feeding that as an input (i.e., conditioning factor) to a diffusion model in stage 2 (Oommen et al., 2024). Similarly, Fan et al. (2024) employ a two-stage pipeline of first generating low-resolution Kolmogorov flow turbulence simulations using a hybrid DNS neural solver on coarse grids and leveraging a conditional diffusion model to super-resolve for high frequencies and fine-grained structures in the second stage. Poli et al. (2022) study frequency-domain models for PDE solving with an aim to accelerate the training process. They propose a method to learn solution operators directly in the frequency domain, which is made possible with a variance-preserving weight initialization scheme. Closest to our work in terms of achieving local convolution and modeling high-frequency features in FNO is NO-LIDK (Liu-Schiaffini et al., 2024), which designs two different local layers for learning the differential and integral operators that capture local and high-frequency features. Hence, we consider this as the relevant and competitive baseline. Although LOGLO-FNO has a high-level similarity to NO-LIDK by introducing architectural modifications to FNO with an additional parallel branch for local convolutions, it differs from NO-LIDK in the manner it achieves local convolutions. It does so by decomposing the input domain into sub-domains and then individually applying spectral convolutions on these sub-domains, thereby effectively capturing localized features. Moreover, an additional parallel branch specifically designed for modeling and providing an inductive bias of high-frequency features is introduced.

Chen et al. (2021) develop a multi-scale vision transformer for image classification tasks and explore different fusion methods aimed at an effective fusion of multi-scale features from the respective branches. Dai et al. (2021) explore attention mechanisms for aggregating the local and global context features and demonstrate their effectiveness on image classification and object localization tasks in computer vision. Specifically, they propose a multi-scale channel attention module and attentional feature fusion blocks for fusing multi-scale features in deep neural networks.
# B Benchmark PDEs and Datasets

We experiment with 2D and 3D transient PDEs that exhibit high frequencies, turbulence behaviour, and rich local patterns due to their inherent nature (e.g., coupled variables), controllable parameters (i.e., PDE coefficients, Reynolds number, forcing terms), chaotic dynamics, shocks, discontinuities, porosity of the medium, and higher-order derivatives.

## B.1 2D Incompressible Navier-Stokes (INS)

**Kolmogorov Flow.** Following prior literature (Li et al., 2022b; George et al., 2024; Liu-Schiaffini et al., 2024; Lanthaler et al., 2024), we adopt 2D Kolmogorov Flow, which is a form of Navier-Stokes PDE, as one of the test cases.

$$ \frac{\partial u}{\partial t} + u \cdot \nabla u - \frac{1}{Re} \Delta u = -\nabla p + \sin(ny)\hat{x}, \quad \nabla \cdot u = 0, \quad \text{on } [0, 2\pi]^2 \times (0, \infty) \quad (7) $$

We use the dataset configuration of Li et al. (2022a) for Re5000 that exhibits high-frequency structures due to the flow being fully turbulent. It consists of 100 samples, each with a spatial and temporal resolution of 128×128 and 500 snapshots, respectively. As with George et al. (2024), we exclude the first 100 timesteps to let the trajectories reach the attractor, obtaining a total of 40,000 pairs of samples, out of which we use 36K input-output pairs for training and 4K for testing.

## B.2 2D Compressible Navier-Stokes (CNS, PDEBench)

In order to study a problem exhibiting turbulence and also capable of describing shock wave formation and propagation, we experiment with the 2D CNS equations (Eq. 8a to 8c) from PDEBench (Takamoto et al., 2022). More specifically, we use the sonic regime dataset (Mach=1.0) where the initial conditions comprise turbulent velocity with uniform mass density and pressure, and the boundary conditions are periodic.

$$ \begin{array}{l} \text{mass density (mass per unit volume)} \\ \partial_t \rho + \nabla \cdot (\rho \mathbf{v}) = 0, \hfill (8a) \\ \rho(\partial_t \mathbf{v} + \mathbf{v} \cdot \nabla \mathbf{v}) = -\nabla p + \eta \Delta \mathbf{v} + \left(\zeta + \frac{\eta}{3}\right) \nabla(\nabla \cdot \mathbf{v}), \hfill (8b) \\ \text{pressure} \\ \partial_t \left(\epsilon + \frac{\rho v^2}{2}\right) + \nabla \cdot \left[\left(p + \epsilon + \frac{\rho v^2}{2}\right) \mathbf{v} - \mathbf{v} \cdot \mathbf{\sigma}'\right] = 0, \hfill (8c) \\ \text{internal energy} \\ \text{viscous stress tensor} \end{array} $$

The dataset (Mach=1.0) contains 1000 trajectories, each comprising 21 timesteps. Following the data split of the PDEBench* repository, we use 900 samples for training and the rest for testing. To facilitate the 1-step training and testing tasks, we construct the input-output pairs by slicing the solution fields (i.e., density, pressure, velocity-x, and velocity-y) at successive timesteps along the length of the trajectory, yielding 18K samples for training and 2K for testing. Due to the high spatial resolution of the data of 512 × 512, we subsample it to 256 × 256 throughout our experiments. However, we use the full spatial resolution to evaluate spatial ZSSR capabilities.

<footnote>*https://github.com/pdebench/PDEBench</footnote>
## B.3 2D Wave Equation (Wave-Gauss, Poseidon)

As an additional challenging PDE problem modeling a different physical phenomenon of the propagation of acoustic waves through a spatially varying medium, we consider the time-dependent 2D Wave Equation, specifically the so-called Wave-Gauss, from the PDEgym†† benchmark of Poseidon (Herde et al., 2024).

$$ \frac{\partial^2 u(x, y, t)}{\partial t^2} - (c(x, y))^2 \left( \frac{\partial^2 u(x, y, t)}{\partial x^2} + \frac{\partial^2 u(x, y, t)}{\partial y^2} \right) = 0, \quad \text{in } D \times (0, T) \qquad (9) $$

spatially varying propagation field

The initial condition $u^0(x, y)$ of the displacement $u$, and the propagation field $c(x, y)$ are both generated using a sum of several Gaussians, and hence the name.

$$ u_0(x, y) = \sum_{i=1}^{n} g_i(x, y), \quad x, y \in (0, 1) $$

$$ g_i(x, y) = \exp \left( - \frac{(x_{c,i} - x)^2 + (y_{c,i} - y)^2}{2s_i^2} \right), \quad x, y \in (0, 1) $$

where $s_i$ is the standard deviation of the $i^{th}$ Gaussian in the propagation field $c$.

$$ c(x, y) = c_0 + \sum_{i=1}^{4} f_i(x, y), \quad x, y \in (0, 1) $$

$$ f_i(x, y) = v_i \cdot \exp \left( - \frac{(x_i + dx_i - x)^2 + (y_i + dy_i - y)^2}{2\sigma_i^2} \right), \quad x, y \in (0, 1) $$

where $v_i$ and $\sigma_i$ are the amplitude and standard deviation of the $i^{th}$ Gaussian, respectively.

The dataset has a total of 10,512 trajectories of 16 timesteps each, with a spatial resolution of 128 × 128, of which we use 10,212 trajectories for training, 60 trajectories for validation, and 240 trajectories for testing, following the data split provided by the authors. Since we perform 1-step training and evaluation, we construct input-output pairs by sampling subsequent timesteps in the length of the trajectory, yielding us 153.18K training, 900 validation, and 3.6K testing sample pairs. The autoregressive rollout is evaluated against the full trajectory length by predicting one future timestep at a time.

## B.4 Compressible Euler four-quadrant Riemann problem 2D (CE-RP, Poseidon)

In order to test the capabilities of LOGLO-FNO in modeling shock waves and discontinuities, we consider the highly challenging Compressible Euler four-quadrant Riemann problem 2D from Poseidon (Herde et al., 2024). The four-quadrant Riemann problem is a generalization of the Sod shock tube to 2D space.

$$ D_{i,j} = \left\{ (x, y) \in \mathbb{T}^2 \mid \frac{i-1}{p} \le x < \frac{i}{p}, \frac{j-1}{p} \le y < \frac{j}{p} \right\}, $$

where $\mathbb{T}^2$ is the 2D torus, and considering a unit square $D = [0, 1]^2$, it can be divided into 2 × 2 square sub-domains when $p = 2$. The underlying solution operator $S(t, \rho_0, v_{x,y}^0, p_0) = [\rho(t), v_{x,y}(t), p(t)]$, solving the

<footnote>††https://camlab-ethz.github.io/poseidon/#Time-dependent</footnote>
compressible Euler equations with periodic boundary conditions, has four fields, viz. density, pressure, and velocity.

The dataset consists of 10k trajectories, with each sample containing 21 snapshots at a spatial resolution of 128 × 128. Following the pre-defined train/val/test split of Herde et al. (2024), we use the initial 9640 trajectories for training, the next 120 for validation, and the ultimate 240 for testing. As with other PDE problems we consider, we slice successive timesteps as input-output pairs starting from the initial condition of a trajectory. This yields us 192.8k training, 2.4k validation, and 4.8k testing data pairs.

## B.5 2D Diffusion-Reaction (PDEBench)

2D Diffusion-Reaction (Takamoto et al., 2022) is a challenging SciML benchmark problem, modeling biological pattern formation. It is challenging because of the nonlinear coupling of the solution variables, viz. activator $u(x,y,t)$ and inhibitor $v(x,y,t)$ from the time-dependent PDE,

$$ \frac{\partial u}{\partial t} = D_u \partial_{xx} u + D_u \partial_{yy} u + R_u(u, v), \quad \frac{\partial v}{\partial t} = D_v \partial_{xx} v + D_v \partial_{yy} v + R_v(u, v), \quad (-1, 1)^2 \times (0, 5] \quad (10) $$

where $D_u$ and $D_v$ are diffusion coefficients for the activator and inhibitor, respectively, and the reaction functions $R_u$ and $R_v$ in Eqn. 10 are defined by the Fitzhugh-Nagumo equations as,

$$ R_u(u, v) = u - u^3 - k - v, \quad R_v(u, v) = u - v, \quad (11) $$

The simulation data is generated by setting $D_u = 1 \cdot 10^{-3}$, $D_v = 5 \cdot 10^{-3}$, and $k = 5 \cdot 10^{-3}$ with no-flow Neumann boundary condition. The training and test sets consist of 900 and 100 trajectories, respectively, where each simulation is of shape (128 × 128 × 101 × 2), indicating the solutions for 101 timesteps and a channel each for the activator ($u$) and inhibitor ($v$).

## B.6 3D Turbulent Radiative Mixing Layers (TRL3D, The Well)

In three-dimensional astrophysical environments, dense cold gas clouds (the “cold phase”) move through a hotter, diffuse medium (the “hot phase”), generating turbulent mixing at their interfaces. This 3D turbulent mixing produces a multiscale intermediate-temperature layer, where gas rapidly cools radiatively due to the enhanced cooling efficiency at intermediate temperatures. As energy is lost via photon emission, the mixed gas condenses and accretes onto the cold phase.

* Growth vs. Dissolution of Cold Clumps

- When radiative cooling dominates turbulent mixing ($t_{\text{cool}} \ll t_{\text{mix}}$), the cold phase grows as mixed gas rapidly cools and accretes onto cold clouds.

- When mixing dominates ($t_{\text{cool}} \gg t_{\text{mix}}$), the cold phase evaporates into the hot medium.

* Cooling-Mass Transfer Relation: The volume-integrated cooling rate $\dot{E}_{\text{cool}}$ and mass transfer rate $\dot{M}$ from hot to cold phases scale as

$$ \dot{E}_{\text{cool}} \propto \dot{M} \propto v_{\text{rel}}^{3/4} t_{\text{cool}}^{-1/4}, $$

where $v_{\text{rel}}$ is the 3D relative velocity between phases, and $t_{\text{cool}}$ is the cooling timescale.

These 3D simulations explicitly model the competition between turbulent mixing (driven by shear, Kelvin-Helmholtz Instabilities (KHI), and turbulence) and radiative cooling, providing a more complete picture of multiphase gas dynamics in astrophysical environments (e.g., galactic halos, stellar winds, or the interstellar medium) (Fielding et al., 2020). The equations underlying these simulations are given by Ohana et al. (2024),
<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /></figure>

<figure><img src="image_12.png" /></figure>

<figure><img src="image_13.png" /></figure>

# C Baseline Neural Operator Models

We choose the following list of seven diverse, competitive, and recent state-of-the-art baselines.

## C.1 Modern U-Net

We benchmark the modern version of U-Net (https://github.com/pdearena/pdearena/tree/main/pdearena/modules) from PDEArena (Gupta & Brandstetter, 2023). Considering that we do not focus on testing the generalization capabilities of neural operators on a diverse set of PDE parameters, the parameter conditioning is skipped. The U-Net architecture models multi-scale spatio-temporal phenomena through a series of downsampling and an equal number of upsampling blocks, with the skip-connections passing information from the downsampling to the upsampling pass. We employ four levels of downsampling and set the channel multipliers to (1, 2, 2, 3, 4) to maintain parameter parity with other baselines. Following prior studies (Lippe et al., 2023), the network directly predicts the residual to the next step instead of the actual solution, and then we down weight this output with the factor $\frac{3}{10}$. Note that this setting improves the results of the modern U-Net model on Kolmogorov Flow significantly (1-step nRMSE REL. % DIFF of 22%) compared to the reported results of NO-LIDK (Liu-Schiaffini et al., 2024) and hence is a strong baseline. The hyperparameters are listed in Appendix O.1.

## C.2 FNO: Fourier Neural Operators

As the original FNO architecture (Li et al., 2021) has been improved and optimized over the years, we utilize the current state-of-the-art implementation of FNO (https://github.com/neuraloperator/neuraloperator) from Kossaifi et al. (2024a) for Kolmogorov Flow 2D, CNS Turb 2D, Wave-Gauss 2D, and Compressible Euler four-quadrant Riemann problem 2D PDEs. However, we omitted tensor factorization techniques in our experiments so as to be able to directly compare our results with NO-LIDK (Liu-Schiaffini et al., 2024), which was also devoid of any such data compression strategies. We use the implementation of Takamoto et al. (2022) for the Diffusion-Reaction 2D problem following Liu-Schiaffini et al. (2024) for a fair comparison. The hyperparameters are provided in Appendix O.2.
### C.3 F-FNO: Factorized Fourier Neural Operators

Tran et al. (2023) propose to improve the original FNO architecture of Li et al. (2021) by factorizing the FFT computation for data with spatial dimensions of two or greater, among other contributions such as skip connections, Markov assumption, Gaussian noise, cosine learning rate schedule, and hence the model Factorized FNO. The factorization leads to a massive drop in the number of trainable parameters, resulting in a lightweight and efficient model. Therefore, we use this as one of the baselines. Having borrowed their open-source implementation (https://github.com/alasdairtran/fourierflow) and adapted to our PDE problems, we provide the hyperparameters we use in Appendix O.3.

### C.4 U-FNO: An Enhanced FNO for Multiphase Flow

Wen et al. (2022) propose U-FNO as an improved version of FNO specifically targeted for modeling multiphase flows. The network consists of six layers in total, three of which are standard Fourier layers from Li et al. (2021) and three UNet layers added to the Fourier layers. We adapt the open-source implementation (https://github.com/gegewen/ufno) provided by Wen et al. (2022) for 3D spatial data to our experiments on 2D spatial data. We tune the model hyperparameters, viz., learning rate, modes, and channel dimension, to approximately match the number of trainable parameters as the other baselines while also reaching a competitive accuracy. The full set of hyperparameters is provided in Appendix O.6.

### C.5 LSM: Latent Spectral Models

Wu et al. (2023) introduce a multi-scale neural operator architecture with the use of a hierarchical projection network mapping the high-dimensional input space into latent dimensions and propose to model the dynamics in this low-dimensional latent space. The essential component of LSM is the so-called Neural Spectral Block, which consists of an encode-process-decode style of processing. Considering the robustness of MSE to LSM's hyperparameters (cf. Appendix G in Wu et al. (2023)), such as scales, number of latent tokens, and number of basis operators, we use 5 scales, 4 latent tokens, and 12 basis operators in the hierarchical projection network. This yields 64, 128, 256, 512, and 512 channels for the five scales in question, starting with the full spatial resolution and initial channel dimensions of 64 for the 2D Kolmogorov Flow, CNS Turb 2D, and Wave-Gauss 2D problems. As for the Diffusion-Reaction 2D PDE, we use the initial channel dimensions of 32. Therefore, the channel dimensions for the five scales starting with the full incoming spatial resolution would be 32, 64, 128, 256, and 256, respectively. The other hyperparameters are provided in Appendix O.4.

### C.6 Transolver: Fast Transformer Solver for PDEs on General Geometries

Wu et al. (2024) design a transformer-based neural operator model, namely Transolver, to solve PDEs on both regular and irregular domains. The idea stems from the observation that self-attention over individual mesh points is unnecessary, expensive, and not scalable because it is not uncommon to have real-world data spanning tens of millions of points or higher, resulting from the discretization of the continuous physical fields. Towards this end, Transolver introduces a so-called 'Physics Attention' mechanism where the mesh points belonging to similar physical states are grouped into *slices* and the self-attention is computed over these *slices*. The resultant model achieves a near-linear complexity and scales to reasonably sized mesh points (e.g., 16,384 – 128×128). Therefore, to include a transformer-based model in the baselines, we benchmark this model on the 2D Kolmogorov Flow, CNS Turb 2D, Wave-Gauss 2D, and Diffusion-Reaction 2D PDEs by adapting the implementation provided in https://github.com/thuml/Transolver/tree/main/PDE-Solving-StandardBenchmark. However, we note that Transolver needs an extended training time of 500 epochs for convergence on the challenging Kolmogorov Flow problem, while the other baselines converge to an optimal loss value in just 136 epochs. The model architecture and training hyperparameters are listed in Appendix O.5.

### C.7 NO-LIDK: Neural Operators with Localized Integral and Differential Kernels

Noticing the deficiencies of FNO in learning local and finer scale features due to the absence of support for local receptive fields, Liu-Schiaffini et al. (2024) propose two additional discretization-invariant convolutional
layers (one for the differential operator and another for the local integral kernel operator) that can be placed in parallel to the (global) Fourier layers of (spherical) FNO. These additional pathways enrich the architecture with capabilities for local convolutions, which are realized through localized integral and differential kernels. The authors test the efficacy of these layers by conducting experiments on a wide range of two-dimensional PDE problems, such as Darcy Flow, turbulent Navier-Stokes, and Diffusion-Reaction on the 2D planar domain, and the Shallow Water Equations on the spherical computational domain. Their results significantly improve over the baseline FNO, ranging from 34% to 87% in terms of nRMSE, demonstrating the effectiveness and importance of these local operations for operator learning. Since our work also achieves local convolutions, albeit by decomposing the input spatial resolution into sub-domains (e.g., patches), we consider the NO-LIDK model as a closely related and competitive baseline. Considering that we report additional evaluation metrics, we train three variants of NO-LIDK (namely, (i) NO-LIDK* denoting only the presence of local integral kernel layers, (ii) NO-LIDK$^{\diamond}$ representing only the existence of differential kernel layers, and (iii) NO-LIDK$^{\dagger}$ indicating the use of both local integral kernel layers and differential kernel layers, in addition to the global FNO branch) on the INS Kolmogorov Flow 2D dataset and reproduce the reported results. Following the authors' suggestion, we use a radius cutoff of $0.05\pi$ for the local integral kernel layers and set the domain length to $[2\pi, 2\pi]$ (cf. Eqn 7). We borrow the other hyperparameters, such as the number of Fourier modes, hidden channels, epochs, learning rate, scheduler, and scheduler steps, for each of the model variants from their paper. The specific values used for each of these are provided in Appendix O.7, which also lists the training and model configuration settings, including the radius cutoff values for the CNS Turb 2D, Wave-Gauss 2D, and Compressible Euler four-quadrant Riemann problem 2D PDEs.

# D Spectral Space Loss Functions

In this section, we focus on two loss functions that operate entirely in the frequency domain.

## D.1 Spectral Patch High-frequency Emphasized Residual Energy (SPHERE) Loss

In congruence with the local branch of LOGLO-FNO, we propose a spectral loss localized to each of the sub-domains $P_m$ of the domain $D$ (§3.1).

Let $u_{pred}$ and $u_{gt}$ be the predictions and ground truths of non-overlapping patches of the solution in the domain, respectively. First, we compute the residual in the physical space: $\Delta u = u_{pred} - u_{gt}$. Second, these residuals of patches are transformed into the frequency domain using 2D DFT realized through FFT: $\Delta \hat{u}(k_x, k_y) = \mathcal{F}(\Delta u)$.

The SPHERE loss is then computed as the weighted squared magnitude of Fourier coefficients as

$$ \text{SPHERE} = \mathcal{W}(k_x, k_y) \odot |\Delta \hat{u}(k_x, k_y)|^2 \quad (12) $$

where the weight function $\mathcal{W}(k_x, k_y)$ emphasizing high-frequencies is defined as,

$$ \mathcal{W}(k_x, k_y) = 1 + \alpha \left( \frac{\text{FM}}{\max(\text{FM}) + \epsilon} \right)^p, \quad \text{FM} = \sqrt{k_x^2 + k_y^2} \quad (13) $$

whereby $\alpha$ and $p$ control the emphasis on high-frequencies. For our experiments, we use $p = 2$.

The loss is finally mean or sum reduced over the channel dimensions and the number of patches, and added to the MSE loss during training.
```python
def compute_frequency weights(alpha, p, psize):
    kx = torch.flt.fltffreq(psize, d=1.0).reshape(1, 1, 1, psize)
    ky = torch.flt.fltffreq(psize, d=1.0).reshape(1, 1, psize, 1)
    freq_magnitude = torch.sqrt(kx**2 + ky**2)

    # Normalize and define frequency weighting function
    freq_magnitude /= (freq_magnitude.max() + 1e-8)
    return 1 + alpha * (freq_magnitude ** p)


def extract patches(x, psize):
    # Reshape to (nb*nt*nc, 1, nx, ny) to apply unfold over spatial dims
    nb, nx, ny, nt, nc = x.shape
    x_reshp = x.permute(0, 3, 4, 1, 2).reshape nb * nt * nc, 1, nx, ny)

    # Extract patches using unfold
    patches = F.unfold(x_reshp, kernel_size=psize, stride=psize)

    # Reshape back to (nb, nt, nc, num patches, patch size, patch size)
    numPtch = patches.shape[-1]
    patches = patches.view nb, nt, nc, numPtch, psize, psize)

    return patches


def SPHERELoss(preds, target, alpha, p, patch_size, reduction="mean"):
    # extract non-overlapping patches
    pred patches = extract patches(pred, patch_size)
    target patches = extract patches(target, patch_size)

    # Compute difference (i.e., error) in real space once
    diff patches = pred patches - target patches

    # Use vmap to apply FFT over temporal and channel dims
    def fft and weight(diff):
        diff FFT = torch.flt.FFT2(diff, norm="ortho")
        weight = compute_frequency weights(alpha, p, patch_size)
        return weight * (diff FFT.real ** 2 + diff FFT.imag ** 2)

    # Apply vmap once along both (nt, nc) dimensions
    diff loss = torch.vmap(
        torch.vmap(fft and weight, in dims=1),
        in dims=2
    )(diff patches)

    # Aggregate spectral loss over all patches, time, and channels
    spect loss = diff loss.mean(dim=(1, 2, 3))

    if reduction == "mean":
        return spect loss.mean()
    else:
        return spect loss.sumspect loss)
```

Unlike the function `compute_frequency weights(...)` shown here for expediency, in practice, we compute the weight only once per patch size and cache it for efficiency reasons.

## D.2 Radially Binned Spectral Energy Errors as a Frequency-aware Loss

We elaborate on the frequency-aware loss term based on radially binning of energy spectra errors introduced in §3.2 by considering a 2D spatiotemporal domain.

The detailed procedure to compute the frequency loss for 2D spatial data is shown in the algorithm below.
<caption>**Algorithm 1** Frequency-aware Loss based on Radially Binned Spectral Errors</caption>

1: **Input:** Prediction $\tilde{\mathbf{Y}} \in \mathbb{R}^{N_b \times N_c \times N_x \times N_y \times N_t}$ and Target $\mathbf{Y} \in \mathbb{R}^{N_b \times N_c \times N_x \times N_y \times N_t}$, $N_x$ and $N_y$ be the spatial resolutions, $N_b$ the batch size, $N_t$ the total timesteps, $N_c$ the number of physical fields, $L_x$ and $L_y$ the length of the spatial domain along the respective axes of the 2D domain, and $\mathbf{x}$ and $\mathbf{y}$ be the 2D mesh grid of frequency indices. Let $iL$ and $iH$ be the radial bin cutoff indices, i.e., $iL$ is the count of low-frequency bins, and $iH$ is the count of low+mid-frequency bins.

2: $\mathbf{E}_F(b, c, k_x, k_y, t) \leftarrow | \mathcal{F}(\mathbf{Y}_{b,c,t} - \tilde{\mathbf{Y}}_{b,c,t})(k_x, k_y) |^2$ & {Energy Spectra Error (ESE)}
3: $\mathcal{R}(k_x, k_y) \leftarrow [\sqrt{\mathbf{x}^2 + \mathbf{y}^2}]$ & {Binned radial distances}
4: $\mathcal{M} \leftarrow \max_{k_x, k_y}(\mathcal{R}(k_x, k_y))$ & {Max radius}
5: $\mathbf{S}_{rad}(b, c, r, t) \leftarrow \sum_{\mathcal{R}(k_x, k_y)=r} (k_x, k_y) \text{ s.t. } \mathbf{E}_F(b, c, k_x, k_y, t)$ for $r = 0 \dots \mathcal{M}$ & {Sum ESE into bin $r$, per $b, c, t$}
6: $\bar{\mathbf{S}}_{rad}(c, r, t) \leftarrow \text{mean}_b(\mathbf{S}_{rad}(b, c, r, t))$ & {Average summed ESE per bin over batch}
7: $\bar{\mathbf{E}}_F(c, r, t) \leftarrow \sqrt{\bar{\mathbf{S}}_{rad}(c, r, t)} \cdot \left(\frac{L_x}{N_x}\right) \cdot \left(\frac{L_y}{N_y}\right)$ & {Normalize 1D radial error spectrum}
8: $\text{err}_F^{\text{low}}(c, t) \leftarrow \text{mean}_{r \in [0, iL-1]}(\bar{\mathbf{E}}_F(c, r, t))$ & {Low-frequency band errors}
9: $\text{err}_F^{\text{mid}}(c, t) \leftarrow \text{mean}_{r \in [iL, iH-1]}(\bar{\mathbf{E}}_F(c, r, t))$ & {Mid-frequency band errors}
10: $\text{err}_F^{\text{high}}(c, t) \leftarrow \text{mean}_{r \in [iH, \mathcal{M}]}(\bar{\mathbf{E}}_F(c, r, t))$ & {High-frequency band errors}

The concrete steps are listed in the PyTorch-style pseudocode below. Since we use `torch fft fftn` in the 'backward' normalization mode, no normalization term is included. Let `preds` and `target` be the predictions and ground truth, respectively, and the 2D spatial data has a single channel and timestep. The computation for multi-channel and multi-timestep data is straightforward since they are obtained independently for each physical variable and timestep in the input.

```python
def RadialBinnedSpectralLoss(preds, target):
    # input data shape and params
    nb, nc, nx, ny, nt = target.size()
    iLow, iHigh = 4, 12
    Lx, Ly = 1.0, 1.0

    # Compute error in Fourier space
    err
    err
    err FFT = torch CFR(phys, dim=[2, 3])
    err FFT_SQ = torch .abs( err FFT)**2
    err FFT_SQ_h = err FFT_SQ[Ellipsis, :nx//2, :ny//2, :]}

    # Create radial indices
    x = torch arange nx//2
    y = torch arange ny//2
    X, Y = torch meshgrid(x, y, indexing="ij")
    radii = torch sqrt(X**2 + Y**2). floor() .to( torch int ) # Radial dist.
    max radius = int( torch max radii))

    # flatten radii for binary mask
    radii flat = radii .flatten() # (nx//2 * ny//2)

    # Spatially flatten Fourier space error; (nb, nc, nx//2 * ny//2, nt)
    err FFT_SQ_flat = err FFT_SQ_h contiguous() .reshapenb, nc, -1, nt)

    # initialize output tensor to hold the Fourier error
    # for each radial bin at distance r from the origin
    err_F vect_full = torch zeros nb, nc, max radius + 1, nt)

    # Apply 'index_add_ ' for all radii and accumulate the errors
    valid_r = radii flat <= max radius # binary mask to find valid radii
```
<figure><img src="image_14.png" /><figcaption>Figure 10: Visualization of the radially binned energy of spectral errors of LOGLO-FNO predictions on the same random trajectory and timestep (as that of the one for baseline FNO) from the test set of Kolmogorov Flow ($Re = 5k$) (Li et al., 2022b). The left plot depicts the radial bins and the location of the boundaries of mid and high-frequency groups. Note that we show only every other radial bin. The right plot visualizes the radially binned spectral error as a line plot over the radial distance from the DC component. The dashed blue and green vertical lines indicate the starting locations of the mid and high-frequency regions, respectively.</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>Figure 11: Visualization of the radially binned energy of spectral errors of NO-LIDK predictions on the same random trajectory and timestep (as that of the one for baseline FNO) from the test set of Kolmogorov Flow ($Re = 5k$) (Li et al., 2022b). The left plot depicts the radial bins and the location of the boundaries of mid and high-frequency groups. Note that we show only every other radial bin. The right plot visualizes the radially binned spectral error as a line plot over the radial distance from the DC component. The dashed blue and green vertical lines indicate the starting locations of the mid and high-frequency regions, respectively.</figcaption></figure>
<figure><img src="image_16.png" /><figcaption>Figure 12: Visualization of the radially binned energy of spectral errors of modern U-Net predictions on the same random trajectory and timestep (as that of the one for baseline FNO) from the test set of Kolmogorov Flow ($Re = 5k$) (Li et al., 2022b). The left plot depicts the radial bins and the location of the boundaries of mid and high-frequency groups. Note that we show only every other radial bin. The right plot visualizes the radially binned spectral error as a line plot over the radial distance from the DC component. The dashed blue and green vertical lines indicate the starting locations of the mid and high-frequency regions, respectively.</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>Figure 13: Visualization of the radially binned energy of spectral errors of LSM predictions on the same random trajectory and timestep (as that of the one for baseline FNO) from the test set of Kolmogorov Flow ($Re = 5k$) (Li et al., 2022b). The left plot depicts the radial bins and the location of the boundaries of mid and high-frequency groups. Note that we show only every other radial bin. The right plot visualizes the radially binned spectral error as a line plot over the radial distance from the DC component. The dashed blue and green vertical lines indicate the starting locations of the mid and high-frequency regions, respectively.</figcaption></figure>
## E High-Frequency Feature Adaptive Gaussian Noise

In many machine learning and signal processing tasks, high-frequency (HF) features often capture fine-grained details or noise-like patterns in the data. Introducing controlled noise based on these features during training can improve model robustness, prevent overfitting, and enhance generalization. It has been shown in prior surrogate modeling investigations that adding a small amount of Gaussian noise helps with long rollouts (Pfaff et al., 2021; Stachenfeld et al., 2022) and stabilized training (Tran et al., 2023). However, traditional Gaussian noise, which is static and independent of the input data, may not adequately capture the variability inherent in turbulent PDEs and physics simulation data rich in high frequencies. To address this shortcoming, we propose a novel method to generate dynamic Gaussian noise that adapts to the statistical properties of the input HF features. Specifically, we obtain the high-frequency feature adaptive Gaussian noise by scaling (α) the standard Gaussian noise N(0, 1) based on the mean (μ) and standard deviation (σ) of the HF features.

$$ \mathbf{N}_{\text{dynamic}} = \mu + \alpha \cdot \sigma \cdot \mathbf{N}, \quad \mathbf{N} \sim \mathcal{N}(0, 1) \qquad (14) $$

### High-Frequency Feature Adaptive Gaussian Noise

Let $\mathbf{X}^{hf} \in \mathbb{R}^{N_b \times N_x \times N_y \times (N_t \cdot N_c)}$ denote the input tensor of HF features, where:

* $N_b$ is the batch size,

* $N_x$ and $N_y$ are the resolutions of the spatial dimensions, and

* $N_t \cdot N_c$ represents the combined temporal and channel dimensions.

The high-frequency feature adaptive Gaussian noise $\mathbf{N}_{\text{dynamic}}$ is then computed as follows:

1. Compute (per sample) Mean ($\mu_b$) and Standard Deviation ($\sigma_b$) of High-Frequency Features

$$ \mu_b = \frac{1}{N_x \cdot N_y \cdot N_t \cdot N_c} \sum_{i=1}^{N_x} \sum_{j=1}^{N_y} \sum_{k=1}^{N_t} \mathbf{X}_{b,i,j,k}^{\text{hf}} $$

$$ \sigma_b = \sqrt{\frac{1}{N_x \cdot N_y \cdot N_t \cdot N_c} \sum_{i=1}^{N_x} \sum_{j=1}^{N_y} \sum_{k=1}^{N_t} (\mathbf{X}_{b,i,j,k}^{\text{hf}} - \mu)^2 + \epsilon} $$

where $\epsilon$ is a small constant added for numerical stability, and $\mu$ and $\sigma$ are obtained by stacking the per sample statistics along the batch dimension.

2. Generate Standard Gaussian Noise

$$ \mathbf{N} \sim \mathcal{N}(0, 1) $$

3. Scale Noise Dynamically

$$ \mathbf{N}_{\text{dynamic}} = \mu + \alpha \cdot \sigma \cdot \mathbf{N} $$

where $\alpha$ is a small value such as 0.025 and $\mathbf{N}_{\text{dynamic}}$ has the same shape as the input $\mathbf{X}^{hf}$.

$\mathbf{N}_{\text{dynamic}}$ can now be added to the batch of inputs to the global and local branches during the training phase of LOGLO-FNO.
# F Complexity Analysis of Local and Global Spectral Convolutions

## F.1 2D Spectral Convolutions

**Global branch.** FLOPs for FFT and IFFT computations (Cooley & Tukey, 1965) on the full 2D spatial resolution.

### Global Branch FLOPs Calculation for FFT & IFFT in Global Spectral Convolution 2D

Let $\mathbf{X} \in \mathbb{R}^{N_b \times N_c \times N_x \times N_y}$ denote the input tensor for the 2D global spectral convolution module, where:

* $N_b$ is the batch size,

* $N_x$ and $N_y$ are the resolutions of the spatial dimensions,

* $N_c$ represents the total number of hidden channels.

$$ \text{FLOPs}_{\text{FFT}} = 5 \cdot N_b \cdot C_{\text{in}} \cdot N_x \cdot N_y \cdot \log_2(N_x \cdot N_y) $$

$$ \text{FLOPs}_{\text{IFFT}} = 5 \cdot N_b \cdot C_{\text{out}} \cdot N_x \cdot N_y \cdot \log_2(N_x \cdot N_y) $$

where $c_{\text{in}}$ and $c_{\text{out}}$ are typically of the same dimension and are referred to as either width or embedding/hidden channel dimension.

Therefore, FFT computation on the global branch with the full 2D spatial resolution has a complexity of $O(N_x \cdot N_y \cdot \log_2(N_x \cdot N_y))$ per channel, making it expensive for large $N_x$ and $N_y$ such as 2048 × 2048 or higher spatial resolutions.

**Local branch.** FLOPs for FFT and IFFT computations (Cooley & Tukey, 1965) on the domain decomposed 2D spatial resolution, for instance, with non-overlapping patches.

### Local Branch FLOPs Calculation for FFT & IFFT in Local Spectral Convolution 2D

Let $\hat{\mathbf{X}} \in \mathbb{R}^{N_b \times N_c \times N_p \times P_{\text{size}} \times P_{\text{size}}}$ denote the input tensor for the 2D local spectral convolution module, where:

* $N_p$ is the number of patches obtained by $\frac{N_x \cdot N_y}{P_{\text{size}}^2}$,

* $N_x$ and $N_y$ are the resolutions of the spatial dimensions,

* $P_{\text{size}} \times P_{\text{size}}$ is the patch size (e.g., 16 × 16),

* $N_c$ represents the width or the number of hidden channels.

$$ \begin{aligned} \text{FLOPs}_{\text{FFT}} &= 5 \cdot N_b \cdot C_{\text{in}} \cdot N_p \cdot P_{\text{size}} \cdot P_{\text{size}} \cdot \log_2(P_{\text{size}} \cdot P_{\text{size}}) \\ &= 5 \cdot N_b \cdot C_{\text{in}} \cdot N_x \cdot N_y \cdot \log_2(P_{\text{size}} \cdot P_{\text{size}}) \end{aligned} $$

$$ \begin{aligned} \text{FLOPs}_{\text{IFFT}} &= 5 \cdot N_b \cdot C_{\text{out}} \cdot N_p \cdot P_{\text{size}} \cdot P_{\text{size}} \cdot \log_2(P_{\text{size}} \cdot P_{\text{size}}) \\ &= 5 \cdot N_b \cdot C_{\text{out}} \cdot N_x \cdot N_y \cdot \log_2(P_{\text{size}} \cdot P_{\text{size}}) \end{aligned} $$

where $c_{\text{in}}$ and $c_{\text{out}}$ are typically set to be of the same dimension (e.g., 128).

Thus, the FFT and IFFT computations (Cooley & Tukey, 1965) on the local branch operating on the patches has a computational complexity of $O(N_x \cdot N_y \cdot \log_2(P_{\text{size}} \cdot P_{\text{size}}))$ per channel. Since $O(N_x \cdot N_y \cdot \log_2(P_{\text{size}} \cdot P_{\text{size}}))$
$P_{size}) \ll O(N_x \cdot N_y \cdot \log_2(N_x \cdot N_y))$ in practice, the FFT computation is significantly cheaper in the local branch. Moreover, it is highly parallelizable when computing FFTs since each patch can be processed independently, leveraging modern accelerators such as GPUs and TPUs.

## F.2 3D Spectral Convolutions

**Global branch.** FLOPs for FFT and IFFT computations (Cooley & Tukey, 1965) on the full 3D spatial resolution.

### Global Branch FLOPs Calculation for FFT & IFFT in Global Spectral Convolution 3D

Let $\mathbf{X} \in \mathbb{R}^{N_b \times N_c \times N_x \times N_y \times N_z}$ denote the input tensor for the 3D global spectral convolution module, where:

* $N_b$ is the batch size,

* $N_x$, $N_y$, and $N_z$ are the resolutions of the spatial dimensions,

* $N_c$ represents the total number of hidden channels.

$$ \text{FLOPs}_{\text{FFT}} = 5 \cdot N_b \cdot C_{\text{in}} \cdot N_x \cdot N_y \cdot N_z \cdot \log_2(N_x \cdot N_y \cdot N_z) $$

$$ \text{FLOPs}_{\text{IFFT}} = 5 \cdot N_b \cdot C_{\text{out}} \cdot N_x \cdot N_y \cdot N_z \cdot \log_2(N_x \cdot N_y \cdot N_z) $$

where $c_{\text{in}}$ and $c_{\text{out}}$ are typically of the same dimension and are referred to as either width or embedding/hidden channel dimension.

Therefore, FFT computation on the global branch with the full 3D spatial resolution has a complexity of $O(N_x \cdot N_y \cdot N_z \cdot \log_2(N_x \cdot N_y \cdot N_z))$ per channel, making it highly expensive for large values of $N_x$, $N_y$, and $N_z$ such as 512 × 512 × 512 or further higher spatial resolutions.

**Local branch.** FLOPs for FFT and IFFT computations (Cooley & Tukey, 1965) on the domain decomposed 3D spatial resolution, for instance, with non-overlapping patches.
## Local Branch FLOPs Calculation for FFT & IFFT in Local Spectral Convolution 3D

Let $\hat{\mathbf{X}} \in \mathbb{R}^{N_b \times N_c \times N_p \times P_{size} \times P_{size} \times P_{size}}$ denote the input tensor for the 3D local spectral convolution module, where:

* $N_p$ is the number of patches obtained by $\frac{N_x \cdot N_y \cdot N_z}{P_{size}^3}$,

* $N_x, N_y$, and $N_z$ are the resolutions of the spatial dimensions,

* $P_{size} \times P_{size} \times P_{size}$ is the patch size (e.g., 16 × 16 × 16, 32 × 32 × 32, etc.),

* $N_c$ represents the width or number of hidden channels.

$$ 
\begin{aligned}
\text{FLOPs}_{\text{FFT}} &= 5 \cdot N_b \cdot C_{\text{in}} \cdot N_p \cdot P_{\text{size}} \cdot P_{\text{size}} \cdot P_{\text{size}} \cdot \log_2(P_{\text{size}} \cdot P_{\text{size}} \cdot P_{\text{size}}) \\
&= 5 \cdot N_b \cdot C_{\text{in}} \cdot N_x \cdot N_y \cdot N_z \cdot \log_2(P_{\text{size}} \cdot P_{\text{size}} \cdot P_{\text{size}})
\end{aligned}
$$

$$ 
\begin{aligned}
\text{FLOPs}_{\text{IFFT}} &= 5 \cdot N_b \cdot C_{\text{out}} \cdot N_p \cdot P_{\text{size}} \cdot P_{\text{size}} \cdot P_{\text{size}} \cdot \log_2(P_{\text{size}} \cdot P_{\text{size}} \cdot P_{\text{size}}) \\
&= 5 \cdot N_b \cdot C_{\text{out}} \cdot N_x \cdot N_y \cdot N_z \cdot \log_2(P_{\text{size}} \cdot P_{\text{size}} \cdot P_{\text{size}})
\end{aligned}
$$

where $c_{\text{in}}$ and $c_{\text{out}}$ are typically set to be of the same dimension (e.g., 64, 128, etc.).

Thus, the FFT and IFFT computations (Cooley & Tukey, 1965) on the local branch operating on 3D patches has a computational complexity of $O(N_x \cdot N_y \cdot N_z \cdot \log_2(P_{\text{size}} \cdot P_{\text{size}} \cdot P_{\text{size}}))$ per channel. Since $O(N_x \cdot N_y \cdot N_z \cdot \log_2(P_{\text{size}} \cdot P_{\text{size}} \cdot P_{\text{size}})) \ll O(N_x \cdot N_y \cdot N_z \cdot \log_2(P_{\text{size}} \cdot P_{\text{size}} \cdot P_{\text{size}}))$ in practice, the FFT computation is significantly cheaper in the local branch. Moreover, it is highly parallelizable when computing FFTs since each patch can be processed independently, leveraging modern accelerators such as GPUs and TPUs.

# G Detailed Quantitative Results and Analysis

## G.1 Evaluation Metrics

Following prior research efforts (Takamoto et al., 2022; Wan et al., 2023), we evaluate our models on a wide range of metrics, viz. RMSE, nRMSE, fRMSE (low, mid, high), MaxError, MELR, and WLR. In addition, we introduce REL. % DIFF (RPD) to report relative performance improvement w.r.t a baseline. For the sake of completeness, we explain the computation of the metrics briefly.

### G.1.1 Normalized Root Mean Square Error (nRMSE) a.k.a. Normalized L₂ a.k.a. Relative L₂ Error

The normalized $L_2$ norm of a vector $\mathbf{e} = (e_1, e_2, \dots, e_n)^T$ is given by:

$$ \text{Normalized } L_2 = \frac{\|\mathbf{e}\|_2}{\|\mathbf{y}\|_2} = \frac{\sqrt{\sum_{i=1}^{n} e_i^2}}{\sqrt{\sum_{i=1}^{n} y_i^2}} $$

where $\mathbf{y} = (y_1, y_2, \dots, y_n)^T$ is the reference vector.

The normalized RMSE is defined as:

$$ \text{Normalized RMSE} = \frac{\sqrt{\frac{1}{n} \sum_{i=1}^{n} e_i^2}}{\sqrt{\frac{1}{n} \sum_{i=1}^{n} y_i^2}} = \frac{\sqrt{\sum_{i=1}^{n} e_i^2}}{\sqrt{\sum_{i=1}^{n} y_i^2}} $$
The factor $\frac{1}{\sqrt{n}}$ cancels out in the numerator and denominator, leading to:

$$ \text{Normalized RMSE} = \text{Normalized } L_2. $$

Given a prediction tensor $\hat{\mathbf{Y}} \in \mathbb{R}^{N_b \times N_c \times N_x \times N_y \times N_t}$ and a target tensor $\mathbf{Y} \in \mathbb{R}^{N_b \times N_c \times N_x \times N_y \times N_t}$, where $b = 1, \dots, N_b$, $c = 1, \dots, N_c$, $x = 1, \dots, N_x$, $y = 1, \dots, N_y$, and $t = 1, \dots, N_t$. The nRMSE is computed as follows:

1. Mean Squared Error over the spatial dimensions:

$$ \text{MSE}_{b,c,t} = \frac{1}{N_x \cdot N_y} \sum_{x=1}^{N_x} \sum_{y=1}^{N_y} (\hat{\mathbf{Y}}_{b,c,x,y,t} - \mathbf{Y}_{b,c,x,y,t})^2 $$

2. Root Mean Squared Error over the spatial dimensions:

$$ \text{RMSE}_{b,c,t} = \sqrt{\frac{1}{N_x \cdot N_y} \sum_{x=1}^{N_x} \sum_{y=1}^{N_y} (\hat{\mathbf{Y}}_{b,c,x,y,t} - \mathbf{Y}_{b,c,x,y,t})^2} $$

3. Normalization Factor:

$$ \text{Normalization Factor}_{b,c,t} = \sqrt{\frac{1}{N_x \cdot N_y} \sum_{x=1}^{N_x} \sum_{y=1}^{N_y} \mathbf{Y}_{b,c,x,y,t}^2} $$

4. Normalized RMSE (nRMSE):

$$ \text{nRMSE}_{b,c,t} = \frac{\text{RMSE}_{b,c,t}}{\text{Normalization Factor}_{b,c,t}} $$

5. Mean nRMSE over the batch, channel, and temporal dimensions:

$$ \text{nRMSE} = \frac{1}{N_b \cdot N_c \cdot N_t} \sum_{b=1}^{N_b} \sum_{c=1}^{N_c} \sum_{t=1}^{N_t} \text{nRMSE}_{b,c,t} $$

### G.1.2 Max Error

Considering 2D spatial data, we can deduce that Takamoto et al. (2022) define the maximum or the worst case error in the entire test set as,

$$ \text{MaxError} = \frac{1}{N_c \cdot N_t} \sum_{c=1}^{N_c} \sum_{t=1}^{N_t} \left( \max_{b,x,y} \left| \hat{\mathbf{Y}}_{b,c,x,y,t} - \mathbf{Y}_{b,c,x,y,t} \right| \right) $$
### G.1.3 RMSE at the Boundaries of the Spatial Domain (bRMSE)

Again, taking 2D spatial data as an exemplary case, we write the formula for bRMSE from Takamoto et al. (2022) as,

1. Compute Sum of Squared Errors (SSE) on the Boundaries of the Spatial Domain:

$$ \text{SSE}_{b,c,t} = \sum_{y=1}^{N_y} (\hat{\mathbf{Y}}_{b,c,1,y,t} - \mathbf{Y}_{b,c,1,y,t})^2 + \sum_{y=1}^{N_y} (\hat{\mathbf{Y}}_{b,c,N_x,y,t} - \mathbf{Y}_{b,c,N_x,y,t})^2 + \\ \sum_{x=1}^{N_x} (\hat{\mathbf{Y}}_{b,c,x,1,t} - \mathbf{Y}_{b,c,x,1,t})^2 + \sum_{x=1}^{N_x} (\hat{\mathbf{Y}}_{b,c,x,N_y,t} - \mathbf{Y}_{b,c,x,N_y,t})^2 \\ \text{\% bottom horizontal boundary} \quad \text{\% top horizontal boundary} \\ \text{\% left vertical boundary} \quad \text{\% right vertical boundary} $$

2. Compute Per-Sample Boundary RMSE:

$$ \text{bRMSE}_{b,c,t} = \sqrt{\frac{\text{SSE}_{b,c,t}}{2N_x + 2N_y}} $$

3. Compute Final bRMSE:

$$ \text{bRMSE} = \frac{1}{N_b \cdot N_c \cdot N_t} \sum_{b=1}^{N_b} \sum_{c=1}^{N_c} \sum_{t=1}^{N_t} \text{bRMSE}_{b,c,t} $$

Or more succinctly put,

$$ \text{bRMSE} = \frac{1}{N_b N_c N_t} \sum_{b=1}^{N_b} \sum_{c=1}^{N_c} \sum_{t=1}^{N_t} \sqrt{\frac{1}{2N_x + 2N_y} \left[ \sum_{y=1}^{N_y} (\Delta_{b,c,1,y,t}^2 + \Delta_{b,c,N_x,y,t}^2) + \sum_{x=1}^{N_x} (\Delta_{b,c,x,1,t}^2 + \Delta_{b,c,x,N_x,t}^2) \right]} $$

$$ \text{where } \Delta_{b,c,x,y,t} = (\hat{\mathbf{Y}}_{b,c,x,y,t} - \mathbf{Y}_{b,c,x,y,t}) $$

### G.1.4 RMSE of Conserved Physical Quantities (cRMSE)

We again consider 2D spatial data for illustrative purposes here since computing the same metric for 3D spatial data is straightforward. Takamoto et al. (2022) define cRMSE as follows.

1. Compute the Error of Spatial Sum (Δs):

For each sample b, channel c, and timestep t, calculate the difference between the spatial sum of the predicted field and the spatial sum of the target field.

$$ \Delta_{s,b,c,t} = \sum_{x=1}^{N_x} \sum_{y=1}^{N_y} (\hat{\mathbf{Y}}_{b,c,x,y,t} - \mathbf{Y}_{b,c,x,y,t}) $$
2. Compute Normalized cRMSE:

$$ \overline{\text{cRMSE}}_{c,t} = \frac{1}{N_x \cdot N_y} \sqrt{\frac{1}{N_b} \sum_{b=1}^{N_b} (\Delta \mathbf{s}_{b,c,t})^2} $$

3. Compute Final cRMSE:

$$ \text{cRMSE} = \frac{1}{N_c \cdot N_t} \sum_{c=1}^{N_c} \sum_{t=1}^{N_t} \overline{\text{cRMSE}}_{c,t} $$

Alternatively, as a one-line expression:

$$ \text{cRMSE} = \frac{1}{N_c \cdot N_t \cdot N_x \cdot N_y} \sum_{c=1}^{N_c} \sum_{t=1}^{N_t} \sqrt{\frac{1}{N_b} \sum_{b=1}^{N_b} \left( \sum_{x=1}^{N_x} \sum_{y=1}^{N_y} (\hat{\mathbf{Y}}_{b,c,x,y,t} - \mathbf{Y}_{b,c,x,y,t}) \right)^2} $$

### G.1.5 Variance-scaled Root Mean Square Error (vRMSE)

Ohana et al. (2024) introduce vRMSE, which is merely RMSE normalized by the variance of the ground truth. Considering 3D spatial data,

$$ \text{MSE}_{b,c,t} = \frac{1}{N_x \cdot N_y \cdot N_z} \sum_{x=1}^{N_x} \sum_{y=1}^{N_y} \sum_{z=1}^{N_z} (\hat{\mathbf{Y}}_{b,c,x,y,z,t} - \mathbf{Y}_{b,c,x,y,z,t})^2 $$

$$ \text{RMSE}_{b,c,t} = \sqrt{\frac{1}{N_x \cdot N_y \cdot N_z} \sum_{x=1}^{N_x} \sum_{y=1}^{N_y} \sum_{z=1}^{N_z} (\hat{\mathbf{Y}}_{b,c,x,y,z,t} - \mathbf{Y}_{b,c,x,y,z,t})^2} $$

$$ \text{Mean}_{b,c,t} = \frac{1}{N_x \cdot N_y \cdot N_z} \sum_{x=1}^{N_x} \sum_{y=1}^{N_y} \sum_{z=1}^{N_z} \mathbf{Y}_{b,c,x,y,z,t} $$

$$ \text{Var}_{b,c,t} = \frac{1}{N_x \cdot N_y \cdot N_z} \sum_{x=1}^{N_x} \sum_{y=1}^{N_y} \sum_{z=1}^{N_z} (\mathbf{Y}_{b,c,x,y,z,t} - \text{Mean}_{b,c,t})^2 $$

$$ \text{vRMSE} = \frac{1}{N_b \cdot N_c \cdot N_t} \sum_{b=1}^{N_b} \sum_{c=1}^{N_c} \sum_{t=1}^{N_t} \left( \frac{\text{RMSE}_{b,c,t}}{\sqrt{\text{Var}_{b,c,t} + \epsilon}} \right) $$

$$ \text{vRMSE} = \frac{1}{N_b \cdot N_c \cdot N_t} \sum_{b=1}^{N_b} \sum_{c=1}^{N_c} \sum_{t=1}^{N_t} \left( \sqrt{\frac{\text{MSE}_{b,c,t}}{\text{Var}_{b,c,t} + \epsilon}} \right) $$

where the mean and variance are computed over the spatial dimensions in Mean$_{b,c,t}$ and Var$_{b,c,t}$, respectively. In other words, the mean and variance are computed for each sample, timestep, and channel independently. The vRMSE score is finally computed by taking a mean over the batch, channel, and temporal dimensions, yielding a single scalar value.
### G.1.6 Mean Energy Log Ratio (MELR) and Weighted Log Ratio (WLR)

Let $K$ be the set of wavenumbers (frequency bins) being evaluated, $|K|$ the cardinality of $K$, and $E_{\text{pred}}(k)$ and $E_{\text{ref}}(k)$ the energy of the predicted and reference (i.e., ground truth) spectrum at wavenumber $k$.

Wan et al. (2023) defines the MELR metric as,

$$ \text{MELR} = \sum_{k \in K} w_k \left| \log \left( \frac{E_{\text{pred}}(k)}{E_{\text{ref}}(k)} \right) \right|, \qquad (15) $$

with the energy spectrum being defined as,

$$ E(k) = \sum_{|\underline{k}|=k} |\hat{u}(\underline{k})|^2 = \sum_{|\underline{k}|=k} \left| \sum_{i} u(x_i) \exp(-j2\pi \underline{k} \cdot x_i / L) \right|^2 \qquad (16) $$

where $L$ is the length of the physical domain, $u$ is the snapshot of the system state, $\hat{u}(\underline{k})$ is the Fourier coefficient at wave-vector $\underline{k}$, $k$ is the magnitude of the wave-vector $\underline{k}$ (i.e., for 2D spatial data, $k = |\underline{k}| = \sqrt{k_x^2 + k_y^2}$).

Treating each wavenumber's log ratio equally and assuming $N_b$ samples in the test set, we arrive at the formula for the unweighted MELR or simply MELR,

$$ \text{MELR} = \frac{1}{N_b} \sum_{b=1}^{N_b} \left( \frac{1}{|K|} \sum_{k \in K} \left| \log \left( \frac{E_{\text{pred}}^{(b)}(k)}{E_{\text{ref}}^{(b)}(k)} \right) \right| \right) \qquad (17) $$

WLR is then the weighted version of MELR with the weights $w_k$ computed using the reference energy spectra,

$$ \text{WLR} = \frac{1}{N_b} \sum_{b=1}^{N_b} \left( \sum_{k \in K} \left( \frac{E_{\text{ref}}^{(b)}(k)}{\sum_{j \in K} E_{\text{ref}}^{(b)}(j)} \right) \left| \log \left( \frac{E_{\text{pred}}^{(b)}(k)}{E_{\text{ref}}^{(b)}(k)} \right) \right| \right) \qquad (18) $$

### G.1.7 Relative Percentage Difference (Rel. % Diff)

We report REL. % DIFF, which indicates the relative improvement (negative value) or degradation (positive value) of errors over the baseline. Denote the proposed model's score of a chosen evaluation metric (e.g., fRMSE) as $\hat{p}$ and the baseline model's score as $\hat{b}$. Then, the REL. % DIFF is defined as,

$$ \text{REL. % DIFF} := \frac{(\hat{p} - \hat{b})}{\hat{b}} \times 100 \qquad (19) $$
## G.2 Detailed Quantitative Results with Full Set of Metrics

### G.2.1 1-step Training and Evaluation

To expand on the abridged results presented in the main paper, we present the results with the full set of evaluation metrics for the setting of 1-step training of baselines and our proposed LOGLO-FNO model on four 2D PDE problems and one 3D PDE in the sections that follow.

**Kolmogorov Flow 2D.** Supplementing the results in the main paper, Table 8 shows the full set of evaluation metrics for 1-step and 5-step AR rollouts on Kolmogorov Flow test dataset.

Table 8: 1-step and 5-step AR evaluation of LOGLO-FNO compared with SOTA baselines on the test set of 2D Kolmogorov Flow (Li et al., 2022b). REL. % DIFF indicates improvement (-) or degradation (+) with respect to FNO. LOGLO-FNO uses 40 and (16, 9) modes in the global and local branches, respectively, whereas the width is set as 65. NO-LIDK\* denotes using only localized integral kernel, NO-LIDK\diamond means only differential kernel, and NO-LIDK\dagger means employing both. Transolver\* indicates a longer training time of the model for 500 epochs due to convergence issues at shorter training epochs of 136.

<table><thead><tr><th>Model</th><th>RMSE (↓)</th><th>nRMSE</th><th>bRMSE</th><th>cRMSE</th><th>fRMSE(L)</th><th>fRMSE(M)</th><th>fRMSE(H)</th><th>MaxError (↓)</th><th>MELR (↓)</th><th>WLR (↓)</th></tr></thead><tbody><tr><td colspan="11" style="text-align:center; font-weight:bold;">1-step Evaluation</td></tr><tr><td>U-Net</td><td>7.17 · 10<sup>-1</sup></td><td>1.3 · 10<sup>-1</sup></td><td>1.47 · 10<sup>0</sup></td><td>1.74 · 10<sup>-2</sup></td><td>2.24 · 10<sup>-2</sup></td><td>3.57 · 10<sup>-2</sup></td><td>4.39 · 10<sup>-2</sup></td><td>2.01 · 10<sup>1</sup></td><td>1.64 · 10<sup>-1</sup></td><td>1.48 · 10<sup>-2</sup></td></tr><tr><td>Transolver<sup>*</sup></td><td>7.69 · 10<sup>-1</sup></td><td>1.39 · 10<sup>-1</sup></td><td>1.42 · 10<sup>0</sup></td><td>7.65 · 10<sup>-3</sup></td><td>2.74 · 10<sup>-2</sup></td><td>4.48 · 10<sup>-2</sup></td><td>4.65 · 10<sup>-2</sup></td><td>1.52 · 10<sup>1</sup></td><td>1.43 · 10<sup>-1</sup></td><td>1.83 · 10<sup>-2</sup></td></tr><tr><td>FNO</td><td>8.08 · 10<sup>-1</sup></td><td>1.47 · 10<sup>-1</sup></td><td>7.94 · 10<sup>-1</sup></td><td>1.1 · 10<sup>-2</sup></td><td>1.36 · 10<sup>-2</sup></td><td>2.05 · 10<sup>-2</sup></td><td>4.7 · 10<sup>-2</sup></td><td>1.46 · 10<sup>1</sup></td><td>5.2 · 10<sup>-1</sup></td><td>2.83 · 10<sup>-2</sup></td></tr><tr><td>F-FNO</td><td>7.53 · 10<sup>-1</sup></td><td>1.37 · 10<sup>-1</sup></td><td>7.41 · 10<sup>-1</sup></td><td>1.5 · 10<sup>-2</sup></td><td>1.49 · 10<sup>-2</sup></td><td>2.15 · 10<sup>-2</sup></td><td>4.36 · 10<sup>-2</sup></td><td>1.42 · 10<sup>1</sup></td><td>4.74 · 10<sup>-1</sup></td><td>2.28 · 10<sup>-2</sup></td></tr><tr><td>LSM</td><td>7.49 · 10<sup>-1</sup></td><td>1.36 · 10<sup>-1</sup></td><td>1.47 · 10<sup>0</sup></td><td>1.36 · 10<sup>-2</sup></td><td>2.59 · 10<sup>-2</sup></td><td>4.42 · 10<sup>-2</sup></td><td>4.64 · 10<sup>-2</sup></td><td>2.05 · 10<sup>1</sup></td><td>1.43 · 10<sup>-1</sup></td><td>1.68 · 10<sup>-2</sup></td></tr><tr><td>U-FNO</td><td>6.13 · 10<sup>-1</sup></td><td>1.12 · 10<sup>-1</sup></td><td>1.0 · 10<sup>0</sup></td><td><b>7.09 · 10<sup>-3</sup></b></td><td>1.27 · 10<sup>-2</sup></td><td>2.22 · 10<sup>-2</sup></td><td>3.71 · 10<sup>-2</sup></td><td>1.64 · 10<sup>1</sup></td><td>1.38 · 10<sup>-1</sup></td><td>1.09 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK<sup>*</sup></td><td>7.25 · 10<sup>-1</sup></td><td>1.33 · 10<sup>-1</sup></td><td>1.12 · 10<sup>0</sup></td><td>9.05 · 10<sup>-3</sup></td><td>1.45 · 10<sup>-2</sup></td><td>2.69 · 10<sup>-2</sup></td><td>4.55 · 10<sup>-2</sup></td><td>1.65 · 10<sup>1</sup></td><td>1.85 · 10<sup>-1</sup></td><td>1.54 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK<sup>o</sup></td><td>6.13 · 10<sup>-1</sup></td><td>1.11 · 10<sup>-1</sup></td><td>5.95 · 10<sup>-1</sup></td><td>1.46 · 10<sup>-2</sup></td><td>1.65 · 10<sup>-2</sup></td><td>2.29 · 10<sup>-2</sup></td><td>3.91 · 10<sup>-2</sup></td><td>1.5 · 10<sup>1</sup></td><td>9.57 · 10<sup>-2</sup></td><td>1.1 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK<sup>†</sup></td><td>5.86 · 10<sup>-1</sup></td><td>1.07 · 10<sup>-1</sup></td><td><b>5.64 · 10<sup>-1</sup></b></td><td>1.05 · 10<sup>-2</sup></td><td>1.44 · 10<sup>-2</sup></td><td>2.46 · 10<sup>-2</sup></td><td>3.82 · 10<sup>-2</sup></td><td>1.47 · 10<sup>1</sup></td><td><b>7.11 · 10<sup>-2</sup></b></td><td><b>1.0 · 10<sup>-2</sup></b></td></tr><tr><td>LOGLO-FNO</td><td>5.89 · 10<sup>-1</sup></td><td><b>1.07 · 10<sup>-1</sup></b></td><td>6.74 · 10<sup>-1</sup></td><td><b>7.23 · 10<sup>-3</sup></b></td><td><b>1.21 · 10<sup>-2</sup></b></td><td><b>1.81 · 10<sup>-2</sup></b></td><td><b>3.54 · 10<sup>-2</sup></b></td><td><b>1.33 · 10<sup>1</sup></b></td><td>1.29 · 10<sup>-1</sup></td><td>1.06 · 10<sup>-2</sup></td></tr><tr><td>REL. % DIFF</td><td>-27.1 %</td><td>-27.21 %</td><td>-15.12 %</td><td>-34.31 %</td><td>-11.22 %</td><td>-11.88 %</td><td>-24.64 %</td><td>-8.99 %</td><td>-75.12 %</td><td>-62.63 %</td></tr><tr><td colspan="11" style="text-align:center; font-weight:bold;">5-step Autoregressive Evaluation</td></tr><tr><td>U-Net</td><td>1.51 · 10<sup>0</sup></td><td>2.65 · 10<sup>-1</sup></td><td>2.31 · 10<sup>0</sup></td><td>5.39 · 10<sup>-2</sup></td><td>6.53 · 10<sup>-2</sup></td><td>1.04 · 10<sup>-1</sup></td><td>9.38 · 10<sup>-2</sup></td><td>1.81 · 10<sup>1</sup></td><td>2.13 · 10<sup>-1</sup></td><td>3.52 · 10<sup>-2</sup></td></tr><tr><td>Transolver<sup>*</sup></td><td>1.91 · 10<sup>0</sup></td><td>3.36 · 10<sup>-1</sup></td><td>2.59 · 10<sup>0</sup></td><td><b>6.12 · 10<sup>-3</sup></b></td><td>7.44 · 10<sup>-2</sup></td><td>1.51 · 10<sup>-1</sup></td><td>1.18 · 10<sup>-1</sup></td><td>2.36 · 10<sup>1</sup></td><td>2.45 · 10<sup>-1</sup></td><td>4.92 · 10<sup>-2</sup></td></tr><tr><td>FNO</td><td>1.33 · 10<sup>0</sup></td><td>2.35 · 10<sup>-1</sup></td><td>1.34 · 10<sup>0</sup></td><td>1.46 · 10<sup>-2</sup></td><td>3.37 · 10<sup>-2</sup></td><td>5.80 · 10<sup>-2</sup></td><td>8.37 · 10<sup>-2</sup></td><td>1.60 · 10<sup>1</sup></td><td>6.18 · 10<sup>-1</sup></td><td>4.93 · 10<sup>-2</sup></td></tr><tr><td>F-FNO</td><td>1.29 · 10<sup>0</sup></td><td>2.28 · 10<sup>-1</sup></td><td>1.27 · 10<sup>0</sup></td><td>2.28 · 10<sup>-2</sup></td><td>3.60 · 10<sup>-2</sup></td><td>5.52 · 10<sup>-2</sup></td><td>8.12 · 10<sup>-2</sup></td><td>1.50 · 10<sup>1</sup></td><td>5.37 · 10<sup>-1</sup></td><td>3.99 · 10<sup>-2</sup></td></tr><tr><td>LSM</td><td>1.81 · 10<sup>0</sup></td><td>3.18 · 10<sup>-1</sup></td><td>2.76 · 10<sup>0</sup></td><td>3.87 · 10<sup>-2</sup></td><td>7.6 · 10<sup>-2</sup></td><td>1.44 · 10<sup>-1</sup></td><td>1.12 · 10<sup>-1</sup></td><td>2.08 · 10<sup>1</sup></td><td>2.04 · 10<sup>-1</sup></td><td>4.39 · 10<sup>-2</sup></td></tr><tr><td>U-FNO</td><td>1.15 · 10<sup>0</sup></td><td>2.03 · 10<sup>-1</sup></td><td>1.45 · 10<sup>0</sup></td><td>6.3 · 10<sup>-3</sup></td><td><b>2.69 · 10<sup>-2</sup></b></td><td>5.33 · 10<sup>-2</sup></td><td>7.35 · 10<sup>-2</sup></td><td>1.56 · 10<sup>1</sup></td><td>2.01 · 10<sup>-1</sup></td><td>2.51 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK<sup>*</sup></td><td>1.36 · 10<sup>0</sup></td><td>2.39 · 10<sup>-1</sup></td><td>1.8 · 10<sup>0</sup></td><td>1.1 · 10<sup>-2</sup></td><td>3.14 · 10<sup>-2</sup></td><td>6.86 · 10<sup>-2</sup></td><td>8.91 · 10<sup>-2</sup></td><td>1.59 · 10<sup>1</sup></td><td>2.23 · 10<sup>-1</sup></td><td>2.93 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK<sup>o</sup></td><td>1.17 · 10<sup>0</sup></td><td>2.04 · 10<sup>-1</sup></td><td>1.12 · 10<sup>0</sup></td><td>1.92 · 10<sup>-2</sup></td><td>3.71 · 10<sup>-2</sup></td><td>5.96 · 10<sup>-2</sup></td><td>7.6 · 10<sup>-2</sup></td><td>1.61 · 10<sup>1</sup></td><td>1.42 · 10<sup>-1</sup></td><td>2.12 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK<sup>†</sup></td><td>1.17 · 10<sup>0</sup></td><td>2.05 · 10<sup>-1</sup></td><td>1.14 · 10<sup>0</sup></td><td>1.23 · 10<sup>-2</sup></td><td>3.31 · 10<sup>-2</sup></td><td>5.94 · 10<sup>-2</sup></td><td>7.74 · 10<sup>-2</sup></td><td>1.56 · 10<sup>1</sup></td><td><b>1.03 · 10<sup>-1</sup></b></td><td><b>2.18 · 10<sup>-2</sup></b></td></tr><tr><td>LOGLO-FNO</td><td><b>1.09 · 10<sup>0</sup></b></td><td><b>1.92 · 10<sup>-1</sup></b></td><td><b>1.12 · 10<sup>0</sup></b></td><td>8.99 · 10<sup>-3</sup></td><td><b>2.80 · 10<sup>-2</sup></b></td><td><b>4.55 · 10<sup>-2</sup></b></td><td><b>6.93 · 10<sup>-2</sup></b></td><td><b>1.26 · 10<sup>1</sup></b></td><td>1.67 · 10<sup>-1</sup></td><td><b>2.07 · 10<sup>-2</sup></b></td></tr><tr><td>REL. % DIFF</td><td>-18.26 %</td><td>-18.39 %</td><td>-16.12 %</td><td>-38.21 %</td><td>-16.86 %</td><td>-21.62 %</td><td>-17.26 %</td><td>-21.31 %</td><td>-73.04 %</td><td>-58.02 %</td></tr></tbody></table>
**CNS Turb 2D.** Table 9 presents the full set of evaluation metrics for 1-step and 5-step autoregressive rollouts on the test split of CNS Turb 2D dataset.

<table><caption>Table 9: 1-step and 5-step AR evaluation of LOGLO-FNO compared with SOTA baselines on the test set of CNS Turb 2D (Takamoto et al., 2022). REL. % DIFF indicates improvement (-) or degradation (+) with respect to FNO. LOGLO-FNO uses 40 and (16, 9) modes in the global and local branches, respectively, whereas the width is set as 65. NO-LIDK\* denotes using only localized integral kernel, NO-LIDK\diamond means only differential kernel, and NO-LIDK\dagger means employing both.</caption><thead><tr><th>Model</th><th>RMSE (↓)</th><th>nRMSE</th><th>bRMSE</th><th>cRMSE</th><th>fRMSE(L)</th><th>fRMSE(M)</th><th>fRMSE(H)</th><th>MaxError (↓)</th><th>MELR (↓)</th><th>WLR (↓)</th></tr></thead><tbody><tr><td colspan="11" style="text-align:center; font-weight:bold;">1-step Evaluation</td></tr><tr><td>U-Net</td><td>3.73 · 10<sup>-2</sup></td><td>4.9 · 10<sup>-2</sup></td><td>9.27 · 10<sup>-2</sup></td><td>3.06 · 10<sup>-3</sup></td><td>2.47 · 10<sup>-3</sup></td><td>2.9 · 10<sup>-3</sup></td><td>1.48 · 10<sup>-3</sup></td><td>2.1 · 10<sup>0</sup></td><td>1.37 · 10<sup>-1</sup></td><td>7.39 · 10<sup>-3</sup></td></tr><tr><td>Transolver*</td><td>3.13 · 10<sup>-1</sup></td><td>4.22 · 10<sup>-1</sup></td><td>3.08 · 10<sup>-1</sup></td><td>1.1 · 10<sup>-2</sup></td><td>4.77 · 10<sup>-2</sup></td><td>4.11 · 10<sup>-2</sup></td><td>4.47 · 10<sup>-3</sup></td><td>3.66 · 10<sup>0</sup></td><td>5.01 · 10<sup>-1</sup></td><td>3.33 · 10<sup>-1</sup></td></tr><tr><td>FNO</td><td>3.69 · 10<sup>-2</sup></td><td>4.82 · 10<sup>-2</sup></td><td>3.64 · 10<sup>-2</sup></td><td>1.42 · 10<sup>-3</sup></td><td>1.28 · 10<sup>-3</sup></td><td>1.78 · 10<sup>-3</sup></td><td>1.57 · 10<sup>-3</sup></td><td>1.96 · 10<sup>0</sup></td><td>2.78 · 10<sup>-1</sup></td><td>7.8 · 10<sup>-3</sup></td></tr><tr><td>LSM</td><td>6.11 · 10<sup>-2</sup></td><td>8.08 · 10<sup>-2</sup></td><td>1.67 · 10<sup>-1</sup></td><td>3.83 · 10<sup>-3</sup></td><td>6.17 · 10<sup>-3</sup></td><td>6.55 · 10<sup>-3</sup></td><td>2.01 · 10<sup>-3</sup></td><td>2.43 · 10<sup>0</sup></td><td>2.7 · 10<sup>-1</sup></td><td>1.66 · 10<sup>-2</sup></td></tr><tr><td>U-FNO</td><td>3.43 · 10<sup>-2</sup></td><td>4.49 · 10<sup>-2</sup></td><td>4.92 · 10<sup>-2</sup></td><td>3.42 · 10<sup>-3</sup></td><td>2.24 · 10<sup>-3</sup></td><td>2.01 · 10<sup>-3</sup></td><td>1.43 · 10<sup>-3</sup></td><td>2.0 · 10<sup>0</sup></td><td><b>1.32</b> · 10<sup>-1</sup></td><td>9.5 · 10<sup>-3</sup></td></tr><tr><td>NO-LIDK*</td><td>4.81 · 10<sup>-2</sup></td><td>6.4 · 10<sup>-2</sup></td><td>4.77 · 10<sup>-2</sup></td><td>1.72 · 10<sup>-3</sup></td><td>1.5 · 10<sup>-3</sup></td><td>2.14 · 10<sup>-3</sup></td><td>1.98 · 10<sup>-3</sup></td><td>2.35 · 10<sup>0</sup></td><td>2.98 · 10<sup>-1</sup></td><td>1.31 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK&deg;</td><td>4.18 · 10<sup>-2</sup></td><td>5.54 · 10<sup>-2</sup></td><td>4.13 · 10<sup>-2</sup></td><td>1.46 · 10<sup>-3</sup></td><td>1.35 · 10<sup>-3</sup></td><td>1.88 · 10<sup>-3</sup></td><td>1.76 · 10<sup>-3</sup></td><td>2.28 · 10<sup>0</sup></td><td>2.0 · 10<sup>-1</sup></td><td>9.49 · 10<sup>-3</sup></td></tr><tr><td>NO-LIDK&dagger;</td><td>4.2 · 10<sup>-2</sup></td><td>5.57 · 10<sup>-2</sup></td><td>4.16 · 10<sup>-2</sup></td><td>1.56 · 10<sup>-3</sup></td><td>1.37 · 10<sup>-3</sup></td><td>1.89 · 10<sup>-3</sup></td><td>1.77 · 10<sup>-3</sup></td><td>2.26 · 10<sup>0</sup></td><td>2.08 · 10<sup>-1</sup></td><td>9.54 · 10<sup>-3</sup></td></tr><tr><td><b>LOGLO-FNO</b></td><td><b>3.23 · 10<sup>-2</sup></b></td><td><b>4.24 · 10<sup>-2</sup></b></td><td><b>3.34 · 10<sup>-2</sup></b></td><td><b>1.24 · 10<sup>-3</sup></b></td><td><b>9.01 · 10<sup>-4</sup></b></td><td><b>1.1 · 10<sup>-3</sup></b></td><td><b>1.37 · 10<sup>-3</sup></b></td><td><b>1.86 · 10<sup>0</sup></b></td><td><b>1.71 · 10<sup>-1</sup></b></td><td><b>5.5 · 10<sup>-3</sup></b></td></tr><tr><td>REL. % DIFF</td><td>-12.31%</td><td>-11.91%</td><td>-8.3%</td><td>-12.54%</td><td>-29.57%</td><td>-38.1%</td><td>-12.38%</td><td>-4.79%</td><td>-38.39%</td><td>-29.53%</td></tr><tr><td colspan="11" style="text-align:center; font-weight:bold;">5-step Autoregressive Evaluation</td></tr><tr><td>U-Net</td><td>1.15 · 10<sup>-1</sup></td><td>1.44 · 10<sup>-1</sup></td><td>1.98 · 10<sup>-1</sup></td><td>7.13 · 10<sup>-3</sup></td><td>1.42 · 10<sup>-2</sup></td><td>1.3 · 10<sup>-2</sup></td><td>3.08 · 10<sup>-3</sup></td><td>2.87 · 10<sup>0</sup></td><td>1.37 · 10<sup>-1</sup></td><td>2.13 · 10<sup>-2</sup></td></tr><tr><td>Transolver*</td><td>5.29 · 10<sup>-1</sup></td><td>6.69 · 10<sup>-1</sup></td><td>5.3 · 10<sup>-1</sup></td><td>2.7 · 10<sup>-2</sup></td><td>9.21 · 10<sup>-2</sup></td><td>6.0 · 10<sup>-2</sup></td><td>4.7 · 10<sup>-3</sup></td><td>4.23 · 10<sup>0</sup></td><td>3.54 · 10<sup>0</sup></td><td>1.25 · 10<sup>0</sup></td></tr><tr><td>FNO</td><td>6.34 · 10<sup>-2</sup></td><td>7.78 · 10<sup>-2</sup></td><td>6.21 · 10<sup>-2</sup></td><td>2.45 · 10<sup>-3</sup></td><td>3.76 · 10<sup>-3</sup></td><td>5.37 · 10<sup>-3</sup></td><td>2.4 · 10<sup>-3</sup></td><td>2.45 · 10<sup>0</sup></td><td>2.22 · 10<sup>-1</sup></td><td>9.36 · 10<sup>-3</sup></td></tr><tr><td>LSM</td><td>2.12 · 10<sup>-1</sup></td><td>2.67 · 10<sup>-1</sup></td><td>3.74 · 10<sup>-1</sup></td><td>1.91 · 10<sup>-2</sup></td><td>3.4 · 10<sup>-2</sup></td><td>2.5 · 10<sup>-2</sup></td><td>4.16 · 10<sup>-3</sup></td><td>3.16 · 10<sup>0</sup></td><td>2.68 · 10<sup>-1</sup></td><td>5.34 · 10<sup>-2</sup></td></tr><tr><td>U-FNO</td><td>6.84 · 10<sup>-2</sup></td><td>8.42 · 10<sup>-2</sup></td><td>7.91 · 10<sup>-2</sup></td><td>5.46 · 10<sup>-3</sup></td><td>5.38 · 10<sup>-3</sup></td><td>5.9 · 10<sup>-3</sup></td><td>2.54 · 10<sup>-3</sup></td><td>2.57 · 10<sup>0</sup></td><td><b>1.13</b> · 10<sup>-1</sup></td><td>1.49 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK*</td><td>6.95 · 10<sup>-2</sup></td><td>8.55 · 10<sup>-2</sup></td><td>6.86 · 10<sup>-2</sup></td><td>2.83 · 10<sup>-3</sup></td><td>4.14 · 10<sup>-3</sup></td><td>5.92 · 10<sup>-3</sup></td><td>2.62 · 10<sup>-3</sup></td><td>2.83 · 10<sup>0</sup></td><td>2.06 · 10<sup>-1</sup></td><td>1.15 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK&deg;</td><td>6.41 · 10<sup>-2</sup></td><td>7.88 · 10<sup>-2</sup></td><td>6.32 · 10<sup>-2</sup></td><td>2.64 · 10<sup>-3</sup></td><td>3.82 · 10<sup>-3</sup></td><td>5.35 · 10<sup>-3</sup></td><td>2.44 · 10<sup>-3</sup></td><td>2.49 · 10<sup>0</sup></td><td>1.37 · 10<sup>-1</sup></td><td>9.57 · 10<sup>-3</sup></td></tr><tr><td>NO-LIDK&dagger;</td><td>6.44 · 10<sup>-2</sup></td><td>7.91 · 10<sup>-2</sup></td><td>6.35 · 10<sup>-2</sup></td><td>2.63 · 10<sup>-3</sup></td><td>3.82 · 10<sup>-3</sup></td><td>5.38 · 10<sup>-3</sup></td><td>2.47 · 10<sup>-3</sup></td><td>2.57 · 10<sup>0</sup></td><td>1.45 · 10<sup>-1</sup></td><td>1.01 · 10<sup>-2</sup></td></tr><tr><td><b>LOGLO-FNO</b></td><td><b>5.43 · 10<sup>-2</sup></b></td><td><b>6.67 · 10<sup>-2</sup></b></td><td><b>5.48 · 10<sup>-2</sup></b></td><td><b>2.1 · 10<sup>-3</sup></b></td><td><b>2.76 · 10<sup>-3</sup></b></td><td><b>3.97 · 10<sup>-3</sup></b></td><td><b>2.12 · 10<sup>-3</sup></b></td><td><b>2.25 · 10<sup>0</sup></b></td><td><b>1.29 · 10<sup>-1</sup></b></td><td><b>6.67 · 10<sup>-3</sup></b></td></tr><tr><td>REL. % DIFF</td><td>-14.35%</td><td>-14.31%</td><td>-11.8%</td><td>-14.23%</td><td>-26.7%</td><td>-26.09%</td><td>-11.42%</td><td>-8.09%</td><td>-41.8%</td><td>-28.75%</td></tr></tbody></table>
**Wave-Gauss 2D.** In Table 10, we present results with the full set of evaluation metrics on the test set of Wave-Gauss 2D dataset.

<table><caption>Table 10: 1-step, 5-step, and 15-step AR evaluation of LOGLO-FNO compared against SOTA baselines on the test set of Wave-Gauss 2D (Herde et al., 2024). REL. % DIFF indicates improvement (-) or degradation (+) with respect to FNO. LOGLO-FNO uses 20 and (16, 9) modes in the global and local branches, respectively, whereas the width is set to 32. NO-LIDK* denotes using only localized integral kernel, NO-LIDK◇ means only differential kernel, and NO-LIDK† means employing both.</caption><thead><tr><th>Model</th><th>RMSE (↓)</th><th>nRMSE</th><th>bRMSE</th><th>cRMSE</th><th>fRMSE(L)</th><th>fRMSE(M)</th><th>fRMSE(H)</th><th>MaxError (↓)</th><th>MELR (↓)</th><th>WLR (↓)</th></tr></thead><tbody><tr><td colspan="11" style="text-align: center; font-weight: bold;">1-step Evaluation</td></tr><tr><td>Transolver*</td><td>2.01 · 10<sup>-1</sup></td><td>2.29 · 10<sup>-1</sup></td><td>7.29 · 10<sup>-2</sup></td><td>1.97 · 10<sup>-2</sup></td><td>3.51 · 10<sup>-2</sup></td><td>2.76 · 10<sup>-2</sup></td><td>2.2 · 10<sup>-3</sup></td><td>2.94 · 10<sup>0</sup></td><td>1.29 · 10<sup>0</sup></td><td>1.1 · 10<sup>-1</sup></td></tr><tr><td>FNO</td><td>3.76 · 10<sup>-2</sup></td><td>4.03 · 10<sup>-2</sup></td><td>1.91 · 10<sup>-2</sup></td><td>2.82 · 10<sup>-3</sup></td><td>7.5 · 10<sup>-3</sup></td><td>5.45 · 10<sup>-3</sup></td><td>6.22 · 10<sup>-4</sup></td><td>1.31 · 10<sup>0</sup></td><td>9.66 · 10<sup>-1</sup></td><td>9.01 · 10<sup>-3</sup></td></tr><tr><td>LSM</td><td><b>2.83</b> · 10<sup>-2</sup></td><td><b>3.08</b> · 10<sup>-2</sup></td><td>3.04 · 10<sup>-2</sup></td><td>2.69 · 10<sup>-3</sup></td><td><b>4.91</b> · 10<sup>-3</sup></td><td>3.86 · 10<sup>-3</sup></td><td>8.54 · 10<sup>-4</sup></td><td><b>9.99</b> · 10<sup>-1</sup></td><td>1.2 · 10<sup>0</sup></td><td><b>6.2</b> · 10<sup>-3</sup></td></tr><tr><td>U-FNO</td><td>3.23 · 10<sup>-2</sup></td><td>3.49 · 10<sup>-2</sup></td><td>2.09 · 10<sup>-2</sup></td><td>3.89 · 10<sup>-3</sup></td><td>6.62 · 10<sup>-3</sup></td><td>4.52 · 10<sup>-3</sup></td><td>7.24 · 10<sup>-4</sup></td><td>1.2 · 10<sup>0</sup></td><td>1.41 · 10<sup>0</sup></td><td>6.82 · 10<sup>-3</sup></td></tr><tr><td>NO-LIDK*</td><td>3.62 · 10<sup>-2</sup></td><td>3.93 · 10<sup>-2</sup></td><td>1.91 · 10<sup>-2</sup></td><td>2.94 · 10<sup>-3</sup></td><td>6.95 · 10<sup>-3</sup></td><td>5.1 · 10<sup>-3</sup></td><td>8.22 · 10<sup>-4</sup></td><td>1.2 · 10<sup>0</sup></td><td>1.27 · 10<sup>0</sup></td><td>8.28 · 10<sup>-3</sup></td></tr><tr><td>NO-LIDK°</td><td>3.33 · 10<sup>-2</sup></td><td>3.58 · 10<sup>-2</sup></td><td>1.68 · 10<sup>-2</sup></td><td><b>2.57</b> · 10<sup>-3</sup></td><td>6.49 · 10<sup>-3</sup></td><td>4.78 · 10<sup>-3</sup></td><td>7.56 · 10<sup>-4</sup></td><td>1.3 · 10<sup>0</sup></td><td>1.31 · 10<sup>0</sup></td><td>7.32 · 10<sup>-3</sup></td></tr><tr><td>NO-LIDK†</td><td>3.25 · 10<sup>-2</sup></td><td>3.5 · 10<sup>-2</sup></td><td>1.68 · 10<sup>-2</sup></td><td>2.7 · 10<sup>-3</sup></td><td>6.31 · 10<sup>-3</sup></td><td>4.61 · 10<sup>-3</sup></td><td>7.4 · 10<sup>-4</sup></td><td>1.19 · 10<sup>0</sup></td><td>1.28 · 10<sup>0</sup></td><td>7.04 · 10<sup>-3</sup></td></tr><tr><td><b>LOGLO-FNO</b></td><td><b>2.9</b> · 10<sup>-2</sup></td><td><b>3.11</b> · 10<sup>-2</sup></td><td><b>1.17</b> · 10<sup>-2</sup></td><td>2.75 · 10<sup>-3</sup></td><td><b>6.43</b> · 10<sup>-3</sup></td><td><b>3.84</b> · 10<sup>-3</sup></td><td><b>3.44</b> · 10<sup>-4</sup></td><td>1.12 · 10<sup>0</sup></td><td><b>9.29</b> · 10<sup>-1</sup></td><td>6.82 · 10<sup>-3</sup></td></tr><tr><td>REL. % DIFF</td><td>-22.97%</td><td>-22.79%</td><td>-38.59%</td><td>-2.55%</td><td>-14.27%</td><td>-29.54%</td><td>-44.65%</td><td>-14.79%</td><td>-3.77%</td><td>-24.33%</td></tr><tr><td colspan="11" style="text-align: center; font-weight: bold;">5-step Autoregressive Evaluation</td></tr><tr><td>Transolver*</td><td>3.45 · 10<sup>-1</sup></td><td>3.62 · 10<sup>-1</sup></td><td>1.31 · 10<sup>-1</sup></td><td>6.09 · 10<sup>-2</sup></td><td>7.73 · 10<sup>-2</sup></td><td>3.47 · 10<sup>-2</sup></td><td>2.93 · 10<sup>-3</sup></td><td>3.66 · 10<sup>0</sup></td><td>2.07 · 10<sup>0</sup></td><td>1.79 · 10<sup>-1</sup></td></tr><tr><td>FNO</td><td>6.91 · 10<sup>-2</sup></td><td>6.75 · 10<sup>-2</sup></td><td>2.06 · 10<sup>-2</sup></td><td>6.5 · 10<sup>-3</sup></td><td>1.6 · 10<sup>-2</sup></td><td>8.35 · 10<sup>-3</sup></td><td>8.57 · 10<sup>-4</sup></td><td>2.23 · 10<sup>0</sup></td><td>1.5 · 10<sup>0</sup></td><td>1.93 · 10<sup>-2</sup></td></tr><tr><td>LSM</td><td><b>4.33</b> · 10<sup>-2</sup></td><td><b>4.34</b> · 10<sup>-2</sup></td><td>3.63 · 10<sup>-2</sup></td><td><b>5.48</b> · 10<sup>-3</sup></td><td><b>8.64</b> · 10<sup>-3</sup></td><td>5.68 · 10<sup>-3</sup></td><td>1.1 · 10<sup>-3</sup></td><td>2.12 · 10<sup>0</sup></td><td>1.72 · 10<sup>0</sup></td><td><b>1.24</b> · 10<sup>-2</sup></td></tr><tr><td>U-FNO</td><td>6.37 · 10<sup>-2</sup></td><td>6.26 · 10<sup>-2</sup></td><td>2.55 · 10<sup>-2</sup></td><td>1.44 · 10<sup>-2</sup></td><td>1.6 · 10<sup>-2</sup></td><td>7.64 · 10<sup>-3</sup></td><td>9.27 · 10<sup>-4</sup></td><td>2.54 · 10<sup>0</sup></td><td>2.0 · 10<sup>0</sup></td><td>1.82 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK*</td><td>6.05 · 10<sup>-2</sup></td><td>5.96 · 10<sup>-2</sup></td><td>1.91 · 10<sup>-2</sup></td><td>7.76 · 10<sup>-3</sup></td><td>1.34 · 10<sup>-2</sup></td><td>7.92 · 10<sup>-3</sup></td><td>1.19 · 10<sup>-3</sup></td><td>2.65 · 10<sup>0</sup></td><td>1.99 · 10<sup>0</sup></td><td>1.81 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK°</td><td>1.17 · 10<sup>-1</sup></td><td>1.21 · 10<sup>-1</sup></td><td>2.23 · 10<sup>-2</sup></td><td>1.14 · 10<sup>-2</sup></td><td>2.43 · 10<sup>-2</sup></td><td>1.12 · 10<sup>-2</sup></td><td>4.07 · 10<sup>-3</sup></td><td>2.96 · 10<sup>0</sup></td><td>2.42 · 10<sup>0</sup></td><td>4.02 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK†</td><td>1.02 · 10<sup>-1</sup></td><td>1.06 · 10<sup>-1</sup></td><td>2.16 · 10<sup>-2</sup></td><td>1.38 · 10<sup>-2</sup></td><td>2.1 · 10<sup>-2</sup></td><td>1.02 · 10<sup>-2</sup></td><td>3.85 · 10<sup>-3</sup></td><td>2.95 · 10<sup>0</sup></td><td>2.41 · 10<sup>0</sup></td><td>3.02 · 10<sup>-2</sup></td></tr><tr><td><b>LOGLO-FNO</b></td><td><b>5.1</b> · 10<sup>-2</sup></td><td><b>4.81</b> · 10<sup>-2</sup></td><td><b>1.12</b> · 10<sup>-2</sup></td><td><b>6.35</b> · 10<sup>-3</sup></td><td><b>1.29</b> · 10<sup>-2</sup></td><td><b>5.43</b> · 10<sup>-3</sup></td><td><b>4.57</b> · 10<sup>-4</sup></td><td><b>1.62</b> · 10<sup>0</sup></td><td><b>1.48</b> · 10<sup>0</sup></td><td><b>1.39</b> · 10<sup>-2</sup></td></tr><tr><td>REL. % DIFF</td><td>-26.2%</td><td>-28.76%</td><td>-45.43%</td><td>-2.41%</td><td>-19.52%</td><td>-34.91%</td><td>-46.6%</td><td>-27.16%</td><td>-1.74%</td><td>-27.85%</td></tr><tr><td colspan="11" style="text-align: center; font-weight: bold;">15-step Autoregressive Evaluation</td></tr><tr><td>Transolver*</td><td>4.33 · 10<sup>-1</sup></td><td>4.96 · 10<sup>-1</sup></td><td>2.11 · 10<sup>-1</sup></td><td>1.2 · 10<sup>-1</sup></td><td>1.15 · 10<sup>-1</sup></td><td>3.64 · 10<sup>-2</sup></td><td>3.5 · 10<sup>-3</sup></td><td>3.84 · 10<sup>0</sup></td><td>1.73 · 10<sup>0</sup></td><td>2.36 · 10<sup>-1</sup></td></tr><tr><td>FNO</td><td>1.21 · 10<sup>-1</sup></td><td>1.34 · 10<sup>-1</sup></td><td>3.86 · 10<sup>-2</sup></td><td>1.76 · 10<sup>-2</sup></td><td>3.13 · 10<sup>-2</sup></td><td>1.32 · 10<sup>-2</sup></td><td>1.39 · 10<sup>-3</sup></td><td>2.42 · 10<sup>0</sup></td><td>1.16 · 10<sup>0</sup></td><td>3.53 · 10<sup>-2</sup></td></tr><tr><td>LSM</td><td><b>7.66</b> · 10<sup>-2</sup></td><td><b>8.49</b> · 10<sup>-2</sup></td><td>5.72 · 10<sup>-2</sup></td><td><b>1.29</b> · 10<sup>-2</sup></td><td><b>1.79</b> · 10<sup>-2</sup></td><td>9.33 · 10<sup>-3</sup></td><td>1.61 · 10<sup>-3</sup></td><td>1.98 · 10<sup>0</sup></td><td>1.29 · 10<sup>0</sup></td><td>2.63 · 10<sup>-2</sup></td></tr><tr><td>U-FNO</td><td>1.15 · 10<sup>-1</sup></td><td>1.27 · 10<sup>-1</sup></td><td>4.4 · 10<sup>-2</sup></td><td>3.08 · 10<sup>-2</sup></td><td>3.14 · 10<sup>-2</sup></td><td>1.24 · 10<sup>-2</sup></td><td>1.17 · 10<sup>-3</sup></td><td>2.67 · 10<sup>0</sup></td><td>1.54 · 10<sup>0</sup></td><td>3.32 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK*</td><td>9.98 · 10<sup>-2</sup></td><td>1.13 · 10<sup>-1</sup></td><td>3.14 · 10<sup>-2</sup></td><td>1.81 · 10<sup>-2</sup></td><td>2.61 · 10<sup>-2</sup></td><td>1.21 · 10<sup>-2</sup></td><td>1.38 · 10<sup>-3</sup></td><td>3.09 · 10<sup>0</sup></td><td>1.57 · 10<sup>0</sup></td><td>3.22 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK°</td><td>1.14 · 10<sup>0</sup></td><td>1.48 · 10<sup>0</sup></td><td>4.61 · 10<sup>-1</sup></td><td>4.2 · 10<sup>-1</sup></td><td>2.88 · 10<sup>-1</sup></td><td>5.88 · 10<sup>-2</sup></td><td>4.24 · 10<sup>-2</sup></td><td>1.65 · 10<sup>1</sup></td><td>3.2 · 10<sup>0</sup></td><td>3.0 · 10<sup>-1</sup></td></tr><tr><td>NO-LIDK†</td><td>7.89 · 10<sup>-1</sup></td><td>1.03 · 10<sup>0</sup></td><td>3.1 · 10<sup>-1</sup></td><td>1.77 · 10<sup>-1</sup></td><td>1.75 · 10<sup>-1</sup></td><td>6.09 · 10<sup>-2</sup></td><td>2.93 · 10<sup>-2</sup></td><td>1.94 · 10<sup>1</sup></td><td>3.09 · 10<sup>0</sup></td><td>2.06 · 10<sup>-1</sup></td></tr><tr><td><b>LOGLO-FNO</b></td><td><b>9.19</b> · 10<sup>-2</sup></td><td><b>1.02</b> · 10<sup>-1</sup></td><td><b>2.25</b> · 10<sup>-2</sup></td><td><b>1.58</b> · 10<sup>-2</sup></td><td><b>2.61</b> · 10<sup>-2</sup></td><td><b>9.24</b> · 10<sup>-3</sup></td><td><b>7.38</b> · 10<sup>-4</sup></td><td><b>1.86</b> · 10<sup>0</sup></td><td><b>1.04</b> · 10<sup>0</sup></td><td><b>2.54</b> · 10<sup>-2</sup></td></tr><tr><td>REL. % DIFF</td><td>-23.83%</td><td>-24.5%</td><td>-41.61%</td><td>-10.45%</td><td>-16.84%</td><td>-30.19%</td><td>-46.99%</td><td>-23.46%</td><td>-10.16%</td><td>-28.0%</td></tr></tbody></table>
**Compressible Euler four-quadrant Riemann problem 2D.** In Table 11, we present the preliminary 1-step and varying timesteps of rollout evaluation results from the experiments we conducted on the highly challenging Compressible Euler four-quadrant Riemann problem 2D problem, which exhibits shock waves and discontinuities.

<table><caption>Table 11: 1-step and varying timesteps (5, 10, 15, 20) autoregressive rollout evaluations of LOGLO-FNO compared against SOTA baselines on the test set of Compressible Euler four-quadrant Riemann problem 2D (Herde et al., 2024). REL. % DIFF indicates improvement (-) or degradation (+) with respect to FNO. LOGLO-FNO uses 40 and (16, 9) modes in the global and local branches, respectively, whereas the width is set to 65. NO-LIDK\* denotes using only the localized integral kernel with a radius cutoff value of 0.0078125, and NO-LIDK\circ means only the differential kernel.</caption><thead><tr><th>Model</th><th>RMSE (↓)</th><th>nRMSE</th><th>bRMSE</th><th>cRMSE</th><th>fRMSE(L)</th><th>fRMSE(M)</th><th>fRMSE(H)</th><th>MaxError (↓)</th><th>MELR (↓)</th><th>WLR (↓)</th></tr></thead><tbody><tr><td colspan="11" style="text-align: center; font-weight: bold;">1-step Evaluation</td></tr><tr><td>FNO</td><td>2.02 · 10<sup>-2</sup></td><td>2.08 · 10<sup>-2</sup></td><td>2.38 · 10<sup>-2</sup></td><td>8.46 · 10<sup>-4</sup></td><td>9.42 · 10<sup>-4</sup></td><td>1.03 · 10<sup>-3</sup></td><td>1.41 · 10<sup>-3</sup></td><td>1.58 · 10<sup>0</sup></td><td>8.8 · 10<sup>-2</sup></td><td>3.09 · 10<sup>-3</sup></td></tr><tr><td>NO-LIDK\*</td><td>3.56 · 10<sup>-2</sup></td><td>3.58 · 10<sup>-2</sup></td><td>3.94 · 10<sup>-2</sup></td><td>1.14 · 10<sup>-3</sup></td><td>1.5 · 10<sup>-3</sup></td><td>2.35 · 10<sup>-3</sup></td><td>2.47 · 10<sup>-3</sup></td><td>2.35 · 10<sup>0</sup></td><td>7.26 · 10<sup>-2</sup></td><td>5.67 · 10<sup>-3</sup></td></tr><tr><td>NO-LIDK\circ</td><td>1.59 · 10<sup>-2</sup></td><td>1.66 · 10<sup>-2</sup></td><td>1.83 · 10<sup>-2</sup></td><td>1.47 · 10<sup>-3</sup></td><td>1.05 · 10<sup>-3</sup></td><td>8.54 · 10<sup>-4</sup></td><td>1.11 · 10<sup>-3</sup></td><td>1.4 · 10<sup>0</sup></td><td>4.34 · 10<sup>-2</sup></td><td>2.54 · 10<sup>-3</sup></td></tr><tr><td>LOGLO-FNO</td><td>1.48 · 10<sup>-2</sup></td><td>1.5 · 10<sup>-2</sup></td><td>1.87 · 10<sup>-2</sup></td><td>6.36 · 10<sup>-4</sup></td><td>5.79 · 10<sup>-4</sup></td><td>5.09 · 10<sup>-4</sup></td><td>9.67 · 10<sup>-4</sup></td><td>1.31 · 10<sup>0</sup></td><td>5.42 · 10<sup>-2</sup></td><td>1.75 · 10<sup>-3</sup></td></tr><tr><td>REL. % DIFF</td><td>-26.65%</td><td>-27.9%</td><td>-21.29%</td><td>-24.82%</td><td>-38.56%</td><td>-50.71%</td><td>-31.58%</td><td>-16.9%</td><td>-38.45%</td><td>-43.26%</td></tr><tr><td colspan="11" style="text-align: center; font-weight: bold;">5-step Autoregressive Evaluation</td></tr><tr><td>FNO</td><td>3.39 · 10<sup>-2</sup></td><td>3.43 · 10<sup>-2</sup></td><td>5.62 · 10<sup>-2</sup></td><td>2.69 · 10<sup>-3</sup></td><td>2.58 · 10<sup>-3</sup></td><td>2.7 · 10<sup>-3</sup></td><td>2.35 · 10<sup>-3</sup></td><td>2.49 · 10<sup>0</sup></td><td>1.18 · 10<sup>-1</sup></td><td>8.3 · 10<sup>-3</sup></td></tr><tr><td>NO-LIDK\*</td><td>3.91 · 10<sup>-2</sup></td><td>4.02 · 10<sup>-2</sup></td><td>6.5 · 10<sup>-2</sup></td><td>2.79 · 10<sup>-3</sup></td><td>3.19 · 10<sup>-3</sup></td><td>3.58 · 10<sup>-3</sup></td><td>2.47 · 10<sup>-3</sup></td><td>2.7 · 10<sup>0</sup></td><td>1.08 · 10<sup>-1</sup></td><td>9.75 · 10<sup>-3</sup></td></tr><tr><td>NO-LIDK\circ</td><td>3.08 · 10<sup>-2</sup></td><td>3.13 · 10<sup>-2</sup></td><td>4.89 · 10<sup>-2</sup></td><td>2.66 · 10<sup>-3</sup></td><td>2.5 · 10<sup>-3</sup></td><td>2.38 · 10<sup>-3</sup></td><td>2.12 · 10<sup>-3</sup></td><td>2.31 · 10<sup>0</sup></td><td>8.17 · 10<sup>-2</sup></td><td>7.52 · 10<sup>-3</sup></td></tr><tr><td>LOGLO-FNO</td><td>2.31 · 10<sup>-2</sup></td><td>2.34 · 10<sup>-2</sup></td><td>4.07 · 10<sup>-2</sup></td><td>1.95 · 10<sup>-3</sup></td><td>1.64 · 10<sup>-3</sup></td><td>1.35 · 10<sup>-3</sup></td><td>1.56 · 10<sup>-3</sup></td><td>1.99 · 10<sup>0</sup></td><td>8.11 · 10<sup>-2</sup></td><td>4.85 · 10<sup>-3</sup></td></tr><tr><td>REL. % DIFF</td><td>-31.82%</td><td>-31.75%</td><td>-27.61%</td><td>-27.55%</td><td>-36.7%</td><td>-50%</td><td>-33.4%</td><td>-20.26%</td><td>-31.44%</td><td>-41.58%</td></tr><tr><td colspan="11" style="text-align: center; font-weight: bold;">10-step Autoregressive Evaluation</td></tr><tr><td>FNO</td><td>5.06 · 10<sup>-2</sup></td><td>4.76 · 10<sup>-2</sup></td><td>6.33 · 10<sup>-2</sup></td><td>4.14 · 10<sup>-3</sup></td><td>4.58 · 10<sup>-3</sup></td><td>4.79 · 10<sup>-3</sup></td><td>3.22 · 10<sup>-3</sup></td><td>3.48 · 10<sup>0</sup></td><td>1.11 · 10<sup>-1</sup></td><td>1.04 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK\*</td><td>6.15 · 10<sup>-2</sup></td><td>5.87 · 10<sup>-2</sup></td><td>7.68 · 10<sup>-2</sup></td><td>4.63 · 10<sup>-3</sup></td><td>5.88 · 10<sup>-3</sup></td><td>6.35 · 10<sup>-3</sup></td><td>3.59 · 10<sup>-3</sup></td><td>3.67 · 10<sup>0</sup></td><td>9.65 · 10<sup>-2</sup></td><td>1.35 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK\circ</td><td>4.72 · 10<sup>-2</sup></td><td>4.52 · 10<sup>-2</sup></td><td>5.6 · 10<sup>-2</sup></td><td>4.39 · 10<sup>-3</sup></td><td>4.55 · 10<sup>-3</sup></td><td>4.37 · 10<sup>-3</sup></td><td>3.02 · 10<sup>-3</sup></td><td>3.55 · 10<sup>0</sup></td><td>7.95 · 10<sup>-2</sup></td><td>9.75 · 10<sup>-3</sup></td></tr><tr><td>LOGLO-FNO</td><td>3.43 · 10<sup>-2</sup></td><td>3.25 · 10<sup>-2</sup></td><td>4.44 · 10<sup>-2</sup></td><td>3.16 · 10<sup>-3</sup></td><td>2.9 · 10<sup>-3</sup></td><td>2.74 · 10<sup>-3</sup></td><td>2.19 · 10<sup>-3</sup></td><td>2.67 · 10<sup>0</sup></td><td>7.47 · 10<sup>-2</sup></td><td>6.28 · 10<sup>-3</sup></td></tr><tr><td>REL. % DIFF</td><td>-32.33%</td><td>-31.75%</td><td>-29.77%</td><td>-23.71%</td><td>-36.74%</td><td>-42.7%</td><td>-31.99%</td><td>-23.2%</td><td>-32.65%</td><td>-39.84%</td></tr><tr><td colspan="11" style="text-align: center; font-weight: bold;">15-step Autoregressive Evaluation</td></tr><tr><td>FNO</td><td>6.87 · 10<sup>-2</sup></td><td>6.37 · 10<sup>-2</sup></td><td>7.35 · 10<sup>-2</sup></td><td>5.55 · 10<sup>-3</sup></td><td>6.91 · 10<sup>-3</sup></td><td>7.19 · 10<sup>-3</sup></td><td>4.11 · 10<sup>-3</sup></td><td>3.98 · 10<sup>0</sup></td><td>1.07 · 10<sup>-1</sup></td><td>1.31 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK\*</td><td>8.44 · 10<sup>-2</sup></td><td>7.92 · 10<sup>-2</sup></td><td>9.08 · 10<sup>-2</sup></td><td>6.68 · 10<sup>-3</sup></td><td>9.04 · 10<sup>-3</sup></td><td>9.36 · 10<sup>-3</sup></td><td>4.61 · 10<sup>-3</sup></td><td>4.09 · 10<sup>0</sup></td><td>9.57 · 10<sup>-2</sup></td><td>1.79 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK\circ</td><td>6.81 · 10<sup>-2</sup></td><td>6.49 · 10<sup>-2</sup></td><td>6.91 · 10<sup>-2</sup></td><td>6.51 · 10<sup>-3</sup></td><td>7.78 · 10<sup>-3</sup></td><td>7.24 · 10<sup>-3</sup></td><td>4.08 · 10<sup>-3</sup></td><td>4.4 · 10<sup>0</sup></td><td>8.97 · 10<sup>-2</sup></td><td>1.35 · 10<sup>-2</sup></td></tr><tr><td>LOGLO-FNO</td><td>4.76 · 10<sup>-2</sup></td><td>4.45 · 10<sup>-2</sup></td><td>5.16 · 10<sup>-2</sup></td><td>4.49 · 10<sup>-3</sup></td><td>4.49 · 10<sup>-3</sup></td><td>4.53 · 10<sup>-3</sup></td><td>2.91 · 10<sup>-3</sup></td><td>3.49 · 10<sup>0</sup></td><td>7.4 · 10<sup>-2</sup></td><td>8.02 · 10<sup>-3</sup></td></tr><tr><td>REL. % DIFF</td><td>-30.73%</td><td>-30.12%</td><td>-29.86%</td><td>-19.17%</td><td>-35.04%</td><td>-36.95%</td><td>-29.04%</td><td>-12.33%</td><td>-30.94%</td><td>-38.74%</td></tr><tr><td colspan="11" style="text-align: center; font-weight: bold;">20-step (full trajectory) Autoregressive Evaluation</td></tr><tr><td>FNO</td><td>8.41 · 10<sup>-2</sup></td><td>7.86 · 10<sup>-2</sup></td><td>8.54 · 10<sup>-2</sup></td><td>6.68 · 10<sup>-3</sup></td><td>8.94 · 10<sup>-3</sup></td><td>9.24 · 10<sup>-3</sup></td><td>4.8 · 10<sup>-3</sup></td><td>4.46 · 10<sup>0</sup></td><td>1.08 · 10<sup>-1</sup></td><td>1.54 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK\*</td><td>1.03 · 10<sup>-1</sup></td><td>9.75 · 10<sup>-2</sup></td><td>1.06 · 10<sup>-1</sup></td><td>8.5 · 10<sup>-3</sup></td><td>1.18 · 10<sup>-2</sup></td><td>1.18 · 10<sup>-2</sup></td><td>5.36 · 10<sup>-3</sup></td><td>4.41 · 10<sup>0</sup></td><td>1.01 · 10<sup>-1</sup></td><td>2.16 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK\circ</td><td>8.79 · 10<sup>-2</sup></td><td>8.48 · 10<sup>-2</sup></td><td>8.67 · 10<sup>-2</sup></td><td>8.82 · 10<sup>-3</sup></td><td>1.14 · 10<sup>-2</sup></td><td>9.97 · 10<sup>-3</sup></td><td>4.95 · 10<sup>-3</sup></td><td>4.99 · 10<sup>0</sup></td><td>9.92 · 10<sup>-2</sup></td><td>1.74 · 10<sup>-2</sup></td></tr><tr><td>LOGLO-FNO</td><td>6.02 · 10<sup>-2</sup></td><td>5.68 · 10<sup>-2</sup></td><td>6.15 · 10<sup>-2</sup></td><td>5.76 · 10<sup>-3</sup></td><td>6.12 · 10<sup>-3</sup></td><td>6.25 · 10<sup>-3</sup></td><td>3.55 · 10<sup>-3</sup></td><td>3.95 · 10<sup>0</sup></td><td>7.66 · 10<sup>-2</sup></td><td>9.85 · 10<sup>-3</sup></td></tr><tr><td>REL. % DIFF</td><td>-28.4%</td><td>-27.74%</td><td>-28%</td><td>-13.77%</td><td>-31.53%</td><td>-32.37%</td><td>-26.12%</td><td>-11.43%</td><td>-28.92%</td><td>-35.93%</td></tr></tbody></table>

We observe that LOGLO-FNO significantly and consistently outperforms Base FNO and the state-of-the-art NO-LIDK baselines on both 1-step and autoregressive rollout evaluations, up to the full trajectory length of 20 timesteps.
**Turbulent Radiative Layer 3D (TRL3D).** To complement the results in the main paper, we present the full set of evaluation metrics for the 1-step evaluation on the test set of Turbulent Radiative Layer 3D problem in Table 12.

Table 12: 1-step evaluation of LOGLO-FNO compared with state-of-the-art baselines on the test set of Turbulent Radiative Mixing Layer 3D dataset (Fielding et al., 2020; Ohana et al., 2024). REL. % DIFF indicates an improvement (-) or degradation (+) with respect to base FNO, which uses a spectral filter size of 12 and 48 hidden channels. LOGLO-FNO uses 12 and (16, 16, 17)\* modes in the global and local branches, respectively, whereas the width is set to 52. (\*This is because the Turbulent Radiative Layer 3D dataset has 2× more sampling points on the z-axis relative to the x and y axes resolutions – (128 × 128 × 256).) Base FNO\*‡ means our implementation of FNO (see Figure 1).

<table><thead><tr><th rowspan="2">Model</th><th colspan="8">1-step Evaluation</th></tr><tr><th>RMSE (↓)</th><th>nRMSE</th><th>bRMSE</th><th>cRMSE</th><th>fRMSE(L)</th><th>fRMSE(M)</th><th>fRMSE(H)</th><th>MaxError (↓)</th></tr></thead><tbody><tr><td>LSM</td><td>5.67 · 10<sup>-1</sup></td><td>8.39 · 10<sup>-1</sup></td><td>1.25 · 10<sup>0</sup></td><td>1.41 · 10<sup>-1</sup></td><td>2.4 · 10<sup>-1</sup></td><td>9.7 · 10<sup>-2</sup></td><td>3.43 · 10<sup>-2</sup></td><td>1.59 · 10<sup>1</sup></td></tr><tr><td>NO-LIDK<sup>o</sup></td><td>1.04 · 10<sup>0</sup></td><td>4.34 · 10<sup>-1</sup></td><td>4.59 · 10<sup>0</sup></td><td>2.17 · 10<sup>-2</sup></td><td>6.87 · 10<sup>-2</sup></td><td>1.22 · 10<sup>-1</sup></td><td>1.23 · 10<sup>-1</sup></td><td>1.85 · 10<sup>1</sup></td></tr><tr><td>Base FNO<sup>‡</sup></td><td>2.84 · 10<sup>-1</sup></td><td>3.06 · 10<sup>-1</sup></td><td>7.05 · 10<sup>-1</sup></td><td>2.04 · 10<sup>-2</sup></td><td>4.87 · 10<sup>-2</sup></td><td>4.58 · 10<sup>-2</sup></td><td>2.89 · 10<sup>-2</sup></td><td>1.12 · 10<sup>1</sup></td></tr><tr><td><strong>LOGLO-FNO</strong></td><td><strong>2.45 · 10<sup>-1</sup></strong></td><td><strong>2.65 · 10<sup>-1</sup></strong></td><td><strong>6.33 · 10<sup>-1</sup></strong></td><td><strong>1.76 · 10<sup>-2</sup></strong></td><td><strong>4.45 · 10<sup>-2</sup></strong></td><td><strong>3.68 · 10<sup>-2</sup></strong></td><td><strong>2.48 · 10<sup>-2</sup></strong></td><td><strong>1.06 · 10<sup>1</sup></strong></td></tr><tr><td>REL. % DIFF</td><td>-13.55%</td><td>-13.27%</td><td>-10.2%</td><td>-13.8%</td><td>-8.61%</td><td>-19.57%</td><td>-14.01%</td><td>-5.17%</td></tr></tbody></table>

### G.2.2 Autoregressive Training and Evaluation

In Table 13, we present the evaluation results with the full set of evaluation metrics for the fully autoregressive training on the Diffusion-Reaction 2D problem.

Table 13: Fully autoregressive evaluation of LOGLO-FNO compared with SOTA baselines on the test set of challenging 2D Diffusion-Reaction coupled problem from PDEBench (Takamoto et al., 2022). We also report the REL. % DIFF to indicate the error improvement (-) or degradation (+) w.r.t FNO.

<table><thead><tr><th>Model</th><th>RMSE (↓)</th><th>nRMSE</th><th>bRMSE</th><th>cRMSE</th><th>fRMSE(L)</th><th>fRMSE(M)</th><th>fRMSE(H)</th><th>MaxError (↓)</th><th>MELR (↓)</th><th>WLR (↓)</th></tr></thead><tbody><tr><td>U-Net</td><td>6.1 · 10<sup>-2</sup></td><td>8.4 · 10<sup>-1</sup></td><td>7.8 · 10<sup>-2</sup></td><td>3.9 · 10<sup>-2</sup></td><td>1.7 · 10<sup>-2</sup></td><td>8.2 · 10<sup>-4</sup></td><td>5.7 · 10<sup>-2</sup></td><td>1.9 · 10<sup>-1</sup></td><td>χ</td><td>χ</td></tr><tr><td>Transolver</td><td>1.74 · 10<sup>-2</sup></td><td>2.63 · 10<sup>-1</sup></td><td>3.05 · 10<sup>-2</sup></td><td>3.37 · 10<sup>-3</sup></td><td>2.99 · 10<sup>-3</sup></td><td>2.03 · 10<sup>-3</sup></td><td>5.51 · 10<sup>-4</sup></td><td>1.11 · 10<sup>-1</sup></td><td>7.19 · 10<sup>-1</sup></td><td>1.15 · 10<sup>-1</sup></td></tr><tr><td>U-FNO</td><td>1.4 · 10<sup>-2</sup></td><td>2.6 · 10<sup>-1</sup></td><td>2.0 · 10<sup>-2</sup></td><td>4.3 · 10<sup>-3</sup></td><td>3.4 · 10<sup>-3</sup></td><td>1.6 · 10<sup>-3</sup></td><td>2.6 · 10<sup>-4</sup></td><td>7.8 · 10<sup>-2</sup></td><td>4.5 · 10<sup>-1</sup></td><td>7.9 · 10<sup>-2</sup></td></tr><tr><td>FNO</td><td>5.2 · 10<sup>-3</sup></td><td>8.3 · 10<sup>-2</sup></td><td>1.5 · 10<sup>-2</sup></td><td>1.2 · 10<sup>-3</sup></td><td>6.2 · 10<sup>-4</sup></td><td>5.6 · 10<sup>-4</sup></td><td>2.4 · 10<sup>-4</sup></td><td>7.3 · 10<sup>-2</sup></td><td>2.96 · 10<sup>-1</sup></td><td>1.3 · 10<sup>-2</sup></td></tr><tr><td>F-FNO</td><td>4.3 · 10<sup>-3</sup></td><td>7.0 · 10<sup>-2</sup></td><td>7.9 · 10<sup>-3</sup></td><td>2.8 · 10<sup>-3</sup></td><td>9.6 · 10<sup>-4</sup></td><td>4.7 · 10<sup>-4</sup></td><td>1.3 · 10<sup>-4</sup></td><td>5.3 · 10<sup>-2</sup></td><td>2.0 · 10<sup>-1</sup></td><td>1.3 · 10<sup>-2</sup></td></tr><tr><td>LSM</td><td>2.81 · 10<sup>-2</sup></td><td>4.47 · 10<sup>-1</sup></td><td>3.45 · 10<sup>-2</sup></td><td>5.92 · 10<sup>-3</sup></td><td>7.17 · 10<sup>-3</sup></td><td>2.4 · 10<sup>-3</sup></td><td>3.67 · 10<sup>-4</sup></td><td>1.32 · 10<sup>-1</sup></td><td>3.43 · 10<sup>-1</sup></td><td>2.08 · 10<sup>-1</sup></td></tr><tr><td>NO-LIDK (loc. int)</td><td><strong>3.6 · 10<sup>-3</sup></strong></td><td><strong>6.3 · 10<sup>-2</sup></strong></td><td><strong>1.0 · 10<sup>-2</sup></strong></td><td><strong>4.8 · 10<sup>-4</sup></strong></td><td><strong>4.0 · 10<sup>-4</sup></strong></td><td><strong>4.6 · 10<sup>-4</sup></strong></td><td><strong>1.5 · 10<sup>-4</sup></strong></td><td><strong>5.0 · 10<sup>-2</sup></strong></td><td>χ</td><td>χ</td></tr><tr><td><strong>LOGLO-FNO</strong></td><td><strong>3.89 · 10<sup>-3</sup></strong></td><td><strong>6.4 · 10<sup>-2</sup></strong></td><td><strong>5.2 · 10<sup>-3</sup></strong></td><td><strong>4.6 · 10<sup>-4</sup></strong></td><td><strong>2.8 · 10<sup>-4</sup></strong></td><td><strong>3.2 · 10<sup>-4</sup></strong></td><td><strong>1.9 · 10<sup>-4</sup></strong></td><td><strong>2.2 · 10<sup>-2</sup></strong></td><td><strong>1.6 · 10<sup>-1</sup></strong></td><td><strong>7.9 · 10<sup>-3</sup></strong></td></tr><tr><td>REL. % DIFF</td><td>-25.19%</td><td>-22.75%</td><td>-65.13%</td><td>-61.67%</td><td>-54.84%</td><td>-42.86%</td><td>-20.83%</td><td>-69.97%</td><td>-44.09%</td><td>-36.98%</td></tr></tbody></table>
## G.3 Further Improvements with Attentional Feature Fusion, Multi-scale Channel Attention, and SPHERE Loss

**2D Kolmogorov Flow.** In this section, we explore ways to further improve the results by considering advanced feature fusion strategies. The global and local branches in LOGLO-FNO models input features on different scales, whereas the HFP module learns different latent features that are representative of high frequencies. Therefore, we propose to employ sophisticated feature fusion modules targeted towards a seamless aggregation of these multi-scale and diverse features, as opposed to a simple summation (Figure 1). Towards this end, we adopt the multi-scale channel attention and attentional feature fusion modules introduced in Dai et al. (2021) to fuse the features of the local and global branches and the HFP branch. The results of an experiment with this fusion in LOGLO-FNO, as well as penalizing the SPHERE loss on the Kolmogorov Flow 2D dataset, are presented in Table 14. We term this model as LOGLO-FNO+. Comparing the results in Table 1 and Table 14, we observe that these improvements lead to an additional 12% in 1-step nRMSE and (6%, 11%) for the mid-and high-frequency errors, respectively. The MELR and WLR errors are also down by 8% and 7%, respectively. In a similar manner, we see consistent improvements in 5-step nRMSE (8%), fRMSE-mid (2%), fRMSE-high (8%), MELR and WLR (3% each) over the LOGLO-FNO model not using feature fusion and the SPHERE loss.

Table 14: 1-step and 5-step AR evaluation of LOGLO-FNO+ employing multi-scale feature fusion and patch-based energy spectra loss (SPHERE) compared with SOTA baselines on the test set of 2D Kolmogorov Flow (Li et al., 2022b). We also report the REL. % DIFF to indicate the improvement (-) or the degradation (+) with respect to FNO. The number of modes used in the global branch is 40, and the local branch uses a patch size of 16 × 16. The number of hidden channels has been set as 65. NO-LIDK† indicates the usage of both differential kernel and localized integral kernel layers in its architecture, NO-LIDK* denotes the model employing only the local integral kernel layer, and NO-LIDK$^{◦}$ stands for the use of only the differential layer as an additional parallel layer to the global spectral convolution layer of base FNO (see Table 4 in Liu-Schiaffini et al. (2024)). Transolver* indicates a longer training time of the model for 500 epochs due to convergence issues at shorter training epochs of 136. We boldface the best result and underline the second-best result.

<table><thead><tr><th rowspan="2">Eval Type</th><th rowspan="2">Model</th><th rowspan="2">RMSE (↓)</th><th rowspan="2">nRMSE</th><th rowspan="2">bRMSE</th><th rowspan="2">cRMSE</th><th rowspan="2">fRMSE(L)</th><th rowspan="2">fRMSE(M)</th><th rowspan="2">fRMSE(H)</th><th rowspan="2">MaxError (↓)</th><th rowspan="2">MELR (↓)</th><th rowspan="2">WLR (↓)</th></tr><tr><th colspan="10">1-step Evaluation</th></tr></thead><tbody><tr><td rowspan="10">1-step</td><td>U-Net</td><td>7.17 · 10<sup>-1</sup></td><td>1.3 · 10<sup>-1</sup></td><td>1.47 · 10<sup>0</sup></td><td>1.74 · 10<sup>-2</sup></td><td>2.24 · 10<sup>-2</sup></td><td>3.57 · 10<sup>-2</sup></td><td>4.39 · 10<sup>-2</sup></td><td>2.01 · 10<sup>1</sup></td><td>1.64 · 10<sup>-1</sup></td><td>1.48 · 10<sup>-2</sup></td></tr><tr><td>Transolver*</td><td>7.69 · 10<sup>-1</sup></td><td>1.39 · 10<sup>-1</sup></td><td>1.42 · 10<sup>0</sup></td><td>7.65 · 10<sup>-3</sup></td><td>2.74 · 10<sup>-2</sup></td><td>4.48 · 10<sup>-2</sup></td><td>4.65 · 10<sup>-2</sup></td><td>1.52 · 10<sup>1</sup></td><td>1.43 · 10<sup>-1</sup></td><td>1.83 · 10<sup>-2</sup></td></tr><tr><td>FNO</td><td>8.08 · 10<sup>-1</sup></td><td>1.47 · 10<sup>-1</sup></td><td>7.94 · 10<sup>-1</sup></td><td>1.1 · 10<sup>-2</sup></td><td>1.36 · 10<sup>-2</sup></td><td>2.05 · 10<sup>-2</sup></td><td>4.7 · 10<sup>-2</sup></td><td>1.46 · 10<sup>1</sup></td><td>5.2 · 10<sup>-1</sup></td><td>2.83 · 10<sup>-2</sup></td></tr><tr><td>F-FNO</td><td>7.53 · 10<sup>-1</sup></td><td>1.37 · 10<sup>-1</sup></td><td>7.41 · 10<sup>-1</sup></td><td>1.5 · 10<sup>-2</sup></td><td>1.49 · 10<sup>-2</sup></td><td>2.15 · 10<sup>-2</sup></td><td>4.36 · 10<sup>-2</sup></td><td>1.42 · 10<sup>1</sup></td><td>4.74 · 10<sup>-1</sup></td><td>2.28 · 10<sup>-2</sup></td></tr><tr><td>LSM</td><td>7.49 · 10<sup>-1</sup></td><td>1.36 · 10<sup>-1</sup></td><td>1.47 · 10<sup>0</sup></td><td>1.36 · 10<sup>-2</sup></td><td>2.59 · 10<sup>-2</sup></td><td>4.42 · 10<sup>-2</sup></td><td>4.64 · 10<sup>-2</sup></td><td>2.05 · 10<sup>1</sup></td><td>1.43 · 10<sup>-1</sup></td><td>1.68 · 10<sup>-2</sup></td></tr><tr><td>U-FNO</td><td>6.13 · 10<sup>-1</sup></td><td>1.12 · 10<sup>-1</sup></td><td>1.0 · 10<sup>0</sup></td><td><b>7.09</b> · 10<sup>-3</sup></td><td>1.27 · 10<sup>-2</sup></td><td>2.22 · 10<sup>-2</sup></td><td>3.71 · 10<sup>-2</sup></td><td>1.64 · 10<sup>1</sup></td><td>1.38 · 10<sup>-1</sup></td><td>1.09 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK*</td><td>7.25 · 10<sup>-1</sup></td><td>1.33 · 10<sup>-1</sup></td><td>1.12 · 10<sup>0</sup></td><td>9.05 · 10<sup>-3</sup></td><td>1.45 · 10<sup>-2</sup></td><td>2.69 · 10<sup>-2</sup></td><td>4.55 · 10<sup>-2</sup></td><td>1.65 · 10<sup>1</sup></td><td>1.85 · 10<sup>-1</sup></td><td>1.54 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK<sup>◦</sup></td><td>6.13 · 10<sup>-1</sup></td><td>1.11 · 10<sup>-1</sup></td><td>5.95 · 10<sup>-1</sup></td><td>1.46 · 10<sup>-2</sup></td><td>1.65 · 10<sup>-2</sup></td><td>2.29 · 10<sup>-2</sup></td><td>3.91 · 10<sup>-2</sup></td><td>1.5 · 10<sup>1</sup></td><td>9.57 · 10<sup>-2</sup></td><td>1.1 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK<sup>†</sup></td><td>5.86 · 10<sup>-1</sup></td><td>1.07 · 10<sup>-1</sup></td><td>5.64 · 10<sup>-1</sup></td><td>1.05 · 10<sup>-2</sup></td><td>1.44 · 10<sup>-2</sup></td><td>2.46 · 10<sup>-2</sup></td><td>3.82 · 10<sup>-2</sup></td><td>1.47 · 10<sup>1</sup></td><td><b>7.11</b> · 10<sup>-2</sup></td><td>1.0 · 10<sup>-2</sup></td></tr><tr><td>LOGLO-FNO</td><td><b>5.89</b> · 10<sup>-1</sup></td><td><b>1.07</b> · 10<sup>-1</sup></td><td>6.74 · 10<sup>-1</sup></td><td><b>7.23</b> · 10<sup>-3</sup></td><td><b>1.21</b> · 10<sup>-2</sup></td><td>1.81 · 10<sup>-2</sup></td><td>3.54 · 10<sup>-2</sup></td><td><b>1.33</b> · 10<sup>1</sup></td><td>1.29 · 10<sup>-1</sup></td><td>1.06 · 10<sup>-2</sup></td></tr><tr><td>LOGLO-FNO+</td><td><b>4.88</b> · 10<sup>-1</sup></td><td><b>8.87</b> · 10<sup>-2</sup></td><td><b>5.57</b> · 10<sup>-1</sup></td><td>8.78 · 10<sup>-3</sup></td><td><b>1.31</b> · 10<sup>-2</sup></td><td><b>1.68</b> · 10<sup>-2</sup></td><td><b>3.04</b> · 10<sup>-2</sup></td><td><b>1.35</b> · 10<sup>1</sup></td><td><b>7.56</b> · 10<sup>-2</sup></td><td><b>8.76</b> · 10<sup>-3</sup></td></tr><tr><td colspan="2">REL. % DIFF w/o fusion</td><td>-27.1 %</td><td>-27.21 %</td><td>-15.12 %</td><td>-34.31 %</td><td>-11.22 %</td><td>-11.88 %</td><td>-24.64 %</td><td>-8.99 %</td><td>-75.12 %</td><td>-62.63 %</td></tr><tr><td colspan="2">REL. % DIFF w/ fusion</td><td>-39.6 %</td><td>-39.78 %</td><td>-29.86 %</td><td>-20.28 %</td><td>-3.87 %</td><td>-18.13 %</td><td>-35.39 %</td><td>-7.91 %</td><td>-83.54 %</td><td>-69.03 %</td></tr><tr><td colspan="12" style="text-align: center;">5-step Autoregressive Evaluation</td></tr><tr><td rowspan="10">5-step AR</td><td>U-Net</td><td>1.51 · 10<sup>0</sup></td><td>2.65 · 10<sup>-1</sup></td><td>2.31 · 10<sup>0</sup></td><td>5.39 · 10<sup>-2</sup></td><td>6.53 · 10<sup>-2</sup></td><td>1.04 · 10<sup>-1</sup></td><td>9.38 · 10<sup>-2</sup></td><td>1.81 · 10<sup>1</sup></td><td>2.13 · 10<sup>-1</sup></td><td>3.52 · 10<sup>-2</sup></td></tr><tr><td>Transolver*</td><td>1.91 · 10<sup>0</sup></td><td>3.36 · 10<sup>-1</sup></td><td>2.59 · 10<sup>0</sup></td><td><b>6.12</b> · 10<sup>-3</sup></td><td>7.44 · 10<sup>-2</sup></td><td>1.51 · 10<sup>-1</sup></td><td>1.18 · 10<sup>-1</sup></td><td>2.36 · 10<sup>1</sup></td><td>2.45 · 10<sup>-1</sup></td><td>4.92 · 10<sup>-2</sup></td></tr><tr><td>FNO</td><td>1.33 · 10<sup>0</sup></td><td>2.35 · 10<sup>-1</sup></td><td>1.34 · 10<sup>0</sup></td><td>1.46 · 10<sup>-2</sup></td><td>3.37 · 10<sup>-2</sup></td><td>5.8 · 10<sup>-2</sup></td><td>8.37 · 10<sup>-2</sup></td><td>1.6 · 10<sup>1</sup></td><td>6.18 · 10<sup>-1</sup></td><td>4.93 · 10<sup>-2</sup></td></tr><tr><td>F-FNO</td><td>1.29 · 10<sup>0</sup></td><td>2.28 · 10<sup>-1</sup></td><td>1.27 · 10<sup>0</sup></td><td>2.28 · 10<sup>-2</sup></td><td>3.60 · 10<sup>-2</sup></td><td>5.52 · 10<sup>-2</sup></td><td>8.12 · 10<sup>-2</sup></td><td>1.50 · 10<sup>1</sup></td><td>5.37 · 10<sup>-1</sup></td><td>3.99 · 10<sup>-2</sup></td></tr><tr><td>LSM</td><td>1.81 · 10<sup>0</sup></td><td>3.18 · 10<sup>-1</sup></td><td>2.76 · 10<sup>0</sup></td><td>3.87 · 10<sup>-2</sup></td><td>7.6 · 10<sup>-2</sup></td><td>1.44 · 10<sup>-1</sup></td><td>1.12 · 10<sup>-1</sup></td><td>2.08 · 10<sup>1</sup></td><td>2.04 · 10<sup>-1</sup></td><td>4.39 · 10<sup>-2</sup></td></tr><tr><td>U-FNO</td><td>1.15 · 10<sup>0</sup></td><td>2.03 · 10<sup>-1</sup></td><td>1.45 · 10<sup>0</sup></td><td><b>6.3</b> · 10<sup>-3</sup></td><td><b>2.69</b> · 10<sup>-2</sup></td><td>5.33 · 10<sup>-2</sup></td><td>7.35 · 10<sup>-2</sup></td><td>1.56 · 10<sup>1</sup></td><td>2.01 · 10<sup>-1</sup></td><td>2.51 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK*</td><td>1.36 · 10<sup>0</sup></td><td>2.39 · 10<sup>-1</sup></td><td>1.8 · 10<sup>0</sup></td><td>1.1 · 10<sup>-2</sup></td><td>3.14 · 10<sup>-2</sup></td><td>6.86 · 10<sup>-2</sup></td><td>8.91 · 10<sup>-2</sup></td><td>1.59 · 10<sup>1</sup></td><td>2.23 · 10<sup>-1</sup></td><td>2.93 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK<sup>◦</sup></td><td>1.17 · 10<sup>0</sup></td><td>2.04 · 10<sup>-1</sup></td><td>1.12 · 10<sup>0</sup></td><td>1.92 · 10<sup>-2</sup></td><td>3.71 · 10<sup>-2</sup></td><td>5.96 · 10<sup>-2</sup></td><td>7.60 · 10<sup>-2</sup></td><td>1.61 · 10<sup>1</sup></td><td>1.42 · 10<sup>-1</sup></td><td>2.12 · 10<sup>-2</sup></td></tr><tr><td>NO-LIDK<sup>†</sup></td><td>1.17 · 10<sup>0</sup></td><td>2.05 · 10<sup>-1</sup></td><td>1.14 · 10<sup>0</sup></td><td>1.23 · 10<sup>-2</sup></td><td>3.31 · 10<sup>-2</sup></td><td>5.94 · 10<sup>-2</sup></td><td>7.74 · 10<sup>-2</sup></td><td>1.56 · 10<sup>1</sup></td><td><b>1.03</b> · 10<sup>-1</sup></td><td>2.18 · 10<sup>-2</sup></td></tr><tr><td>LOGLO-FNO</td><td><b>1.09</b> · 10<sup>0</sup></td><td><b>1.92</b> · 10<sup>-1</sup></td><td><b>1.12</b> · 10<sup>0</sup></td><td>8.99 · 10<sup>-3</sup></td><td><b>2.8</b> · 10<sup>-2</sup></td><td>4.55 · 10<sup>-2</sup></td><td><b>6.93</b> · 10<sup>-2</sup></td><td><b>1.26</b> · 10<sup>1</sup></td><td>1.67 · 10<sup>-1</sup></td><td><b>2.07</b> · 10<sup>-2</sup></td></tr><tr><td>LOGLO-FNO+</td><td><b>9.81</b> · 10<sup>-1</sup></td><td><b>1.72</b> · 10<sup>-1</sup></td><td><b>1.02</b> · 10<sup>0</sup></td><td>1.05 · 10<sup>-2</sup></td><td>2.9 · 10<sup>-2</sup></td><td><b>4.43</b> · 10<sup>-2</sup></td><td><b>6.27</b> · 10<sup>-2</sup></td><td><b>1.31</b> · 10<sup>1</sup></td><td><b>1.17</b> · 10<sup>-1</sup></td><td><b>1.90</b> · 10<sup>-2</sup></td></tr><tr><td colspan="2">REL. % DIFF w/o fusion</td><td>-18.26 %</td><td>-18.39 %</td><td>-16.12 %</td><td>-38.21 %</td><td>-16.86 %</td><td>-21.62 %</td><td>-17.26 %</td><td>-21.31 %</td><td>-73.04 %</td><td>-58.02 %</td></tr><tr><td colspan="2">REL. % DIFF w/ fusion</td><td>-26.37 %</td><td>-26.63 %</td><td>-23.43 %</td><td>-27.99 %</td><td>-14.03 %</td><td>-23.58 %</td><td>-25.09 %</td><td>-15.62 %</td><td>-77.76 %</td><td>-61.43 %</td></tr></tbody></table>

Note that the terminology REL. % DIFF w/ fusion in the Table 14 above indicates the use of SPHERE loss in addition to feature fusion. This detail has been omitted in the row for an uncluttered presentation.
## G.4 Significance of Adding More Modes in the Fourier Layers of Global Branch

### G.4.1 Kolmogorov Flow 2D

In this study, we analyze the influence of adding more modes in the global branch while keeping the number of modes constant in the local branch, which is done by using a single local branch with a patch size of 16 × 16. This setting yields the local spectral convolution branch with 9 modes, including the Nyquist frequency for the y-axis of the 2D spatial data. Further, we also fix the embedding dimension to 48. Therefore, the number of modes in the global branch is the only varying factor. The experiments are conducted on the turbulent 2D Kolmogorov Flow dataset (Li et al., 2022b).

**1-step Evaluation.** Here, we evaluate the baseline FNO and LOGLO-FNO models on 1-step errors. The LOGLO-FNO models have been trained with a single local branch (patch size 16 × 16) in addition to the global branch. Figures 14, 3, 15, and 16 plot xRMSE, MaxError, and the energy spectra (MELR and WLR) metrics.

<figure><img src="image_18.png" /><figcaption>Figure 14: Comparison of FNO vs. LOGLO-FNO showing 1-step nRMSE and MaxError (↓) on the test set of Kolmogorov Flow ($Re = 5k$) (Li et al., 2022b) for varying number of modes in global branch and full set of modes (i.e., (16, 9) for patch size 16 × 16) in local branch.</figcaption></figure>

In Figure 14, we observe that as we increase the modes in the global branch, it generally leads to a reduction in nRMSE and MaxError for both models, except for a slight increase in MaxError when utilizing the full set of modes, potentially due to overfitting. Also, note that LOGLO-FNO outperforms base FNO significantly, particularly when the modes used in the global branch are fewer (i.e., sparse modes). We hypothesize that the local branch compensates in this case when there is a lack of expressivity. Even when using the full set of available modes, LOGLO-FNO yields improved error. This behavior shows that local convolutions always supplement global convolutions.

Figure 3 in the main paper visualizes the low, mid, and high-frequency band-classified errors for varying numbers of modes in the global branch. A similar trend as observed for nRMSE also holds here. Additionally, we observe that the error improvement for LOGLO-FNO over FNO for the high-frequency is more pronounced than for low and mid-frequencies. Note the logarithmic scale on the y-axis. Further, FNO tends to overfit when employing the full set of modes (fRMSE (Low) & fRMSE (Mid)), suggesting potential overfitting and corroborating the findings of Lanthaler et al. (2024). In contrast, LOGLO-FNO does not exhibit this behavior and achieves the lowest error when utilizing the full set of modes. Notably, LOGLO-FNO with