Alias-Free Mamba Neural Operator
Authors: Jianwei Zheng
LiweiNo
Ni Xu
Junwei Zhu
XiaoxuLin
Xiaoqin Zhang
Venue: NeurIPS 2024 poster (2024)
Keywords: Mamba
Neural Operator
Primary area: deep_learning_architectures
TL;DR: A neural operator that introduces a novel Mamba integration.

Abstract: Benefiting from the booming deep learning techniques, neural operators (NO) are considered as an ideal alternative to break the traditions of solving Partial Differential Equations (PDE) with expensive cost.
Yet with the remarkable progress, current solutions concern little on the holistic function features--both global and local information-- during the process of solving PDEs.
Besides, a meticulously designed kernel integration to meet desirable performance often suffers from a severe computational burden, such as GNO with $O(N(N-1))$, FNO with $O(NlogN)$, and Transformer-based NO with $O(N^2)$.
To counteract the dilemma, we propose a mamba neural operator with $O(N)$ computational complexity, namely MambaNO.
Functionally, MambaNO achieves a clever balance between global integration, facilitated by state space model of Mamba that scans the entire function, and local integration, engaged with an alias-free architecture. We prove a property of continuous-discrete equivalence to show the capability of
MambaNO in approximating operators arising from universal PDEs to desired accuracy. MambaNOs are evaluated on a diverse set of benchmarks with possibly multi-scale solutions and set new state-of-the-art scores, yet with fewer parameters and better efficiency.

Alias-Free Mamba Neural Operator
Jianwei Zheng, Wei Li, Ni Xu, Junwei Zhu, Xiaoxu Lin, and Xiaoqin Zhangg
Zhejiang University of Technology, Hangzhou, Zhejiang
Abstract
Benefiting from the booming deep learning techniques, neural operators (NO) are considered as an ideal alternative to break the traditions of solving Partial Differ- ential Equations (PDE) with expensive cost. Yet with the remarkable progress, current solutions concern little on the holistic function features- global and local information- during the process solving PDEs. Besides, meticulously de- signed kernel integration to meet desirable performance often suffers from severe computational burden, such as GNO with O(N(N 1)), FNO with O( N)), and Transformer-basee based NO with O(2)). To counteract the dilemma, we propose a mamba neural operator with O(N) computational complexity, namely MambaNO. Functionally, MambaNO achieves clever balance between global integration, facilitated by state space model of Mamba that scans the entire function, and local integration, engaged with alias-free architecture. We prove property of continuous-diserte equivalence to show capability of MambaNO in approxi- mating operators arising from universal PDEs to desired accuracy. MambaNOs are evaluated on a diverse set of benchmarks with possibly multi-scale solutions and set new state-of-tee offtc--f-th-aarrr scores, yet with fewer parameters and better efficiency.
Introduction
Numerous scientific and engineering problems entail recurrently resolving intricate Partial Differential Equation (PDE) I for various parameter values, including morphology across biomes 22, physical system state 3 and Radcliffe solvers, such as Finite Element Methods (FEM) and Finite Difference Methods (FDM), necessitate an equilibrium between the speed of obtaining solutions and the level of refinement due to the requirement of resolving equations through domain discretization. Another disadvantage of these methods lies in the unavoidable resolving stage when the initial conditions of PDEs are varied. Recently, data-driven methods have emerged as an alternative for the development of faster, more robust, and more accurate solvers due the natural ability directly ascertain the trajectory of an PDE family from the data, instead of single instance of the equation.
oscillation 44, name just few. Nonetheless, traditional
These techniques, collectively termed operator learning neural operator, strive approximate a well-behaved mapping from input function spaces, such as initial and boundary conditions, to solutions of some PDEs valued also within function spaces. As burgeoning subject, most well- known NOs have been substantiated, encompassing Operator networks [5] DeepONets (DON) [6],
Graph Neural Operator (GNO) (7). Fourier Neural Operator (FNO) g [0], Transformer-based learning architectures
12), and the most recent Convolution Neural Operators (CNO) [13].
Despite the substantial success of recently introduced operator learning frameworks, existing algo- rithms continue to demonstrate specific limitations to various degrees. The focus of operator network learning on mapping between finite-dimense dimenssona function spaces, yet the truthful NO often engages with functions owning infinite dimensions. The practice of DON cannot take inputs at any point,
Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

while GNO delivers very low efficiency when dealing with heterogeneous data. Transformer based architectures suffer from slow inference speeds due the quadratic complexity of attention computa- tion, which serves as a special kernel integration. The use of Galerkin-style attention [4] can only alleviate rather than solve this problem. FNO accelerates the efficiency of the model by converting global convolution operation, which is a special form of kernel integration, into frequency domain multiplication through Fourier transformation. However, FNO may not respect the framework of alias-free operator learning, mentioned in (15), and suffers aliasing errors, a fact already identified in [6]. The most recent CNO has established a new state-of-the-art score in this field, which relies on U-Net as the core architecture and proceeds with several specified operations to circumvent aliasing errors. However, as a local operator, CNO falls short in capturing global information, which is of vital importance for functions. Another recently reported model, Mamba [17 [8], has also attracted great deal of attention due to its capability in capturing global information with linear complexity. Unfortunately, the connection of state space model in Mamba to the integral kernel in NOs is far from intuitive. In addition, Mamba often produces artifacts [19] or pixel adhesion (20, raising the challenge of seeking continuous-disere have plagued the naive use of Mamba in the context of NO learning. In this work, we attempt to marry the current merits yet with the disadvantages suppressed. The practical contributions are threefold.
equivalence (CDE), as highlighted in [15]. Both these two
We propose novel integral form as the NO kernel, namely mamba integration, costing only ((V) computational complexity and enabling the grasp of global function information. On that basis, with the local function feature further furnished by convolution, we present an avant-garde garde Mamba Neural Operator (MambaNO), behaving as deep PDE sovler.
Apart from proving that MambaNO is intrinsically a representation--euiival -eqivvvlnn neural opera- tor in the sense of [21. we also provide a universality result to demonstrate that MambaNO can approximate continuous operators, fitting a large class of PDEs, to desired accuracy.
We test Ma on full set of benchmarks that span across massive PDEs ranging from linear elliptic and hyperbolic to nonlinear parabolic and hyperbolic equations, with potentially multiscale solutions. It is evidenced that MambaNO outperforms competing baselines on all benchmarks, both when testing in-distribution and in out-of-distribution cases, yet with reduced model parameters and computational costs. Codes are available P
Hence, offer new Mamba-based operator learning model incorporating more holistic features, with favorable properties in theory and excellent performance in practice.
2 Related Work
2.1 Learnable PDE Solvers
The development of deep learning has revitalized various fields [22]. In recent years, numerous investigations have been conducted on the application of learning networks to solve PDEs 23 24). A prevalent treatment involves initially encoding the data spatially, followed by the employment of diverse schemes for temporal evolution. For example, physics-informa approaches utilize PDE supervision to approximate the solution (25), commonly parameterized by neural networks, within a consolidated space-time domain. Yet, the accuracy is negatively correlated with solving efficiency. Moreover, retraining is required across different instances of a PDE.
To design more robust and efficient algorithms, Lu et al. 6 pioneeringly introduced the practical implementation of universal operator approximation theorem [5], which further be integrated with prior knowledge of the system [26]. Independently, infinite-dimensional solution operators approximated by iteratively learnable integral kernels (7). Such kernel can be parameterized by message-passing (27). neural networks (28), attention mechanism [1] (2],, convolution in Fourier domain [8] or in bandlimited function space [13]. Following this, we further present novel form
called as mamba integration, capturing global function information in O(N) time complexity.
2.2 Alias-free Framework
Aligned with the discussion in 30 a neural operator or an operator learning architecture should hold the ability in managing functions as both inputs and outputs. Yet, in digital environments, engagement

---

and computation with functions are implemented via discrete representations, e.g., grid point values, cell averages, or generally coefficients of a specific basis, instead of the original continuous entities. Recently, the alias-free framework [5]. in which the aliasing error is zero, was proposed. We provide a detailed definition in the Supplementary Material (AT) Within this framework, continuous functions should be consistently, or uniquely and stably, derived from the discrete counterparts, e.g., point evaluations, basis coefficients, etc., at any resolution, hammering at reconciling potential inconsistencies between operators and their discrete representations. Models that do not follow this framework, such as CNNs, may generate inconsistent outputs as the resolution varies, and thus Representation equivalent Neural Operators (ReNO), whose definition is given in SM Similarly, the property of continuous-discette equivalence (CDE), proposed by [15] that measures the consistency between discrete and continuous representations, cannot be guaranteed. In addition, the higu-freyuency information introduced by point-wiss wist activation functions also leads to similar inconsistencies 31 20) Therefore, FNO cannot be considered as a ReNO that meets the alias-free framework. More recently, CNO was elaborated as the first instance that conforms to the alias-free framework (13), paving the way for the implementation of our MambaNO.
3 Mamba Neural Operator
Setting. The core purpose here involves learning a mapping between two infiniteddimessional spaces via finite array of observed input-output pairings from this correspondence. Generally, the problem concerned can be formally delineated as follows. Let X' = HD),,, Rdx) and y (D, gd be Sobolev spaces functions defined on 2-dimensional bounded domains = T and g : X + y be non-linear map. By constructing a parametric map Gg. the practice would be to build an approximation of gf from data pairs (u;,9 (.(()(()) i=1 E x VxVi i.e.,
(3.1) with parameters from the dinteedimensonall space R. by seeking Øt R so that Got gt. The practical learning of Go can be naturally addressed through the empirical-risk minimization problem,
: y, E
N Σ (g+) - Go i=1
min (u) - Go(w)) BERP
min DERP
u eE
(3.2)
N
Given that our methodology is conceived within the dimensional approximation enjoys a shared set of network parameters which are consistent in infinite environment devoid of approximations.
context, each finite-
Bandlimited Approximation. Since the original Sobolev space is too large to allow for any form of continuous-discred equivalence (A.I). we select subspace B instead of the original H. On that basis, the possibility of achieving equivalence between the underlying operator and its discrete representations would be basically guaranteed. In this respect, we elaborate the space of bandlimited functions defined by
(D) = e L²((D) : suppp S [--w, w]?),
(3.3)
in which > 0 denotes the frequency bound and f represents the Fourier transform of f. Note that a bandlimited function can approximate the original function with arbitrary precision (depending on w), then a bandlimited operator mapping between bandlimited functions can also approximate the original operator with arbitrary precision. In other words, for any & > 0, there exist w and continuous operator Go- (D) B. such that g Goll E, with ||-I| pertaining to the corresponding operator norm. In addition, let Pur denote a certain Fourier projection, (v) is capable of discarding the any function in that space.
components higher than frequency w, where g H(D)
Definition of MambaNO. As the underlying operator maps between the spaces of bandlimited functions, the operator approximation architecture shall be constructed in a structure- preseervig fashion. That is to say, it is dedicated to form corresponding mapping in-between bandlimited functions, thus respecting the CDE property. To that end, a Mamba Neural Operator (MambaNO) is engineered approximate the operator Go (D) - B,, Following the common paradigm of 32 33 our practical elaboration also lies in a compositional mapping.
most NOs 12
Go = Q o Pr (r + KT- + by by-1)) .... (m (Wo + Ko + bn)) o P,
(3.4)
3

---

Mamba Integration
MambaNO
Activatiin
(
HxWx1
Activation
HsW&C
Conv Integrationx2
b
Element wise Multiplication
W, Mamba Integration22
Mamba Intogration22
Element- wise Addition
Downsampling
Upsampling
C1
Convid
MLP
H w
2'2"20
Conv Integrationx2
W Lb.
State Space Model
W
Mamba Integratinn2
Mamba Integrationx2
Coneoo
Downsampling
Upsampling
Conv Integration
Act
H.W.
Conv Integration<?
W +b
W, b
Mamba Iategration32
Mamba Integraatonx22
Upsampling Downsampling
Comto
Downsampling
Upsampling
Act
H. W
Conv Integration
8
W 4
W
Mamba Integration22
Mamba Integrattonx2
Activation/Act
V
24
-o(v)
H w SC
E
Figure The overall architecture MambaNO.
where
P {u E (D, Rdx)} + {wo E Bu (D, [R))) Q {ur (D, RdT)} {u B (D,R"))
(3.5a)
(3.5b)
the lifting and projection mappings respectively, W. E Rd++idd, are linear operators in matrix form, b, Rd++1 are bias functions, 1)t are activation functions, and Pt are either upsampling or downsampling operators in each layer. Note that all these operations are performed locally or
pointwisely, following the principle of discretization invariance [34] to some extent. The general
formulation the kernel integral operator K,, {vr : pd {v441 parameterized by such as:
R Rd-11 is
(K)(v))aa(<)> - K2 (x(x), er (v))
Vx,y E D,
(3.6)
in which the parameter of kernel K2 is learnt from given data. For instance, FNO 8 35
employs
convolution as the primary operation, while Transformer-basee neural operators I 36
leverage
attention mechanisms. In this work, a new integral form fitting Mamba architecture is crafted.
Mamba Integration. For simplicity of the exposition, omit the subscript from Eq. (3.6)
hereafter, which is originally used to denote the number iterations during integration flow.
(K)(());a)(x>>>>>
K1 y, v(r), v(v))o(y)d
(3.7)
Following the tradition of FNO 8 37 first assume that K₁ R², x RR. Rd4 1xd, concerns little on the spatial variables (u(x), v vly)), but only on the input pair (x, y). Then, we let
K, u) = CE Be Be-Ay
(3.8)
where A, B and C temporarily constant parameters, for easier deduction purpose. To further ensure a possible employment the scanning pattern used in Mamba [77, we set the integration
interval y E (=o0, 2) instead of the entire definition domain D, hence Eq. 3.7 becomes
(K((()))))())= = f (CEAR. Be Be-Ayuu(y)dd
(3.9)

---

Clearly, 4z is independent of the integral variable from which Eq. 3.9 can be rewritten as: (K((()):a)(2)) Chh(2), with h(x) eAw
(3.10)
Furthermore, with simple operation of differential performed on I, we can get h (1) = Ah(x) + Bu(x),
(3.11)
which together with Eq. (3.10) leads
N/ (x) - Al(x) + Bu(x), u(x) -Ch(x),
(3.12)
where u(x) (z) More details of the deduction can be found in SM By now, we have seamlessly married the computation of kernel integral in Eq. (3.7) with State Space Model (SSM) 38 Drawing inspiration from the theory of continuous systems, the goal of Eq. (3.12) is to map two-dimensional function, denoted v(x:), u(x) through the hidden space (xr). Within this context, serves as the evolution parameter, while and C act as the projection parameters. To integrate Eq. (3.12) into deep learning paradigm, discretization process initially necessary. Note this transformation is crucial to align the model with the sampling rate of the underlying signal embodied in the input data, enabling computationally efficient operations. To address the drift and diffusion effects within most PDEs, unlike the Monte Carlo approximation used in conventional integrals, our approach employs the Scharfetter- Gummeel method [39], which approximates the matrix exponential using Bernoulli polynomials, and can be formally defined as follows:
Á = exp(AA))
(3.13)
B = (AA) (exp(AA) - 1) AB,
where A is a timescale parameter converting continuous parameters and B into their discrete counterparts A and B. The discrete representation of Eq. (3.12) can be formulated as follows:
h(*x)= =Ah(zx-1) + Be(xz), u(xx)) =Ch(xx),
(3.14)
More details can be found in SM A.3 Recall that our intention is integrate the two-dimensional function in a scanning manner for integration, capturing global information with O(N) complexity. In Eq. (.14). Mxx), serving as the hidden space, encapsulates relevant information about the
integrated points before Therefore, through Carleman bilinearization, we can construct a kernel approximate the nonlinear state space evolution 440 41), i.e., the output can be derived through global convolution:
K = (CB, CAB. CA
(3.15)
u(rx) = v(zx) K.
where K E R denotes a structured convolution kernel and k is the sampling points of input v.
Convolution Integration. Mamba integration enjoys global receptive field, yet believe that introducing local convolution integration would further bring benefits in capturing more holistic features. To commence with, the convolution integration for Kr (D) (L)) defined as:
Kx)((x) = = ( yyf(yd
Σ Kig Kigl (1)-
Vo E D,
(3.16)
where E Bw, is discrete kernel with size N, zi is the resultant grid points. Thus, the convolution operator can be intuitively parameterized in physical space, deviating far from the treatments of Fourier transformation and then followed by matrix multiplication, as FNO (8].
Upsampling, Downsampling, and Activation Operators. Ideal upsampling recognized as not altering the continuous representation, simply transforming the original function to larger bandwidth- limited space. In other words, just viewing the function from a limiied space as belonging to higher-bandwiee bandwidth space does not actually change any values of the function. The upsampling operators for some T6 > w are defined as,
Uw Bw, - (D(D), (2) f(x), Vx E D.
(3.17)
5

---

Cross- Scan
ONN) complexity
Figure 2:
The cross-scan operation integrates pixels from four directions with O(N) complexity.
On the other side, for certain w < w, the concerned function f E B can be downsampled to lower band Be by setting D B. (D) - [g, defined by
W0
/ hm ())(dd,, Var E D,
Dww (x)
(3.18)
where bw is an interpolation sine filter, playing the role of removing high-free high-frequency components exceeding the lower band limit w and converting the function from higher band-limited space B. to a lower band-limited space (D). As known, applying the activation function point- wise directly would break the band-limits of the underlying function space, potentially introducing arbitrarily righer-frequence information and causing aliasing errors [20]. We aim to apply the activation function within sufficiently large band-limited w space minimize the introduction of
high-frequency information and thereby reduce aliasing errors. To this end, we define the activation
operators in 3.4) as,
nee (D) B (D), f = ( (r Ye D.
(3.19)
Recall that the activation operator within a broader band-limited space confined in a 'sandwich-llke" structure, as shown in Eq. (344) with lifting P and projection Q lying at the edges and kernel integrations occupying the middle position.
Algorithm SSM Block (Mamba Integration)
Input: v(z), continuous funtion with shape [sampling points (N), dimension (D))
Parameters: A, an evolution parameter; A, timescale parameter; B and C, projection parameters Linear Projection Layer: Linear(-)
Output: u(x),, a function also with shape [sampling points, dimension)
I: A, B, C' Linear(v), Linear(v), Linear(v)
2: A AA)(AA)
3: B = (AA) (exp(AA) - I) AB
4: u(x) (A, B, (x())) X is discrete sequence that contains 22,"2111 TNN. 4.1: h(*x) = 1 + Bu(xx)
4.2: u(xx)) = Ch((aR)
4.3: w(x) (u(x1), ((12) [((N))
5: return u(x)
MambaNO Architecture. Given bandlimited functions as inputs and outputs, all the concerned ingredients can be assembled in a U-shaped operator architecture, which is graphically given in Fig. I As seen, the input function, say u E B. (D, RRxx) ) is first lifted and then processed through a series of layers. Five main layers are used, i.e., convolution integration (3.16). Mamba integration (3.15). activation layer (3.19). upsampling layers (3.17). and downsampling layers (3.18). Each layer is fed with band-limited function, and another binield function holding the same band returns as the output. As the entire operation flow runs solely in the channel width, the underlying bandlimits are confirmed since the spatial resolution left unchanged. Thus, MambaNO assumes function input and throws it into a set of encoders, where the input is space-wise downsampled but channel-wise widened. Then, the function passed through set of decoders, where the channel width shrunk, yet the space resolution is enlarged. In the meantime, the encoder and decoder layers sharing the same spatial resolution and bandlimits are coupled with resnet architecture. Thus, as we go deeper into the encoder flow, transferring high-ffrequecy information via skip connections is allowed, avoiding

---

which to be totally filtered out with the sinc operation. In other words, the high-frequenc information not just produced by the activation function but also altered through the intermediate modules.
Practically, the constant parameters A, B, and C. are better learned from the data, allowing attractive model adaptability. This together with the shifted integral interval, as shown in Eq. 3..)) enables cross-scan operation within the state space. As illustrated in Fig. 2 we choose to unfold sampling points into sequences along with rows and columns and then proceed with scanning along four different directions, i.e., top-left to bottom-right, bottom-right to top-left, top-right to bottom-left, and bottom-left to top-right. These sequences are further processed by the SSM block for kernel integration, ensuring that information from various directions is thoroughly scanned, thus capturing diverse function features. Afterwards, the sequences from the four directions are merged, restoring the output function. The pseudo-code for the SSM block is presented in Algorithm I
Continuous-diserete Equivalence for MambaNO. We have defined MambaNO (3.4) an mapping operator in-between bandlimited functions and that runs in a scanning pattern within a state space. In practice, like any other computational methods, MambaNO shall be performed in a discrete manner, with discretized variants of individual layers specified in AM.. Given the practical implementations of each elementary block, i.e., convolution, up- and down-sampling, activation, etc., we then prove the following proposition, whose details are given in SM B.2
Proposition 3.1 Mamba Neural Operator G (D. RAX) (D, is a Representation equivalent Neural Operator (ReNO). That is, MambaNO enjoys the CDE property.
More details on the notion of ReNOs can be found in SM A.2 As ReNO, MambaNO is representation-equivalee
allowing migration between grids of different scales, yet with little aliasing errors. This property, which can also be called resolution-discrete invariance, as significant characteristic of the neural operator, is highlighted in Ref. (8.
Complexity Analysis of MambaNO. Given a discrete counterpart u. E R#X of a two- dimensional continuous function, one can reshape it to get E RND with N H x W. For simplicity, we assume the sequence dimensions of A, A, B,C are all D. As shown in step 1 of Algorithm|| 1. the complexity of generating three learnable projections O(3N The dicretization steps 2 and 3 involve four matrix multiplications, which cost (²) Step 4 is the state space model with (N) complexity [42). In summary, the computations of Algorithm are all linear with the sequence length, i.e., O(N). Please refer to SM C for the time complexity of other models.
4 Experiments and Analysis
4.1 Experimental Settings
Training Details and Baselines. For fairness and reliability, all experiments are consistently con- ducted standardized platform with NVIDIA RTX 3090 GPU and 2.40GHz Intel(R) Xeon(R) Silver 4210R CPU. Several well-known PDE solvers are used as the competing baselines, such as CNO 13 FNO 8 DeepONet (DON) (). Galerkin Transformer (GT) 14. as well the very typical ResNet
43 and [4] architectures.
Representative PDE Benchmarks (RPB). As standard set of benchmarks for machine learning of PDEs, RBP focuses solely on wivedimensional PDEs since conventional numerical methods have already yielded quite pleasing outcomes miedimensional functions; however, procuring training data for those in three dimensions or higher is immensely cost-prohibitive. With these considerations in mind, RBP covers Poisson Equation, Wave Equation, Transport Equation, Allen-Cath Equation, Navier-Stokes Eqns, Darcy flow, and Flow past airfoils, which are defined on Cartesian domains. We have roughly listed the related information of the equations in SM
4.2
In and
Results
The test results for both the in- and diitr---iitiiittion evaluations from all competing models are shown in Table Specific for in-distribution experiments, it can be easily observed that, among all competitors, except for the Allen-Cahn equation, CNO enjoys an evident superiority compared to others. Moreover, our MambaNO, benefiting from the introduction of both global and local integrations, outperforms even further. Taking the Poisson equation as an instance, CNO performs twenty times better than FNO, while MambaNO further reduces the error by one-third. On the other
7

---

Table I: Relative median L test errors for various benchmarks and models.
In/Out
GT
Unet
ResNet
DON
FNO
CNO
MambaNO
Poisson
In
4.09%
1.05%
0.63%
19.07%
7.35%
0.31%
0.17%
Equation
Out
3.47%
1.55%
1.34%
11.18%
8.62%
0.33%
0.21%
In Equation Out
Wave
0.91 %
0.96%
0.70%
1.43%
0.65%
0.40%
0.38%
1.97%
2.24%
2.50%
3.12%
1.95%
1.29%
1.22%
Smooth
In 1.18%
0.59%
0.47%
1.38%
0.34%
0.29%
0.26%
Transport Out
666.07%
2.97%
2.73%
119.61%
1.97%
0.35%
0.34%
Discontinuous
In 1.70%
1.44%
1.41%
6.35%
1.26%
1.11%
1.08%
Transport
Out 27270.96%
1.62%
1.54%
140.73%
3,47%
1.31%
1.21% %
Allen-Cahn
In 1,30%
1.38%
2.36%
22.97%
0.87%
0.91 %
0.72%
Equation
Out 3.03%
3,28%
3.91% %
20.75%
2.18%
2.33%
2.11%
Navier-Stokes
In 4.61%
4.94%
4.10%
12.95%
3.97%
3.07%
2.74%
Equation Out 17.23%
16.98%
15.04%
23.39%
14.8 14.89%
10.94%
5.95%
Darcy
In 0.86%
0.54%
0.42%
1.13%
0.80%
0.38%
0.33%
Flow Out 1.17%
0.64%
0.60%
1.61 %
1.11 1%
0.50%
0.44%
Compressible Euler Out
In 2.33%
0.72%
1.89%
2.15% 3.08%
0.49%
0.39%
0.34%
3.14%
0.91%
2.20%
0.74%
0.63%
0.61%
Input
Ground Truth
Prediction
Prediction CNO
Prediction MambaNO
ea
oa
en
0A
04
0A
02
02 100
880
9800
98o 0.5
0.0
0.5
Input
Ground Truth
Prediction FNO
Prediction CNO
Prediction MambaNQ
0.5
we
08
ea
0.4
0A
B4
GA
0A
502 025 650
0.75
1.30
9g
0.00
625
050
0.7
1.00
9.00
025
6.50
0.75
1:00
0g
8co
6.5
0.50
675
1.00
880
025
0.0
0.5
1.00
Figure Visual predictions on representative in- (top row) and distr--disttiuution (bottom row).
hand, can that UNet and ResNet also behave well in some cases. This not only verifies the feasibility of convolution as kernel integration, but also shows that deep learning can directly ascertain the trajectory of an equation family from the data. However, the limitations also evident, lying specifically in the fact that little consideration is given on the alias-free framework. Therefore, when migrating to out-distribution experiments, the experimental results concerned show a significant decline. We can see that in the out-distribution experiments of the Navier-Stokes equation, their performance drops more than that of CNO, which another instance that adheres the alias-free modeling. MambaNO again sets a new state-of-the-ar score in this case, which not only follows this framework, but also benefits from the global integration. More intuitively, Fig. 3 plots the in-distribution and out-of-distributio visualization of the Navier-Stokes equation, taking the three best- perfoorrrrmig models, e.g., FNO, CNO, and MambaNO, as examples. Our first observation that, whether in the in-distribution or out-distribution environments, the predictions of MambaNO are the closest the ground truth. The regions within the black box provide the most clear comparisons. Note that FNO is limited by the pointwise activation function that introduces aliasing errors, and CNO limited by the convolution integral paying little attention to global information, while MambaNO has made improvements on both these issues.
4.3 Resolution Invariance
In the left and central segments of Fig. we compare UNet, FNO, CNO, and MambaNO vis-a-vis the metric of varying errors across resolutions, which is an important property for robust operator learning. The selected equation is the Navier-Stokes benchmark. In this figure, we can see that both UNet and FNO are practically not discrete invariant. With resolution changing, sharp decline

---

17.5
5.5
5.0
4.5
GT. SLUIN
Test Erro (in %)
8 4.0
7.2MM
MantoNo, JAON
12.5
4.5
3.5
Nect Emer
4.0
3.0
3.5
3.0
2.5 32
96
128
2.5
32
64
96
128
211
2
Testing Resolution
Tasting Resolution
Number of training samples
Figure Left Center: Test errors vs. Resolutions. Right: Errors vs. Training samples.
happens in the performance of UNet. For FNO, when the resolution is less than 64, there 25% decrease in performance. As resolution increases, a modest fluctuation in errors also occurs. Thanks the alias-free framework, CNO and MambaNO enjoy much flatter curves along with different resolutions, validating the advantage of respecting CDE property. Between these two, MambaNO further enjoys much lower errors, which is again attributed the combination of convolution and mamba integration, introducing the more holistic function features.
Variations in model performance against different data scales also evaluated Navier-Stokes. The right segment of Fig. 4
provides the test errors in the log domain. As shown, MambaNO consistently achieves optimal performance compared GT, FNO, and CNO, regardless of the number of samples used. More encouragingly, the performance lead is yet achieved with the least model parameters.
5 Ablation Study
To affirm the benefit from combining both global and local information, Table discloses results pertaining to the ablation of Mamba and Convolution integration. Based our final configuration, the mamba integration is replaced with convolution integral, thus obtaining baseline I with pure local terms. Similarly, ensure a fair comparison, the parameters of all three competitors are configured at the same level. As expected, due the lack of partial information, the performance of pure convolution integration and pure Mamba integration drops by 14% and 32%, respectively. In terms of efficiency, while pure convolution treatment enjoys less inference time and FLOPs in this case, it cannot be ensured in practice since more network layers are needed to enlarge the receptive field. Due to space limitations, have placed more ablation experiments in SM E
can also obtain baseline 3, which holds pure mamba integration. To
Table 2: The ablation results by using different components.
indicates our default choice.
Configuration
Test errors 4
Time 4
Params +
FLOPs J
1. Pure Convolution integration
1.14
0.83
0.98 x
0.79
2. Mamba-Convolatton integration*
1.00 x
1.00 X
1.00 x
1.00
3. Pure Mamba integration
1.32
1.14
1.02 x
1.19
Conlusion
We propose MambaNO, novel neural operator for solving PDEs, which incorporates Mamba integration and convolution integration to capture global and local function information holistically, yet in linear complexity. The basic design principle is to marry the integral kernel with the state space model. On that basis, with respect to spaces of bandlimited functions, CDE property is naturally satisfied authentically assimilate the innate operators, as opposed the discrete representation surrogation. A suite of experiments conducted on representative PDE benchmarks demonstrate that MambaNO outperforms the recent baselines, such as GT, FNO, and CNO, on many practical generalization, metrics namely test errors, running efficiency, resolution invariance, out-of-distribun and data scaling. Limitation: We have presented MambaNO for operators in a fundamental two- dimensional Cartesian domain. The expansion to three-space dimensions is theoretically intuitive, but computationally laborious. This intensifies the computational load for global integration, which inherently escalates the model parameters.

---

Acknowledgements
This work was supported in part by the Pioneer and Leading Goose R&D Program of Zhejiang under Grant 2023C01241, the National Natural Science Foundation of China under Grant 62276232, the Key Program of Natural Science Foundation of Zhejiang Province under Grant LZ22F030012 and Zhejiang Students' Technology and Innovation Program under Grant 2024R403B071.
References
[1] Kamyar Azizzadenesheli, Nikola Kovachki, Zongyi Li, Miguel Liu-Schiaffini, Jean Kossaifi, and Anima Anandkumar. Neural operators for accelerating scientific simulations and design. Nature Reviews Physics, pages 1-9, 2024.
[2] Alexander R Cobb, René Dommain, Kimberly Yeap, Cao Hannan, Nathan C Dadap, Bodo Bookhagen, Paul H Glaser, and Charles Harvey. A unified explanation for the morphology of raised peatlands. Nature, 625(7993):79-844 2024.
[3] Kevin Course and Prasanth B Nair. State estimation of a physical system with unknown governing equations. Nature, 622(7982):261-267, 2023.
[4] Ralf Konietzka, Alyssa A Goodman, Catherine Zucker, Andreas Burkert, João Alves, Michael Foley, Cameren Swiggum, Maria Koller, and Núria Miret-Roig. The radcliffe wave is oscillating. Nature, pages 1-2, 2024.
[5] Tianping Chen and Hong Chen. Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Trunsuctions on Neurul Networks, 6(4):911-913) 1995.
[6] Lu Lu, Pengzhan Jin, Guofei Pang. Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3):218-22) 2021.
[7] Anima Anandkumar, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Nikola Kovachki, Zongyi Li, Burigede Liu, and Andrew Stuart. Neural operator: Graph kernel network for partial differential equations. In International Conference on Learning Representations, 2020.
[8] Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, Anima Anandkumar, al. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations, 2020.
[9] Boris Bonev, Thorsten Kurth, Christian Hundt, Jaideep Pathak, Maximilian Baust, Karthik Kashinath, and Anima Anandkumar. Spherical fourier neural operators: Learning stable dynamics on the sphere. In International Conference on Machine Learning, pages 2806-2823. PMLR, 2023.
[10] Gege Wen, Zongyi Li, Kamyar Azizzadenesheli, Anima Anandkumar, and Sally M Benson. U-fno-an enhanced fourier neural operator-based deep-learning model for multiphase flow. Advances in Water Resources, 163:104180, 2022.
[11] Zijie Li, Kazem Meidani, and Amir Barati Farimani. Transformer for partial differential equations' operator learning. Transactions on Machine Learning Research, 2022.
[12] Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying, Yinpeng Dong, Songming Liu, Ze Cheng. Jian Song, and Jun Zhu. Gnot: A general neural operator transformer for operator learning. In International Conference on Machine Learning, pages 12556-12566 PMLR, 2023.
[13) Bogdan Raonic, Roberto Molinaro, Tim De Ryck, Tobias Rohner, Francesca Bartolucci, Rima Alaifari, Siddhartha Mishra, and Emmanuel de Bézenac. Convolutional neural operators for robust and accurate learning of pdes. Advances in Neural Information Processing Systems, 36, 2024.
[14] Shuhao Cao. Choose a transformer: Fourier or galerkin. Advances Neural Information Processing Systems, 2224924-249440 2021.
10

---

[15] Francesca Bartolucci, Emmanuel de Bezenac, Bogdan Raonic, Roberto Molinaro, Siddhartha Mishra, and Rima Alaifari. Representation equivalent neural operators: a framework for alias-free operator learning. Advances in Neural Information Processing Systems, 36, 2024.
[16] VS Fanaskov and Ivan Oseledets. Spectral neural operators. In Doklady Mathematics, volume
108, pages $226-S232. Springer, 2023.
[17] Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model. arXiv preprint arXiv:2401.0944 2024.
[18) Rui Xu, Shu Yang. Yihui Wang, Bo Du, and Hao Chen. A survey on vision mamba: Models, applications and challenges. arXiv preprint arXiv:2404.18861 2024.
[19] Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. Mip-splatting: Alias-free 3d gaussian splatting. arXiv preprint arXiv:2311.16993, 2023.
[20] Tero Karras, Miik Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Alias-free generative adversarial networks. Advances in Neural Information Processing Systems, 34:852-866 863,, 2021.
[21] Francesca Bartolucci, Emmanuel de Bézenac, Bogdan Raonić, Roberto Molinaro, Siddhartha Mishra, and Rima Alaifari. Are neural operators really neural operators? frame theory meets operator learning. SAM Research Report, 2023, 2023.
[22] Yueqian Quan, Honghui Xu, Renfang Wang. Qiu Guan, and Jianwei Zheng. Orsi salient object detection via progressive semantic flow and uncertainty- aware refinement. IEEE Transactions on Geoscience and Remote Sensing, 62:5608013 -5608025, 2024.
[23] Lu Lu, Xuhui Meng. Zhiping Mao, and George Em Karniadakis. Deepxde: A deep learning library for solving differential equations. SIAM review, 63(1):208-22) 228, 2021.
[24] George Em Karniadakis, loannis G Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang. and Liu Yang. Physics-informed machine learning. Nature Reviews Physics, 3(6):422-440) 2021.
[25] Zongyi Li, Hongkai Zheng, Nikola Kovachki, David Jin, Haoxuan Chen, Burigede Liu, Kamyar Azizzadenesheli, and Anima Anandkumar. Physics-informat neural operator for learning partial differential equations. ACM/JMS Journal of Data Science, 2021.
[26] Sifan Wang, Hanwen Wang, and Paris Perdikaris. Learning the solution operator of parametric partial differential equations with physics-informed deeponets. Science Advances, 7(40): eabi8605, 2021.
[27] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International Conference on Machine Learning, pages 1 PMLR, 2017.
[28] Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Andrew Stuart, Kaushik Bhattacharya, and Anima Anandkumar. Multipole graph neural operator for parametric partial differential equations. Advances in Neural Information Processing Systems, 33:6755-6766. 2020.
[29] Wenhui Peng, Zelong Yuan, Zhijie Li, and Jianchun Wang. Linear attention coupled fourier neural operator for simulation of three-dimensional turbulence. Physics of Fluids, 35(1), 2023.
[30] Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces with applications to pdes. Journal of Machine Learning Research, 24(89): 2023.
[31] Hagay Michaeli, Tomer Michaeli, and Daniel Soudry. Alias-free convnets: invariance via polynomial activations. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 16333-16342, 2023.
fractional shift

---

[32] Xiaoyu Zhao, Xiaoqian Chen, Zhiqiang Gong, Weien Zhou, Wen Yao, and Yunyang Zhang. Recfno: a iopluutioninvariian flow and heat field reconstruction method from sparse obser- vations via fourier neural operator. International Journal of Thermal Sciences, 195:108619, 2024.
[33] Makoto Takamoto, Timothy Praditia, Raphael Leiteritz, Daniel MacKinlay, Francesco Alesiani, Dirk Pflüger, and Mathias Niepert. Pdebench: An extensive benchmark for scientific machine learning. Advances in Neural Information Processing Systems, 35:1596-161,, 2022.
[34] Samira Kabri, Tim Roith, Daniel Tenbrinck, and Martin Burger. Resolution-invariant image classification based on fourier neural operators. In International Conference on Scale Space and Variational Methods in Computer Vision, pages 236-249. Springer, 2023.
[35] Ning Liu, Siavash Jafarzadeh, and Yue Yu. Domain agnostic fourier neural operators. Advances in Neural Information Processing Systems, 36, 2024.
[36] Min Wei and Xuesong Zhang. Super-resolution neural operator. In Proceedings the Conference Computer Vision and Pattern Recognition, pages 18247-18256, 2023.
the IEEE/CVF
[37] Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar. Fourier neural operator with learned deformations for pdes on general geometrics. Journal of Machine Learning Research, 24(388): 2023.
[38] Albert Gu, Karan Goel, and Christopher Re. Efficiently modeling long sequences with structured state spaces. In International Conference on Learning Representations, 2021.
[39] AA Kulikovsky. A more accurate scharfetter- germmmelerr-gmmml algorithm of electron transport for semiconductor and gas discharge simulation. Journal of computational physics, 119(1):149-
155, 1995.
[40] Yousef Saad. Analysis of some krylov subspace approximations to the matrix exponential operator. SIAM Journal on Numerical Analysis, 29(1):209-228, 1992.
[41] Yoshinobu Kawahara. Dynamic mode decomposition with reproducing kernels for koopman spectral analysis. Advances in neural information processing systems, 29, 2016.
[42] Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXi::2312.00752, 20233
[43] Kaiming He, Xiangyu Zhang, Shaoging Ren, and Jian Sun. Deep residual learning for image the IEEE conference on Computer Vision and Pattern Recognition,
recognition. In Proceedings the
pages 770-778, 2016.
[44] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assiste Intervention-MICCA 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings. part III 18, pages 234-2411 Springer, 2015.
[45] Zongyi Li, Nikola Kovachki, Chris Choy, Boyi Li, Jean Kossaifi, Shourya Otta, Moham- mad Amin Nabian, Maximilian Stadler, Christian Hundt, Kamyar Azizzadenesheli, al. Geometry-informed neural operator for large-scale 3d pdes. Advances in Neural Information Processing Systems, 36, 2024.
12

---

Supplementary Material Alies-Free Mamba Neural Operator
A Definition
A.1 Operator Aliasing Error
Setting. Let U Dom U C H + IC be an operator between two dissociable Hilbert spaces, and let v { his and Φ {Ōx be frame sequences for H. and K, respectively, with synthesis terms To and To. Their closed linear spans are represented by My := span( : i E I} and Me = span k Kj. The pseudo-inverses Th and T initially given on My and Ma. respectively, can in practice be extended to the entire Hilbert spaces, i.c. T. H. + (I) and Té K. + (K).. "Operator aliasing" describes the relationship established by identifying the continuous operator U and its discrete counterpart u: (I) + e (K), which is determined by the input and output frame sequences (V, Φ). Then, we can build the operator Ta ouo' y H + K,, whose definition clearly depends on the choices of the frame sequences that we make on the continuous level. Any mapping can be interpreted as a discrete representation of an underlying continuous operator U. We now give the definition of the Operator Aliasing Error:
Definition: Operator A Aliasing Error. The aliasing error operator e eU, u, V, D): Dom U C is given by
- K.
s(U, u. Vr 4) U - o
(A.1)
and the corresponding scalar error is ||E(U, u, V, 1) with ||-|| denoting the operator norm.
An aliasing error of zero implies that the operator U can be perfectly represented by first discretizing the function with T applying u, and then reconstructing with Ta. If the aliasing error zero, we say that (U, u, V b) satisfies continuous-diserete equivalence (CDE). implying that accessing the discrete representation u is exactly the same as accessing the underlying continuous operator U.
How does such error occur? And how should we avoid it? Consider the operator U(f) = If
an operator from BR to B2n- The choice to discretize inputs and outputs on the same grid { ( Jez corresponds to choosing v = 4 = {sinc(25lx n.) me),, and to defining the discrete mapping u: (Z) + (Z) by u(v) T U o Tw (v) = O U. where @ denotes the entrywise product.
Then, for every f E Ba such that U(f) E B2n \ Ba, we have
<UU, u, V. ()) = U()- - To To
= - - # 0.
(A.2)
Aliasing error occurs in this process, U introduces new frequencies that exceed the bandwidth w. and these high frequencies cannot be represented by the sequence framework V = + (sinc n))eez. We can rectify this by sampling the output functions on grid with twice the resolution of the input grid. This corresponds to choosing = {sinc (4f2x - mez and to defining u = T U Tv. which simply maps samples from grid points {
n 20 nez into squared samples from the double
resolution grid { i nez This effectively removes aliasing since the equality U To a T satisfied. Furthermore, sampling the input and output functions with arbitrarily higher sampling rate, i.e., representing the functions with respect the system {sine(2Aæ n)]nez with S2 > 20, yields aliasing error since (sine(202x n)) meez constitutes frame for B2n Bn- The activation function that introduces high-frequency information can serve as a prime example of our above content.
We can determine different discretized versions u and u' of the continuous operator U by choosing different sequence frameworks V, & and V &', respectively. Therefore, the consistency between operations u and u' can be evaluated using the following error function.
Definition: Representation equivalence error. The representation equivalence error function r(u, u') ( (K), defined as:
T(u, ul) = u Tt. To o' o T Tw: and the corresponding scalar error is [l, u with |I-I1 denoting the operator norm.
(A.3)
13

---

A.2 Representation equivalent Neural Operators
Representation equivalent Neural Operators. (U, u) Representation equivalent Neural Operator for every frame-sequence pair V 2 that satisfies Dom U S My and Ran U SM. no aliasing errors occur, i.e., the aliasing enror operator is identical to zero:
E(U, u, V ②
or s(U,u) =0.
(A.4)
In this case, all discrete representations u( 1) are equivalent, demonstrating that they uniquely determine the same underlying operator U, whenever continuous-dicrrete equivalence property holds at the level of the function spaces. This also implies that we can unambiguously determine the discrete u(V V b) To U To consistent with the continuous counterpart U by fixing the representations V, concerned with input and output functions. practice, we shall have access to different representation grids of the inputs and outputs, and their associated representation error is zero, c.g., Tu, u') = (please refer to SM. A.3) where u' denotes u( D'). Now, we establish a connection between aliasing and representation equivalence.
In addition, assume that and u' are separate discrete forms of U, concerning frame sequences (L) ② and (I', ('), respectively. Then, the transformation between these discrete forms can be implemented through the formula:
u( V 4') = T To wV, 3 o T Tw
(A.5)
A.3 Discrete layers for MambaNO
Setting. Assume that the discrete multichannel signal obtained after uniformly sampling continuous signal a E (D) is: as E exxxxx then a, (. c) refers to the (i, coordinate of the c-th channel of the signal, where i, 1...s and 1...d.
Convolution Integral. First, we consider the convolution integral of a single channel, that is c 1. And, the discrete version of the convolution kernel corresponding Eq. (3.16) is K R In order to ensure that the input and output signals have the same spatial dimension 8 x 8, we choose to perform zero- zero-padeel parodpg of length & on the original discrete signal, where & = (k 1)/2. Then the discrete form of the convolution integral for single channel is:
K(aa) (a, * [i.
[w[m, n] ā_(i m.j n). i,j = l...s,
(A.6)
m,n==
where * represents the discrete convolution, and a, E 2()++2) represents the discrete signal after zero-padding.
Then the convolution of multi-channel discrete signal a₂ E Rexexdon and discrete convolution kernel Kw E k Xdout can be defined
& dum
(a, * K Hi.j..1
Σ
[mlm,n,c ā,li m.j n, c). 1.j=1...s,
(A.7)
m,n=-
where corresponds to the index of the output channel and to the index of the input channel.
Mamba Integral. It can be deduced from Eq.
3.10 that
F(x)= = 1=)))))
B(e-Ay)or (Ke-ArDo(y)dy
I" Ble B(e Ble-AAuuu(yl (o)))()
=F(O) +
(A.8)
-h(0) /-" Ble B(e-Anno(y)dy
Continuing the derivation, we can obtain
(xx) =h(0) /" B(e-ANy Ale--Ayy(u)dds
(A.9)
M(x) - + +e ex { Ble B(e-Avyo(y)dy
14

---

Assume z4 and Fx+1 are two adjacent sampling points after discretization, we will show below how go from h(xx) h(0x+1). Firstly, we provide their definitions below:
h(xx)) - A (10) + e Ann Io B(e B(e-Ao)u(y)dy.
four B(e B(e (e-AA)y(yddy
h(0++1) = e A + e
= =ellatttezt--0a) ((0) + cdleat(ounn-aa) (mm Ble B(e-Av)u(y)dg)
2K (e ((0)) + cA !
( B(e (le-An)u(y)dg)
B(e B(ee (ooA))u())dll emaus
xx
F4+1
((A +
(A.10)
Assume A Ik+1 2g. the step length parameter for the time interval, then
h(2x+1) e + [ ( B(ellann=-))o(g)ddee
(A.11)
According to the concept of zero-order hold, when A approaches 0, we can regard the value of function in the interval [Ttk 2x+1] constant v(xx+1), then Eq. (A.1l) can be written as:
/ ox+s
h(*K+1) (AA
dy
(()((()))) Bu(xx+ 1)
(
1o+7
(e Au) (e-Aul Auljdy,
IN
=e^~n(ox) (u Be (e-AoL-) -c-Axx))
(A.12)
=e^Ah(rr) + Bu( A
(c
(GA(ex-1-0N)) 1),
=(dA)(x)) ABU(EE+1)
AA (c - 1).
According to the Scharfetter-Gummet method 39 let = eAn B (AA) -1 (cA1- -1) AB,
then:
h((++1) Ah(xx) + Bu(I++1).
(A.13)
Lastly, with Eq. 3310 combined, obtain:
h( =A((zz) + Bu(xx+1).
(A.14)
u ((x+1) -Ch(x++1).
This is easy from Eq. (A.14) to Eq. 3.14
Upsampling and Downsampling Operators. For w > 0, let hn be the interpolation windowed-sinc filter defined as
(no, .1 sinc(2warg) sinc(2wen)) (00, E R
(A.15)
For a discrete, single-channel signal ag E Raxx let (n)(n) be its periodic extension into infinite length, e.g., a. [n] a, mod s) for n E Z. The discrete upsampling Ug Rx + R Nax Ns an integer factor N E N of the signal a, E Ix is defined by:
by
1. First, the signal is transformed into ag E NoxNs achieved by interspersing each pair of signal samples from a, with N samples of zero value. The formula is defined as follows:
ag N = (1) Is()) a,íd mod s. mod s), i,j= Ns.
(A.16)
where 5 = (1, +,, 1,... (N 1)s + 1) and Is is the indicator function.
15

---

2. Second, we use the interpolation filter ha/2 to convolve with the output of the first step. The formula is defined follows:
Us (as)
Σ T.,EZ
Ta,tNaln, m] h1/2(is ns, ms), i,j 1...... Ns.
(A.17)
Please note that the discretization step includes an additional convolution of an interpolation filter, which more than the up-sampling process of the continuous function. This is because the introduction of zero-value samples expands the signal size while introducing dis- continuity, namely high-frequened components. In order to eliminate these uig-frequency components, we need to apply interpolation filters remove these high-frequenge noises.
The discrete downsampling D..N I _ R*/Nxe/N by an integer factor N E N&s/N E N of the signal a₂ E R defined by:
1. First, we use the interpolation filter ha/2N to convolve with ag. The formula is defined as follows:
Σ
ān m| ha/(22) ns,js is ms), i 11......// s/N. (A.18)
M.MER
2. Second, resample the discrete signal 0s,a/N once every N points. The formula is defined as follows:
Da,n(a,)(i, a.iny=l(i 1)s + 1, 1)8 + 1],
i.j=1...884 (A.19)
The execution of multi-channel discrete upsampling and downsampling is accomplished by separate applications of the respective single-channel operators.
Activation layer. We define the discretized activation function layer as
1/s(a.) Dx,N o (a,)
(A.20)
where 17 R + IR is activation function applied point-wise. Through experimentation, we found that setting N achieves the best balance between efficiency and effectiveness.
B Proofs
B.1 Proof of Equation 3.12
Start with Eq. (3.10) and differentiate both sides with respect to a::
1 (x) Ae / B(e B(e~Ar)u(y)e Ax Be A7 u())
Ae Ar / B(e B(e-ddyu(y)dy Bc(x),
(B.1)
=AM(x) + Br(x).
Together with Eq.
3..9 we can obtain Eq. (3.12).
B.2
Proof that MambaNO a ReNO.
Setting. Follow Eq. 3.4 in MambaNO, one layer is defined as:
Pr (rr KT-1 br-1)).
(B.2)
Among them, only Pr. Tr, and Kr-1 involve mapping between bandlimited spaces, while WT-1 and by-1 are mappings within the bandlimited space. Mappings within the band-limited space do not introduce aliasing errors. Therefore, we only need to discuss the first three operators, e.g., kernel integration operators Kr-1, non-linear activation operators 77,, and projection operators Pr. In the following texts, we define Tw.. and 7 as the synthesis operators and the pseudo-inverses operator between B,, and (z22)
16

---

Kernel Integration Operators. Firstly, the form of the convolution integration is as follows:
[x)((x)=
km,ns Zm,n), E R
(B.3)
17,,12
where l E N, kimn E C, and 2m,n = form of finite sum, and each term f(z 2m.n) ) can regarded as a translation f. Since f is in still in B (IR2). Actions such multiplying by finite number
(2 2w Zw mm.ez The convolution operator is defined in the
E (RR2), these translations of constant coefficients km. and taking the sum will not cause the function to escape from this space. From another perspective, if the functions in B (TR?) have certain properties, such as smoothness and rapid decay, then the weighted sum and movement in convolution will not change the basic properties of the function. Therefore, we present the commutative diagram of the convolution integral:
B
Kw
Bw
Tv m
T
(7.2 (T(22)
E kd
where Σ f is an abbreviated expression form of Σm f 2m,n).
Secondly, the output of the mamba integration (also known as SSM) can be defined as global convolution integral like Eq. (3.15. Similar to the defined convolution integral, such global integral is a well-defined operator from B to itself.
B
SSM
B
Tv
Tt
(2 (
f (222
where K is global convolution kernel in Eq. (3.55..
Non-linear Activation Operators. Now, we assume the activation function 7h. as an operator from B., to Bw given that w Moreover, Dw, B, B,, serves orthogonal projection from By to Bw. while Uw By, - B is a natural embedding from B,, to B. Non-linear activation functions may introduce frequency components beyond the original band-limited space. To reserve spectral space for these potentially introduced high-frequency components, we decide to first boost the frequency of the hund-limited space B.,, where the original function f is located, that is Uw (J). (f) After applying the activation function, in order to maintain system consistency and signal band-limited characteristics, we use D to filter frequency components exceeding w, where D w can be seen as a kind of band-limited filter. Now we can provide the corresponding exchange diagram.
Ma,w
Da.
B
B
B
B
To
Tt
Toe
Tb
e (222
UnN
(2
(2
DAN
(2
In the diagram, Uw and DiT, are discretized as
Un T o olw Tw.. D=,v= D Tw
(B.4)
Therefore, the transformation between the continuous and discrete representations of the activation function a is consistent. This means that the operations we do on the discrete level will not lose information or introduce errrr when returning to the continuous level.
Projection Operators. As can be seen in Eq. (B.4). the co-existence of up- up-sampling and down- sampling operators ensures the consistency of band-limited space. Thus, in our implementation, we use the U-shaped architecture, which has an equal number of up-sampling and down-sampling operators. In summary, this exact correspondence between its constituent continuous and discrete operators establishes MambaNO as an example of Representation equivalent neural operators or ReNOs A.2). thus proving Proposition 3.1 of the main text.
17

---

C Time Complexity Analysis of Other Models
Given a discrete counterpart RHxWxC of two-dimensional continuous function, we can reshape u to get R RNC with N = H x W.
Table 3: Computational complexity of standard deep learning models. is the sampling points of a continuous function.
Models
Range
Complexity
GNO (kernel)
radius r
O(N(N - 1)))
Transformer
global
O(N2)
FNO (FFT)
global
((NlogN)
CNO
local
O(N)
MambaNO
global+local
O(N)
Graph neural operator (GNO). The integral formula of GNO is:
(KC())(<x)= = [ (u(e)
K(x, y) v(y)dy.
(C.I)
Σ (EU(
K(,, vly)gy
where K is a kernel (lypically parametrized by neural network) and qy E R are suitable quadrature weights. While the GNO can represent local integral operators by picking suitably small neighbor- hood (z) C D, the evaluation of the kernel and aggregation in each neighborhood ((x:) is slow and memory-intee intensive for general kernels k D x - R". For each point, we need to visit each of its adjacent points. As reported in [4), the time complexity of GNO is O(N(N 1))).
Transformer. The integral formula of Transformer- NO is:
(I(())(x))= /
(C.2a)
(K(v))(()) 20 ((vox), ((ys)))((s),
(C.2b)
(Z)())(
exp
Vd.
W. Wy)..
(C.2c)
exp (W,v(2,).M (.wtte))))
Vd,
from which could be perceived as employing the softmax function onto three transformed vectors with length nt,, Evidently, Eq. (C.2c) agrees closely with the widely adopted self-attention mechanism within transformers, in which the matrices Wg: Wk, W. Rd. xd: respectively represent the learning transformation of queries, keys, and values. As can be seen from Eq. (C.2c), it is necessary to go through two nested loops length n, N. Therefore, the time complexity of Transformer-base baseed NO is (
FNO. FNO replaces the global convolution in the time domain with multiplication operations in the frequency domain, therefore, its kernel integral is:
/ K K(r, k(z,g)o(yddy,
(k(v))(z))
( K(x y)v(yddy,
(C.3)
=F-1 (Ro (Fur) (z),
in which RG designs N multiplications, parametrized by dh therefore its time complexity is O(N). However, the time complexity of Fast Fourier Transform ONNlogN), so the total time complexity is ONNogN).
18

---

Time bilggratinn (sermdd
0.5
Transfomer: O(N)
(coradd
0.5
GNO: OININ-I)
0.4
EXOGFFIHONNGGN))
0.4
CNO: O(N)
MambaNO:
0.3
0.3
0.2
Tlme
0.1
0.0
256
64
192
256
Grid Resolution
Grid Resolution
Figure 5: Time of integration vs. Grid resolution.
CNO. The local convolution operator defined by
(K()))(x)) [ k(x
- xx(x-1)))0
y)b(y)dy
k Σ
(C.4)
Kaju(z 2zy);
4-=1
where f E Bu. is a discrete kernel with size E N, By the resultant grid points. Evaluating each point involves k22 multiplications. So the total time complexity is O(NA²) O(N).
From a quantitative perspective, as shown Fig. 5 we have measured the time required for different integral forms different numbers of grids. In which, the specific time complexity of GNO depends on the choice of degree, which is up to 1 in a graph with N points. The graphical representation illustrates that despite the time complexity of O(N) being concurrett for both, the duration necessitated for kernel integration in MambaNO is slightly lower than that required CNO.
D Representative PDE Benchmarks & Numerical Results
Poisson Equation. The prototype presented represents linear elliptic PDE,
-Au f, in D, u|aD 0.
(D.1)
The solution operator gt, mapping the source term f to the solution u, can be denoted as gt H Let us consider the source term:
f(x,8) = K2
Σ 5-=1
aij (62 (2) sin(wia) sin(Tjy), V(z,y) D,
(D.2)
with - 0055. While training the models, we set K = 16 in Eq. D.2) and select dij possess
a uniform distribution between [-1, Beyond in-distribution testing, we also conduct out-of-
distribution testing by configuring K 20.
For the training phase, we produce 1024 samples, and for the evaluation phase, we furnish 256 samples each for both in-distribution and out-of-distributin testing. These are attained by sampling the precise solution, u, resolution of 64 points on / 00, 1]2. validation set, composed of 128 samples, is also constructed for model selection purpose. The data utilized for training is normalized to the range [0. with the same normalization constants the training data. Fig. 6 random in-distribution testing sample and an dut-if-distribution testing sample, well as predictions made by FNO, CNO, and MambaNO, are depicted. When handling the Poisson equation, FNO is obviously inferior to CNO and MambaNO. In terms of handling details, MambaNO is superior to CNO.
Wave Equation. The given prototype represents linear hyperbolic PDE,
Uu è² Au 0, in D (O, 2o(x, y) f(z v).
(D.3)
19

---

Input
Ground Truth
Prediction FNO
Prediction CNO
Prediction MambaNO
10
14
24
88
22
822
02
P8
0.50
$0.00 622
Bap B75
Input
Ground Truth
Prediction FNO
Prediction CNO
Prediction MambaNO
28
84
oB
en
84
0.4
0
0
0
00
023
0:50
0.75
1.00
0.00
6:2
050
6.75
1.50
025
350
0.75
1000
8.00
623
0350
0Js
1.0
880
025
Figure 6: Visual predictions on representative in- (top row) and out-of-distribution (bottom row). (Poisson Equation.)
Input
Prediction
Prediction
Prediction MambaNO
0.6
06
0.6
06
06
Be
0.2
0.2
0.2
0.2
888 "80
3.00
0000 0.5 1.00
Input
Ground
Prediction
Prediction CNO
Prediction MambaNO
1.0
10
1.-
0s
ub
ue
ub
be
0.2
0.
0.2
880
0.00
0.75 1.00
Figure 7: Visual predictions on representative in- (top row) and out-of-distribution (bottom row).
(Wave Equation.)
with constant propagation speed 0.1. This represents multiscale standing wave exhibiting periodic pulsations over time, which are dependent on the variable K. Considering the initial conditions defined by Eq. (D.2) where 1 and K 24, it becomes possible to accurately compute the exact solution. The exact value at time > 0 given by
K
u(x,
01 (i2 +ン sin((iaa) sin(rjy) cos Call/+}3
V(z,)) E D.
K2
(D.4)
Our goal is to approximate the function at the moment when T = 5. For generating training and in-distribution test instances, configure 7 = 5, K 24, and assign aij uniform distribution value from 11, Conversely, for out-of-distribution evaluation, we adjust the decay exponent of 0.85 and K 32.
the modes
total of 512 samples are generated for the training set, and 256 samples each are generated for in-distribution and out-of-distribution testing by sampling the precise solution a 64 x 64 resolution. We generate 128 samples to create separate validation set. All training data are normalized to constants those employed for the training data. Fig. 7 a random in-distribution testing sample and an out-of- distribution testing sample, as well as predictions made by FNO, CNO, and MambaNO, are given. The predictive performance of CNO is better than FNO, but our MambaNO even better.
fit within the [0, 1] interval, and the testing data normalized using the
Transport Equation. The PDE is represented as follows,
+ Vu = 0, u(t 0) = S.
(D.5)
20

---

Input
Ground
Prediction
Prediction CNO
Prediction MambaNQ
1.0
1.6
0.8
0.
08-
0.
06
02
02
0.3
02
0.00
8.00
"8co 025
0.0
1.00
Input
Ground Truth
Prediction FNO
Prediction CNO
Prediction MambaNO
1.0
1.0
146
1.0
0.6
06
0.6
06
02
02
8.50
0.75
00
0.35
0.5
0.50
1.00
035
Figure 8: Visual predictions representative in- (top row) and (Smooth Transport.)
(bottom row).
with given velocity field and initial data By defining the constant velocity field as (Ua, (2y) = (0.2, 0.2), the solution u(x, y, 1) is obtained as w(x, y, () f(x Ugl, v,y)) We consider two distinctive training data types, i.e., Smooth Transport and Discontinuous Transport. Furthermore, in both instances, we generate 512 samples for training, 256 for validation, and 256 for both in-distribution and dustt--distiiuuttinn testing, all derived from the precise solution. Every sample is normalized to fit within the interval [0, 1].
Smooth Transport. The first kind comprises smooth initial data, configured as radially symmetric Gaussian. Its centers are chosen randomly and uniformly from the range (0.2, 0.4)². and the corre- sponding variance is consistently drawn from the set (0.003, 0.009). Formally, the initial conditions are given by
1 exp ( (2 -1
f(z) ((t))ddet(E)
I = == v), = (perPy). (D.6)
where oll such that a 2(0.003, 0.009) and pas My (0.2,, 0.4). I is an identity matrix and U((-) is the uniform distribution. Every sample is normalized to fit within the interval [o, I]. testing, the centers of the Gaussian inputs are uniformly sampled from the range (0.4, 0.6) implying that μı and fy are distributed uniformly between 0.4 and 0.6. The data is generated 64 x 64 resolution. In Fig. dittr--distiitutionn testing sample, as well as predictions made by FNO, CNO, and MambaNO, are provided. The figures further validate the conclusions that MambaNO outperforms CNO and FNO.
For
random in-distribution testing sample and an
Discontinuous Transport. The second variety is a discontinuous initial data, designed as the indicator function of radial disk. Here, the centers are uniformly pulled from (0.2, 0.4)², and the radii chosen uniformly from the span (0.1, 0.2). For out-of-distribution testing, the disk centers are pulled uniformly from (0.4, 0.6) Formally, the initial conditions are given by
f(x) = (s) (z), I (2, f (HarHyy)
(D.7)
where r U(0.1,0.2). Pz: My ~ ((0.2,0.4), I is an indicator function, and Sr(u) = {x ||5- ||11=1112 r the sphere of radius with the center μa. Due the infinite spectral content in discontinuous data, the aliasing error tends to persist throughout data sampling. To alle- viate this, firstly, we generate samples at 128 x 128 resolution. Then, by performing a frequency domain downsampling of the generated data, we obtain the actual samples at resolution of 64 x 64. In Fig. a random in-distribution testing sample and an -fi-s--diitiiuutionn testing sample, as well as predictions made by FNO, CNO, and MambaNO, are depicted. The accompanying figures substanti- the conclusions in Table I concurrently demonstrating a consistent superiority of MambaNO over both CNO and FNO in testing scenarios.
Clen-Cahn Equation. This serves a prototype for nonlinear parabolic PDEs,
U1 Au 2²u(u² 1),
(D.8)
with reaction rate of = 220. This refers to the underlying operator gt f +> u(., T) which maps the initial conditions denoted by f to the final solution recorded at final time T equal to 0.0002.
21

---

Input
Prediction
Prediction CNO
Prediction MambaNO
1.0
1.6
0.6
0.
0.8
08
06
02
02
0.2
02
0.00
8.00
80c 0.5
0.0
Input
Ground Truth
Prediction FNO
Prediction CNO
Prediction MambaNO
1.0
1.0
146
1.0
0.6
0.
0.6
06
o8
02
02
8.50
00.00
0:35
0.50
1.00
0.55
Figure 9: Visual predictions representative in- (top row) and out-of- (bottom row). (Discontinuous Transport.)
Input
Ground Truth
Prediction FNO
Prediction CNO
Prediction MambaNO
88
e8
0.4
B.4
6A
0A
0.55 0:50 0.75 1.00
0.00
6.25
050
075
1.00
08.0
025
650
075
1:00
8:0
6.8
0.0
6775
1.00
%80 025
050
075
10
Input
Prediction
Prediction
Prediction MambaND
08
8
06
66
0G
06
04
02
0.2
02
02
800
8.00
080
0.5
0.50
0.5
Figure 10: Visual predictions on representative in- (top row) and out-of-distribution (bottom row). (Allen-Cahn Equation.)
The prescribed initial conditions for both training and in-distribution testing conform to the format given in Eq. (.2)). where equals 1, K equals 24, and coefficients diy uniformly drawn from the range 1, 1]. For out-of-distribution testing, is set to 16, and the initial decay, denoted by r. is randomly selected from uniform distribution within the range [0.85, 1.15) of the modes in Eq. (D.2). This setup facilitates the evaluation of the model's 's capacity to infer varying dynamics of the system. Training and testing data generated using finite difference scheme
grid, with resolution of 642. Specifically, space is uniformly discretized at resolution s² 64 x 64, with Ar defined as 1/s. Subsequently, given that explicit method is employed, the time domain uniformly discretized with time step At approximately equal to 5.47 x 10-7 N is set equal to [T/At] + 1. Here, U,,, signifies u(iAz, Az. nAt) for i,j = 0. 1, 1.... s and = 0, 1.... N. Additionally, zero-valued ghost cells are incorporated at the boundaries. The finite difference scheme delineated follows:
At (U + U7 + U U211, 4UT,)) AIs (U U3 1),
U Ug
(D.9)
for i, 1, and n 0, 1,.,,..,N
N. The selected AL meets the CFL condition, ensuring
At < [Ax)² each of in-distribution and out-t--diittibuttin testing samples, maintaining a resolution of x 64. Normalization of the training data is performed within the range [0. 1]. The testing data are normalized using same constants as those used for training data. Fig. 10, a random in-distribution testing sample and out-of-distribution testing sample, well as predictions made by FNO, CNO,
2e The generation process yields 256 training samples, 128 validation samples, and 128
and MambaNO, are provided. Reaffirming the conclusions drawn from Table T these figures again demonstrate that MambaNO leads a slight edge compared to CNO and FNO.
22

---

Navier-Stokes Equation. The referenced PDEs provide model for the motion of incompressible fluids, with the formative principles being
Ue + (u Vju Vp vuu, div 0,
(D.10)
for the torus D T2, subject to periodic boundary conditions and viscosity of 4 x 10 which are applied solely to Fourier modes of an amplitude 2 12. This approach effectively models fluid flow under conditions of an extremely high Reynolds number. This refers to the underlying operator GT + u(-, 7) which maps the initial conditions denoted by f to the final solution u recorded final time T' equal to Initially, we adopt the following conditions.
tanh 2n 0.25 P for y =a(x) -
uo y)
tanh 2n 0.75-y
otherwise
(D.11)
Po y) =
where og: [0, 1] + R is a variable of the initial data provided by
(x(2)
Ogk sin (2nk2 - Bx))
(D.12)
The random variables a% and 3g are independently and identically distributed, with uniform distri- bution on the intervals [0. 1] and [0, 2T] respectively. The parameters and are set as 5 = 0.025 and p = 10. For the smoothing parameter, we select p 0.1. Our initial conditions reference the widely recognized thin shear layer issue. Here, the evolution of the shear layer occurs via vortex shedding, leading to a complex array of vortices. The creation of training and in-distribution testing samples employs spectral viscosity method, initiating with sinusoidal perturbation of the shear layer. The layer exhibits a thickness of p = 0.1 and comprises 10 perturbation modes, each uniformly sampled from the interval [-1, 1]. For the experiments conducted outside of the distribution, reduce p to = 0.09 and relocate the shear layers closer to the midpoint of the domain, positioning them at y 0.3 and y 0.7, opposed to the locations y 0.25 and y 0.75 in the original initial condition. This examination assesses the model's 's capacity to accommodate a flow regimen with higher count, and variable locations, of the shed vortices. For the training set, we produce total of 750 samples and generate an additional 128 samples for use in the validation set, as well in-distribution and diiti--distribution testing. The Navier-Stokes equations are simulated with spectral viscosity method on a 128 x 128 resolution to generate both the training and testing data, 64 x 64 resolution. The target is discern the operator
and then these data are downsampled that bridges the initial velocity to velocity at ''' = 1. The training data are normalized within the range [0, 1]. whilst the testing data employ the training data's normalization constants for its own normalization. In Fig. 3 a random in-distribution testing sample and an out-of-distribution test sample are given, as well as predictions made by FNO, CNO, and MambaNO. From the figure, we can see that MambaNO obviously performs better than FNO and CNO, particularly in the parts within the black box.
Darcy flow. The steady-state Darcy flow is described by the second order linear elliptic PDE,
(aVu) f. in D, ulen 0.
(D.13)
In this equation, represents the diffusion coefficient, while solution operator G symbolized as a + u, establishes a mapping from the diffusion coefficient a (expressed as push forward of a Gaussian process) the solution In this context, the input is the diffusion coefficient a, denoted as ~ #. Here, l signifies a Gaussian Process with a zero mean and squared exponential kernel
denotes the forcing term. The
[ y/ 12
k(,, v) o² exp
22 = 0.1.
(D.14)
The function 2 : RR- + R assigns value of 12 to positive components on the real line and 3 to the negative elements. For this particular scenario, the push-forward measure is characterized by a pointwise definition. The in-distribution and out-of-distribution samples are characterized by variations in the length scales of the Gaussian process. For in-distribution testing, a length scale of I = 0.1 is utilized. While for out-of-distribution tests, employ length scale of / = 0.05. For
23

---

Input
Prediction
Prediction CNO
Prediction MambaNO
100
1.6
0.6
06
06-
08
06
02
02
03
02
02
0.00
8.0
8co 025
050
075
Input
Ground Truth
Prediction FNO
Prediction CNO
Prediction MambaNO
10
e8
06
o6
04
02
02
80p
0.25
030
0.75
1.00
0000
5.5
0.0
0.75
1.00
8.00
0.5
5:30
0.75
1:00
880
6.25
0.50
6.75
100
0.00
Figure 11: Visual predictions representative in- (top row) and out-of-distribution (bottom row). (Darcy flow.)
Input
Ground Truth
Prediction FNO
Prediction CNO
Prediction
0.8
08
88
08
ои
04
0.4
04
044
0:50
0.75
8.00
5.25
0.7 1.00
0.5 1:00
0:50 ars
880 6.2
6.75
1.00
98 02s
u5e u.s
100
Input
Ground Truth
Prediction
Prediction CNO
Prediction MambaNO
1.0
1.0
1.6
10
0.5
0.
08
e6
02
02
b3
02
02
000
fm
025 0.50
Figure 12: Visual predictions on representative in- (top row) and out-of-distribution (bottom row). (Flow past airfoils.)
the training session, 256 samples are generated. Additionally, the study sees the generation of 128 samples for validation resolution of 64 x 64. In Fig. I1. a random in-distribution testing sample and an out-of- testing sample, well predictions made by FNO, CNO, and MambaNO, listed. The graphical representation clearly demonstrates that the performance of FNO cannot catch up those of CNO or MambaNO. In finer details, our model exhibits superior performance compared to CNO, thus corroborating the values presented in Table I
in-distribution, and nuitof-distribution testing. The data corresponds to
Flow past airfoils. This flow is represented utilizing the compressible Fuler equations in model,
+ Fiv 0, pv, E F [pv, pv + pl, (E' + p)]u]t.
(D.15)
The density velocity v, pressure p, and total energy E are interrelated the modular framework, guided ideal gas equation of state,
p 1
5 plu
(D.16)
where 7 1.4. Other significant variables related to flow complicate matters, such the speed of |a] The airfoils under consideration are characterized by perturbing to the established shape of the RAE2822 airfoil, implemented using Hicks-Henne Bump functions. Specifically, in alignment with standard protocols in aerodynamic shape optimization, we examine a reference airfoil shape where upper and lower surface of the airfoil are situated at (z, (2/c) and (z, (2/c)) respectively. Here, represents the chord length while y and Uler correspond to the renowned RAE2822 airfoil. The reference shape subsequently undergoes
sound a = / and the Mach number M =

---

perturbations through the application of the Hicks-Henne Bump functions,
10
10
(E) = (L) +
Σ
di Bi()),
) = ved +
Σ a; B,(
1=1
which satisfies
((( sin 8 )))
1n2
91
In 14 Ini'
{
d. - - 0z + 1) x 10-3
a U = -0 0.5) 10-3
i = 1,,,,, 10,
4 [0, 1)
We can now formally categorize the airfoil shape as S = E y S y²} and consequently, designate the shape function function. The foundational operator in focus, G + p. transitions the shape function f into the flow's 's density at the steady state of compressible Euler equations. The freestream boundary conditions
X[s] y). with X symbolizing the characteristic
T 11 M 0.729, 0 =1 = 2.31
are employed, with the solution operator projecting the shape function onto the persistent state density distribution. The data ultimately interpolated onto a Cartesian grid of dimensions 128 x 128 on the underlying domain D = [-0.75, 1.75|². Unit values are subsequently allocated to the density p(z, y) for all coordinates (x, y) within the set S. The forms of the training data samples align with 20 bump functions, with coefficients & uniformly drawn from [0, 1J2. Testing beyond the range of distribution conducted with 30 bump functions. Throughout the training and assessment stages, the discrepancy between the acquired solution and the actual reality is solely computed for the points (z, y) that exist outside the airfoil shape S. We produce 750 samples for the training set and 128 samples each for the validation set, in-distribution testing set, and out-of-distrbbution testing set. In this controlled examination, the data are not standardized, In Fig. 12] random ni-distribution testing sample and an out-of -i-ttiiibiiiioi testing sample, as well as predictions made by FNO, CNO, and MambaNO, are depicted. Whether in general or in local details, MambaNO has achieved the best performance.
E
More Ablation Studies
Please note, all the experimental data are relative to default choice, which is marked with From Fig. [ it can be seen that MambaNO assumes a function input and throws into a set of encoders, where the input is space-wisely downsampled but channel-wisely widened. The depth of the encoder determines how many scale-varied feature functions are within the model. Regarding the Depth, we design the ablation experiments concerned, with the results shown in Table 4 When Depth = 4, the model achieves the best balance among performance, inference time, parameters, and FLOPs.
As mentioned in the main body of the text, we expand the band-limited space of the original function using an upsampling operation before employing the activation operator. Therefore, the ablation experiments investigating the optimal value of upsampling factor N, are also conducted. The results are given in Table As can be seen, the case with no upsampling, i.c., NN,=l, suffers severe aliasing errors. while fourfold upsampling provides minor improvement, it significantly increases the computational burden. Therefore, we consider that a twofold upsampling rate achieves a well balance of efficacy and efficiency.
Meanwhile, in our cross-scan module, we used four different directional scans for the function. We wanted to understand the specific impact of different directional scans on model performance enhancement, so we conducted related ablation experiments, the results are as shown in the Table 5) We use the variable direction to represent the number of cross-scan directions in the experiment. Evidently, we observe a superior effect when the direction 4, thus motivating our selection of this value.
25

---

Finally, we conducted relevant ablation experiments on the number of integrations in the encoder and decoder, as shown in the 6 Here, [z. m. represents the number of integrations the encoder or decoder in Fig. [ from top bottom. When we discuss the encoder, the decoder is fixed the default selection with *; similarly, when discussing the decoder, the encoder is the default selection. Our model is the best choice among all configurations.
Table 4: The ablation results by using different parameters. indicates our default choice.
Parameter
Test errors
Time
Params +
FLOPs
Depth =2
6.07x x
0.63 x
0.08x
0.61 x
Depth =
2.94x
0,81 x
0.28x
0.82 x
Depth 4*
1.00
1.00
1.00x
1.00x
Depth =
1.30x
1,21x
3.73x
1.17x
Ny=1 y=2$
1,50x
0.78
0.87
0,83x
1.00x
1,00
1.00
1.00x
N
0.97
2.33x
1.52x
1.33x
Table The ablation results by using different amounts of directions in Cross-Scan, 'indicates our default choice
Cross-Scan Directions
Test errors
Time
Params|
FLOPs
a. direction=l
1.51
0.74x
0.81
0.75
b. direction=2
1.22x
0.85x
0.87 x
0.84
c. direction=4
1.00x
1,00
1.00x
1.00
Table 6: The ablation results by using different amounts of integration.
indicates default choice.
Encoder's Integration Layers
Test errors.
Time.
Params.
FLOPP$
Conv Integration
Mamba Integration
[1,1,1,1]
a. [1,1,1,1]
1.91x 1.00 1.05x
0.77x
0.65x
0.67x
b. [2,2,2,2]
b. [2,2,2,2]*
1.00x
1.00x
1.00x
[3,3,3.3]
c. [3,3,3,3]
1.94x
1.35x
1.33x
Decoder's Mamba Integration Layers
Test errors.
Time.
Params.
FLOPs|
a. [1,1,1,1]
1.58x
0.71
0.86x
0.85x
[2,2,2,2]
1.00x
1.00
1.00
1.00x
c. [3,3,3,3]
1.20x
1.53x
1.14x
1.15x
More Comparative Experiments
In Table the occasional performance decline observed with increasing model capacity can be attributed to underfitting due to insufficient data samples. To address this, we conducted comparative experiments on a larger dataset (10,000 samples. with 7,000 used for training), evaluating the same four O(N) complexity algorithms: Oformer [11], GNOT [12. CNO 13], and our MambaNO.
The quantitative results are presented in Table and we also provide efficiency metrics for more in-depth comparison. As shown in Fig. [3] the accuracy of our model has further improved. Overall, MambaNO still leads in performance among the four O(N) competitors, while ranking second in terms of epochs used and training time, and third in terms of GPU memory usage (close to CNO).
G Hyperparameters
For all competing methods, we used the optimal parameters recommended in the original papers. As for the proposed MambaNO, most of the hyperparameters, including encoder depth, upsampling factor, scanning direction, and integration layers, have been ablated, with results and practical settings
26

---

Table 7: (Left) Relative median L errors for OFormer, GNOT, CNO and MambaNO on the 2DCFD dataset (The compressible Navier-Stokes equations in 'PDEBENCH: An Extensive Benchmark for Scientific Machine Learning', ) (Right) MambaNO's performance on the Navier- Stokes equations under different dimension hyperparameters.
In/Out
OFormer
GNOT
CNO
MambaNO
Dimensions
16
32
48
In
1.71%
1.42%
1.35%
1.29%
Navier Stoker
In
4.34%
2.74%
2.70%
3.21%
Out
3.32%
3.22%
2.57%
2.13%
Equation
Out
8.52%
5.95%
5.92%
7.14%
GPU memory usage
15736M
5718M
8433M
8858M
GPU memory usago
5094M
8854M
15536M
22364M
Epochs
329
817
999
407
Epochs
999
802
851
907
Total Time
4.57h
5.22h
4.93h
4,75h
Total Time
3.1
3.27h
6.18h
15,02h
Ground Truth
Pedictionn OParme
hedistion GIOT
Tredittion CND
Prediction
E3p LD
BAY KSP Ko 100
No Ro0 Ko 1o
2o Ms so hediction
RO Bx Predstion
Nudittion
Predction Nambati
No
Figure 13: Visual predictions representative in- (top row) and out-of-distribution (bottom row).
Table MambaNO best-pentt rest-performing hyperparameters configuration for different benchmarks. 1/. 7
and w represent the learning rate, scheduler gamma, and weight decay, respectively. Convolution Integration and Mamba Integration denote the quantities of each in a single layer of kemel integration.
Size
Fyocha
Dimensions
Convolation Integration
Integration
(M)
FI.PP
Poisson Equation
1000
0.001
0.98
Ic:06
16
1.10
0.30
Wine Equition
1000
01001
4.33
Transport
Disconimoss Tranppott
1000
lc-06
Allen Cahn
1000
0.001
0.98
lc:06
32
2
2
5:40
1.23
Navie--Sokee
16
1000
0.001
0.9
1c-10
16
2
2
1.38
0.36
Dancy Flow
16
1000
0.001
0.98
le-06
32
2
2
5.40
1.23
Compressible Ealer
16
1000
0.001
0.98
1e:10
32
3
2
6.97
1.53
provided in subsection of the supplemental material. The remaining training parameters are listed in Table 8 In terms of different PDEs, most parameters were fixed across all experiments, with some fine-tuned for better performance.
H Broader Impacts
Our work will improve the speed and accuracy of solving PDE, which is beneficial promote the progress of related disciplines such as mathematics and physics; it has no negative impact on society.
27

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Abstract and Introduction are usually readers' first impressions paper, so they should accurately reflect the contribution and scope of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Deliberate discussion of the limitations of work is an important part of scientific research, which helps readers understand the scope research and possible directions for improvement.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
28

---

Answer: [Yes]
Justification: For each theoretical result, the paper provides a full set of assumptions and a complete (and conrect) proof in the main text or supplementary materials.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have described the entire process of the experiment in great detail in the main text and supplementary materials, and submitted the code of the model we proposed.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
29

---

reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
We have submitted the model code and part of the experimental dataset.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https SaweeeeeoooSSeeeeeiiiiii.e
/nips
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We have described the related content of the experiment in great detail in the main text and supplementary materials, and conducted ablation experiments on the selection of model hyperparameters.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: This is because our experimental design or data analysis method does not include the calculation or display of error bars, and believe that such information is not critical for our research question.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
30

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: The resources used for the experiments have been thoroughly explained in the main text.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics cttiics/ttptt...
Answer: [Yes]
Justification:
We fully adhere to the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discussed potential societal impacts in supplementary materials.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
31

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper does not carry such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
The paper properly credits the creators original owners of the assets used, such as code, data, and models. And we have correctly cited the relevant literature.
Guidelines:
The answer NA means that the paper does
existing assets. The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
32

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provide the code alongside the compressed file.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
33

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
34


Reviews:
[{"review_id": "NMPQQyRdyS", "reviewer": "Reviewer_oeYD", "summary": "The paper introduces Mamba Neural Operator (MambaNO) for solving Partial Differential Equations (PDEs) efficiently. Unlike existing methods, which are computationally expensive and often neglect global and local feature integration, MambaNO offers O(N) complexity. It balances global and local integration through a state-space model and alias-free architecture. MambaNO demonstrates accurate approximation of operators for PDEs and achieves state-of-the-art performance on benchmarks with fewer parameters and better efficiency.", "questions": "1. Could the author clarify the main applications of this type neural operator? And do they face the similar constraint of LLM models? In addition, why autoregressive models like MAMBA is more helpful than standard ML models? What are the benefits of causal dependency in these applications. \n2. Is the main application about function approximation problem or generic PDE solving? If it includes generic PDE solving, the author should also provide experiments on that.", "limitations": "1. In general for Neurips submission, the author should frame the paper towards generic compute scientist than specialist in this field. So, they should give clear definition, application and motivation of problem. The current intro and background are written in either very vague (related work) or techinical (mamba operator setting) manner.\n2. I am not an expert in neural operator learning. I would leave the judgment of effectiveness (e.g. figure 3) and novelty to other reviewers.", "rating": 5, "confidence": 1, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper is in general well-motivated and provides an promising MAMBA-based alias-free framework for solving PDE. \n2. The paper is well-written with nice visualization with some experimental verifications.", "weaknesses": "1. Since alias-free is an important property of the framework, it should move it from the appendix to the main body and provide a succinct summarization. Also, the paper should clarify the meaning of alias-free in the main body. Does it mean zero alias error? Can we view them as the reconstruction error for function approximation. \n2. The plots (figure 1,2,3) need better explanation of the setting. It is unclear from the plots what tasks they refer to and what does the those colorful patches/separation mean. \n3. Since MAMBA is a work that mainly address the system concerns, like memory and computation saving via kernal computation, the paper should make very clear comparison with the original MAMBA to explain their design choices. How does the MAMBA techniques improves the neural operator settings and why other methods cannot achieve it. E.g. Are there memory benefit. Furthermore, the paper makes several extensions to the original MAMBA work in Figure 1, like down/up sampling, conv integration, and the autoencoder style architecture. The paper should makes better justification why they adopt this way of stacking modules. \n4. Since the integration of MAMBA as the neural operator is not unique, the author should perform ablation study on the alternative ways of stacking MAMBA layers. Further ablation studies like varying the different MAMBA module inherent parameters will be helpful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1. Alias-free is truly an important property of the framework. However, two reasons lead us to move it into the appendix. First, our innovation lies in proposing MambaNO that follows alias-free property. The alias-free framework is innovated by other work [1], not ours. Therefore, we put more effort in clarifying our own ideas, yet with most of the detailed discussions on “alias-free” left in the appendix. Note that a brief introduction of this framework was already given in the main texts. Second, an alias-free NO means it is a representation-equivalent neural operator (ReNO), enjoying discrete-continuous equivalence. Therefore, we have also provided an explicit proposition claiming that MambaNO is actually a ReNO. This implies that the discrete implementations within the neural network are equivalent to the underlying continuous operations, minimizing aliasing errors as much as possible. As you stated, it does mean zero alias error and can be viewed as the reconstruction error for function approximation.\n\n2. Most previous works [2][3][4] have used the fluid-like patterns to visualize the variables of PDEs. In this work, we simply follow this typical scheme to demonstrate the performance of our methods. As suggested, more explanations will be given in latter version.\n\n3. Note that a new Mamba variant is not our core effort. Instead, we attempt to propose a new neural operator for PDEs. Along this line, we found that the kernel integral in traditional NOs bears certain similarities to state-space models, leading to the proposal of MambaNO. Compared with the original NOs, such as FNO[2] and CNO[3], rather than MAMBA, the proposal enjoys both local and global dependencies. The advantages of memory benefit and linear computation are also inherited. \n\n4. In fact, we have provided ablation experiments on the number of stacking MAMBA layers, as shown in Table 6 of supplementary material. Besides, the ablations of varying  inherent parameters by using different scanning mechanisms are also given, as shown in Table 5 of supplementary material. As suggested, we would move these contents into the main texts for clarity.\n\n5. As in most previous works [1][2][3][4], the primary application of NOs is solving PDEs. The constraints that need to be considered during application include, but are not limited to, the initial and boundary conditions of PDEs [5], Continuous-Discrete Equivalence in band-limited function spaces [1], and discretization invariance [2]. We believe these constraints are more theoretically dependent, different from LLMs those are more data driven. To the best of our knowledge, LLM has never been used in solving PDEs. The performance of MambaNO stems from multi-scale information and a scanning mechanism that handles PDE boundary and initial conditions. We believe that causal dependency is essential for the interaction between boundary and initial conditions.\n\n6. The primary application is the generalized solution of PDEs. Considering that 1D PDEs are relatively simple, we provide experimental data and visualization results for eight representative 2D PDEs in the original text to demonstrate MambaNO's potential for solving PDEs. Due to time and computational resource constraints, irregular shapes and higher-dimensional problems remain to be explored.\n\n[1]. Representation equivalent neural operators: a framework for alias-free operator learning. in NeurIPS 2024.\n\n[2]. Fourier Neural Operator for Parametric Partial Differential Equations. in ICLR 2021.\n\n[3]. Convolutional neural operators for robust and accurate learning of PDEs. in NeurIPS 2024.\n\n[4]. Transformer for Partial Differential Equations’ Operator Learning. Transactions on Machine Learning Research, 2022.\n\n[5]. Pdebench: An extensive benchmark for scientific machine learning. in NeurIPS 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BOoN0sPod7", "reviewer": "Reviewer_Jj9b", "summary": "The authors present a new operator architecture based on Mamba and convolutional integration. They show theoretically and empirically that this neural operator is discretization invariant and alias-free. The proposed architecture outperforms baselines across a variety of 2D benchmarks.", "questions": "How could you see adapting this model to larger, irregular systems? Resampling\nto a regular grid is likely not a feasible strategy with complex geometries or 3D\nproblems.\n- I am curious about the model’s capacity to learn across multiple PDEs or physics\nscenarios. Most large PDE models use some sort of attention mechanism or transformer architecture. [1, 2, 3] How do you see your model fitting into this ecosystem, where a pretrained model can be fine-tuned?\n1. Zhongkai Hao, Chang Su, Songming Liu, Julius Berner, Chengyang Ying, Hang Su, Anima Anandkumar, Jian Song, Jun Zhu, DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training, https://arxiv.org/abs/ 2403.03542\n2. Michael McCabe, Bruno Régaldo-Saint Blancard, Liam Holden Parker, Ruben Ohana, Miles Cranmer, Alberto Bietti, Michael Eickenberg, Siavash Golkar, Geraud Krawezik, Francois Lanusse, Mariel Pettee, Tiberiu Tesileanu, Kyunghyun Cho, Shirley Ho, Multiple Physics Pretraining for Physical Surrogate Models, https://arxiv.org/abs/2310.02994\n3. Maximilian Herde, Bogdan Raonić, Tobias Rohner, Roger Käppeli, Roberto Molinaro, Emmanuel de Bézenac, Siddhartha Mishra, Poseidon: Efficient Foundation Models for PDEs, https://arxiv.org/abs/2405.19101", "limitations": "-\tThe convolutional integration assumes a fixed grid, which makes this neural operator incompatible with irregularly discretized meshes. For practical physical applications, this could be a significant limitation.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The theoretical justification for the architecture is strong.\n- The empirical results show good performance across a variety of PDEs.\n- The proposed computational complexity of O(N) is great.\n- There are good ablation studies and discretization invariance results.\n- There is a good description of the dataset and methods, as well as the\ncomputational complexity analysis.", "weaknesses": "The lack of statistical significance of the results makes the benchmark comparisons somewhat less convincing.\n- I could not find hyperparameters, model details, or training details of the baseline models, which makes it difficult for me to evaluate if those baselines are comparable to the default configuration of the MambaNO.\n- The architecture may be less scalable than others, some of the ablation studies present decreased performance with increasing model capacity. If the authors could comment if this is caused by overfitting or model limitations, that would be great.\n- The proposed computational complexity is great, however, it would be great if it was supported by empirical timing experiments or comments on the efficiency of the model. Although faster, does the model need more epochs/training time compared to baselines? If the authors could comment on the training details of there different models that would help.\nOverall, the proposed architecture seems to perform well and is based on theoretically sound analysis. However, the lack of reproducibility and transparency makes the results less convincing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1. As suggested, we have provided the experimental data and their p-values in the uploaded PDF, showing that the differences between the experimental results are statistically significant. In other words, these differences are not due to any random fluctuations. Note that such tables are too redundant and often contain many repetitions since most p-values are <0.001, so we have omitted the presentation of p-values in the original submission. For all competing methods, we used the best parameters suggested in their original papers. All the selected parameters and the trained models will be released at GitHub for comparisons and evaluations. For the proposed MambaNO, most of the hyperparameters, including encoder depth, upsampling factor, scanning direction, and integration layers, have been ablated with results and practical settings given in subsection E of supplemental material. However, we omitted the training parameters in our previous submission. As suggested, they are now provided in Table 2 of the uploaded PDF. In terms of different PDEs, most of them are fixed in all the experiments, yet with partial of them fine-tuned for better performance. \n\n2. The phenomenon of occasionally performance declines with increasing model capacity can be attributed to underfitting (insufficient data samples). Thanks for your reminding, we have re-conducted comparative experiments on a larger dataset (10,000 samples, with 7,000 used for the training set) among four algorithms of the same O(N) complexity, i.e., Oformer[1], GNOT[2], CNO[3], and our MambaNO, the first two of which are suggested by another reviewer. The experimental results are provided in Table 3 of the uploaded PDF. As seen, the accuracy of our model has been further boosted. Note the efficiency metrics are also provided for a deeper comparison.\n\n3. As suggested, we have provided the training loss curves for FNO[4], CNO, and MambaNO in Fig. 2 of the uploaded PDF. As seen, FNO suffers from poorer performance that results from early convergence. The convergence trends of MambaNO and CNO, both with a time complexity of O(N), are basically consistent. With a deeper inspection, we can see that the performance fluctuation of MambaNO is more stable than CNO, with finally lower errors given same epochs. Therefore, neither more epochs nor more training time is required for MambaNO. As suggested, more training details are given in Table 2 of uploaded PDF file. As for better reproducibility and transparency, we indeed only provided the trained model for evaluation. During current stage, the attachment can not be updated, nor any link to website is allowed to provide. However, we promise to release the complete codes at GitHub in the coming days.\n\n4. For irregular geometries, GNOT [2] encodes shape features using an encoder into K and V, which are then combined with cross-attention. RecFNO [5] uses MLP to learn irregular or sparse observations and reshapes the obtained features into a regular shape. These approaches provide insights into how our model can handle irregular problems. We need to use an encoder to integrate irregular shape features and transform them into a form that can be used for subsequent processing. GINO [6] combines GNO and FNO to solve 3D problems. I believe that our MambaNO can also compensate for shortcomings in irregular shapes and 3D problems in this way. Additionally, for 3D problems, designing an efficient integral scanning scheme is also an issue to be discussed. So far, we have not attempted to pre-train the model on multiple PDEs and then fine-tune it. However, these three fine-tuning related papers have broadened our perspective. Thanks. Perhaps pre-training on different PDEs would be more beneficial for our model's performance on downstream tasks. We believe this is an interesting and worthwhile direction to explore. Thanks again for your valuable insights!\n\n5. Yes, fixed-grid convolutions may not be suitable for irregular discrete grids. However, as mentioned in response to point 4, there are already related works that address irregular shapes [2][5][6]. We believe that MambaNO also owns potential for handling irregular shapes, which will be a focus of our future research.\n\n[1]. Transformer for Partial Differential Equations’ Operator Learning. Transactions on Machine Learning Research, 2022.\n\n[2]. Gnot: A general neural operator transformer for operator learning. In ICML 2023.\n\n[3]. Convolutional neural operators for robust and accurate learning of PDEs. in NeurIPS 2024.\n\n[4]. Fourier Neural Operator for Parametric Partial Differential Equations. in ICLR 2021.\n\n[5]. RecFNO: A resolution-invariant flow and heat field reconstruction method from sparse observations via Fourier neural operator. International Journal of Thermal Sciences, 2024.\n\n[6]. Geometry-informed neural operator for large-scale 3d pdes. in NeurIPS 2024.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BwSduoH9Bu", "reviewer": "Reviewer_eofX", "summary": "This paper proposes a novel neural operator structure that applies the Vision Mamba architecture to neural operators. Additionally, it introduces an activation operator to mitigate the impact of standard neural network activation functions on bandlimited functions, thus reducing aliasing error. The method shows promising results when compared with several neural operator methods on both in-distribution and out-of-distribution benchmarks.", "questions": "No", "limitations": "No", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The strengths of this paper are as follows:\n\n1. The writing is exceptionally clear, making it easy to follow and effectively explaining the complexity reduction benefits brought by the Mamba structure.\n\n2. The method achieves impressive results, demonstrating the advantages of the Mamba structure. The paper also evaluates the method on datasets with varying resolutions, showing that the proposed approach maintains consistent performance across different resolutions.", "weaknesses": "The main drawback of this paper is the lack of experiments and benchmarks. The selected datasets and baselines for the experiments are not sufficiently representative. The authors should consider referring to datasets from PDEBench [1], or at least citing them.\nSince the primary advantage of the Mamba NO is complexity reduction, the paper should also compare it with other low-complexity models in transformers, such as linear attention models (e.g., OFormer, GNOT).\n\n**References**\n1. PDEBENCH: An Extensive Benchmark for Scientific Machine Learning (https://arxiv.org/abs/2210.07182)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "In the original texts, we presented the results of eight representative two-dimensional partial differential equations, demonstrating that MambaNO enjoys better accuracy yet with O(N) complexity. Thanks for your reminding of the new datasets and the two outstanding models. As suggested, we have added several experiments on 2DCFD dataset from PDEBench[1], comparing with the mentioned OFormer[2] and GNOT[3], as well as one original competing method CNO[4]. The numerical and visualized results are given respectively in Table 3 and Fig. 1 of the uploaded PDF file. It is shown that MambaNO still leads the performance among the four O(N) competitors, while ranking second in terms of used epochs and training time, as well as third in terms of GPU memory usage (close to CNO). \n\nNote that the main contribution of MambaNO is not simply complexity reduction. In fact, we propose a novel form of kernel integral that conducts holistic feature learning both locally and globally. On that basis, the theoretical analysis on the obeyance of discretization invariance and continuous-discrete equivalence is also provided. Moreover, the performance again is achieved with only O(N) complexity.\n\nFinally, we will cite this mentioned dataset and models in later version. The experimental results on the complete PDEBench dataset will be also added, given chance of revision. Thanks for your valuable suggestions!\n\n\n[1]. Pdebench: An extensive benchmark for scientific machine learning. in NeurIPS 2022.\n\n[2]. Transformer for Partial Differential Equations’ Operator Learning. Transactions on Machine Learning Research, 2022.\n\n[3]. Gnot: A general neural operator transformer for operator learning. in ICML 2023.\n\n[4]. Convolutional neural operators for robust and accurate learning of PDEs. in NeurIPS 2024.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pdmCZkqFq7", "reviewer": "Reviewer_5UBc", "summary": "This paper introduces a novel neural operator called MambaNO (Mamba Neural Operator) for solving PDEs. The key contributions atleast to me are:\n\n1. A new integral form called \"mamba integration\" with O(N) computational complexity that captures global function information. This is so cool! I give props to the authors for coming up with this!\n2. An alias-free architecture that combines mamba integration with convolution to capture both global and local function features.\n3. Theoretical analysis proving MambaNO is a representation-equivalent neural operator (ReNO) and can approximate continuous operators for a large class of PDEs.\n4. Extensive empirical evaluation demonstrating state-of-the-art performance across various PDE benchmarks.", "questions": "1. How do you envision extending the cross-scan operation to 3D or higher-dimensional PDEs? What challenges do you anticipate?\nHave you conducted any preliminary experiments on 3D PDEs to assess the scalability of MambaNO?\n2. Can you provide more insight into the relationship between the state space model used in Mamba and the specific requirements of PDE solving?\n3. While MambaNO achieves O(N) complexity, how does its practical runtime and memory usage compare to other methods, especially for large-scale problems?\n4. How well does MambaNO generalize to PDEs with very different characteristics from those in the training set? Are there certain types of PDEs where it struggles? Have you explored transfer learning approaches, where a model trained on one class of PDEs is fine-tuned for another?", "limitations": "They have presented the limitations!", "rating": 7, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. Novelty: The paper introduces a new approach to neural operators by adapting the Mamba architecture, which has shown promise in other domains. The combination of global (mamba integration) and local (convolution) operators is innovative in this context.\n2. Theoretical foundation: The authors provide a solid theoretical analysis, including proofs of representation equivalence and approximation capabilities. This adds depth to the empirical results and helps understand why the proposed method works.\n3. Comprehensive experiments: The evaluation is thorough, covering a wide range of PDE types and comparing against multiple state-of-the-art baselines. The inclusion of both in-distribution and out-of-distribution tests strengthens the claims of generalization ability.\n4. Performance improvements: The reported improvements in accuracy and efficiency are substantial across different PDE types, which is impressive given the diversity of the benchmarks.\n5. Alias-free framework: By adhering to an alias-free framework, the authors address an important issue in neural operators, potentially improving the model's stability and generalization capabilities.\n6. Efficiency: The O(N) complexity of the mamba integration is a significant advantage, especially for high-dimensional problems.", "weaknesses": "1. Hyperparameter sensitivity: The paper doesn't provide a comprehensive analysis of how sensitive the model is to various hyperparameters, such as the number of layers or the dimensionality of the state space.\n2. Potential limitations in handling multi-scale phenomena: To me atleast there are many PDEs that exhibit behavior across multiple scales. While the U-shaped architecture with up and downsampling operations addresses this to some extent, the paper doesn't deeply analyze how effectively MambaNO can handle problems with significantly different scales of behavior.\n3. Lack of discussion on boundary condition handling: The paper doesn't provide a detailed explanation of how different types of boundary conditions are handled within the MambaNO architecture, which is a critical aspect of PDE solving.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1. Most of the hyperparameters, including encoder layer, upsampling factor, scanning direction, and integration depth, have been ablated with results and practical settings given in subsection E of supplemental material. Note that for most NO applications, the dimension of state space is simply set as 16 or 32, hence we omitted its ablation in our previous submission. As suggested, the concerned experiments are provided in the uploaded PDF. The practical selections are dependent on the balance of efficacy and efficiency.\n\n2. Yes, handling different scales is a key issue in solving PDEs. MambaNO addresses the problem of different behavioral scales not only through the up- and down- samplings of its U-shaped architecture. Additionally, MambaNO combines global (Mamba integration) and local (convolution integration) treatments to learn holistic features, thereby better handling multi-scale phenomena. Besides, the ablations on U-net layers and the replacement of Mamba or convolution integration with naive modules are also given in the subsection E of supplemental material. However, the texts (deep analysis) on how effectively MambaNO can handle problems with significantly different scales are indeed insufficient, which will be fulfilled in later versions. In addition, we conjecture that introducing adaptive mesh refinement or incorporating physics-guided priors would tap the potential of enhancing the capabilities in multi-scale applications.\n\n3. Boundary conditions play a crucial role in solving PDEs because they define the behavior of the solution at the boundaries of the computational domain. We believe MambaNO's unique scanning mechanism can better handle the initial and boundary conditions of PDEs, allowing the model to utilize this information during the forward inference process. Reducing the scanning directions would also decrease the model's effectiveness, as demonstrated in our ablation study in supplemental material. In fact, techniques such as data augmentation, inputting more boundary information, adaptive boundary conditions, and multi-task learning can significantly enhance the model ability to handle different boundary conditions. We will continue to explore the limits of MambaNO's capability to handle various boundary conditions in the near future.\n\n4. To extend the cross-scan operation to 3D or higher-dimensional PDEs, we need to generalize the scanning process to handle additional dimensions. This involves iterating along multiple spatial axes and potentially managing increased computational complexity. In 3D case, this means scanning along the x, y, and z axes while ensuring the consistency and accuracy of the numerical solution.\nAnticipated challenges \nComputational Complexity: The increase in dimensions significantly raises computation and memory demands. \nData Handling: Managing large datasets in higher dimensions can be complex. This includes efficient storage, retrieval, and manipulation of data.\nAccuracy and Stability: Increased dimensions would heighten the potential for numerical errors and instabilities.\nUnfortunately, we have not yet conducted experiments related to 3D PDEs. Inspired by GINO [1], which combines GNO and FNO for 3D PDEs, we may next consider integrating GNO and MambaNO to address the 3D challenges.\n\n5. The state-space model is a mathematical framework that represents a physical system in terms of input, output, and state variables, which are related through first-order differential equations. PDEs typically need to be solved within specific spatial and temporal domains, requiring precise handling of boundary and initial conditions. The state-space model is particularly well-suited for managing these conditions. Specifically, the state-space model uses a scanning-like mechanism to handle information from initial and boundary conditions, integrating this information into the state equations to determine the system's state at the next time point. This approach is logical and intuitive because the state-space model systematically tracks the time evolution of the system's states, ensuring that initial and boundary conditions are correctly processed at each time step.\n\n6. As suggested, we have conducted experiments on larger dataset such as 2D CFD in PDEBench [2] and compared MambaNO with CNO[3] and other two O(N) competitors, including GNOT[4] and OFormer[5]. The numerical and visual results are given in Table 3 and Fig. 1 of the uploaded PDF, respectively. Evidently, the performance of MambaNO is ahead of other three competitors. In terms of memory usage, stopping epoch, and training time, our MambaNO ranks third, second, second, respectively. Note that all these four are with O(N) complexity.\n\n7. We believe that generalizing to very different PDEs would be a tremendous advance in this field. In the original texts, we provide both in-distribution and out-of-distribution test results for the same PDE, verifying that MambaNO has optimal generalization across different distributions. However, we have not yet conducted experiments on significantly different PDEs. We consider that the generalization capability for irregular or higher-dimensional PDEs still needs improvement. In the future, we will further explore transfer learning to enhance the ability of the algorithm in handling different PDEs.\n\n[1]. Geometry-informed neural operator for large-scale 3d pdes. in NeurIPS 2024.\n\n[2]. Pdebench: An extensive benchmark for scientific machine learning. in NeurIPS 2022.\n\n[3]. Convolutional neural operators for robust and accurate learning of PDEs. in NeurIPS 2024.\n\n[4]. Gnot: A general neural operator transformer for operator learning. in ICML 2023.\n\n[5]. Transformer for Partial Differential Equations’ Operator Learning. Transactions on Machine Learning Research, 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
At the outset, we would like to thank all four reviewers for their thorough and patient reading of our article. Overall, four reviewers have all complimented some aspects of our study. 

Specifically, Reviewer 1: The paper is well-motivated and presents a promising MAMBA-based alias-free framework for solving PDEs, and it is well-written with effective visualizations and experimental verifications.

Reviewer 2: The paper provides a strong theoretical justification for the architecture, with empirical results showing good performance across a variety of PDEs. The proposed computational complexity of O(N) is impressive, and the paper includes good ablation studies and discretization invariance results, as well as a detailed description of the dataset, methods, and computational complexity analysis.

Reviewer 3: The paper is exceptionally clear, making it easy to follow and effectively explaining the complexity reduction benefits brought by the Mamba structure. It achieves impressive results, demonstrating the advantages of the Mamba structure and evaluates the method on datasets with varying resolutions, showing that the proposed approach maintains consistent performance across different resolutions.

Reviewer 4: The paper introduces a new approach to neural operators by adapting the Mamba architecture, which has shown promise in other domains. The combination of global (Mamba integration) and local (Convolution  integration) operators is innovative in this context. Firstly, the authors provide a solid theoretical analysis, including proofs of representation equivalence and approximation capabilities, adding depth to the empirical results and helping to understand why the proposed method works. Secondly, the evaluation is thorough, covering a wide range of PDE types and comparing against multiple state-of-the-art baselines. The inclusion of both in-distribution and out-of-distribution tests strengthens the claims of generalization ability. The reported improvements in accuracy and efficiency are substantial across different PDE types, which is impressive given the diversity of the benchmarks. Thirdly, by adhering to an alias-free framework, the authors address an important issue in neural operators, potentially improving the model's stability and generalization capabilities. Additionally, the O(N) complexity of the Mamba integration is a significant advantage, especially for high-dimensional problems.

Besides, their criticism and constructive suggestions will enable us to improve the quality of our article. If our paper is finally accepted, we will incorporate all the changes that we outline below in a camera-ready version (CRV) of our article. As allowed by the conference, we are uploading a one page PDF that contains figures and tables on numerical experiments which support our arguments below. With this context, We proceed to answer the points raised by each of the reviewers individually, below.

Yours sincerely,

Authors of "Alias-Free Mamba Neural Operator".

Meta-review: This paper adapted the SSM arch to PDE operator learning problems. It receives generaly favorable reviews. However, here are some missing references in terms of mathematics. The authors should acknowledge these previous work in the camera-ready version.
- Rather than the "zero-order hold" the authors mentioned, the technique used to approximate the matrix exponential takes advantage of the Bernoulli polynomial approximation of $e^{xA}$, sometimes known as the Scharfetter–Gummel schemes in computational math.
- The original authors of Mamba SSM did not acknowledge this, but the construction of the kernel in (3.15) to approximate a nonlinear state space evolution is called Carleman bilinearization (or simply linearization sometimes), and the resulting form traces back to Krylov subspace, e.g., check Y. Saad's 1992 SINUM paper *Analysis of some Krylov subspace approximations to the matrix exponential operator*. The Krylov subspace has been widely applied to dynamical system in the form of the evolution of state spaces ever since, with application applied to ML especially in Koopman operators, e.g., Y. Kawahara NIPS 2016.
Decision: Accept (poster)