Coordinate Transform Fourier Neural Operators for Symmetries in Physical Modelings
Authors: Wenhan Gao
Ruichen Xu
Hong Wang
Yi Liu
Venue: Accepted by TMLR ()

Abstract: Symmetries often arise in many natural sciences; rather than relying on data augmentation or regularization for learning these symmetries, incorporating these inherent symmetries directly into the neural network architecture simplifies the learning process and enhances model performance. The laws of physics, including partial differential equations (PDEs), remain unchanged regardless of the coordinate system employed to depict them, and symmetries sometimes can be natural to illuminate in other coordinate systems. Moreover, symmetries often are associated with the underlying domain shapes. In this work, we consider physical modelings with neural operators (NOs), and we propose an approach based on coordinate transforms (CT) to work on different domain shapes and symmetries. Canonical coordinate transforms are applied to convert both the domain shape and symmetries. For example, a sphere can be naturally converted to a square with periodicities across its edges. The resulting CT-FNO scheme barely increases computational complexity and can be applied to different domain shapes while respecting the symmetries.

Published in Transactions on Machine Learning Research (10/2024)
Coordinate Transform Fourier Neural Operators for Symmetries in Physical Modelings
Wenhan Gao Stony Brook University
wewban.gnoo@stonnrooocco
Ruichen Xu Stony Brook Universiby
ruichern.zuu@sttnngrrrokk..t
Hong Wang" Stony Brook University
wanghong//7000@mmal..om
Yi Liu Stony Brook University
ji.liu./@sstnnsbrook.ddd edu
Reviewed on OpenReview: https: //
// openreview. net/ forum? id- PMD7A77K3
Abstract
Symmetrics often arise in many natural sciences; rather than relying on data augmentation regularization for learning these symmetries, incorporating these inherent symmetries directly into the neural network architecture simplifies the learning process and enhances model performance. The laws of physics, including partial differential equations (PDFs), remain unchanged regardless of the coordinate system employed to depict them, and sym- metries sometimes can be natural to illuminate in other coordinate systerss. Moreover, symmetries often are associated with the underlying domain shapes. In this work, we con- sider physical modelings with neural operators (NOs), and propose approach based on coordinate transforms (CT) to work different domain shapes and symmetries. Canon- ical coordinate transforms are applied to convert both the domain shape and symmetries. For example, a sphere can be naturally converted to a square with periodicities across its edges. The resulting CT-FNO scheme barely increases computational complexity and can be applied different domain shapes while respecting the symmetries. The code and data. are publicly available http:://ggtaabb..c///eaaaagggggu
1 Introduction
In numerous fields, such as electromagnetism (Bermúdez et al. 2014 researchers seek to study the behavior of physical systems under various parameters, such as different initial conditions, boundary values, and forcing functions. Traditional numerical mothods can be excessively time-consuming for simulating such physical systems. class of data-driven surrogate models, termed neural operators (NOs), provide an efficient alternative (Li et al. 2021, Rahman al.] 2023 Raonic al., [0223 Chen al. 2023). Neural operators approximate the mapping from parameter function space solution function space. Once trained, obtaining solution for new instance of the parameter requires only a forward pass of the network, which can be several orders of magnitude faster than traditional numerical methods (Li et al. 2021 A prevalent field of application is solving classes of Partial differential equations (PDEs). PDEs are widely used in modoling physical phenomena; for example, Navior-Stokos equations in fluid dynamics (Serrano et al. 2023) and Schrödinger equations (Dirac & M. 1981) in quantum mechanics. Although solving parametric PDEs is a major application of neural operators, they have wider applicability in general physical modeling, even without known physics formulations, such as in climate modeling (Bonev et al. 2023).
* Work done during internship Stony Brook University.

---

Published in Transactions on Machine Learning Research (10/2024)
Circular
Rectangular
Rectangular
Circular
FNO Layers
Rotation
Translation
Translation
Rotation
FNO Layers
a(x)
Polar Transform
Inverse Transform
Figure 1: Overview of CT-FNO. We first apply an appropriate coordinate transform so that the domain becomes rectangular, and the corresponding symmetry becomes translation symmetry. Then, FNO layers, which are translationally equivariant, operate in a different coordinate system predict output solutions. Finally, an inverse coordinate transform is applied back to the original coordinate system.
Various symmetries can be naturally observed the dynamics of physical phenomena. Symmetries within the underlying physical phenomena are reflected both in the physical model and its solution operators. Symmetries in physical systems can manifest in various ways, indicating specifie properties within the math- ematical formulation of the physical systems. For instance, in climate modeling, we observe rotational equivariance as the Earth rotates. In PDEs, if a PDE describing physical phenomenon remains unchanged when the spatial coordinates or time are shifted, it displays translational symmetry. Common symmetries include translation symmetries, rotation symmetries, scale symmetries, Galilean symmetries, and more. In- corporating these symmetries into network architectures can enhance model generalization, interpretability, and roduce lcarning complexity (Cohen Welling
2016 2017
Weiler al. 2023 Zhang ct al. 2023
Several group equivariant architectures have been proposed and explored in diverse applications to incorpo- rate symmtries (Kofinas et al. 2021 Weiler & Cesa 2019: Cohen et al. 2019 Bekkers et al. 2018 Fuchs et al. 2020). Group convolutional neural networks have achieved significant success in numerous symmetry groups, encompassing Euclidean groups E(n), scaling groups, and others. Nevertheless, group convolutions can be computationally expensive for general continuous groups (Cohen & Welling, 2017), and for some symmetry classes, particularly those arising PDEs, performing group convolutions can even be intractable.
In this work, we introduce the Coordinate Transform Fourier Neural Operator (CT-FNO) address symme- tries within physical modelings. Employing canonical coordinate transform, demonstrate how different symmetries can be seamlessly transformed into translation symmetries. Leveraging the properties of the convolution, Fourier layers in FNO exhibit equivariance to cyclic translations. Consequently, the CT-FNO proposed herein can effectively harness symmetries, offer generalization across various domain shapes, and does not introduce any new learnable parameters. Our contribution can be summarized in four parts: (1) We propose the CT-FNO framework, which extends FNO into various domains while respecting the underlying symmetries. (2) It is mathematically justified that CT-FNO is capable of approximating the behavior of an operator within specified precision, and this lays the foundation for our CT-FNO frame- work. (3) Experimental results reveal that CT-FNO can effectively maintain symmetries. In instances where symmetries exist, CT-FNO consistently outperforms FNO with minimum increase in computation cost. (4) Experimental results also demonstrates generalization applicability of CT-FNO across different domain shapes while respecting the underlying symmetrics, which not achievable by existing architectures.
2
Related Work
2.1 Neural PDE Solvers
Recently, neural PDE solvers have shown great success as an alternative to Iraditional numerical methods for solving PDE problems in many areas practical engineering and life sciences (Sirignano et al., 2020 Pathak
jet al. 2022. Zhang al.) 2022 Azizzadenehheli et al. (024). Traditionally, solving a PDE involves scoking smooth function satisfying the derivative relationships within the equations. Based on this view, Physics-
Informed Neural Networks (PINNs) have been developed to tackle PDEs individually. Another perspective
2

---

Published in Transactions on Machine Learning Research (10/2024)
is to regard differential operators as mappings between function spaces. Stemming from this perspective, neural operators have been proposed for solving families of PDFs (Anandkumar et al. 2019 9: Lu et al. 2021
Brandstetter et al. 2022). Neural operators parameterize solution operators in infinite dimensions and map the input paramcter functions to their respective solutions. Neural operators can be applied beyond PDEs; they can serve as surrogates for integral operators, derivative operators, or general function-to-function mappings (Fanaskov & Osclodets 2022). Among neural operators, Fourier neural operators (FNOs) (Li eet al. 2021) have drawn lots of attention. FNO is type of kernel integral solver in which the kernel integral is imposed to be convolution (Kovachki et al. 2023). Global convolution is performed via Fourier layers in the frequency domain. By leveraging low-frequency modes and truncating high-frequency modes, FNO captures global information and incurs low computational costs and achieves discretization invariance.
2.2 Equivariant Architectures
The concept of equivariance emerge in many machine learning tasks, especially in the field of computer vision. Convolutional Neural Networks (CNNs) are known to be equivariant to translations, characteristic that has propelled CNNs' significant success. Group convolution has been studied to achieve equivariance beyond translation (Cohen & Welling) 2016) such as rotation and scaling. A regular group convolution, within a discrete group, convolves input features with multi-channel filters in the group space; group action on the imput feature corresponds to cyelic shifts between channels. Later works extend equivariance to continuous groups, for example, utilizing steerable filters (Cohen & Welling 2017 Weiler & Cesa 2019)
and B-spline interpolation (Bekkers) 2020). Other than group convolution, Esteves et al. (2018) brings up an interesting idca of achieving rotation and scale equivariance by using a log-polar sampling grid. For symmetries in PDEs, Group FNO (G-FNO) (Helwig et al. 2023) parameterized convolution kernels in the Fourier-trausfomeed group space, extending group equivariant convolutions to the frequency domain. The resulting G-FNO architecture is equivariant to rotations, reflections, and translations. Nevertheless, group convolutions can be computationally expensive; for some symmetries arising in PDEs or physical dynamics in general, performing group convolutions can even be intractable. Symmetries such as rotation and translation are commonly studied in computer vision. However, those that exist in PDEs or physical dynamics, in general, are under-studied. Our work, instead of utilizing group convolution, offers an alternative approach to incorporating symmetries in neural operators through canonical coordinate transformation.
2.3 Relations with Prior Work
G-FNO (Helwig et al. 2023) is recent method that performs group convolution in the frequency domain. As the Fourier transform is factually rotationally equivariant, G-FNO also achieves equivariance in the physical domain. However, in both FNO and G-FNO, in order to benefit from the power of FFT, the domain has to be roctangular, which limits the symmetrics it can have. For examplc, with rotation symmetrics, a rectangular domain cannot be rotationally equivariant to arbitrary rotations. Unlike in computer vision tasks, where the convolution kernels are usually local and can have local symmetries, the kernel in FNO and G-FNO is global. Thus, the existence of local symmetries is unclear. For PDEs, the symmetries they can have often associate with the underlying domain shapes. Therefore, we propose CT-FNO to adapt to various domain shapes while also respecting the symmetries that the underlying domain shape can embody. Although both our work and G-FNO aims to incorporate symmetrics, we look at a different angle that takes both symmctrics and domain shapes into account, which cannot be solved solely by group convolution.
3 Preliminaries
3.1 Group Equivariance
Equivariance is a property of an operator, such as a neural network or a layer within one, such that if the input transforms, the output transforms correspondingly in a predictable way. A more complete review of necessary group theory and equivariance concepts within the scope of this work is included in Appendix A
Definition 3.1 (Equivariance). An operator Φ. X + y is said to be equivariant to a group G if actions of G on X and v, respectively denoted by I," :X - X' and 1. y + V, satisfy
Vg G: (T())) ( n) JEX,
3

---

Published in Transactions on Machine Learning Research (10/2024)
where X' and y are subspaces of a function space. In other words, the operator & commutes with actions of G.
In the context of operator learning, Lg and Lg be considered transformations, such as rotation, of the input paramcter function and the output solution function, respectively. Φ on the other hand, can be thought of as the solution operator that maps an input function to its corresponding output solution. Equivariance to various groups, including but not limited to SE(n), SIM(n), can be achieved by group- convolutions (Cohen & Welling 2016; WWiler al.) 2023). Let v(h) and s(h) be real valued functions on group G with (h) = (g h), the group convolution is defined as:
/
(U*K) (h)
J9EG v(g)x hh) dg.
where e can be regarded as the input feature and the convolution kernel.
Note that the first group convolution layer should be treated differently if the input functions not defined on G. For semi-direct product groups, RR x H, the first layer is lifting convolution that raises the feature map dimensions to the group space (Bekkers 2020).
Integrability over a group and the ioentification of the suitable measure, dg, are necessary for group con- volution. It has been shown that with the measure dg, group convolution consistently maintains group equivariance. For all G,
( v (a (g h1 dg
gEG
I v(b)s ((ab) (())) db
J u(b)x (b-la -1) db
=(U*R) (a (a-
= La (U (h).
Similar arguments can show that the first layer, sometimes referred to as the lifting convolution layer, also upholds group equivariance.
3.2 Fourier Neural Operators
Fourier neural operators learns to map an input function the solution function in infinite-dimensionaal spaces. Inspired by the kernel method for PDEs, each Fourier layer consists of fixed non-linearity and kernel integral operator K. modeled by network parameters, defined (Ao)(x:) = J n(x,, y)u(y)dy. As special case of integral kernel operator, translation invariance is imposed on the kernel, (k(2, v) <(x v)). which is a natural choice from the perspective of fundamental solutions. Thus, the integral kernel operator in FNO is defined as convolution operator
(Ku)(x) = [fe <(x yo)e-ydd
(1)
From the famous convolution theorem, convolution in physical space can be carried out as element-wise multiplication in the frequency domain,
(Ko)(x)= = (F Fo)
(2)
where and FF1 are the Fourier transform and inverse, respectively. Instead of learning the kernel the physical space, FNO directly learns FK in the frequency domain.
4 Coordinate Transform Neural Operators
In FNOs, the kernel integral operator a convolution operator over the translation group R",, equipped with its canonical additive group structure. This assumes that the kernel is translation invariant, thereby making the operator itself translation equivariant. Following this idea, FNO can be extended to respect
4

---

Published in Transactions on Machine Learning Research (10/2024)
(a)
(b)
(c)
(d)
(c)
(l)
Figure 2: Examples of shapes to which CT-FNO can be applied. (a) Sphere: Polar Coordinates (b) Torus:
Polar Coordinates (c) Ellipse: Polar Coordinates (d) Spherical Sector: Polar Coordinates (e) Cylinder:
Cylindrical Coordinates (f) Hyperbolic Plane: Hyperbolic Coordinates
other symmetries through canonical coordinate transform. The geometry and boundaries of the domain often dictate certain symmetries that the solution operators might exhibit. For example, if the domain is square, rotations other than multiple of 90 degrecs do not make sense. Traditional numerical schemes frequently maintain specific symmetry properties of the underlying PDE in different coordinate systems. Symmetries in physics, including PDEs, correlate with the shape and characteristics of the domain. For example, if the domain is circular, it does not make sense to think about (global) translation symmetries anymore. This motivates the pursuit of formulating a toordinate-transfor neural operator method that can be applied to different domain shapes and respect the symmetry of the underlying domaín shape. In spite of that, we propose the Coordinate Transform Fourier Neural Operators (CT-FNO) framework; CT-FNO is an extension of the FNO architecture in different coordinate system. A coordinate transformation is first applied to the underlying PDE systems such that the domain shape and symmetries can be natural for FNOs to handle. Then the PDE solution operator is approximated by the neural operator.
4.1 Universal Approximation with Coordinate Transform
In this subsection, we provide justification for applying FNO with coordinate transformation. Although it is intuitive that FNO can be applied to different coordinate systems, given that the laws of physics are not dependent on the coordinate system to which they adhere, one may still question the mathematical rigor of such operations. Therefore, we provide the following corollary to justify the use of coordinate transformation in FNOs. A prior work has established the universal approximation theorem for FNOs (Kovachki et al. 2021) ) importantly, the theorem asserts that, under certain conditions, FNOs maintain continuity as a mapping between Sobolev spaces; thus, FNOs can approximate the behavior of operator within specified precision over the considered function space (a given compact subset of a Sobolev Space). A complete treatment of this theoretical aspect is included in Appendix B In this work, we discuss modification to FNOs for the purpose of coordinate transformation, which is a parameter mapping; therefore, we present the following corollary.
Corollary 4.1 (Universal Approximation with Parameter Mapping). Let s,8' 2 0. Assuming that G, a continuous operalor mapping from HT(T,, to H² (Td,Rd.), holds true. For any compuct subsel contained in Ha (Tá:Rd)) Rd=) and any given positive e, there exists a modified Fourier Neural Operator, denoted .N. This operator N, mapping from (T((T, Rd.). to H (Td, incorporates the continuous and compact operators Pa and Pu on H'd) and H" (TA;Ra.), respectively. The structure of N" is per equation 10 and it maintains continuâty as an operator from Ha to Ha,, thereby ensuring the following condition is met:
sup ([ - ~ (a)ll SC. aek
This corollary incorporates additional compact operators P. and P., on the respective Sobolev spaces as parameter mappings (coordinate transform and its inverse), and such compact operators maintain the con- tinuity property of FNOs; thus, the modified FNOs can still approximate the behavior of an operator within a specified precision. Consequently, this corollary enables us to consider CT-FNNO by providing theoretical guarantee that CT-FNO can approximate operators in another coordinate system.
5

---

Published in Transactions on Machine Learning Research (10/2024)
58
15
20
30
(a)
(b)
(c)
Figure 3: (a): Plot of a function in Cartesian coordinates. (b): The resulting frequency information after applying 1D FFT to the function in polar coordinates for the angular axis. Rows the coefficients of the Fourier modes of the rings. This is generated from high-resolution grid in polar coordinates. (c): Plot of the same function in polar coordinates.
4.2 Coordinate Transforms and Symmetries
In this subsection, we introduce coordinate transformation and provide an overview of applying coordinate transforms adapt FNO to various domains while respecting the underlying symmetries. For illustration, we take rotation symmetries and circular domains in 2D as an example. While we focus on 2D circular domains for simplicity, the concept of coordinate transformation readily extends higher dimensions and other domain shapes. A coordinate transform, in mathematics, is mapping that relates coordinates in one coordinate system to coordinates in another coordinate system. Symmelries also transform along with coor- dinate systems. For instance, if the model is isotropic, we would consider convolution over the rotation groups SO(2). To perform convolution over SO(2), an approach outlined in Helwig et al. 20233 involves adapting the workflow of group-equivvaiian CNNs: rotating the convolution kernels to incorporate a dimension for rotations in the resulting feature maps:
(Ko) = (2.)))((z.))
(3)
(O) sin(w)
where Rs
(-sin(0) (i) cos(
E S0(2).
Letting æ = r (0)s = (0), the input function v(x, y) and the kernel <(0; y) are transformed polar coordinates as vr, 8) and s(r, 0), respectively. Although written in different coordinate system, the underlying physical dynamics remain unchanged. We adapt to the kernel convolution defined in Li et al.
2022 for polar coordinates:
- (KC)) = | ((t0)))
-)u((r de'
The natural domain shape and symmetry to consider are circular domains with rotation symmetries, denoted by Rgg E SO(2). Through the use of canonical coordinates for abelian Lie-groups (Segman et al. 1992) convolution equation ! respects the rotation symmetry. It is worth noting that while we have rotational symmetries, it does not make sense to translate the radial axis; in other words, the angular axis is periodic, but the radial axis is not. We apply padding to practically solve this issue. Moreover, the function basis for transformation should be treated differently these two axes are not equivalent. However, empirically, we observe that the FFT basis is still effective demonstrated in Section 5
Therefore, we have extended FNOs to circular domains while respecting the underlying symmetry associated with them. After transforming them into polar coordinates, the domain's 's shape nicely converts to a rectangle, enabling us to leverage the power of Fast Fourier Transforms (FFTss to compute the convolution:
(Ku)g F (Fx.
where & denotes (r, f).
Analogously, FNO can be extended to other domain shapes while respecting the underlying symmetries. As illustrated by examples in Figure CT-FNO can handle various domain shapes. Tt is important to note that

---

Published in Transactions on Machine Learning Research (10/2024)
Angular Arts
Angular Axis
Radius AxiS
(a)
(b)
(c)
(d)
Figure Different Sampling Methods (a): Uniform grid sampling in ratio of 1 for the radial and
augular axes, (b): Uniform grid sampling in ratio of 1 6. (c)(d): Sampled points in polar coordinates
corresponding to (a) and (b), respectively.
this list contains only a small portion of the shapes CT-FNO can operate on. It is not possible to provide a complete list, as there are numerous examples. These shapes, commonly found in numerical PDE literature, have broad applications across various natural sciences and engineering fields (Adler al. 2013 Leng et al, 2016; Nissen-Meyer al. 2014). It should be mentioned that there are numerous real-life applications that are naturally modeled in polar coordinates, and achieving rotational equivariance is desirable. Examples of such applications include global weather forecasting, water pipe modeling, electromagnetism, and more. Notably, previous work, such as Bonev et al. (2023), has successfully applied FNO to spherical domains for atmospheric dynamics forecasting. In our study, we generalize the application of FNO to different domain shapes Uhrough coordinate transformation. In another line of research, it has been concluded that some symmetries become manifest only in new coordinate system that must be discovered (Liu & Tegmark, 2022), and group convolution might not be suitable for such symmetrics. It is essential to note that CT- FNO has the potential to handle more complex domain geometries or to incorporate hidden symmetries within the underlying PDEs through canonical coordinate transformation.
4.3 Sampling Grid for Different Coordinate Systems
In this subsection, we discuss the importance of the sampling grid under different coordinate systems and propose sampling grid for circular domains as an example address this issue. Although operator learning is a lask involving the mapping of functions, which is grid-indeepndeet,, oflentimes, we only have access to function values at some finite collocation points. In FNO, to harness the power of FFTs, the sampling grid must be roctangular and equidistant. If the provided grid points are not equidistant, interpolation is applied to obtain uniform grids (equidistant grids) (Liu et al. 2023ch. If the PDE is described in a Cartesian coordinate system, and the inference for the solution is uniform a Cartesian coordinate system, it critical to choose a sampling grid that represents the function well. Taking polar coordinates as an example, this can also be observed in the frequency domain. If we apply FFT on the angular axis, as shown in Figure the magnitude of high-frequency coefficients increases as increases. For different applications, different samplings might be preferred, but the message to convey is that sampling grids have to be carefully handled after transformation.
To mediate this issue, we propose sampling an equidistant grid on the radius axis and the angular axis based on a ratio of 1: 6; this is to respect the geometric nature of a ring in Cartesian coordinate systems, where the ratio of the ring's length to its radius is 2 : 1. For example, 40 equidistant points are sampled over the radius axis, 240 equidistant points should be sampled over the angular axis. This 1 6 ratio heuristic made for simplicity; a 2n sampling and taking the closest integer for the angular axis can also be used. An illustration of different sampling grids is provided in Figure it is clear that the sampling on the right, which follows the 1: 6 rule, better represents the function in Cartesian coordinates under a similar number of total sampling points, Although this construction seems simple, it plays a crucial role in reducing the error of interpolation if the points come from a uniform distribution in Cartesian coordinates or an equidistant grid in Cartesian coordinates. We demonstrate this simple yet important aspect in Section 5.3.2

---

Published in Transactions on Machine Learning Research (10/2024)
5
Experiments
In this section, we evaluate the proposed CT-FNO based on the following criteria:
Symmetry Preservation Study: We demonstrate that CT-FNO is effective in preserving symmetries.
Generalizability Study: We showcase CT-FNO's broader applicability in terms of operating on other domain shapes while respecting the symmetries inherent in the domain shape.
5.1 Diffusion of Heat on a Cylinder
In this example, we show that the resulting CT-FNO architecture can work well on domains to which FNO or GFNO cannot be directly applied and outperform baselines that are applicable to such domains. Moreover, we demonstrate CT-FNO's ' capability to preserve symmetries.
Simulation Description. We consider the conduction of heat over time in an isolated medium defined the surface of cylinder. There is no heat flux across the top and bottom surfaces of the cylinder. As result, this system is invariant under translation and equivariant under rotations. We simulate such data with a 3D heat equation on the surface of cylinder:
OU (x. y, 1)
aAU(x, (1,,0.2,0) (2) [0
Ot aU (x, M. 2,
= 0, EDDD, 0z
where :=
<.<<6,, 6), U denotes the temperature, a = 1 denotes the thermal diffusivity of the matcrial, the axial coordinate, and denotes time.
= {(x, 2. E 123 + y² = 1,0 < <<<<<<<>>
Setup. We provide details on generating initial conditions in Ap- pendix Solutions are obtained by transforming this system into cylindrical coordinates and employing second-order finite differ- ence scheme for spatial domain and an implicit Euler scheme for time, using a spatial grid of size 128 x 128 and a temporal step- size of 0.01. The resolutions are down-sampled to 64 x 64 for training, with point locations stored in both Cartesian and po- lar coordinates for further usage. A total of 1100 different initial conditions are generated, and the corresponding solutions ob- tained. The dataset is divided into 1000 training data samples and 100 testing data samples, referred to as the Normal Testing Set. Additionally, each testing sample in the Normal Testing Set is rotated by a random degrec within the discretization tolerance, resulting in the Rotated Testing Set. is worth noting that rotations will alter the distribution of the data, potentially leading to suboptimal performance due to out-of-distribution issues. Therefore, it is desirable to have an cquivariant model to be robust under the Rotated Testing Set. Timing results are recorded NVIDIA Tesla V100 32GB GPU. All results are averaged over 10 different runs, and the mean and standard deviation of relative & error are reported. This is the setting for all subsequent experiments unless otherwise specified.
(b) Figure 5: (a): Sample Initial Condition (b): Sample Solution at t =
(a)
As the origin is fixed for all generated data; in other words, all spatial collocation positions are the same, and there no translation to be considered. Therefore, even for models that are not translation-invariant the performance will not be affected negatively. However, CT-FNO does maintain both translation invariance and rotation equivariance as natural result of coordinate transformation. Since the domain is not rectangular, FNO cannot be directly applied without sacrificing the power FFTs, We compare CT-FNO with baselines applicable to cylindrical surfaces. The results will be compared with GNO (Anandkumar et al. 2019), GINO [Li et al. 2023), and DeepOnet (Lu et al.) 2021). For GNO, GINO, and DeepOnets, Cartesian coordinates are used for node connectivity based on radius or as collocation positions. It worth noting that GNO can also be made equivariant by using relative information, e.g., distances; we included the comparison here. where we also show that this PDE is equivariant rotations and included more results and details for equivariant GNOs.
More experimental details can be found in Appendix C
8

---

Published in Transactions on Machine Learning Research (10/2024)
Table 1: Results on Diffusion of Heat on a Cylinder. CT-FNO achieves the best performance under the
normal testing set and it outperforms all other baselines to greater extend when symmetries exist. Standard deviations are given in perentheses.
Common Information
Normal Testing Set
Rotated Testing Set
# Par. (M)
(Tain(%)
Time (s)
Test (%)
Test (%) 0.774(0.08)
CT-FNO
2.18
00.47(0.02)
0.00197
0.773(0.08)
DeepOnet
2.40
1.508(0.967)
0.00085
1.659(0.992)
47.783(6.34)
GNO
4.61
3.079(1.37)
0.08062
5.885(0.20)
26.783(4.23)
GNO
4.61
2.763 (1.29)
0.08133
(.279
5.279
GINO 4.63
1.386(0..33)
0.02734
1.95)
33.374(4.99)
Evaluation ivaluation/infeeecee time for batch of 20 input initial conditions.
Compared to other baselines, DeepOnet is not discritization invariant on the input.
Equivariant Adaptation of GNO; details can be found in Appendix C.2
Results and Analysis. In Table | we present the results for the diffusion heat on a cylinder. CT-FNO outperforms all other baselines terms of accuracy under the Normal Testing Set. This result expected, as it is observed in many existing works that FNO outperforms GNO on rectangular domains (Li ct al. 2021; Liu et al. 2023a). Compared to DeepOnet, although CT-FNO is slower in inference time, the dillerence in accuracy is significant. We observe a large standard deviation for DeepOnet; thus, we include results from every single run in Appendix [ We rotate every data sample in the Normal Testing Set to create a new testing set of 100 data samples, denoted as the Rotated Testing Set. Since only CT-FNO is rotation equivariant, outperforms all other baselines to a greater extent. These results suggest. that CT-FNO can extend FNO well various domain shapes while respecting the underlying symmetries.
5.2 Synthetic Operator
In this example, we demonstrate that CT-FNO provides greater generalizability to various domain shapes. Since FNO operates on rectangular domains, we cannot directly compare CT-FNO with FNO or GFNO. However, we consider an inscribed square domain for the other baseline models demonstrate that the performance of CT-FNO is comparable to all other baselines while providing greater generalizability.
Simulation Description. We aim to learn synthetic operator G + where f + R is defined as
f(x,y)
Σ i-=1
aij (2 (2) sin(wii) sin()jy),
K
and u Ω + R defined as
K
u(x, v)
Σ
aij (12 + +2)) sin()ix) sin(xju))
TK
1.j
with / - -0.5, K - 16, and ai are ii.d. uniform on [-1,1] The dataset is created by setting different sets of random numbers dij.
Setup. We define the domain as a circle centered at. (0.5, 0.5) with radius of X for CT-FNO:
a= = ((T,)( E R² (z- 0.5) + (v - 0.5) < 2
For comparison with other baselines, including FNO, G-FNO, and Radial-FNO, we consider the domain be unit box: Ω {(,, v) E (0, 1) bounded by the circular domain considered in CT-FNO as demonstrated in Figure 66 For the square domain, the grids are 64 64, and for the circular domain, the grids are 26 156. We intentionally choose to have circular domain to show that CT-FNO can operate on circular domains, and its performance is comparable with other basclines on rectangular domains. We bound the square domain with the circular domain for testing purposes. In the shared square, the values are identical given the analytical expressions; if CT-FNO can perform well on the bounding circular domains, indicates that CT-FNO performs well on the square domain as well. This operator learning task inspired by Poisson Equation in Raonic et al. (2023): in fact, for the square domain, f and u are the input function and solution pairs for the Poisson Equation in Appendix D
9

---

Published in Transactions on Machine Learning Research (10/2024)
The training dataset contains 1000 input-output function pairs, whereas the testing dataset contains 200 input-ouiput function pairs. It is evident that this operator is equivariant to SO(2) rotations; we randomly select 30% and 60% of the testing input-output function pairs and rotate them by an element in the Ca group for square domains and by an arbitrary degree for the circular domain. These rotated pairs are then added to the testing set, referred to as Testing Set A and Testing Set B, respectively.
Results and Analysis. In Table 2. we present results for the Synthetic Operator. Considering that the square domain for other baselines is bounded by the circular domain for CT- FNO, we may conclude that CT-FNO achieves roughly the same level of accuracy compared to all other baselines. It can be seen that CT-FNO able to operate on circular domains while respecting the underlying rotational symmetries with- out noticeable increase in computational costs. Although we do not observe superior performance of CT-FNO over other baselines, our contribution focuses on the generaliz- ability of CT-FNO to extend FNO to other domain shapes while maintaining the underlying symmetries. Moreover, with such generalization, CT-FNO is robust under arbitrary rotations as the discretization tolerates, whereas other base- lines are constrained to C4 rotations due to the nature of square domains. The experimental findings in this example corroborate with the conclusion from Section I. that CT-FNO, similar to FNO, can approximate operators in another coordinate system with thcoretical guarantees.
Boundary Square Domain
Boundary Circuler Domah
Input Fundtio
Output Solution
Figure Sample input function and output solution. Input function and output solution values are the same in the shared square.
Table 2: Results on 2D Synthetic Operator. CT-FNO achieves rotational equivariance without significantly increasing computational complexily; the training time per epoch is close to that of FNO and much less than that of G-FNO. Moreover, it can be seen that CT-ENO can operate on circular domains while maintaining similar performance as all other basclincs. Standard deviations are given in parentheses.
Common Information
Testing Set A
Testing Set B
# Par. (M)
Train(%)
TPE3 (s)
Test (%)
Test (%)
FNO (C4)
2.37
0.412(0.03)
0.416
22.537(1.58
31.942(1.53
G-FNO (C.)
2.24
0.378
0.709
0.400(0.04)
0.402(0.04)
Radial-FNO (Ct)
2.63
0.386(0.03)
0.540
0.431(0.04)
0.434(0.04)
CT-FNO (Arbitrary) TPE: Average Time Per Epoch since G-FNO will increase the channel width and increase FFT computations.
2.31
(0.09)).06)
0.459
0.403
0.402
5.3 2D Darcy Flow Equation
5.3.1
Symmetry Study
In this example, we specifically demonstrate CT-FNO's capability to preserve rotation symmetries, even for rectangular domains.
Simulation Description. The Darcy flow equation a fundamental equation in fluid dynamics that describes the flow of fluids through porous media. It crucial in various fields like hydrogeology, petroleum engineering, soil mechanics, and environmental science. We consider the steady-state of the 2D Darcy Flow
equation from Li ct al. 2021 given by:
V (a()) = f()
(0, 1)²
u(x) 0 E 0(0 1)²
where a E - (0.,1)2; is the diffusion coefficient and f e La ((0, 1);; RR) is the forcing function that is kept fixed f(x) = 1. We are interested in learning the operator mapping the diffusion cocfficient a(x) the solution u(z). It can be shown that this operator is equivariant the C₁ rotation group (rotational
symmetry group of order 4) shown in Appendix F
10

---

Published in Transactions on Machine Learning Research (10/2024)
Setup. The imput diffusion cocfficient field a(., y) is generated by the Gaussian random field with piecewise function, namely a(æ,)) v) ()(),, where μ is distribution defined by J = N (0, (-A - 9))- The mapping s IR + R takes the value 12 on the positive and 3 on the negative, and the wush-forward is defined point- wise. Solutions are obtained using a sccond-order finite difference scheme on a 241 x 241 grid and then downsampled to 49 x 49. For CT-FNO, we use bilinear interpolation to convert Cartesian grids into 40 240 equidistant grids in polar coordinates.
For this 2D Darcy Flow, apply CT-FNO to achieve rotation equivariance. As the domain is a unit box, we test rotation equivariance under the C1 group. The results will be compared with FNO (Li et al, 2021), G-FNO [Helwig et al. 2023). and radial-FNO (Shen et al. 2022 Helwig et al., 2023), which is a frequeney domain radial kernel that is invariant to rotations. The data comes in an equidistant grid in Cartesian coordinates. To apply CT-FNO, we convert it to polar coordinates using bi-linear interpolation, and the solution will be interpolated back to Cartesian coordinates through bi-linear interpolation. Since the resulting domain is not rectangular in polar coordinates, reflection padding appliod. The reported errors are based on Cartesian coordinates. The training datasel. contains 1000 input-oulput function pairs, whereas the testing dataset contains 200 input-output function pairs, denoted as the Normal Testing Set. We rotate every testing sample in the Normal Testing Set by an clement in the C4 group and denote this testing dataset as the Rotated Testing Set to test rotation equivariance.
Table 3: Results on the 2D Darcy Flow equation. CT-FNO achieves the best performance without signifi- cantly increasing computational complexity; the training time per epoch is close to that of FNO. We rotate the testing input-output function pairs and rotate by an element in the C4 group; these testing data-set is called the Rotated Testing Set. Standard deviations are given in parenthescs.
Common Information
Normal Testing Set
Rotated Testing Set
# Par. (M)
(%a
TPE* (s)
Test (%)
Test (%) (0.10)
FNO G-FNO
2.37
0.333
0.413
(0.04)
2.24
0.315(0.03)
0.710
(.605
(0695)0.05)
Radial-FNO
2.63
(..34(0.04)
0.539
0.713(0.08)
0.713 0.669(0.10)
CT-FNO
2.31
0.276(0.06)
0.462
0.664(0.10)
8 *TPE: We record the average time per epoch since G-FNO increases the channel width and increase FFT computations.
Results and Analysis. In Table 3 we present the results for the Darcy Flow equation. CT-FNO performs slightly better compared to all other base- lines under normal data conditions. However, it does not provide enough evidence that CT-FNO per- forms better Ihan other baselines under normal lest- ing data, and this is expected as the objective of CT-FNO is to target symmetrics and generalizabil- ity to other shapes for which FNO cannot be di- rectly applied. For C4 testing data, the performance of FNO degrades moderately, although not as obvi- ously as in previous examples as the coefficient func- tions are sampled from Gaussian random fields and then mapped to piccewise constants, and thus, the distribution of the rotated testing set does not devi- ate much from the normal testing set.
Table 4: Results for CT-FNO on 2D Darcy Flow un- der different grid samplings. The more it deviates from the proposed grid sampling, the larger the error becomes. CT-FNO is trained on grid values in polar
coordinates obtained by interpolation.
Sampling Grid Size
Test (%)
Train (%)
20 x 120*
0.687
0.276
30 80
0.738
0.301
40 x 60
0.847
0.331
49 x 49
0.966
0.354
15 x 160
0.754
0.359
12 x 200
0.873
0.432
10 x 240
1.057
0.501
8 x 300
1.361
0.616
Proposed Sampling Grid
CT-FNO, G-FNO, and raO are all cquivariant to rotations; thus, their performances are robust against rolations in the testing set. The lesting errors do not change for G-FNO and Radial-FNO; the testing error varies slightly for CT-FNO due to interpolation applied. Equivariant architectures can extract local symmetrics (Cohen & Welling, 2016 2017) and thus improve results even on data without global symmetries. However, in FNO-based architectures, convolution kernels are global; it is unclear whether group global convolution can capture local symmetries. The group-equivariant architecture can still enhance
11

---

Published in Transactions on Machine Learning Research (10/2024)
network performance it provides broader expressive power to the neural network in the frequency domain; however, this increase also raises the computational complexity of FFT due to the larger number of channels.
5.3.2 Ablation Study on Sampling Grid
We compare the results under different sampling grids as discussed in Section 4.3 Bilinear interpolation is applied for all cases. In Table | we demonstrate the impact of using different sampling grids. It can be observed that the proposed 6 rule exhibits superior performance compared to other sampling grids. As the ratio deviates further from 1 : 6, the performance decreases monotonically. Therefore, we conclude that the proposed sampling grid highly effective and important. Analogously, for other domain shapes and coordinate transformations, one can identify a sampling grid similar to this to respect the goometric nature of the Cartesian coordinate system. For this example, the results are averaged over 5 runs.
6 Conclusion, Limitations, and Future Work
In this work, we propose designing an FNO architecture based on coordinate transformation. Specifically, by considering the underlying shape and symmetries of the domains it embeds, we may apply coordinate transformation to scamlessly convert the domain into rectangular domain in which FNO can be applied. Through coordinate transform, we convert rotational symmetries into translation symmetries, which are naturally inherent in FNO architectures. We conduct experiments to evaluate our proposed CT-FNO. Results show that operator learning with coordinate transformation can achieve similar performance to FNO while being able to generalize to various domain shapes while respecting the symmetries.
Limitations. Although this work generalizes FNO various domains, is still not broad enough; it would be interesting to see if coordinate transformation can be applied with other methods work on general domains while respecting the symmetries.
Future Work. In Bonev et al. (002)), the authors have explored the use of a different coordinate system for climate modeling. As our work is a generalization of coordinate transformation for neural operators, it will be interesting to see if CT-FNO can be applied to more practical real physical systems. Moreover, some symmetrics become manifest only in new coordinate system that must be discovered (Liu & Tegmark 2022); is worth exploring whether such symmetries can be captured by CT-FNO.
References
J. Н. Adler, M. Brezina, T. A. Manteuffel, S. F. McCormick, J. W. Ruge, and L. Tang. Island coalescence using parallel first-order system least squares incompressible resistive mggetohhydrodyuamis.. SIAM Journal on Scientific Computing, 35(5):8171-S19)) 2013. doi: 10.1137/120880227
Anima Anandkumar, Kamyar Azizzadencsheli, Kaushik Bhattacharya, Nikola Kovachki, Zongyi Li, Burigede Liu, and Andrew Stuart. Neural operator: Graph kernel network for partial differential equations. In ICLR 2020 Workshop Integration of Deep Neural Models and Differential Equations, 2019. URL
Kamyar Avimadenneshlli, Nikola Kovachki, Zongyi Li, Miguel Liu-Schiafinii, Jean Kossaifi, and Anima Anandkumar. Neural operators for accelerating scientific simulations and design. Nature Reviews Physics, Apr 2024. ISSN 2522-5820.
doi: 10.1008/442544004-077715 URL https://dod.org//1
0008/842254-004-007722-
Frik J Bekkers. B-spline CNNs on lie groups In International Conference on Learning Representations, 2020. URL httts::/ http://aeeeeataeeeeeooooooonnnna
Erik J. Bekkers, Maxime W. Lafarge, Mitko Veta, Koen A. J. Eppenhof, Josien P. W. Pluim, and Remcco Duits. Roto-translation covariant convolutional networks for medical image analysis. In Alejandro F, Frangi, Julia A. Schnabel, Christos Davatzikos, Carlos Alberola-Tópez, and Gabor Fichtinger (ods.). Medical Image Computing and Computer Assisted Intervention MICCAI 2018, pp. 440-448, Cham, 2018. Springer International Publishing.
12

---

Published in Transactions on Machine Learning Research (10/2024)
Alfredo Bermúdezz, Dolores Gómez, and Pilar Salgado. Mathematical Models and Numerical Simulation in Flectromagnetism, volume 74 of UNITEXT. Springer International Publishing, Cham, 2014. ISBN 9783319029481.
doi: 10.1007/978 CRL http:///iin.sprinnee www/cc..co......ccc 1007/
9-8-3-319-00 -19-9999999---8 02949-8
Boris Bonev, Thorsten Kurth, Christian Hundt, Jaideep Pathak, Maximilian Baust, Karthik Kashinath, and Anima Anandkumar. Spherical fourier neural operators: learning stable dynamics on the sphere. In Proccodings of the 40th International Conference on Machine Learning, ICML'23, JMLR.org, 2023.
Johannes Brandstetter, Daniel E. Worrall, and Max Welling. Mcssage passing neural PDE solvers. International Conference Learning Representations, 2022. URL http:tss///pperreiier... http://///pppeeeeeeeeee///rrrrrr net//orum?id
vSix3HPYKSU
Xu Chen, Yongjic FU, Shuo Liu, and Xuan Di Physics-informed neural operator for coupled forward- backward partial differential equations. In 1st Workshop the Synergy of Scientific and Machine Learning Modeling @ ICML2023, 2023. URL https://openreviie: http://ww....to/////doooooooooo.
Taco Cohen and Max Welling. Group equivariant convolutional networks. In Maria Florina Balcan and Kilian Q. Weinberger (ods.), Procoedings of The 33rd International Conference on Machine Learning, volume 48 Proceedings of Machine Learning Research, pp. 290002999, New York, New York, USA, 20-22 Jun 2016. PML.R URL, https::/prooeeddingg mlr ppwss//48/ htt./////o/c...l.ccmm
Taco Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling. Gauge equivariant convolutional net- works and the icosahedral CNN. In International conference on Machine learning, pp. 1321 1330. PMLR, 2019.
Taco S. Cohen and Max Welling, Steerable CNNs. In International Conference Learning Representations, 2017. URL
Dirac and P. A. M. The principles of quantum mechanics. Clarendon Press, Oxford, 1981.
Carlos Esteves, Christine Bllnn-Blanchette, Xiaowei Zhou, and Kostas Daniilidis. Polar transformer networks.
In International Conference on Learning Representations, 2018. URL httpes//opearrviee..etffff.
id=HktR1U1AZ
Vladimir Fanaskov and Oseledets. Spectral neural operators. ArXiv, abs/2205.10573, 2022. URL https:
//api. semanniccont wttpaa/ror//ooops.1020400000
Fabian B. Fuchs, Daniel E. Worrall, Volker Fischer, and Max Welling, Se(3) transforrrsss 3d roto-translation cquivariant attention networks. In Advances in Neural Information Processing Systems 34 (NcurIPS), 2020.
Jacob Helwig, Xuan Zhang, Cong Fu, Jerry Kurtin, Stephan Wojtowytsch, and Shuiwang Ji. Group equiv- ariant Fourier neural operators for partial differential equations. In Proceedings of the 40th International Conference Machine Learning, 2023.
Miltiadis Kofinas, Naveen Shankar Nagaraja, and Efstratios Gavves. Roto-translated local coordinate frames for interacting dynamical systems. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021. URL http:tppps//ppeerreee.ett/eeeeee id=c3RKZas9am
Nikola Kovachki, Samuel Lanthaler, and Siddhartha Mishra. On universal approximation and error bounds for fourier neural operators. The Journal of Machine Learning Research, 22(1):13237-1831) 2021.
Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces with applications pdes. Journal of Machine Learning Research, 24(89):1-97, 2023. URL /tjm://jmlr... .rg hrg/papers/v24/ 21-1524.hta
13

---

Published in Transactions on Machine Learning Research (10/2024)
Kuangdai Leng, Tarje Nissen-Meyer, and Martin van Driel. Efficient global wave propagation adapted to 3-D structural complexity: a seeubospectalsspettaadeeeee approach. Geophysical Journal International. 207(3):1700 1721, 09 2016. ISSN 0956-540X. doi: 10.1093 /0//////////// /tt////..333.3.3 URL httph:t/t///ete .tgg/1..003/ gg1/ggw363
Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Burigede liu, Kaushik Bhattacharya, An- drew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations, 2021. URL https::/openreviert wtwpa/rwwwwwe.eeeornnrrt.e. net/forum? 1d=c8P9NQVtmn0
Zongyi Li, Nikola Borislavov Kovachki, Chris Choy, Boyi Li, Jean Kossaifi, Shourya Prakash Otta, Moham- mad Amin Nabian, Maximilian Stadler, Christian Hundt, Kamyar Azizzadeneshelli, and Anima Anand- kumar. Geometry-informed neural operator for large-scale 3d PDEs. In Thirty-sevnnth Conference on Neural Information Processing Systems, 2023. URL https://openrevier wtw.eeew.eeeeooooeeeeeeeeeeeee http://awwweeeeeeeeeeeeeeooeonnr
Ning Liu, Siavash Jafarzadeh, and Yue Yu. Domain agnostic fourier neural operators. In Thirty-seventh Conference on Neural Information Processing Systems, 2023a. URL. https::/openreviiet hwtp:/neffforrr
1d=ubap5FKbJs
Ning Liu, Yue Yu, Huaiqian You, and Necraj Tatikola. Tno: Invariant neural operators for learning complex physical systems with momentum conservation. In Francisco Ruiz, Jennifer Dy, and Jan-Willen van de Meent (eds.), Proceedings of The 26th International Conference on Artificial Intelligence and Statistios, volume 206 of Proceedings Machine Learning Research, pp. 6822 6838. PMLR, 25 27 Apr 2023b. URL .tml
https://proceediin mlr. www.mlrrreeeeeee wttpppeeeessssvoeeop
Songming Liu, Zhongkai Hao, Chengyang Ying, Hang Su, Ze Cheng, and Jun Zhu. Nuno: general framework for learning parametric pdes with non-uniform data. In Proceedings the 40th International Conference on Machine Learning, TCML.'2 '33. JML.R.org, 2023c.
Yi Liu, Limei Wang, Meng Liu, Yuchao Lin, Xuan Zhang, Bora Oztekin, and Shuiwang Ji. Spherical message passing for 3d molecular graphs. In International Conference on Learning Representations, 2022.
Ziming Liu and Max Tegmark. Machine learning hidden symmetries. Physical Review Letters, 128(18), May 2022. ISSN 1079-7114. doi: 10.1103/phye 122 URL http::/dx.ddo. http///x......////....oo.. 1103/ PhysRevLett. 128. 180201
Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via. deeponet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3))218-22)) March 2021. ISSN 2522-5839.
T. Nessen-Meyer, M. van Driel, S. C. Stähler, K. Hosscini, S. Hempel, L. Auer, A. Colombi, and A. Fournier. Axisem: broadband 3-d seismic wavefields in axisymmetric media. Solid Earth, 5(1): (154-44444, 2014. doi: 10.5194/sc-5-445-2011 2014. URL httptt//ss//ee....mm httpp:///sscccoonnn.csssoorrr/rr
Jaideep Pathak, Shashank Subramanian, Peter Harrington, Sanjeev Raja, Ashesh Chattopadhyay, Morteza Mardani, Thorsten Kurth, David Hall, Zongyi Li, Kamyar Avizzadenesheli, Pedram Hassanzadeh, Karthik Kashinath, and Animashree Anandkumar. Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators. arXiv preprint arXiv:2202.11214 2022.
Md Ashiqur Rahman, Zachary E Ross, and Kamyar Avizzadenesheli. U-NO: U-shaped neural operators. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL httpps://peerevvew..c net/
forum?id==3o0QFscool
Bogdan Raonic, Roberto Molinaro, Tobias Rohner, Siddhartha Mishra, and Emmanuel de Bezenac. Con- volutional neural operators. In ICLR 2023 Workshop on Physics for Machine Learning, 2023. URL
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling E(n) equivariant graph neural networks. CoRR, abs/2102.09844, 2021. URL http://awwww..rr////////./.../// 09844
14

---

Published in Transactions on Machine Learning Research (10/2024)
J. Segman, J. Rubinstein, and Y.Y. Zeevi. The canonical coordinates method for pattern deformation: theoretical and computational considerations. IEFE Transactions on Pattern Analysis and Machine Intelligence, 14(12):1171 1183, 1992. doi: 10.1109/34.177382.
Louis Serrano, Léon Migus, Yuan Yin, Jocelyn Ahmed Mazari, Jean-Noël Vittaut, and patrick gallinari. INFINITY: Neural field modeling for reynolds-averaged navicr-stokes equations. In 1st Workshop the Synergy of Scientific and Machine Learning Modeling @ ICML22023, 2023. URL httpe://openrevier
net/forum?idd+BBSV0BT1
Paul Shen, Michael Herbst, and Venkat Viswanathan. Rotation equivariant operators for machine learning scalar and vector fields, 2022.
Justin Sirignano, Jonathan F. MacArt, and Jonathan B. Freund, DPM: A deep learning PDE augmentation method with application largo-oddy simulation. Journal of Computational Physics, 423:109811, 2020.
ISSN 0021-9991.
Maurice Weiler and Gabriele Cesa. General e2)i-qquvvr steerable cmns. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Bue, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL /ttsps://proceeedngs neurips.cc/papet cc/paper_
pdf
Maurice Weiler, Patrick
Forré,
Erik Verlinde, and
URL Max
Welling
Equivariant and Coordinate Independent Convolutional Networks. 2023.
https://
maurice-weiler. www.or.ggilbb...... io/cnn, http:///ww..aaaaaaaaaaaaoaaaaaaa
Minkai Xu, Jiaqi Han, Aaron Lou, Jean Kossaifi, Arvind Ramanathan, Kamyar Azizzadenesheli, Jure Leskovec, Stefano Ermon, and Anima Anandkumar. Equivariant graph neural operator for modeling 3d dynamics. arXiv preprint arXiv:2401.11037, 2024.
Mingrui Zhang, Jianhong Wang, James B Tlhomole, and Matthew Piggott. Learning to estimate and refine fluid motion with physical dynamics. In Proceedings of the 39th International Conference on Machine Learning, 2022.
Xuan Zhang, Limei Wang, Jacob Helwig, Youzhi Luo, Cong Fu, Yaochen Xie, Meng Liu, Yuchao Lin, Zhao Xu, Keqiang Yan, Keir Adams, Maurice Weiler, Xiner Li, Tianfan Fu, Yucheng Wang, Haiyang Yu, YuQing Xie, Xiang Fu, Alex Strasser, Shenglong Xu, Yi Liu, Yuanqi Du, Alexandra Saxton, Hongyi Ling, Hamnah Lawrence, Hannes Stärk, Shurui Gui, Carl Edwards, Nicholas Gao, Adriana Ladera, Tailin Wu, Elyssa F. Hofgard, Aria Mansouri Tehrani, Rui Wang, Ameya Daigavane, Montgomery Bohde, Jerry Kurtin, Qian Huang, Tuong Phung, Minkai Xu, Chaitanya K. Joshi, Simon V. Mathis, Kamyar Azizzadeneshelli, Ada Fang, Alán Aspuru- Guzik, Erik Bekkers, Michael Bronstein, Marinka Zitnik, Anima Anandkumar, Stefano Ermon, Pietro Liò, Rose Yu, Stephan Günnemann, Jure Leskovee, Heng Ji, Jimeng Sun, Regina Barzilay, Tommi Jaakkola, Connor W. Coley, Xiaoning Qian, Xiaofeng Qian, Tess Smidt, and Shuiwang Ji. Artificial intelligence for science in quantum, atomistic, and continuum systems. arXiv preprint ar arXiv:2307.08422, 2023.
15

---

Published in Transactions on Machine Learning Research (10/2024)
A Group Equivariance Preliminaries
Definition A.1. A group is an algebraic structure that consists of a set G together with binary operator, the group product, that satisfies the following axioms:
Closure: For all g. G' we have h.g E G.
Identity: There exists an identity element E.
Inverse: for cach G there cxists an inverse element g-1 E G such that g 9.995 9...9.1- = e.
Associativity: For each g, h, i G we have (g h) g- (h i).
Definition A.2. A subset H C G of a group G forms a subgroup if it closed under composition and taking inverses:
Composition: for all g, h E H one has gh E H
Inversion: for all g H one has g 5'e
Subgroups are also groups; in other words, they satisfy the group axioms.
Definition A.3. Let (G, and (H, HH) be two groups. Their (outer) direct product (G,-G) x (H, -H) is defined on the Cartesian product G x H of the undorlying scts, equipped with the group product defined as
GxH +G H, (1) (gz, h2) (g) G I2. h1 "H h2)
with the condition that the elements of the factors G and H are independent from each other. If the group H acts on G, the notion of direct product groups is generalized to semi-direct product groups, denoted
G x H.
In this work, we have mentioned several special groups. Hercin, we provide the definitions for the general audience:
The Orthogonal group ((n) is the group of orthogonal matrices in dimensional Euclidean space. These matrices preserve the dot product, representing rotations and reflections.
The Special Orthogonal group SO(n) is subgroup of (nn) and consists of proper rotations in n-dimensional Fuclidean space without reflections.
The Euclidean group F(n)) = TR" x O(n) is the group of isometries of Fuclidean space IR", which includes translations and rotations.
The Special Euclidean group SE(n) = RM SO(n) is subgroup of F(n) and consists of rigid transformations, which include translations and rotations, but the rotations are limited to proper rotations (no reflections).
The cyclic group Ca, also known as the cyclic group of order is defined as the group consists of four elements, denoted by Ca (Ro, Roo, Riso. R2m0); representing rotations of 0%, 2000, 180", and 270", repectivelly.
0000",
The group pd consists of all compositions of translations and rotations in the C4 group.
Definition A.4. Let be a group and X be a set. A (left) group action a mapping from x to that satisfies the following properties:
Identity Element: e = x for all x E X, where is the identity element of G.
Compatibility: (gh) g (h+ x)) for all g, G and X.
16

---

Published in Transactions on Machine Learning Research (10/2024)
B Theoretical Analysis of Parameter Transformation
The concept of FNOs, introduced in the seminal work (Li et al. 2021). holds significant relevance in the
context of the periodic domain Tá. In this setting, FNOs are conceptualized as mapping W A(D; R.) + (ID:: The formulation of this mapping is as follows:
M(a) = Q LL -1-10 c R((a).
(6)
where R and Q represent the lifting and projection operators, respectively, as elaborated in equation S and equation g Further, the non-linear layers Lt within this structure are represented by
(e(v)(z) - [ev(z)+ be(x) + +
+ (P((k) (P(())))))
F(o)(()) (m))
(7)
Here, We E Rd, and be(z) constitute pointwise affine mapping (reflecting weights and biases), while Pe: Zd - Cd.xd, specifies the coefficients for non-local, linear mapping via the Fourier transform.
Moreover, the listing operator R A(D; - U(D; Rdy), where d, du, functions locally and is defined as
R(a)(x) Ra(x), RE Rdoxds
(8)
and the projection operator Q:U(D;R*) + ((DDR operates locally, given by
((v)(z) = Qu(z), Q E Rdxd.
(9)
An adaptation to the FNO architecture is proposed by integrating two additional parameter mapping mod- ules. This leads to revised mapping N A,D,; AdD: 1D, expressed as
(a) : P. Q Lr o... R Pa(a),
(10)
where Pa Ad) + A(D; Rdo) and P., UD, Rd.) + Rd): function parameter mappings.
In accordance with the foundational principles outlined in the universll approximation theorem for FNOs (Kovachki et al. 2021), we extend these concepts through the following theorem:
Theorem B.1 (Universal Approximation for FNOs). Let us consider two non-negative integers, and s', where both S 2 O. Let G be a continuous operator, defined from H' (Td;; Rda) to H' (Td: Given any compact subset K within HT (R4)) and for any arbitrary positive volue of there exists a Fourier Neural Operator, denoted as N, mapping from H HTTT;; Rdd) ) to H (T4, Rdo).. This operator N, structured according the form delineated in equation 6, maintains continuity as mapping from H² to H² Consequently, this results in the following inequality being satisfied:
[19(a) N(aa)l S. aEK
sup
Building upon this theorem, we derive a corollary, that follows immediately from Theorem B.1 to address the modifications involving parameter mapping within the FNO framework:
Corollary B.2 (Universal Approximation with Parameter Mapping). Let s, 2 Assuming that G, continuous operator mopping from HTTT, to H² (Td; holds true. For any compact subset contained in (Td, and any given positive c, there crists modified Fourier Neural Operator, denoted N.. This operalor N", mayping from H*(Td, Rdo). to H HTdd, (Td,Rd.), incorporates the conlinuous and compact operators Pa and Pu on H*(Td, Rda) and H" (Tdd; respectively. The structure of NN is as per equation 10. and it maintains continuity as an operator from H to H thereby ensuring the following condition is met:
sup [[9(a) - -N -N (())l Se. aek
17

---

Published in Transactions on Machine Learning Research (10/2024)
C Diffusion of Heat
C.1 Initial Conditions
We generate the initial conditions
N
M
U(S,0) 0) Uo(E)= = (o( =
Σ
Σ
[Am cos arccos + Bum sin arcsin y)] cos
moZ
SSm
L
N
M
-10(arccos m)2
exp
+ 1,
2m
where Aum: Bum ~ 0.,5) Spm = -(1-1), (
with == and y = 2, and Anm = with Lg 2T and Lg = 6. The initial temperature generated satisfics periodic conditions for continuity around the cylinder's circunference. These initial temperatures distributions are made to demonstrate the effectiveness and importance of the equivariant nature of CT-FNO. The second term introduces higher initial temperature at fixed locations on the cylinder; consequently, rotations will alter the initial heat distribution. In various real-world applications, rotations will have significant impact as well.
Nmm + +
Lg 12
C.2 Details and Additional Results on Equivariant Adaptation of GNO
For the implementation of GNO, we directly adopt the settings from Anandkunar et al. (2019). In this approach, the node features, denoted f, for node i, comprise the function value and the positional vector in Euclidean space (x, coordinates) associated with the node. Similarly, the edge features, denoted Eiy for the edge between nodes i and j, encompass all the function values and positional vectors of nodes and j.
How design more sophisticated equivariant GNNs is a field of study (Satorras et al. 2021 Liu al. 2023b 2022). Moreover, lately, there have been studies on more sophisticated designs of GNOs their variants, such Liu et al. (2023b); Xu et al. (2024), for certain tasks. A straightforward equivariant adaptation of GNO, which use to compare with CT-FNO, is to way that respects the underlying symmetries, in the node and edge features. For node i, we denote its coordinates by Fe, and we take the two points with the maximum and minimum function values, respectively, among its neighbors defined by the cut-off radius as reference points. We denote these two points as Tmni and mmax Now, the node feature of the ith node, fi, comprises the function value and the distances between node and the two reference points:
reference to embed geometric information, in
amin 1 Emin and dinax || Emax
For edge features, Cij, between nodes and j, we include the four distances [dmmin ddpin d'nax, ddpaa] and the pairwise distance, dj || ā,رb, between nodes and j. It is clear that such construction of node and edge features is equivariant to rotations. Therefore, GNO is made equivariant.
C.3 Rotation Equivariance
Lemma C.1. Let D = ((,,v,2) E R2 1,0 6} C R33 be the cylindrical domain considered with the center of the base being the origin and c R20 be the space of time. Suppose that the functions U T R solve the portial differential equation
a (1, U,
aU (1, (1,1,2,1) DXT
G1.
(11)
OU (x, y. 2,t)
=0, =0,
2 DD...
dz
18

---

Published in Transactions on Machine Learning Research (10/2024)
Table 5: Test and train errors from all runs for DeepOnet vary greatly. However, even in the case of the best result, it is still slightly less accurate than FNO.
Test Error
Train Error
1
0.00996
0.00799
2
0.00998
0.00817
3
0.01003
0.00821
0.01033 4
0.00866
0.01040 5
0.01008
0.01097 6
0.01021
7
0.01138
0.01036
8
0.03011
0.02798
9
0.03129
0.02954
10
0.03146
0.02957
Take Rg to be the representation of rotation around z z-ozis acting on D, which can be described by unitary matrices:
cos 0
sin 0 cos
Ra =
sin 0
0
Then the function Un R&D T R also salisfy equation U (,,
11 for all 0 E [0, 2m), where Un (,,
Proof. Equation
11
be written
aU (8,
1 PU(B,
@U(B, 2,t)
Øt
052
0z2
(12)
aU (B, 2,t)
02
where 3 arccos x arcsin y E [0, 2x). The representation of rotation around 2-axis will just result in a
periodic translation in 0. Thus, it sulfices to show that 2, satisfies equation 12 then U(2,3 0, 1,
also satisfies equation 12 which is trivial.
C.4
DeepOnet Results
The results for DeepOnet from every single run are recorded in Table
D Poisson Equation
This protolypical linear elliptic PDE from Raonic al. 2023 is given by,
Au f, in D, aap 0.
The solution operator 9 ++ u, maps the source term f to the solution u. With source term,
K
f(x,y) = K2
Σ oij (2 (2+ sin((ii))sin sin(srjv), V(zy))
with -0.5, D {(x, 3) (0, 1)2), the corresponding exact solution can be analytically computed:
19

---

Published in Transactions on Machine Learning Research (10/2024)
u(z,))
Σ
aij (i2 (
sin((Ti)) sin(#jy))
T r
1.j
we fix K- 16 and choose dij be i.i.d. uniformly distributed from [-1,1]-
E
Rotation Equivariance of the Darcy Flow Equation
Lemma E.1. Let D S I2 be a domain. Suppose that the functions u - IR, a: - R_, and f: - R
solve the partial differential equation
(a(a))Vu(()) = ((x)
(13)
Take R be an orthogonal matrir describing a rotation. Then the functions un : R- R, an R R-1D
R+, and fa R-1D + R also satisfy equation 13
where:
un(x) u (xx)
an(x) = a Rx)
(14)
fa(x): = (RRx
Note that Lemma E.1 most meaningful when the domain D invariant under the rotation R, e.g., the domain circle the domain R², If the domain is unit box, D (0, 1)², it is most meaningful to consider rotation by 0%, 90" 180 270".
Proof. Since fa(x) f() = 1 is invariant under rotation, to show that un, ans and fa satisfy equation 13 it is sufficient to show that V (un))(( = (V aVu) (Rr). We have
8 θ aR(I) uR(#)
((R()))
dz,
dz,
① (a(xx) u( (Rx)
θ Ox,
dx,
2 2
da (Rx) du RuiRis + a (Rxx) Ozj
du Du Ru RuiRgs
(15)
Oxk
duu Ox,
i=1 j= k=1
da
Du (Rx) + al (xx)
82 Rx) +
da [Rx) Ox2
du (Rx)
(Rx)
a (Rx)
(Rx)
dz
dz1
dz
Oxy
(V VV)) (Rx).
20


Reviews:
[{"review_id": "f5G40VJAT5", "reviewer": "Reviewer_pdUD", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I like the idea of using coordinate transform (canonical coordinate) to simplify the network implementation of modeling PDF and operators. That is the symmetry along an arbitrary manifold can be converted into simple translation symmetry by using the canonical coordinate. I believe this is an important extension of the Fourier neural operator.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The present paper introduces coordinate transform into Fourier neural operator together to extend the neural operator into many domains.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I don't have a request for major revisions, but only some minor changes.\n\n- The text above Eq. (1): \"Thus, he\" should be \"Thus, the\"\n- Fig. 3: it is better to add the labels of the x and y axes.\n- The last paragraph in section 4: it is better to use the Jacobian matrix in the change of integration variables to strengthen the math rigor of why the radius and angular axes have a ratio of 1:6.", "broader_impact_concerns": "No.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "LWgwOQVVVE", "reviewer": "Reviewer_5vdP", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n1. The presented results are reasonable and showcase how CT-FNOs can learn over more complex domains without sacrificing accuracy (relative error) while preserving symmetry.\n\n## Weaknesses\n\n1. Although the idea of using a general coordinate transform to extend the applicability of CT-FNOs is a relatively new extension in the Fourier Neural Operator Space, the novelty is limited by a number of existing papers that explore closely related ideas. Perhaps the most related paper is Bonev et al. (2023), which uses a different coordinate system (spherical) for climate modeling via a Spherical Fourier Neural Operator (SFNO) extension of the FNO.\n\n2. The paper does not present analysis of application areas that are now accessible via their presented coordinate transform augmentation and the demonstrated applicability is actually quite narrow; all non-rectangular domains are circular or effectively circular (e.g. the cylinder is a product of a circle and a unit interval). The same 1:6 sampling grid is reused, a study of other domain shapes is not given. To the credit of the authors, they are fairly upfront about this fact and state that broadening the applicability of the FNO is left for future work.\n\n## Verdict\n\nAlthough the presented idea of using a general coordinate transform to extend CT-FNOs is somewhat novel, closely related papers have investigated the use of a spherical coordinate transform to extend the applicability of FNOs. The application areas given are fairly synthetic and no examples are given beyond applications to a circular (or effectively circular) domain. That being said, the presented results for all three examples are reasonable and showcase how CT-FNOs can learn over more complex domains without sacrificing accuracy while preserving symmetry. I believe this paper is a reasonable first step in extending FNOs to work over more complex domains. The claims made in the submission are reasonably well-supported, although some of the language regarding broad applicability should be toned down to reflect the relatively limited scope of studied domains. I do believe the findings of the paper would be interesting to some portion of TMLR's audience, specifically, those working on FNOs. Since both criteria for acceptance to TMLR are satisfied, I recommend acceptance of this paper.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes the Coordinate Transform Fourier Neural Operator (CT-FNO) framework, which extends applicability of the Fourier Neural Operator (FNO) to non-rectangular domains, while preserving underlying symmetry. The paper also provides an extension of an FNO universal approximation theorem to CT-FNOs. Further, the paper conducts experiments that showcase how CT-FNOs are able to effectively maintain symmetries and how they perform as well if not better than a number of pertinent baselines in a variety of contexts. These contexts are learning the diffusion of heat on a cylinder, learning a synthetic operator, and learning the dynamics of a 2D Darcy flow equation.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The writing in this paper is for the most part clear, however English conventions need to be followed more carefully in certain paragraphs. I elaborate on this further with a non-exhaustive list of corrections below. Additionally, the way standard deviations are given for tables is non-standard. Please use either $\\pm$ or explicitly specify that standard deviations are given in parentheses next to each result. Also, what are \"accuracies\" in this context? I assume the authors are referring to relative error of the predicted functions, e.g. mean absolute percentage error (MAPE). Please augment the experimental section to clearly specify the metrics used for the presented results.\n\nThe aforementioned non-exhaustive list of minor corrections is given below:\n\n- Section 2.1, first paragraph: \"Fourier neural operators (FNOs) has drawn lots of attention\" -> \"Fourier neural operators (FNOs) have drawn lots of attention\"\n- Section 2.3, first paragraph: \"For PDEs, the symmetries it can have\" -> \"For PDEs, the symmetries they can have\"\n- Section 3.1, second paragraph: \"$L^\\mathcal{X}_g$ and $L^\\mathcal{Y}_g$ can be considered as a transformation, such as rotation, of the input\" -> \"$L^\\mathcal{X}_g$ and $L^\\mathcal{Y}_g$ can be considered as transformations, such as rotations, of the input\"\n- Section 3.2, first paragraph: \"Thus, he integral kernel\" -> \"Thus, the integral kernel\"\n- Section 4.1, first paragraph: \"In this works, we discuss\" -> \"In this work, we discuss\"\n- Section 4.2, third paragraph: \"for Abelian Lie-groups\" -> \"for abelian Lie-groups\" (the convention is to not capitalize the \"a\" in abelian)\n- Figure 4, caption: The descriptions for (a) and (b) are flipped. Figure (a) is what happens when you use a 1:1 ratio, whereas Figure (b) is what happens when you use the ratio 1:6.", "broader_impact_concerns": "This work deals with broadening the applicability of Fourier Neural Operators. I do not believe the paper requires a statement addressing potential ethical implications of the presented research.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "9YyIlpLDIk", "reviewer": "Reviewer_mw3p", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n\n- Generally clear manuscript\n- Didactic introduction of group convolution and Fourier neural operators\n\n### Weaknesses\n\n- Emphasis that CT-FNO is applicable beyond rotations and in higher dimensional domains, but no experiments demonstrating this", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors present an approach to apply Fourier neural operators (FNO) to non-euclidian data domains, by adding fixed coordinate transforms mapping the data domain to a rectangular input domain of the FNO, and analogously from the FNO output to the target domain. The proposed method is in particular useful to map the translation equivariance of the FNO model to another equivariance arising from the symmetry of the domain, such as a rotation symmetry. Tests with polar and cylindrical coordinates are shown.\n\nI am not closely familiar with the large existing literature on physics-informed ML, and will therefore not comment on the novelty and contribution of the approach.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "### Main requested changes or clarifications\n\n\n1. **Design of the experiments in sect.5.1 (Cylinder)**\n - The authors state for the cylinder: \"Since the domain is not rectangular,\nFNO cannot be directly applied without sacrificing the power FFTs.\"  \n    However, the direct method here seems to be to cut vertically and flatten the cylinder, giving a rectangular grid. And since a FFT assumes periodic boundary conditions, a FNO would be rotationally equivariant both in the angular direction (correctly) and in the vertical direction (incorrectly). Is this not also what is done by the CT-FNO in this case? Possibly I am misunderstanding the setup here.\n - It is stated that the GNO can also be used such that it is equivariant - then this should be done and added to the comparison.\n - It was unclear to me why the performance on the rotated test set should be worse than on the normal test set for the comparison methods. Are the initial conditions of the training set not generated uniformly across angles?\n - How are the vertical boundaries of the grid in CT-FNO treated compared to the angular (periodic) boundary?\n\n\n\n### Optional and minor changes\n\n2. **Vague abstract**  \n  - The abstract remained relatively vague, explaining the concept of symmetries in physics, but very little about the concrete approach and its window of applicability. My recommendation is to rewrite the abstract to be more precise and descriptive.\n  - In the last sentence, the scheme \"generalizes well across different domain shapes\" seems to suggest that a single model could be trained on one domain shape, and then generalize at test time to a new domain shape. (The formulation also reappears in the main text). However, such an experiment, while an interesting possibility, is not part of the presented results.\n  \n3. **Missing general definition of CT-FNO**\n - A general statement of how the coordinate transform is added to the FNO in section 4 would be useful.\n  \n3. **Sampling grid and Fig.4**  \n  - Fig. 4 was visually confusing: panel a) corresponds to d), and panel b) to c) ?\n  - Why are the circular domains mapped to square domains? If the aim is to proportionally represent radial and angular axes, measured by the outside circumference of the circle, why not represent this as a rectangle with aspect ratio $1 : 2 \\pi$ ?\n  - This figure highlights how uniform sampling on the coordinate grid can lead to non-uniform resolution in the physical embedding space (close compared to far from the center of the circle). Could the authors discuss when this is a useful or problematic feature? For example, in a rotating physical system the characteristic angular size of feature fluctuations may vary along the radial coordinate. \n  \n4. **Omitted proof and unclear significance of Corollary 4.1**  \n - Both the main text and Appendix D state Corollary 4.1 (respectively D.2) at length, without explict proof since it would follow immediately from the theorem of Kovachki et al. '21. Since the authors appear to nonetheless assign importance to the extension and do not simply cite the result of Kovachki et al. '21, it might be good to formalize the logic of adding the coordinate transforms in a short proof. \n  \n  \n5. **Discussion of discovering symmetries**  \n - The conclusion raises the question whether CT-FNO could capture and discover symmetries which are not known a priori. However, the coordinate transformation must be chosen by hand in the current approach. Is the idea here to make the coordinate transform learnable?\n\n\n6. **Typos**  \n  - Sect. 5.1: \"More details of experimental details\"\n  - App. E: \"Poisson Equation\" appears doubled", "broader_impact_concerns": "/", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes an extension of Fourier Neural Operators to non-rectangular domains via general coordinate transformations. The resulting method preserves underlying symmetries and is applied in several physical modeling experiments.

The reviewers found the method to be a useful if relatively small improvement over prior work. They also found the scope of the analysis to be somewhat narrow. To enhance the impact of this work, the authors might consider extensions to more diverse and complex domains. Nevertheless, as it stands, the paper is well presented, with its main claims being adequately investigated and its main limitations being clearly described. As such, this paper is appropriate for publication at TMLR and I recommend acceptance.
Decision: Accept (poster)