Domain Agnostic Fourier Neural Operators
Authors: Ning Liu
Siavash Jafarzadeh
Yue Yu
Venue: NeurIPS 2023 poster (2023)
Keywords: Operator-Regression Neural Networks
Neural Operators
Data-Driven Physics Modeling
Geometrical and Topological Shape Changes

Abstract: Fourier neural operators (FNOs) can learn highly nonlinear mappings between function spaces, and have recently become a popular tool for learning responses of complex physical systems. However, to achieve good accuracy and efficiency, FNOs rely on the Fast Fourier transform (FFT), which is restricted to modeling problems on rectangular domains. To lift such a restriction and permit FFT on irregular geometries as well as topology changes, we introduce domain agnostic Fourier neural operator (DAFNO), a novel neural operator architecture for learning surrogates with irregular geometries and evolving domains. The key idea is to incorporate a smoothed characteristic function in the integral layer architecture of FNOs, and leverage FFT to achieve rapid computations, in such a way that the geometric information is explicitly encoded in the architecture. In our empirical evaluation, DAFNO has achieved state-of-the-art accuracy as compared to baseline neural operator models on two benchmark datasets of material modeling and airfoil simulation. To further demonstrate the capability and generalizability of DAFNO in handling complex domains with topology changes, we consider a brittle material fracture evolution problem. With only one training crack simulation sample, DAFNO has achieved generalizability to unseen loading scenarios and substantially different crack patterns from the trained scenario. Our code and data accompanying this paper are available at https://github.com/ningliu-iga/DAFNO.

Domain Agnostic Fourier Neural Operators
Ning Liu" Global Engineering and Materials, Inc. Princeton, NJ 08540 ningliu@umich. edu
Siavash Jafarzadeh" Department of Mathematics Lehigh University Bethlehem, PA 18015 si sij2228lehigh. edu
Yue Yu? Department Mathematics Lehigh University Bethlehem, PA 18015 yuy2148lehigh. edu
Abstract
Fourier neural operators (FNOs) learn highly nonlinear mappings between function spaces, and have recently become popular tool for learning responses of complex physical systems. However, to achieve good accuracy and efficiency, FNOs rely on the Fast Fourier transform (FFT), which is restricted to modeling problems rectangular domains. To lift such restriction and permit FFT on irregular geometries as well as topology changes, we introduce domain agnostic Fourier neural operator (DAFNO), novel neural operator architecture for learning surrogates with irregular geometries and evolving domains. The key idea is to incorporate smoothed characteristic function the integral layer architecture of FNOs, and leverage FFT to achieve rapid computations, in such a way that the geometric information is explicitly encoded in the architecture. In our empirical evaluation, DAFNO has achieved mate-of-the-arr accuracy as compared to baseline neural operator models on two benchmark datasets of material modeling and airfoil simulation. To further demonstrate the capability and generalizability of DAFNO in handling complex domains with topology changes, we consider brittle material fracture evolution problem. With only training crack simulation sample, DAFNO has achieved generalizability to unseen loading scenarios and substantially different crack patterns from the trained scenario. Our code and data accompanying this paper are available at hhttps::/ // //gtttu....... com www.nnngggggg/llia...
Introduction
Deep learning surrogate models provide a useful data-driven paradigm to accelerate the PDE-solving and calibration process of scientific modeling and computing problems. Among others, a wide range of scientific computing applications entail the learning of solution operators, i.c., the learning of infinite dimensional function mappings between any parametric dependence the solution field. prototypical instance is the case of solving Navier-Stokes equations fluid mechanics, where the initial input needs to be mapped to temporal sequence of nonlinear parametric solutions. The
demand for operator learning has sparked the development of neural operator based methods (Li
al.
2020a blc You al. (202aa blc, Goswami al. 2022 Liu al.
2022 Gupta et al.. 2021 Lu
et al.
2019: Cao 2021 Hao et al. 2023; Li et al., 2022b Yin et al.)
2022c with one of the most
popular architectures being Fourier Neural Operators (FNOs) (Li et
2020c You al. 2022c).
The success of FNOs can be mostly attributed to convolution- integral kernel that learns in issouutioniinvariatt manner and the computationally efficient evaluation achieved via Fast Fourier Transform (FFT) (Brigham 1988). While learning in the spectral domain is fast, the latter comes at a cost: the computational domain of the underlying problem needs to be rectangular with uniformly
'Equal contribution Corresponding author
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

meshed grids. This is often intractable as the domain of interest is, more often than not, irregular. An often taken trick for applying FNO to irregular domains is to embed the original domain into a larger rectangular domain and zero-pad or extrapolate on the redundant space (Lu et al. poses two potential problems, one being the possible numerical errors and even instabilitres due to the discontinuity at the original domain boundary (e.g., the Gibbs phenomenon (Gottlieb & Shu 1997)
2022). This
and the other, perhaps more importantly being the fact that the padding/cxtrapolating techniques cannot handle domains with shallow gaps, as is the case in object contact and crack propagation problems. Meanwhile, another line of work emphasizes the learning of a diffeomorphic mapping between the original domain and latent domain with uniform grids which FNO can be applied (Li et al., 2022a). However, in this approach the changes on the boundary and the domain topology can only be informed via the learned diffeomorphism, which results in approximation errors when tested on a new domain geometry and possible failure when a change in topology is involved on the domain geometry.
In this work, we aim to design FNO architectures that explicitly embed the boundary information of irregular domains, which we coin Domain Agnostic Fourier Neural Operator (DAFNO). This is inspired by the recent work in convolution-base bounded domains of arbitrary shapes are explicitly encoded in the nonlocal integral formulation. We argue that, by explicitly embedding the domain boundary information into the model architecture, DAFNO is able to learn the underlying physics more accurately, and the learnt model is generalizable to changes on the domain geometry and topology. Concretely, we construct two practical DAFNO variants, namely, eDAFNO that inherits the explicit FNO architecture (Li et al. 2022a) and iDAFNO that is built upon the implicit FNO (IFNO) architecture characterizing layer-independen kernels YYou et al., 2022c). Moreover, boundary smoothening technique is also proposed resolve the Gibbs phenomenon and retain the fidelity of the domain boundary. In summary, the primary contributions of the current work are as follows:
peridynamics (Jafarzadeh et al. 2022b) in which
We propose DAFNO, a novel Fourier neural operator architecture that explicitly encodes the boundary information of irregular domains into the model architecture, so that the learned operator is aware of the domain boundary, and generalizable to different domains of complicated geometries and topologies.
By incorporating (smoothened) domain characteristic function into the integral layer, our formulation resembles a nonlocal model, such that the layer update acts as collecting interactions between material points inside the domain and cuts the non-physical influence outside the domain. As such, the model preserves the fidelity of the domain boundary as well as the convolution form of the kernel that retains the computational efficiency of FFT.
We demonstrate the expressivity and generalizability of DAFNO across wide range of scientific problems including constitutive modeling of hyperelastic materials, airfoil design, and crack propagation in brittle fracture, and show that our learned operator can handle not only irregular domains but also topology changes over the evolution of the solution.
2 Background and related work
The goal of this work is to construct a neural network architecture to learn common physical models on various domains. Formally, given D := {(g In,, u.ln,) ) a labelled set of function pair observations both defined on the domain x E Qg C Rs. We assume that the input {g)()) } is a set of independent and identically distributed (i.i.d.) random fields from known probability distribution ja on A(R"),, ), a Banach space of functions taking values in Rd,, ux(x) E ((B=)), possibly noisy, is the observed corresponding response taking values in IR.... Taking mechanical response modeling problem for example, Ω2 is the shape of the object of interest, g: g: may represent the boundary, initial, or loading conditions, and (z) can be the resulting velocity, pressure, or displacement field of the object. We assume that all observations can be modeled by a common and possibly unknown governing law, e.g., balance laws, and our goal is to construct surrogate operator mapping, G. from A to il such that
9 ig (x) ux()), Va E 24.
Here, 9 represents the (trainable) network parameter set.
In real-world applications, the domain Ω2 can possess different topologies, e.g., in contact problems
(Benson & Okazawa, 2004, [Simo Laursen T992) and material fragmentation problems (Del
Luycker et al.) 2011 Agwai al. 2011: Silling 2003), and/or evolve with time, as is the case
2

---

in large-deformation problems (Shadden et al. 2010) and stuid-st interaction applications (Kuhl et al. 2003; Kamensky et al., 2017). Hence, it is desired to develop an architecture with generalizability across various domains of complex shapes, so that the knowledge obtained from one geometry can be transferable to other geometries.
2.1 Learning solution operators of hidden physics
In real-world physical problems, predicting and monitoring complex system responses are ubiquitous in many applications. For these purposes, physics- based PDEs and tranditional numerical methods have been commonly employed. However, traditional numerical methods are solved for specific boundary, initial, and loading conditions g on a specific domain Q. Hence, the solutions are not generalizable to other domains and operating conditions.
To provide more efficient and flexible surrogate model for physical response prediction, there has been significant progress in the development of deep neural networks (NNs) and scientific machine
learning models (Ghaboussi et al. 1998, 1991| Carleo al. 2019 Karniadakis et al., 2021 Zhang
et al. 2018 Cai et al. 2002 Pfau et al. 2020; He et al. 2021 Besnard et al. (006). Among others,
neural operators (Li et al. 2020a b You et al. [202aa] Ong et al., [022] Gupta et al., 2021] Lu
et al. 2019 2021b Goswami et al. 2022 Tripura & Chakraborty (022)) show particular promise in learning physies of complex systems: compared with elassical NNs, neural operators are resolution independent and generalizable to different input instances. Therefore, once the neural operator trained, solving for new instance of the boundary/nnitaalloddnng condition with different discretization only requires a forward pass. These advantages make neural operators a useful tool to
many physics modeling problems (Yin et al. 2022a Goswami et al. 2022 Yin et 2022b
Li al.
2020a
Lu et al. 2022 2021a)
2.2 Neural operator learning
Here, we first introduce the basic architecture of the general integral neural operators (Li al. 2020a ble You et al. [0222a c), which are comprised of three building blocks, First, the input function, g(x) E is lifted a higher-ddimnsioonll representation via h h((x) ((g] == P/z, f where
gz))]" +p.
p(s+d,)xxd, and p e Rd define an affine pointwise mapping. Then, the feature vector function h (x) goes through an iterative layer block, where the layer update is defined via the sum of local linear operator, a nonlocal integral kernel operator, and a bias function: h111(()))
J [h (z)) 0, L. is a sequence functions representing the values of the network at each hidden layer. J
Here, h'(x) E IRdh,
Th are the nonlinear operator layers defined by the particular choice of networks. Finally, the output uxx) E U is obtained via a projection layer by mapping the last hidden layer representation h ''(x)) onto U as: u(x) Q ((x))(x) == Q2 (Q1 (Q)h((x) + qn) q2- Q1, Q2. qı and q2 are the appropriately sized matrices and vectors that are part of the learnable parameter set, and an activation function (e.g.. ReLU (He et al. (018)) GeLU).
Then, the system response can be learnt by constructing surrogate operator of ① 0( (xx
Qo J o o... JL P(g] Rs u(x). by solving the network parameter set via an optimization problem:
min (D = min Σ
(C (gn 0), u.)l
deB
Here, C' denotes a properly defined cost functional (e.g., the relative mean square error) on Qz
2.3 Fourier neural operators
The Fourier neural operator (FNO) is first proposed in Li et al. (202cc) with its iterative layer architecture given by a convolution operator:
J [1)))<x) = W (w((())+ d' + x(x y: o'))((y)dd))
where I E Rd dd and è' dd are learnable tensors at the 1-th layer, and e &d is tensor kernel function with parameters v'. When rectangular domain with uniform meshes is considered, the above convolution operation be converted to multiplication operation through discrete Fourier transform:
('))(<) =o (W' + F F [F(n(:))) F()(())(()))
3

---

Figure 1: A schematic of extending an arbitrarily shaped domain Q to a periodic box and its discretized form in 2D.
where F and F=11 denote the Fourier transform and its inverse, respectively, which are computed using the FFT algorithm to each component of h separately. The FFT calculations greatly improve the computational efficiency due their quasilinear time complexity, but they also restrict the vanilla FNO architecture to rectangular domains Ω (Lu et al. 2022).
To enhance the flexibility of FNO in modeling complex geometries, in Lu al. (0222) the authors proposed to pad and/or extrapolate the input and output functions into a larger rectangular domain. However, such paddinggextrappllatig techniques are prone numerical instabilities (Goutleb & Shu 1997), especially when the domain is concave and/or with complicated boundaries. As shown in Lu al. (2022). the performance of dgFNO+ substantially deteriorates when handling complicated domains with notches and gaps. In Geo-FNO (Li et al. (022a).. an additional neural network is employed and trained from data, to continuously map irregular domains onto a latent space of rectangular domains. As result, the vanilla FNO can be employed on the rectangular latent domain. However, this strategy relies on the continuous mapping from the physical domain rectangular domain, hence it is restricted to relatively simple geometries with no topology change.
3 Domain Agnostic Fourier Neural Operators
In this section, we introduce Domain Agnostic Fourier Neural Operator (DAFNO), which features the generalizability to new and unseen domains of arbitrary shapes and different topologies. The key idea is to explicitly encode the domain information in the design while retaining the convolutional architecture in the iterative layer of FNOs. In what follows, we present the eDAFNO architecture based on the standard/explicit FNO model (Li et al., 2020c). while the iDAFNO architecture based on the implicit FNO model (You et al., 2022c) is provided in Appendix A.
Concretely, we enclose the physical domain of interest, Q by a (slightly larger) periodic box T, as shown in Figure [ Next, we define the following domain characteristic function:
E Ω
x(x)
o
x TET
which encodes the domain information of different geometries. Inspired by Jafarzadeh et al. (2022b),
we incorporate the above-encoded domain information into the FNO architecture of (3), by multiply- ing the integrand in its convolution integral with x())x(y):
J'(N) =o (L ((xx))x(e)xat - y: (h(y)) h(x))dy + w'((x) + c
(5)
Herein, we have followed the practice in You et al. (2022a) and reformulated Laplacian operator, which is found to improve training efficacy. By introducing the term (I)) the integrand vanishes when either point or y is positioned inside Ω and the other is positioned outside. This modification eliminates any undesired interaction between the regions inside and outside of Ω. As result, it tailors the integral operator act on Ω independently and is able to handle different domains and topologies. With this modification, the FFT remains applicable, since the convolutional structure of the integral is preserved and the domain of operation yet spans the whole rectangular box T. In this context, (5) can be re-organized as:
3 to nonlocal
J' [h] =a (f k(x y: v (olo'))())))))) - h(x) / k( y: (x)))d + W'h(x) +d)
Note that multiplying (')( dt with (x) does not alter the computational domain. Now that the integration region is rectangular box T. the FFT algorithm and its inverse can be readily applied,

---

Iterative layers
loop for times
Input
Lifting layer
Projection layer
Output
function
h(a)
function
g(x)
P
as (t)
J DAANO
x(x)
Figure 2:
An illustration of the proposed DAFNO architecture. We start from the input function (z).. After lifting, the iterative Fourier layers are built that explicitly embed the encoded domain information, Lastly, we project the last hidden layer representation to the target function space.
and hence we can further express the eDAFNO architecture as:
J'Mh| =0 (
-((=)=(((()(((())))))))) ( ('h(s)++)))))
(6)
where I(o; = F (=(K(((()) -[oo]
An illustration of the DAFNO atchitecture is provided in Figure 2 Note that this architectural modification is performed at the continuum level and therefore is independent of the discretization. Then, the box T can be discretized with structured grids (cf. Figure I. as is the case in standard FNO.
Although the proposed DAFNO architecture in (6 can handle complex generalization in domains, it has potential pitfall: since the characteristic function is not continuous on the domain boundary, its Fourier series cannot converge uniformly and the FFT result would present fictitious wiggling near the discontinuities (i.e., the Gibbs phenomenon (Day et al. 1965)). As a consequence, the introduction X can potentially jeopardize the computational accuracy. To improve the efficacy of DAFNO, we propose to replace the sharp characteristic function, X. with smoothed formulation:
(x) == tanh tand( (z))
0.5) 0.5
Here, the hyperbolic tangent function tanh(=) := exp(2) (pp(2))oxp(-3) dist(ær a22) denotes the (approxi- exp(2 texp())=xpp((2
mated) distance between a and the boundary of domain Ω2 and / controls the level of smoothness, which is treated as tunable hyperparameter. An illustration of the effect of the smoothed x is displayed in Figure 3 with additional plots and prediction results with respect to different levels of smoothness, 3. provided in Appendix B. In what follows, the sign neglected for brevity.
Remark: We point out that the proposed smoothed geometry encoding technique, although simple. is substantially different from existing function padding/cxtrapolation techniques proposed in Lu et al. (2022) who cannot handle singular boundaries (as in the airfoil tail of our example 2) nor notches/shallow gaps in the domain (as in the crack propagation of our example 3). Our proposed architecture in (5) is also more sophiscated than a trivial zero padding at each layer in that the charac- teristic function xx= is multiplied with the integrand, whereas the latter breaks the convolutional structure and hinders the application of FFTs.
4 Numerical examples
In this section, we demonstrate the accuracy and expressivity of DAFNO across a wide variety of scientific problems. We compare the performance of DAFNO against other relevant scientific machine learning models. including FNO (Li al., 2020c). Geo-FNO (Li et al. 2022a). IFNO You et al. 2022ch, F-FNO (Tran et al., 2022). GNO (Li et al. 2020a). DeepONet (Lu et al. 2019). and UNet Roonebberger al., (015). particular, we carry out three experiments on irregular domains, namely, constitutive modeling of hyperelasticity in material science, airfoil design in fluid mechanies, and crack propagation with topology change fracture mechanics. For fair comparison, the hyperparameters of each model are tuned to minimize the error validation datasets, including initial learning rate, decay rate for every 100 epochs, smoothing parameter, and regularization parameter, while the total number of epochs is restricted to 500 for computational efficiency. The
5

---

Computational
Rounday Facoding (--
Gmunn 41x41
eDAFNO Prediction 41xd1
eDAFNO Prediction 161x161
(a)
(b)
(d)
Figure 3: An illustration on hyperelasticity sample: (a) sharp characteristic function (b) smoothed characteristic function X. (c) ground truth, (d) eDAFNO (trained using 41 x41 discretization) predic- tion from the same resolution, and (e) zero-shot super-resolution prediction from eDAFNO (trained using x41 discretization and evaluated directly on 161 161 discretization).
relative 1.2 error is reported as comparison metrics for both the training and test datasets. The experiments of each method are repeated on 5 trials with 5 different random seeds, and the and standard deviation of the errors are reported. Further details and additional results are provided in Appendix B.
4.1
Constitutive modeling of hyperelasticity
We start with a hyperelasticity problem material science that models the fundamental principle of constitutive relations. The high-fidelity synthetic data in this benchmark is governed by
8u dt
(8)
where p denotes the mass density, u and a represent the corresponding displacement and stress fields, respectively. The computational domain is enclosed by unit cell [0, 1]2 of which the center exists randomly shaped void, as described by its radius r = 0.2 T+eap(F) and F ~ N(0, ( + --)
0.2
The bottom edge is fixed and the top edge is subjected to tensile traction of [0, 100). The underlying hyperelastic material of the incompressible Sivlin-Saunders type. For training, we directly adopt the dataset in Li al. (2022a), where total of 1000, 200, 200 samples are selected for training, validation and testing, respectively. For this problem, the input is represented as point clouds and the target is the resulting stress field.
Table The total number of parameters (in millions) of selected models for hyperelasticity dataset.
model
eDAFNO
iDAFNO
FNO
IFNO
Geo-FNO
GNO
DeepONet
UNet
F-FNO
npаram
2.37
0.60
2.37
0.60
3.02
2.64
3.10
3.03
Table Test errors for the hyperelasticity problem, where bold numbers highlight the best method.
Model
of training samples
100
1000
eDAFNO
16.446 10.422 %
4.247 1.247%110.066.
1.094% 0194%=0.0024
Proposed model
iDAFNO
16.669%.10.525.
4.214% 0.058 %
1.207%
FNO w/ mask
+9.467%
7.852%+O.. 130%
4.550%
IFNO w/ mask
00.262%--00.766
7.700%:110.0246;
4.4819:110.2296
Geo-FNO
28.725%.1.. 112.600%
11.343%-14.4444
2.316%-10.2839
Baseline model
GNO
29.305%.10. %
18.574% -..58%%
13.007% 1.0.229%
DeepONet
35.334%==0.179
25.455%
11.998%=t0.786%
F-FNO
55.672%%443.8555
12.135%
3.193% 1.622%
UNet
98, 1111666%%002699
34.467%-42.8588
5,462%==000485
FNO w/ smooth X
5.479%::10.186%
1.415%
Ablation study
IFNO w/ smooth X
17.145%-t0.43285
5.088%:20.146%
1.50%%
Ablation study We first carry out an ablation study by comparing (a) the proposed two DAFNO models with (b) the baseline FNO/IFNO with the sharp characteristic function as input (denoted FNО/IFN w/ mask), and (c) the original FNO/INNO with our proposed smoothened boundary

---

characteristic function as imput (denoted as FNO/IFNN w/ smooth x). In this study, scenarios (b) and (c) aim investigate the effects of our proposed boundary smoothing technique, and by comparing scenarios (a) and (c) we verify the effectiveness of encoding the boundary information in eDAFNO architecture. addition, both FNO- and IFNO-based models are tested, with the purpose to evaluate the model expressivity when using layer-independent parameters in iterative layers. Three training dataset sizes (i.e., 10, 100, and 1000) are employed to explore the effect of the proposed algorithms on small, medium, and large datasets, respectively. The number of trainable parameters are reported Table T Following the common practice as in Li et I (2022a), the hyperparameter choice each model is selected by tuning the number of layers and the width (channel dimension) keeping the total number parameters of the same magnitude.
The results of the ablation study are listed in Table Firstly, by directly comparing the results of FNO (and IFNO) with mask and with smooth boundary encoding, one can tell that the boundary smoothing technique helps to reduce the error. This is supported by the observation that FNO and IFNO with smooth X consistently outperform their counterparts with mask in all data regimes, especially when sufficient amount of data becomes available where huge boost in accuracy can be achieved (by over 300%). On the other hand, by encoding the geometry information into the iterative layer, the prediction accuracy is further improved, where eDAFNO and iDAFNO outperform FNO and IFNO with smoothed X by 22.7% and 20.0% in large-data regime, respectively. Another interesting finding is that eDAFNO is 9.4% more accurate compared iDANNO in the large-data regime, although only a quarter of the total number of parameters is needed in iDAFNO due to its inveeenneeendtntt parameter setting. This effect is less pronounced as reduce the amount of available data for training, where the performance of iDAFNO is similar to that of eDAFNO in the small- and medium-date regimes. This is because İDAFNO has much smaller number of trainable parameters and therefore less likely to overfit with small datasets. Given that the performance of eDAFNO and iDAFNO is comparable, is our opinion that both architectures are useful in different applications. In Figure 3 an example of the computational domain, the smoothened boundary encoding, the ground truth solution, and the eDAFNO prediction demonstrated. To demonstrate the capability of prediction across resolutions, we train eDAFNO using data with 41 x41 grids then apply the model to provide prediction on 161 161 grids-on one can see that eDAFNO can generalize across different resolutions.
Comparison against additional baselines We further compare the performance of DAFNO against additional relevant baselines, including GNO, Geo-FNO, F-FNO, DeepONet, and UNet. Note that the results of GNO, DeepONet, and UNet are obtained using the same settings as in Li et al. (2022a). Overall, the two DAFNO variants are significantly superior other baselines in accuracy, with eDAFNO outperforming GNO, DeepONet, UNet, Geo-FNO, and F-FNO by 1088.9%, 975.0%, 399.3%, 111.7% and 191.9%, respectively, in large-data regime. DAFNO also more memory efficient compared to Geo-FNO (the most accurate baseline), as it foregoes the need for additional coordinate deformation network. As shown Table when the mayer-nndeennde parameter setting in iDAFNO is taken into account, DAFNO surpasses Geo-FNO by 407.49 in memory saving.
4.2 Airfoil design
Computational Domain
Boundary Encoding (B
Ground Truth
eDA FNO Prediction
2.0
25
10
0.6
95
Figure 4:
An illustration on a test sample from the airfoil design problem. From left to right: an illustration of the discretized computational domain, the smoothed boundary encoding (i.e., smoothed X), the ground truth, and the eDAFNO prediction.
In this example, investigate DAFNO's performance in learning transonic flow around various airfoil designs. Neglecting the effect of viscosity, the underlying physics can be described by the Fuler equation:
0p V. 7ppv) =0, Øt V. V-(pu v u =0 Øt + (EE v)) =0, (9)
Opv
OE
7

---

Table 3: Results for the airfoil design problem, where bold numbers highlight the best method.
Model
Train
Test
eDAFNO
0.329%:000020% 0,448%+00.00
0..00%
-000005%
Proposed model
iDAFNO
0.642%++0.000%
cDAFNO irregular grids
0.331%++0.0035
0.659%%+00.077..
Geo-FNO
1.565%10. 1800%
1..5%
Baseline model
F-FNO
0.566%+40.066%
0.794%:t00.025%
FNO w/ mask
2.676%++0.055%
3.725%++0. 108%
UNet w/ mask
2.781%++1.0844
4.957%:+0.059%
with p. p and E' being the fluid density, pressure and the total energy, respectively. and e denoting the corresponding velocity field. The applied boundary conditions are: Poe = 1. Mach number Mo 0.8, and Psc 1 on the far field, with no penetration enforced on the airfoil. The dataset used for training is directly taken from Li et al. (2022a), which consists of variations of the NACA-0012 airfoil and is divided into 1000, 100, 100 samples for training, validation and testing, respectively. For this problem, we aim to learn the resulting Mach number field based given mesh as input. An example of the computational domain, the smoothened boundary encoding, the ground truth, and the eDAFNO prediction is illustrated in Figure 4
We report in Table our experimental observations using eDAFNO and iDAFNO, along with the comparison against FNO, Geo- Seo-FNO, F-FNO and UNet, whose models are directly attained from Li al. (2022a); Tran et al. (2022). We can see that the proposed eDAFNO achieves the lowest error on the test dataset, beating the best result of non Non-DAFNO baselines by 24.9% and Geo-FNO by 63.9%, respectively. Additionally, iDAFNO reaches similar level accuracy with only quarter of the total number of parameters employed, which is consistent with the findings in the previous example. With the properly trained DAFNO models, efficient optimization and inverse design are made possible.
a
Physical space
Computational space
Geo-FNO
eDAFNO
eDAFNO
0008
0.00
0.02
Prediction in physical space
eDAFNO prediction
eDAFNO imregular grids
Figure 5: eDAFNO applied to the airfoil dataset on irregular grids. (a): the highly irregular and adaptive grids in the physical space is firstly deformed (via either analytical mapping f trainable neural network for grid deformation) to uniform grids the computational space, on which eDAFNO is applied learn the underlying physics. The learned prediction is then converted back to the physical space via the inverse mapping f-1 or with another trainable neural network, (b): an illustration of the absolute error distribution of predictions in Geo-FNO, eDAFNO trained uniform grids, and eDAFNO trained on irregular grids.
Aside from the enhanced predictability, DAFNO can also be easily combined with the grid mapping technique Geo-FNO to handle non-uniform grids. particular, no modification on the model architecture is required, where one just needs to include analytical or trainable mapping from

---

non-uniform/ireguuaa mesh grids to uniform mesh grids. As demonstration, we consider irregular grids of the airfoil dataset and use a pre-computed function to map irregular grids to regular grids. In the irregular grid set, we place more grid points near the airfoil to provide a better resolution near the important parts. The test error is provided Table 3 where the test loss using the eDAFNO learned model is 0.659% + 0.007%, which is similar to the DAFNO results on uniform grids. In Figure 5 we demonstrate the irregular mesh and the absolute error comparison across Geo-FNO, DAFNO on regular grids, and DAFNO on irregular grids. One can see that, while both DAFNOs substantially oulperform Geo-FNO, the error contours from eDAFNO with an irregular mesh show a smaller miss-match region near the airfoil, illustrating the flexibility of DAFNO in meshing and its capability in resolving fine- grained features.
4.3
Crack propagation with topology change in domain
In this example, we aim to showcase DAFNO's capability in handling evolving domains by modeling crack propagation in brittle fracture. We emphasize that DAFNO represents the first neural operator that allows for learning with topology change. In the field of brittle fracture, growing crack can be viewed as change in topology, which corresponds to an evolving x(t) in the DAFNO architecture. In particular, we define the following time-dependent characteristic function:
[ x 2t(t) x(x, 1) = 0 a EET(
(10)
where (t()) denotes the time-dependent domain/evolving topology. Employing time-dependent x in (6) keeps the neural operator informed about the evolving topology. In general, the topology evolution rule that determines (2(t) can be obtained from a separate neural network or from physics as the case in the current example. The siddeiiiiitty synthetic data in this example is generated using 2D PeriFast software (Jafarzadeh et al., 2022a). which employs a peridynamics (PD) theory for modeling fracture (Bobaru et al.) 2016). A demonstration of the evolving topology, as well as further details regarding the governing equations and data generation strategies, is provided in Appendix B.3.
In this context, we select eDAFNO as the surrogate model to learn the the internal force density operator. Specifically. let un (z,,t, t). uz t) denote the two components of the displacement field u at time f. and L1. L2 be the two components of the internal force density L[u]. Given u1, U2 and as input data, we train two separate eDAFNO models to predict L1 and L2, respectively. Then, we substitute the trained surrogate models in the dynamic governing equation and adopt Pelocity-Verle time integration to update u for the next time step, whereafter f2 and X are updated accordingly.
The problem of interest is defined on a 40 mm x 40 mm thin plate with a pre-crack of length 10 mm one edge, which is subjected to sudden, uniform, and constant tractions on the top and bottom edges. Depending on the traction magnitude, crack grows at different speeds, may or may not bifurcate, and the final crack patterns can be of various shapes. Our training data is comprised of two parts, one consisting of 450 snapshots from a crack propagation simulation with a fixed traction magnitude of a 4 MPa, and the other consisting of randomized sinusoidal displacement fields and the corresponding L1 Le fields computed by the PD operator. The sinusoidal subset contains 4,096 instances without fracture and is used to diversify the training data and mitigate overfitting on the crack data. For testing. we use the trained models in two scenarios with traction magnitudes different from what used in training, which allows us to evaluate the generalizability of eDAFNO. Note that these tests are highly challenging for the trained models, as the predictions of the previous time step are used as the models' input in the current step, which leads to error accumulation as the simulation marches forward.
Figure 6 displays the test results on the crack patterns under different traction magnitudes, where the low traction magnitude (left column) shows slow straight crack, the mid-level traction results in a crack with moderate speed and a bifurcation event (middle column), and the highest traction leads to rapid crack growth and initiates a second crack from the other side of the plate (right column). The trained eDAFNO model is able to accurately predict the left and right examples while only seeing the middle one, indicating that it has correctly learned the true constitutive behavior of the material and generalized well to previously unseen loading scenarios and the correspondingly changed domain topology. To further verify eDAFNO's generalizability to unseen geometries, in Figure 7 we compare eDAFNO with the baseline FNO w/ smoothed mask model, and plot their relative errors in X and u. As observed, the FNO predictions become unstable at a fairly early time in the two test scenarios.

---

4 6
Ground Truth
eDA Predicttoon
Figure 6: Comparison of the fracture patterns with different loading scenarios between the high-
fidelity solution and eDAFNO prediction.
10
eDAFNO, 4MPP
eDAFNO, o-IMPs FNO with musk, IMPa
eDAFNO, 2MPa
Relative error geometry (mask)
eDAFNO, 2MPs FNO with mask, 2MPa
<DAFNO, 6-6MPs
o-6MPa
Relative error in displacement
FNO with mask, 6-4MPu
FNO with 2MPN
0.8
10
PNO with mask 6MP
0.6
0.4
10-1
0.2
2
3
Time (us)
Time (JuS)
Figure 7: Comparison of relative errors the characteristic function (left) and displacement fields (right) of the evolving topology predicted using the trained eDAFNO and FNO with mask each time step for the training (loading 4 MPa) and testing (loading = MPa and 6 MPa) scenarios.
since creates mask which is out of training distribution, while DAFNO can handle different crack patterns by hard coding it in the architecture, so the results remain stable for a much longer time.
5
Conclusion
We introduce two DAFNO variants to enable FNOs on irregular domains for PDE solution operator learning. By incorporating the geometric information from smoothed characteristic function in the iterative Fourier layer while retaining the convolutional form, DAFNO possesses the computational efficiency from FFT together with the flexibility operate different computational domains. As a result, DAFNO is not only highly efficient and flexible in solving problems involving arbitrary shapes, but also manifests its generalizability on evolving domains with topology change. In two benchmark datasets and a real-world crack propagation dataset, we demonstrate the state-of-the-ar performance of DAFNO. We find both architectures helpful in practice: eDAFNO is slightly more accurate while iDANNO is more computationally efficient and less overfitting with limited data.
Limitation:
Due to the requirement of the standard FFT package, in the current DAFNO focus on changing domains with uniformly meshed grids. However, we point out that this limitation can be lifted by using nonuniform FFT (Grreengard & Lee| 2004) deformation, as shown in the airfoil experiment. Additionally. in our applications, we have focused on applying the same types boundary conditions the changing domains (e.g., all Dirichlet or all Neumann). In this context, another limitation and possible future direction would be on the transferability PDEs with different types of boundary conditions.
additional mapping for grid
10

---

Acknowledgments and Disclosure of Funding
Jafarzadeh would like acknowledge support by the AFOSR grant FA9550-22-1-0197, and Y. Yu would like to acknowledge support by the National Science Foundation under award DMS-1755 1753031 and the AFOSR grant 1A9550-22-1-0197 Portions of this research were conducted on Lehigh University's s Research Computing infrastructure partially supported by NSF Award 2019035.
References
Agwai, A., Guven, I., and Madenci, E. Predicting crack propagation with peridynamics: comparative study. International journal fracture, 171(1):65, 2011.
Benson, D. J. and Okazawa, S. Contact a multi-material Eulerian finite element formulation. Computer Methods in Applied Mechanics and Engineering, 193(39):4277-43 4298,, 2004. ISSN 0045-7825.
Besmard, G., Hild, F., and Roux, S. "finite-element" displacement fields analysis from digital images: application to portevin-le châtelier bands. Experimental mechanics, 46(6):789-88) 803, 2006.
Bobaru, F., Foster, J. T., Geubelle, P. H., and Silling, S. A. Handbook of peridynamic modeling. CRC press, 2016.
Brigham, E. O. The fast Fourier transform and its applications. Prentice-Hall, Inc., 1988.
Cai, S., Mao, Z., Wang, Z., Yin, M., and Karniadakis, E. Physies-informed neural networks (PINNs) for fluid mechanics: A review. Acta Mechanica Sinica, pp. 1-12, 2022.
Cao, S. Choose transformer: Fourier galerkin. Advances in neural information processing systems, 34:24924-24940, 2021.
Carleo, G., Cirac, I., Cranmer, K., Daudet, L.., Schuld, M., Tishby, N., Vogt-Maranto, L., and Zdeborová, L. Machine learning and the physical sciences. Reviews of Modern Physics, 91()) 045002, 2019.
Day, J., Mullineux, N., and Reed, J. Developments in obtaining transient response using fourier transforms: Part i: Gibbs phenomena and fourier integrals. International Journal of Electrical Engineering Education, 3(4):501-506 1965.
De Luycker, E., Benson, D. J., Belytschko, T., Bazilevs, Y., and Hsu, M.-C. X-FEM in isogeometric analysis for linear fracture mechanics. International Journal for Numerical Methods in Engineering, 87:541-566 565, 2011.
Ghaboussi, J., Garrett Jr, J., and Wu, X. Knowledge-based modeling of material behavior with neural networks. Journal engineering mechanics, 1 117(1):32--153, 1991.
Ghaboussi, J., Pecknold, D. A., Zhang, M., and Haj-Ali, R. M. Autoprogressive training of neural network constitutive models. International Journal for Numerical Methods in Engineering, 42(1): 105-126, 1998.
Goswami, S., Bora, A., Yu, Y., and Karniadakis, G. E. Physics-informin neural operators. 2022 arXiv preprint arXh::2007.05748, 2022.
Gottlieb, D. and Shu, C.WW. On the gibbs phenomenon and its resolution. SIAM review, 39(4):
644 668, 1997.
Greengard, L. and Lee, J.-Y. Accelerating the nonuniform fast fourier transform. SIAM review, 46(3): 443-454, 2004.
Gupta, G., Xiao, X., and Bogdan, P. Multtwavelet-based operator learning for differential equations. Advances in Neural Information Processing Systems, 34:24048-24062, 2021.
Ha, Y. and Bobaru, Studies of dynamic crack propagation and crack branching with peridynam- ics. International Journal of Fracture, 162(():229-22) -244, 2010.

---

Hao, Z., Wang, Z., Su, H., Ying, C., Dong, Y. Liu, S., Cheng, Z., Song, J., and Zhu, J. Gnot: A general neural operator transformer for operator learning. International Conference on Machine Learning, pp. 12556-12569. PMIR, 2023.
He, J., Li, L., Xu, J., and Zheng, C. ReLu deep neural networks and linear finite elements. arXiv:1807.030 807.03973, 2018.
He, Q-. Laurence. D. W., Lee, C.-H., and Chen, J.-S. Manifold learning based data-driven modeling for soft biological tissues. Journal of Biomechanics, 117:110124, 2021.
Jafarzadeh, S., Mousavi, F., and Bobaru, F. Perifast/dynamics: matlab code for explicit fast convolution-based peridynamic analysis deformation and fracture. 2022a.
Jafarzadeh, S., Mousavi, F., Larios, A., and Bobaru, F. A general and fast convolution-based method for peridynamies: Applications to elasticity and brittle fracture. Computer Methods in Applied Mechanics and Engineering, 392:1 14666, 2022b.
Kamensky, D.,. Hsu, M.-.,, Yu, Y., Evans, J. A., Sacks, M. S., and Hughes, T. Immersogeometrii cardiovascular stru-structure interaction analysis with divergence-conforee conformmng b-splines. Computer methods in applied mechanics and engineering, 314:408-472, 2017.
Karniadakis, G. E., Kevrekidis, G., Lu, L., Perdikaris, P., Wang. S., and Yang, L. Physics-informed machine learning, Nature Reviews Physics, 3(6):422-440, 2021.
Kuhl, E., Hulshoff, S., and de Borst, An arbitrary Lagrangian Eulerian finite element approach for fluid-structure interaction phenomena. International Journal for Numerical Methods in Engineer- ing,
2003.
Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., and Anandkumar, A. Neural operator: Graph kernel network for partial differential equations. arXiv preprint arXiv:2003.03485 2020a.
Li, Z.,, Kovachki, N., Azizzadenesheli, K., Liu, B., Stuart, A., Bhattacharya, K., and Anandkumar, A. Multipole graph neural operator for parametric partial differential equations. Advances Neural Information Processing Systems, 33:NeurIPS 2020, 2020b.
Li, Z., Kovachki, N. B., Azizzadenesheli, K., Bhattacharya, K., Stuart, A., and Anandkumar, A. Fourier Neural Operator for Parametric Partial Differential Equations. In International Conference on Learning Representations, 2020c.
Li, Z., Huang, D. Z., Liu, B., and Anandkumar, A. Fourier neural operator with learned deformations for PDEs on general geometries. arXiv preprint arXiv:2207.0522 220222299, 2022a.
Li, Z., Meidani, K., and Farimani, A. B. Transformer for partial differential equations' operator learning. arXiv preprint arXiv:2205. 13671, 2022b.
Liu, N., Yu, Y., You, H., and Tatikola, N. INO: Invariant neural operators for learning complex physical systems with momentum conservation. arXiv preprint 11Xi::2212.14365, 2022.
Lu, L.. Jin, P., and Karniadakis, G. Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators. arXiv preprint arXiv: 19ii::910.. 90..0000...9.. 2019.
Lu, I.., He, H., Kasimbeg, P., Ranade, R., and Pathak, J. One-shot learning for solution operators of partial differential equations. arXiv preprint arXiv:2104.0511, 2021a.
Lu, L., Jin, P., Pang, G., Zhang, Z., and Karniadakis, G. E. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3): 218-229, 2021b.
Lu, L., Meng, X., Cai, S., Mao, Z., Goswami, S., Zhang, Z., and Karniadakis, G. E. A comprehensive and fair comparison two neural operators (with practical extensions) based on fair data, Computer Methods in Applied Mechanics and Engineering, 393: 14778, 2022.
12

---

Ong, Y. Z., Shen, Z., and Yang, H. IAE-NET: Integral autoencoders for ionpetiizationnnnvaiiee learning. 03 2022. doi: w.t.://.........2........
Pfau, D., Spencer, J. S., Matthews, A. G., and Foulkes, W. M. C. Ab initio solution the many- electron schrödinger equation with deep neural networks. Physical Review Research, 2(3):033429, 2020.
Ronneberger, O., Fischer, P., and Brox, T. U-net: Convolutional networks for biomedical image segmentation. In International Conference Medical image computing and computer-assisted intervention, pp. 234-241. Springer, 2015.
Shadden, S. C., Astorino, M., and Gerbeau, J.-F. Computational analysis of an aortic valve jet with Lagrangian coherent structures. Chaos, 20:017512, 2010.
Silling, S. Dynamic fracture modeling with a meshfree peridynamic code. Computational Fluid and Solid Mechanics, 2003:641 6444, 2003.
Simo, J. C. and Laursen, T. A. An augmented Lagrangian treatment of contact problems involving friction. Computers & Structures, 42(1))97-116, 1992. ISSN 0045-7949.
Tran, A., Mathews, A., Xie, L., and Ong, C. S. Factorized fourier neural operators. In The Eleventh International Conference Learning Representations, 2022.
Tripura, T, and Chakraborty, S. Wavelet neural operator: neural operator for parametric partial differential equations. arXiv preprint vrXiv:2205.0219) 2022.
Yin, M., Ban, E., Rego, B. V., Zhang, E., Cavinato, C.,, Humphrey, J. D., and Em Karniadakis, G. Simulating progressive intramural damage leading to aortic dissection using DeepONer: an regrator- neural network. Journal of the Royal Society Interface, 29(187)00210077, 2022a.
Yin. M., Zhang, E., Yu, Y., and Karniadakis, G. E. Interfacing finite elements with deep neural operators for fast multiscale modeling of mechanics problems. Computer Methods in Applied Mechanics and Engineering, in press, pp. 11 115027, 2022b.
Yin, Y., Kirchmeyer, M., Franceschi, J.-Y., Rakotomamonjy, A., et al. Continuous pde dynamics forecasting with implicit neural representations. In The Eleventh International Conference on Learning Representations, 2022c.
You, H., Yu, Y., D' D'Elia, M., Gao, T., and Silling, S. Nonlocal kernel network (NKN): A stable and inoolution-independee deep neural network. Journal of Computational Physics, pp. arXiv preprint arXiv:2201. 2200:22000.0211,, 2022a.
You, H., Zhang, Q.. Ross, C. J., Lee, C.-H., Hsu, M.-C., and Yu, Y. A physics-guided neural operator learning approach to model biological tissues from digital image correlation measurements. Journal of Biomechanical Engineering, accepted for publication, pp. arXiv preprint arXiv:2204.00205. 2022b.
You, H., Zhang, Q. Ross, C. Lee, C.-H,, and Yu, Y. Learning deep implicit fourier neural operators (IFNOs) with applications heterogeneous material modeling. Computer Methods in Applied Mechanics and Engineering, 398: 1e:115296, 2022c. doi: https://doi.org/10 http//0..0166..coo 1016//.cma.2022 2022.115996 15296.
Zhang. L., Han, J., Wang, H., Car, R., and Weinan, E. Deep potential molecular dynamics: scalable model with the accuracy of quantum mechanics. Physical Review Letters, 120(14):14301,, 2018.
13


Reviews:
[{"review_id": "LIaqYwRiL2", "reviewer": "Reviewer_FUid", "summary": "Fourier Neural Operators (FNOs) are a popular method for modeling physical systems such as different types of PDEs. However, in order to use the FFT to make FNO efficient, the input needs to be a regular grid. The authors study the question of irregular grid inputs for FNO, as well as problems with changing topologies. They propose to add a smoothed characteristic function in the integral layer architecture. This directly encodes the topology into the architecture, and it allows the architecture to continue to use FFT while allowing irregular grid inputs. The authors apply this domain agnostic method to the regular FNO as well as implicit FNO. The authors run experiments on material modeling, airfoil simulation, and material fractures.", "questions": "Can the authors comment on all of the weaknesses listed above?\n\nAlso, I wonder if Geo-FNO can be combined with DAFNO, to handle more complex topologies and also be able to handle changing topologies?\n\nDo you plan to release your code? It could even be released during the rebuttal with e.g. https://anonymous.4open.science/.\n\nI would raise my score if some or all of the points from the \"weaknesses\" section are addressed.", "limitations": "The authors discuss limitations in Section 5: they only consider changing domains where the grid is uniform, and they only consider problems with the same boundary conditions as the domain changes. I agree that it would be good to add a mapping for grid deformation in future work.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- This is the first neural operator that can handle learning with topology changes. This is a useful property to handle fracture problems, such as material fracture or earthquakes.\n- The technique is relatively simple: adding a characteristic function to the integral kernel and adding a smoothing term, but it is novel and elegant that the characteristic function encodes the topology directly in the architecture. This is how the authors’ technique can learn changing topologies, when, say Geo-FNO learns a fixed mapping and cannot learn changing topologies. \n- The authors experiment on three problems: hyperelasticity, airfoils, and crack propagation, and they also perform ablation studies to separate the characteristic function and the smoothness.", "weaknesses": "- My biggest concern is that the technique may not be able to handle highly irregular topologies or topologies with fine-grained features, for two separate reasons. As shown in Figure 1, the resulting grid from the authors’ technique is still uniform, but with a characteristic added, to tell what is inside or outside the topology. Since the grid is still uniform, this may not work well for a topology that is very irregular or has fine features. This is in contrast to Geo-FNO, which learns a mapping, allowing it to handle highly irregular topologies and also to focus more grid points on the “important” parts of the topology and fewer grid points on the coarse / less important parts.\nFurthermore, the smoothing function could magnify the above problem, making it even harder to model delicate parts of the topology. I think of this as a weakness because the authors say that handling irregular topologies is one of the main contributions of their work. \n- The technique is fairly simple: it boils down to adding a characteristic function with smoothing. But on the other hand, this technique is the first to handle dynamically changing topologies.\n- The authors did not release code or mention releasing code. If they authors released code (anonymously during submission) it would have a bigger impact.\n- Why not compare airfoil with the other baselines? Also, why do the 3rd and 4th panels of Figures 4 look like there is a vertical line discontinuity in the center of the image?\n- Why not compare the cracking to more baselines than just FNO, such as Geo-FNO?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their valuable suggestions.\n\n**Highly irregular topologies or topologies with fine-grained features**: DAFNO can be readily combined with the grid mapping technique in Geo-FNO, to handle non-uniform grids. No modification on the NN architecture is required, and one just needs to include an analytical or trainable mapping from non-uniform/irregular mesh grids to uniform mesh grids. As a demonstration of this capability, we consider irregular grids in the airfoil problem and use a pre-computed function to map irregular grids to regular grids, then we train eDAFNO. In this irregular grid set, we place more grid points near the airfoil, so as to provide a better resolution near the important parts as suggested by the reviewer. The test error is provided in the table below: The test loss using the eDAFNO learned model is 0.659\\%$\\pm$0.007\\%, which is similar to the DAFNO results on uniform grids. In Fig. 1 of the attached pdf file in the global response we demonstrate the irregular mesh, and in Figure 2 we plot the errors of Geo-FNO, DAFNO on regular grids, and DAFNO on irregular grids. One can see that while both DAFNOs substantially outperform Geo-FNO, the error contours from eDAFNO with irregular mesh show a smaller miss-match region near the airfoil, verifying the flexibility of DAFNO in meshing and its capability in resolving fine-grained features. \n\n**Smoothing function**: We treat the smoothing level $\\beta$ as a hyperparameter, and it is tuned according to the validation error, to either smoothening the boundary or keep the original boundary untouched. This is illustrated in Fig. 8 in the appendix of the paper, where the original boundary is not smoothened with a very large smooth level $\\beta$. As such, the $\\beta$ as well as the smoothing level are automatically chosen based on the intrinsic resolution of the topology in training and validation datasets: if the topology prefers fine-grained features, a smaller $\\beta$ (and smoother $\\chi$) will result in a larger validation error and will not be chosen. We will include a discussion about this in the revised manuscript to strengthen the paper.\n\n**Release code**: We have uploaded our DAFNO code of the hyperelasticity problem on anonymous github and sent the link to the Area Chair. Our DAFNO package on other problems will also be made publicly available on github once the paper is accepted.\n\n**Airfoil with the other baselines**: The hyperelasticity problem serves as a thorough comparison against available baselines, after which we picked the baselines that have the best performance for the airfoil problem. This is in consistence with the settings in Geo-FNO. In order to further enrich the baselines, we have added another recent FNO variant, Factorized FNO (F-FNO), in both the elasticity and airfoil problems as a baseline. Additionally, as mentioned above, we added eDAFNO applied to irregular grids to highlight the fact that eDAFNO is not constrained to uniform grids. Our conclusion still holds. DAFNOs consistently outperform all selected baselines. The ``vertical line discontinuity'' in Fig. 4 is a physical phenomenon called shock wave, which reflects the nature of the transonic flow governed by the Euler's equation [1]. As discussed in [2] and [3], the capability of capturing such discontinuity is considered as an important metric in classical numerical methods.\n\n[1] Luk, Jonathan, and Jared Speck. \"Shock formation in solutions to the 2D compressible Euler equations in the presence of non-zero vorticity.\" Inventiones mathematicae 214.1 (2018): 1-169.\n\n[2] Roe, Philip L. \"Characteristic-based schemes for the Euler equations.\" Annual review of fluid mechanics 18.1 (1986): 337-365.\n\n[3] Hennemann, Sebastian, et al. \"A provably entropy stable subcell shock capturing approach for high order split form DG for the compressible Euler equations.\" Journal of Computational Physics 426 (2021): 109935.\n\n**Cracking with Geo-FNO**: The crack example is presented intentionally as a problem where Geo-FNO falls short and cannot be used. The reason is that the Geo-FNO requires a pre-defined or trainable isomorphism between a rectangular domain and the targeted irregular domains. However, in fracture problems the domain undergoes severe topological changes including the emergence of new holes or discontinuities/boundaries (as shown in Fig. 5) which is not known a priori. Such an isomorphism does not exist (intuitively, there is no way to define a mapping between each yellow regions in Fig. 5 to a rectangular domain).\n\nTest errors for the hyperelasticity problem:\n\n| Model| 10 samples | 100 samples | 1000 samples | \n| :------------- | :-----------: | :-----------: | :-----------: | \n|eDAFNO | **16.446\\%**$\\pm$**0.472\\%** | 4.247\\%$\\pm$0.066\\% | **1.094\\%**$\\pm$**0.012\\%**|\n|iDAFNO | 16.669\\%$\\pm$0.523\\% | **4.214\\%**$\\pm$**0.058\\%** | 1.207\\%$\\pm$0.006\\%|\n|FNO w/ mask | 19.487\\%$\\pm$0.633\\% | 7.852\\%$\\pm$0.130\\% | 4.550\\%$\\pm$0.062\\% |\n|IFNO w/ mask | 19.262\\%$\\pm$0.376\\% | 7.700\\%$\\pm$0.062\\% | 4.481\\%$\\pm$0.022\\% |\n|Geo-FNO | 28.725\\%$\\pm$2.600\\% | 10.343\\%$\\pm$4.446\\% | 2.316\\%$\\pm$0.283\\% |\n|GNO | 29.305\\%$\\pm$0.321\\% | 18.574\\%$\\pm$0.584\\% | 13.007\\%$\\pm$0.729\\% |\n|DeepONet | 35.334\\%$\\pm$0.179\\% | 25.455\\%$\\pm$0.245\\% | 11.998\\%$\\pm$0.786\\%|\n|F-FNO | 35.672\\%$\\pm$3.852\\% | 12.135\\%$\\pm$5.813\\% | 3.193\\%$\\pm$1.622\\%|\n|UNet | 98.167\\%$\\pm$0.236\\% | 34.467\\%$\\pm$2.858\\% | 5.462\\%$\\pm$0.048\\%|\n\nResults for the airfoil design problem:\n| Model | Train error | Test error |\n| :------------- | :-----------: | :-----------: |\n|eDAFNO | 0.329\\%$\\pm$0.020\\% | **0.596\\%**$\\pm$**0.005\\%** |\n|iDAFNO | 0.448\\%$\\pm$0.012\\% | 0.642\\%$\\pm$0.020\\% |\n|eDAFNO on irregular grids | 0.331\\%$\\pm$0.003\\% | 0.659\\%$\\pm$0.007\\% |\n|Geo-FNO | 1.565\\%$\\pm$0.180\\% | 1.650\\%$\\pm$0.175\\% |\n|F-FNO | 0.566\\%$\\pm$0.066\\% | 0.794\\%$\\pm$0.025\\% |\n|FNO w/ mask | 2.676\\%$\\pm$0.054\\% | 3.725\\%$\\pm$0.108\\% |\n|UNet w/ mask | 2.781\\%$\\pm$1.084\\% | 4.957\\%$\\pm$0.059\\% |"}, {"review_id": "WPzDmqjeJV", "reviewer": "Reviewer_fNa3", "summary": "In this paper, the authors propose the Domain Agnostic Fourier Neural Operator (DAFNO), an FNO that can deal with irregular boundaries. While the classical FNO is limited by construction to irregular domains, DAFNO simply includes a smoothed function $I(\\cdot)$  of the characteristic function of the domain on which the data is defined. DAFNO is shown to outperform relevant baselines on 2-dimensional problems with irregular boundaries and one problem with topology changing over time.", "questions": "- Can the proposed method be extended to modeling systems ≥ than 3 dimensions?\n- At line 262: “In general, the topology evolution rule that determines Ω(t) can be obtained from a separate neural network or from physics as is the case in the current example”, did you test out a separate neural network in the experiments as well?", "limitations": "The main limitation (about uniformly meshed grids) is mentioned. Other limitations are covered in the “weaknesses” section.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is about a very simple yet effective idea: including a smoothed mask to model boundaries inside the integral operator. This approach is original to my knowledge and effective in terms of strong experimental evaluation relevant to practical applications, including being able to handle topology changes. The paper is clear in its description and well-placed in the current literature on neural operators and deep surrogate models for scientific ML.", "weaknesses": "In my opinion, there is no major weakness in this paper. The main limitation could be that the approach is (particularly) simple, but I believe this does not need to be seen as a negative point. Sometimes, simpler is better. The experimental results are solid but do not encompass more complex settings such as 3D fluid dynamics. Another weakness is the lack of source code, which I believe should be provided, given the simplicity of the setting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their valuable suggestions.\n\n**High-dimensional problems**: DAFNOs are readily applicable to more complex settings and higher-dimensional problems, as neither the characteristic geometric encoding nor the smoothening technique is constrained to a specific dimension. It will just require more memory when it comes to higher dimensions.\n\n**Release code**: We have uploaded our DAFNO code of the hyperelasticity problem on anonymous github and sent the link to the Area Chair. Our DAFNO package on other problems will also be made publicly available on github once the paper is accepted.\n\n**Topology evolution rule as NN**: No yet. Practically, one can calculate the topology evolution rule in the form of fracture energy [1] or damage field [2] using a separate neural network, although that might introduce additional modeling error in long-term propagation. In this work we used physical laws so as to focus on resolving the geometric changes.\n\n[1] Goswami, Somdatta, et al. \"A physics-informed variational DeepONet for predicting crack path in quasi-brittle materials.\" Computer Methods in Applied Mechanics and Engineering 391 (2022): 114587.\n\n[2] You, Huaiqian, et al. \"Learning deep implicit Fourier neural operators (IFNOs) with applications to heterogeneous material modeling.\" Computer Methods in Applied Mechanics and Engineering 398 (2022): 115296."}, {"review_id": "UGSY84PRZh", "reviewer": "Reviewer_UPhM", "summary": "The Fourier Neural Operator (FNO) is a model in the field of neural operators that has successfully interpreted various physical phenomena. However, one of the issues with FNO is its limitation to learn only on rectangular domains. In Geo-FNO, this problem is addressed by lifting irregular domains to the latent space of rectangular domains.\n\nThe authors propose a novel convolution integral kernel through the domain characteristic function X(x) and introduce DAFNO (Domazin-Adapted Fourier Neural Operator). DAFNO outperforms the existing baseline in problems involving airfoil and hyperelasticity.\n", "questions": "Is there no experimental result for the benchmark dataset (Pipe) based on the Navier-Stokes equation in the Geo-FNO paper? I regard it as one of the significant experiments in this domain.\n\nIntuitively, it might be expected that DAFNO, having designed the Iterative layers in parallel, could have larger training and inference times compared to the baseline models like FNO and Geo-FNO. Can you provide any measurement results to demonstrate this?\n", "limitations": "There is no limitations.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Conducting research on a domain-agnostic physics simulator is a crucial endeavor. In particular, the field of neural operators, which the authors have explored, offers a highly efficient approach zto interpreting recent physical information, thus holding significant potential.", "weaknesses": "Could you provide more detailed explanation about the mathematical motivation behind the domain characteristic function? It would be necessary to provide specific explanations for Eq(4) and Eq(5).\n\nDue to the varying model sizes between the baseline models and the proposed model in Table 1, it is difficult to claim that the comparison is fair. A more equitable comparison can be achieved when all model sizes are standardized to be the same.\n\nThe authors proposed a new neural operator model called DAFNO. However, besides modifying the internal structure of the iterative layers in Geo-FNO, specifically by splitting the structure of the iterative layers in Figure 2, it is challenging to identify any other distinct novelties.\n\nAdding more baselines in the field of neural operators could lead to better experiments and improved evaluations. Such as, MWT (Multiwavelet-based neural operator), WNO (Wavelet neural operator)\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their constructive comments.\n\n**Mathematical motivation behind the domain characteristic function**: In order to obtain a truly domain-independent operator, we aim to hard-code domain information into the architecture. The challenge is to maintain the applicability of FFT while hard-coding the bounded, arbitrarily shaped domain. The aim is achieved if we enclose the bounded domain inside the rectangular box (so FFT is applicable), but \"cut\" the interactions between the main domain and the exterior, as if they are two non-interacting bodies. This means that when FNO integral is being computed for a point $x$, integration over points $y$ that fall onto the exterior should be avoided. This is achieve by adding $\\chi(x)\\chi(y)$ inside the integral. $\\chi$ is 1 inside and 0 outside of the domain; as a result, the integrand becomes zero for $(x,y)$ pairs that one falls inside and one falls outside of the domain since $\\chi(x)\\chi(y) = 0$ for such pairs. Hence, the main domain and the exterior are separated. With this simple but effective modification, the integral operator is still defined over a box and the convolution form of the integral is preserved. As a result, FFT remains applicable while the bounded domain info is hard-coded into the architecture. \n\n**Unify model sizes in table 1**: We have updated the total number of parameters in DeepONet to be on the same level as other models ($\\sim$2.5M), and the results are updated accordingly. Here, we point out that the only exceptions are the two implicit models (i.e., iDAFNO and IFNO), which have much fewer parameters. This is because these models are designed to be layer-independent in parameters. In other words, different layers share the same set of parameters, and therefore the total number of parameters stays the same with the increase of layers. To provide a fair comparison between explicit and implicit models, we employ the same architectural hyperparameters (number of layers, width, modes, etc), for eDAFNO, iDAFNO, FNO, and IFNO, while keeping the number of parameters in all explicit models on the same level as other baselines. As a result, implicit models having far fewer parameters than others.\n\nThe total number of parameters and per-epoch runtime (second) in the hyperelasticity problem.\n\n|Model | eDAFNO | iDAFNO | FNO | IFNO | Geo-FNO | GNO | DeepONet | UNet | F-FNO |\n| :------------- | :-----------: | :-----------: | :-----------: | :-----------: | :-----------: | :-----------: | :-----------: | :-----------: | :-----------: | \n|nparams | 2.37M | 0.60M | 2.37M | 0.60M | 3.02M | 2.64M | 3.10M | 3.03M | 3.21M|\n|runtime (s) | 2.00 | 1.70 | 1.81 | 1.62 | 5.12 | 98.37 | 940.12 | 5.04 | 3.41 |\n\nTest errors for the hyperelasticity problem.\n\n| Model| 10 samples | 100 samples | 1000 samples | \n| :------------- | :-----------: | :-----------: | :-----------: | \n|eDAFNO | **16.446\\%**$\\pm$**0.472\\%** | 4.247\\%$\\pm$0.066\\% | **1.094\\%**$\\pm$**0.012\\%**|\n|iDAFNO | 16.669\\%$\\pm$0.523\\% | **4.214\\%**$\\pm$**0.058\\%** | 1.207\\%$\\pm$0.006\\%|\n|FNO w/ mask | 19.487\\%$\\pm$0.633\\% | 7.852\\%$\\pm$0.130\\% | 4.550\\%$\\pm$0.062\\% |\n|IFNO w/ mask | 19.262\\%$\\pm$0.376\\% | 7.700\\%$\\pm$0.062\\% | 4.481\\%$\\pm$0.022\\% |\n|Geo-FNO | 28.725\\%$\\pm$2.600\\% | 10.343\\%$\\pm$4.446\\% | 2.316\\%$\\pm$0.283\\% |\n|GNO | 29.305\\%$\\pm$0.321\\% | 18.574\\%$\\pm$0.584\\% | 13.007\\%$\\pm$0.729\\% |\n|DeepONet | 35.334\\%$\\pm$0.179\\% | 25.455\\%$\\pm$0.245\\% | 11.998\\%$\\pm$0.786\\%|\n|F-FNO | 35.672\\%$\\pm$3.852\\% | 12.135\\%$\\pm$5.813\\% | 3.193\\%$\\pm$1.622\\%|\n|UNet | 98.167\\%$\\pm$0.236\\% | 34.467\\%$\\pm$2.858\\% | 5.462\\%$\\pm$0.048\\%|\n\n**Challenging to identify distinct novelties**: To our best knowledge (and also as pointed out by reviewers FUid and BzBv), our work has for the first time proposed to modify the internal structure of the iterative layers and encode domain geometry. As a result, DAFNOs are the first neural operator that can represent and handle dynamically changing domain topology (also pointed out by reviewer FUid).\n\n**Adding more baselines such as MWT**: We appreciate the reviewer's suggestion. The reason why we did not use them as baselines in our current work is that MWT requires the input grids to be 2 to the integer powers (i.e., $2^N$) and does not fit the benchmark datasets. We will include a discussion in the revised manuscript to acknowledge the contribution of MWT and MNO. Per the reviewer's suggestion, we have added Factorized FNO (F-FNO) as an additional baseline in both the elasticity and airfoil problems (see the table for elasticity problem above, and results of airfoil problem in global response), and the original findings still stand: DAFNOs consistently outperform all baselines.\n\n**Baseline on pipe flow example from Geo-FNO**: Per suggested by the reviewer, we have added an additional test of DAFNO on the pipe flow example. As shown in Fig.  3 of the attached pdf file in the global response, eDAFNO has achieved a similar performance to Geo-FNO: when comparing the relative $L^2$ errors, eDAFNO's test error on the pipe dataset is 0.719\\%, while the test error of Geo-FNO is 0.67\\%. When comparing the maximum absolute error, eDAFNO has 0.051, while Geo-FNO has 0.061. This is probably due to the fact that all pipes have a very simple geometry, which can be accurately represented with the pre-specified mapping in Geo-FNO. We want to further comment that such a pre-specified mapping for grid deformation can also be readily added to DAFNO, as demonstrated in Fig. 1 in the attached pdf file of the global response for the airfoil problem. In that case, DAFNO will be exactly the same as Geo-FNO. \n\n**Computational cost comparison**: We have already reported the comparison in terms of the runtime in Table 4 in the Appendix (which is also copied above). As demonstrated in the table, DAFNOs' runtime is similar to FNO, and beats Geo-FNO by 67\\% in iDAFNO and 61\\% in eDAFNO."}, {"review_id": "BXdYswnmUs", "reviewer": "Reviewer_dnax", "summary": "The paper explores the extension of Fourier Neural Operators (FNOs) to irregular geometries and topology changes. To leverage the computational speed benefits of the fast Fourier transform (FFT) employed by FNOs, they enclose the physical domain with a period box. They then adapt the FNO kernel by multiplying the integrand with a smoothed characteristic function that encodes domain information. This encoding enables their method to generalize to new geometries while preserving the FFT computational efficiency. They outperform several FNO baselines on two benchmark datasets, and showcase the properties of their method on a real-world crack propagation dataset, which contains evolving topology.", "questions": "1. I wonder why Geo-FNO is not in the baselines for the Crack propagation dataset. It would have been an interesting baseline, especially to see if it could adapt to evolving geometries better than a masked FNO. It is claimed that Geo-FNO cannot handle topology change, but once a continuous mapping is learned, it can be applied. I agree that it should not adapt well, but is there a specific reason for its absence in these experimental results? \n\n2. Just to confirm my understanding, for the FNO baseline, the grid is interpolated as in Geo-FNO, and then a mask is given as an additional input?\n\n3. I don’t quite understand the link from equation 5 to the equation before 6. Why is $\\chi(x)$ in factor of $W^l h(x) + c^l$ ? Is it to ensure that DAFNO will not produce results for points outside of the domain? \n\n4. Given the link of the smoothing function to a signed distance function (SDF) with the $\\\\tanh(\\\\beta \\\\text{dist}(x, \\partial \\Omega))$ formulation, is there a reason for not directly using a SDF instead of the smoothing function ? It may have given more (or a different) information to the network, as in different previous works (for e.g. [4]). \n\n5. Out of curiosity, have you experimented with using other smoothing functions besides tanh (for e.g. a logistic function)?\n\n[4]: Guo, Xiaoxiao, Wei Li, and Francesco Iorio. \"Convolutional neural networks for steady flow approximation.\" Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016.", "limitations": "The main limitations are already presented in the article.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well written and clear. The different choices of architecture are well justified. \n- It tackles an interesting and useful problem for real-world application of FNO.\n- The idea of incorporating the domain information in the kernel is simple, but effective and grounded, and the smoothing of the domain characteristic function helps avoiding dicontinituty problems.\n- Experimental results on three datasets are strong.\n- The crack propagation dataset is well-considered and helps highlighting the advantages of DAFNO, especially its ability to adapt to evolving topologies.", "weaknesses": "- The comparison with several baselines is interesting, but given the recent improvements seen over FNO and Geo-FNO, having F-FNO [1] as a baseline seems important. In addition to neural operators, other grid-independent methods that can be used on irregular domains have been developed, for instance [2] implemented a continuous model using Implicit Neural Representations (INRs). It would have been interesting to see a comparison or at least a discussion about the advantage of DAFNO in the light of these recent improvements in the community (for e.g. computational cost?).\n\n- As explained in the limitations and clear from the methodology itself, the architecture struggles to handle non-uniformly meshed grids due to the inability to use FFT on such grids. This limitation significantly restricts the range of problems that can be efficiently tackled using this architecture. For example, if applied to real-world airfoil surrogate modeling problems like [3], it seems that DAFNO would not offer significant improvements over the original FNO.\n\n- While the idea behind DAFNO is interesting and novel, the architectural improvement upon the regular FNO architecture seems limited. The authors address this issue in a ‘remark’ paragraph, but I am not totally convinced that it is a sufficient improvement, especially since the architecture does not appear to efficiently handle non-uniformly meshed grids.\n\n- This is more a remark than a weakness. In Figure 3, the zero-shot super-resolution prediction from eDAFNO does not effectively highlights the resolution invariant properties, since there is no ground truth to compare it to. While some degree of \"deblurring\" is noticeable, it remains unclear if the network successfully captures higher frequency phenomena that may occur at higher resolutions. The appendix, specifically Figure 10, sheds light on this property for the crack propagation dataset. However, Figure 3 does not truly demonstrates this resolution invariant property.\n\n[1]: Tran, Alasdair, et al. \"Factorized Fourier Neural Operators.\" The Eleventh International Conference on Learning Representations. 2022.\n\n[2]: Yin, Yuan, et al. \"Continuous PDE Dynamics Forecasting with Implicit Neural Representations.\" The Eleventh International Conference on Learning Representations. 2023.\n\n[3]: Bonnet, Florent, et al. \"AirfRANS: High Fidelity Computational Fluid Dynamics Dataset for Approximating Reynolds-Averaged Navier–Stokes Solutions.\" Advances in Neural Information Processing Systems 35 (2022): 23463-23478.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their valuable suggestions.\n\n**Comparison with additional baselines**: We have added F-FNO as an additional baseline in both the elasticity and airfoil problems. On the other hand, since INR focuses on learning a time-continuous dynamics model of the underlying flow, and it is not applicable to these two baseline problems. We would like to point out that when comparing F-FNOs with DAFNOs, our original conclusions still stand: DAFNOs consistently outperform F-FNOs in accuracy, with halved computational time.\n\nTest errors for the hyperelasticity problem.\n\n| Model| 10 samples | 100 samples | 1000 samples | \n| :------------- | :-----------: | :-----------: | :-----------: | \n|eDAFNO | **16.446\\%**$\\pm$**0.472\\%** | 4.247\\%$\\pm$0.066\\% | **1.094\\%**$\\pm$**0.012\\%**|\n|iDAFNO | 16.669\\%$\\pm$0.523\\% | **4.214\\%**$\\pm$**0.058\\%** | 1.207\\%$\\pm$0.006\\%|\n|Geo-FNO | 28.725\\%$\\pm$2.600\\% | 10.343\\%$\\pm$4.446\\% | 2.316\\%$\\pm$0.283\\% |\n|F-FNO | 35.672\\%$\\pm$3.852\\% | 12.135\\%$\\pm$5.813\\% | 3.193\\%$\\pm$1.622\\%|\n\nThe per-epoch runtime (second) in the hyperelasticity problem.\n\n|Model | eDAFNO | iDAFNO | FNO | IFNO | Geo-FNO | F-FNO |\n| :------------- | :-----------: | :-----------: | :-----------: | :-----------: | :-----------: | :-----------: | \n|runtime (s) | 2.00 | 1.70 | 1.81 | 1.62 | 5.12 | 3.41 |\n\nTest errors for the airfoil problem.\n\n| Model | Train error | Test error |\n| :------------- | :-----------: | :-----------: |\n|eDAFNO | 0.329\\%$\\pm$0.020\\% | **0.596\\%**$\\pm$**0.005\\%** |\n|iDAFNO | 0.448\\%$\\pm$0.012\\% | 0.642\\%$\\pm$0.020\\% |\n|eDAFNO on irregular grids | 0.331\\%$\\pm$0.003\\% | 0.659\\%$\\pm$0.007\\% |\n|Geo-FNO | 1.565\\%$\\pm$0.180\\% | 1.650\\%$\\pm$0.175\\% |\n|F-FNO | 0.566\\%$\\pm$0.066\\% | 0.794\\%$\\pm$0.025\\% |\n\n**Handling non-uniform grids**: We would like to point out that DAFNO can be readily combined with the grid mapping technique in Geo-FNO, to handle non-uniform grids. As a demonstration of this capability, we consider irregular grids in the airfoil problem and use a pre-computed function to map irregular grids to regular grids, then we train eDAFNO. In this irregular grid set, we place more grid points near the airfoil, so as to provide a better resolution near the important parts as suggested by the reviewer. The test loss using the eDAFNO learned model is 0.659\\%$\\pm$0.007\\%, which is similar to the DAFNO results on uniform grids. In Fig. 1 of the attached pdf file in the global response we demonstrate the irregular mesh, and in Fig. 2 we plot the errors of Geo-FNO, DAFNO on regular grids, and DAFNO on irregular grids. One can see that while both DAFNOs substantially outperform Geo-FNO, the error contours from eDAFNO with irregular mesh show a smaller miss-match region near the airfoil, verifying the flexibility of DAFNO in meshing and its capability in resolving possible fine-grained feature in real-world modeling problems.\n\nThe original paper focused on uniform grids, just to highlight our simple but efficient architecture to embed characteristic domain encoding -- which can be readily combined with the grid deformation technique in Geo-FNO to handle non-uniform grids, in addition to its unique capability in handling topological changes.\n\n**Geo-FNO for crack propagation**: The crack example is presented intentionally as a problem where Geo-FNO falls short and cannot be used. The reason is that the Geo-FNO requires a pre-defined or trainable isomorphism between a rectangular domain and the targeted irregular domains. However, in fracture problems the domain undergoes severe topological changes including the emergence of new holes or discontinuities/boundaries (as shown in Fig. 5) which is not known a priori. Such an isomorphism does not exist (intuitively, there is no way to define a mapping between each yellow regions in Fig. 5 to a rectangular domain).\n\n**FNO baseline**: The reviewer is correct that the grid in the FNO baseline is interpolated as in Geo-FNO, and then a mask is given. The settings as well as the interpolated benchmark datasets are consistent with the Geo-FNO paper.\n\n**Explanation of $\\chi(x)$ in factor of $W^l(x)+c^l$**: Yes, the reviewer is correct. Adding $\\chi(x)$ in factor of $W^l(x)+c^l$ makes the values outside of the main domain zero. Additionally, it allows us to factor out $\\chi(x)$ from the entire equation and present it in the form of Eq. 6.\n\n**Using SDF as the characterization function**: We didn't use signed-distance function directly because there is no physical meaning: the characterization function $\\chi$ in our setting is a smoothed binary representation which aims to provide near-local structural information. By multiplying it in the integral layer, we can eliminate the interactions between points inside and outside the physical domain, similar as in the molecular dynamics method [1]. As pointed out by [2], SDF aims to provide global structural information and its effect is very different from the local information from binary representation. To them, SDF works but binary representation does not, probably due to the fact that their NN architecture is substantially different from ours: SDF was used as an input, while the characteristic function was applied to layer weights in our setting. Moreover, the tanh on distance function aims to provide a smoothing function, while the SDF itself is not smooth.\n\n[1] Hansson, Tomas et al. \"Molecular dynamics simulations.\" Current opinion in structural biology 12.2 (2002): 190-196.\n\n[2] Guo, Xiaoxiao et al. \"Convolutional neural networks for steady flow approximation.\" Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016.\n\n**Other smoothing functions**: We have also tested other smoothing functions, such as the Gaussian filter. The effect is very similar to tanh. This fact was also verified from Table 6 in our ablation study: As far as the characteristic function has a sufficient level of smoothness, the test error does not vary much."}, {"review_id": "LZxvulbTZ7", "reviewer": "Reviewer_BzBv", "summary": "This paper presents a novel method of extending Fourier Neural Operators (FNO) to irregular geometries using an indicator function to represent the shape of the geometric area. This area is then extended to a larger, regular area, thereby allowing FNO to handle irregular geometries. This is indeed an innovative approach, as the use of indicator functions in neural operators to handle irregular areas has not been done before. The methodology bears some similarities to the use of neural networks in  PINNs to learn hard constraints, ensuring regular equations comply with constraints on the geometric boundary. Experimental results indicate that the proposed method achieves commendable performance with fewer parameters.", "questions": "None.", "limitations": "None.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper's idea is innovative, using indicator functions in neural operators to handle irregular geometries is a novel approach. The authors show through experimental results that the proposed method performs well with fewer parameters, a significant advantage.", "weaknesses": "1. The review of related work seems insufficient. The recent development of methods represented by transformers [1,2,3] that handle irregular geometric areas well is not mentioned or cited at all. Moreover, last year's ICLR Factorized FNO [4] also dealt with several non-uniform geometric area problems.\n2. Regarding the technical aspect of the paper, I have a concern. The original irregular grid may be adaptive, implying that we may need a very high resolution when embedding it into a uniform grid. This might mean that the uniform grid's spacing needs to be smaller than the smallest spacing of the original grid, leading to considerable computation waste. The proposed method does not seem to avoid this waste.\n3. I suggest that for datasets like airfoil and hyperelasticity, which have been detailed in other works, there is no need for further extensive description in the main text. From a machine learning researcher's perspective, the dataset's attributes, sizes, and challenges are more important than detailed principles of its generation, especially when these have already been thoroughly discussed in other studies.\n\nConsidering the above points, the authors need first add a few key references, carefully consider the technical questions raised, and then heavily revise the paper. If the core technical challenges (question 2 ) can be addressed, I may consider increasing the score after the rebuttal. Otherwise, I would suggest that the authors continue to revise this work for submission to another conference or journal.\n\n1. Choose a Transformer: Fourier or Galerkin (https://arxiv.org/abs/2105.14995)\n2. Transformer for Partial Differential Equations' Operator Learning (https://arxiv.org/abs/2205.13671)\n3. GNOT: A General Neural Operator Transformer for Operator Learning (https://arxiv.org/abs/2302.14376)\n4. Factorized Fourier Neural Operators (https://arxiv.org/abs/2111.13802)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their constructive comments.\n\n**References on transformer-type neural operators and F-FNO**: We thank the reviewer's kind suggestions, and have added F-FNO [4] as a baseline in our comparison. As shown in the tables below: in both the hyperelasticity and airfoil design problems, DAFNO outperforms F-FNO.  Regarding the transformer-type neural operators, we would like to point out that [1] is based on regular grids and does not handle irregular geometric areas. While [2] and [3] can handle irregular geometric areas, they got officially published in April and July this year, respectively, which were around the same time this manuscript was submitted. We will cite these work in our revised paper.\n\n**Handling non-uniform grids**: DAFNO can be easily combined with Geo-FNO and include either an analytical or trainable mapping from non-uniform/irregular mesh grids to uniform mesh grids. As a demonstration of this capability, we consider irregular grids in the airfoil problem and use a pre-computed function to map them to regular grids, as shown in Fig. 1 in the attached pdf file of the global response. Then, we train eDAFNO on regular grids and map the results back. The test loss on irregular grids using the eDAFNO learned model is 0.659\\%$\\pm$0.007\\%, which is similar to the DAFNO results on uniform grids.\n\nTest errors for the hyperelasticity problem, where bold numbers highlight the best method.\n\n| Model| 10 samples | 100 samples | 1000 samples | \n| :------------- | :-----------: | :-----------: | :-----------: | \n|eDAFNO | **16.446\\%**$\\pm$**0.472\\%** | 4.247\\%$\\pm$0.066\\% | **1.094\\%**$\\pm$**0.012\\%**|\n|iDAFNO | 16.669\\%$\\pm$0.523\\% | **4.214\\%**$\\pm$**0.058\\%** | 1.207\\%$\\pm$0.006\\%|\n|FNO w/ mask | 19.487\\%$\\pm$0.633\\% | 7.852\\%$\\pm$0.130\\% | 4.550\\%$\\pm$0.062\\% |\n|IFNO w/ mask | 19.262\\%$\\pm$0.376\\% | 7.700\\%$\\pm$0.062\\% | 4.481\\%$\\pm$0.022\\% |\n|Geo-FNO | 28.725\\%$\\pm$2.600\\% | 10.343\\%$\\pm$4.446\\% | 2.316\\%$\\pm$0.283\\% |\n|GNO | 29.305\\%$\\pm$0.321\\% | 18.574\\%$\\pm$0.584\\% | 13.007\\%$\\pm$0.729\\% |\n|DeepONet | 35.334\\%$\\pm$0.179\\% | 25.455\\%$\\pm$0.245\\% | 11.998\\%$\\pm$0.786\\%|\n|F-FNO | 35.672\\%$\\pm$3.852\\% | 12.135\\%$\\pm$5.813\\% | 3.193\\%$\\pm$1.622\\%|\n|UNet | 98.167\\%$\\pm$0.236\\% | 34.467\\%$\\pm$2.858\\% | 5.462\\%$\\pm$0.048\\%|\n|FNO w/ smooth $\\chi$ | 17.431\\%$\\pm$0.536\\% | 5.479\\%$\\pm$0.186\\% | 1.415\\%$\\pm$0.025\\% |\n|IFNO w/ smooth $\\chi$ | 17.145\\%$\\pm$0.432\\% | 5.088\\%$\\pm$0.146\\% | 1.509\\%$\\pm$0.018\\% |\n\nTest errors for the airfoil problem.\n\n| Model | Train error | Test error |\n| :------------- | :-----------: | :-----------: |\n|eDAFNO | 0.329\\%$\\pm$0.020\\% | **0.596\\%**$\\pm$**0.005\\%** |\n|iDAFNO | 0.448\\%$\\pm$0.012\\% | 0.642\\%$\\pm$0.020\\% |\n|eDAFNO on irregular grids | 0.331\\%$\\pm$0.003\\% | 0.659\\%$\\pm$0.007\\% |\n|Geo-FNO | 1.565\\%$\\pm$0.180\\% | 1.650\\%$\\pm$0.175\\% |\n|F-FNO | 0.566\\%$\\pm$0.066\\% | 0.794\\%$\\pm$0.025\\% |\n|FNO w/ mask | 2.676\\%$\\pm$0.054\\% | 3.725\\%$\\pm$0.108\\% |\n|UNet w/ mask | 2.781\\%$\\pm$1.084\\% | 4.957\\%$\\pm$0.059\\% |"}]

Author rebuttal:
We thank the reviewers for the constructive comments, for recognizing the importance/usefulness of our work (reviewers FUid, UPhM, dnax), the novelty and elegance of DAFNO's architecture (reviewers FUid, fNa3, dnax, BzBv), DAFNO's role as the first neural operator that can handle dynamically changing topologies (reviewer FUid), effective architecture and strong experimental results (reviewers fNa3, dnax), computational advantage with far fewer parameters (reviewer BzBv), and clarity in writing (reviewers fNa3, dnax).

**Being simple yet flexible**: We want to comment on the simplicity of our DAFNO model: as also pointed out by Reviewer fNa3, being simple does not need to be seen as a negative point. Sometimes simpler is better! It is the performance that matters the most. The simplicity of DAFNO's architecture adds benefits in terms of facilitating implementation. 

**Additional baselines and experiments**: In order to further enrich the baselines to compare to, we have added Factorized FNO (F-FNO) as another baseline in both the elasticity and airfoil problems. Our original conclusion still stands: DAFNOs consistently outperform all selected baselines. We also show the applicability of DAFNO on irregular/adaptive grids using the airfoil dataset, where the non-uniform grids are generated using sinusoidal functions and are highly adaptive in the vicinity of the airfoil. DAFNO achieves very similar accuracy compared to uniform meshes, while the error gets reduced near the airfoil.

**Release code**: We have uploaded our DAFNO code of the hyperelasticity problem on anonymous github and sent the link to the Area Chair. Our DAFNO package on other problems will also be made publicly available on github once the paper is accepted.

Test errors for the hyperelasticity problem, where bold numbers highlight the best method.

| Model| 10 samples | 100 samples | 1000 samples | 
| :------------- | :-----------: | :-----------: | :-----------: | 
|eDAFNO | **16.446\%**$\pm$**0.472\%** | 4.247\%$\pm$0.066\% | **1.094\%**$\pm$**0.012\%**|
|iDAFNO | 16.669\%$\pm$0.523\% | **4.214\%**$\pm$**0.058\%** | 1.207\%$\pm$0.006\%|
|FNO w/ mask | 19.487\%$\pm$0.633\% | 7.852\%$\pm$0.130\% | 4.550\%$\pm$0.062\% |
|IFNO w/ mask | 19.262\%$\pm$0.376\% | 7.700\%$\pm$0.062\% | 4.481\%$\pm$0.022\% |
|Geo-FNO | 28.725\%$\pm$2.600\% | 10.343\%$\pm$4.446\% | 2.316\%$\pm$0.283\% |
|GNO | 29.305\%$\pm$0.321\% | 18.574\%$\pm$0.584\% | 13.007\%$\pm$0.729\% |
|DeepONet | 35.334\%$\pm$0.179\% | 25.455\%$\pm$0.245\% | 11.998\%$\pm$0.786\%|
|F-FNO | 35.672\%$\pm$3.852\% | 12.135\%$\pm$5.813\% | 3.193\%$\pm$1.622\%|
|UNet | 98.167\%$\pm$0.236\% | 34.467\%$\pm$2.858\% | 5.462\%$\pm$0.048\%|
|FNO w/ smooth $\chi$ | 17.431\%$\pm$0.536\% | 5.479\%$\pm$0.186\% | 1.415\%$\pm$0.025\% |
|IFNO w/ smooth $\chi$ | 17.145\%$\pm$0.432\% | 5.088\%$\pm$0.146\% | 1.509\%$\pm$0.018\% |

Test errors for the airfoil problem.

| Model | Train error | Test error |
| :------------- | :-----------: | :-----------: |
|eDAFNO | 0.329\%$\pm$0.020\% | **0.596\%**$\pm$**0.005\%** |
|iDAFNO | 0.448\%$\pm$0.012\% | 0.642\%$\pm$0.020\% |
|eDAFNO on irregular grids | 0.331\%$\pm$0.003\% | 0.659\%$\pm$0.007\% |
|Geo-FNO | 1.565\%$\pm$0.180\% | 1.650\%$\pm$0.175\% |
|F-FNO | 0.566\%$\pm$0.066\% | 0.794\%$\pm$0.025\% |
|FNO w/ mask | 2.676\%$\pm$0.054\% | 3.725\%$\pm$0.108\% |
|UNet w/ mask | 2.781\%$\pm$1.084\% | 4.957\%$\pm$0.059\% |


The total number of parameters and per-epoch runtime (second) in the hyperelasticity problem.

|Model | eDAFNO | iDAFNO | FNO | IFNO | Geo-FNO | GNO | DeepONet | UNet | F-FNO |
| :------------- | :-----------: | :-----------: | :------------- | :-----------: | :-----------: | :------------- | :-----------: | :-----------: | :-----------: |
|nparams | 2.37M | 0.60M | 2.37M | 0.60M | 3.02M | 2.64M | 3.10M | 3.03M | 3.21M|
|runtime (s) | 2.00 | 1.70 | 1.81 | 1.62 | 5.12 | 98.37 | 940.12 | 5.04 | 3.41 |

Meta-review: The paper presents an extension of the Fourier Neural Operator (FNO) designed to handle irregular grids and changing domain topologies while retaining the advantages of using the FFT, as in FNO. To achieve this, the authors adapt the FNO kernel by introducing a smoothed characteristic function that encodes domain transformations. This enables the method to generalize to new geometries. The approach is evaluated in the contexts of material modeling, airfoil simulation, and a dataset of material fracture evolution.


The extension of FNO to accommodate irregular grids has inspired several recent extensions. Reviewers agree that the approach proposed here is novel, simple, and efficient. In response to the reviewers' suggestions, the authors have conducted additional experiments that further support their findings. All the reviewers believe that this represents a valuable enhancement for neural operator deployment and recommend acceptance.
Decision: Accept (poster)