Title: Domain Agnostic Fourier Neural Operators
Venue: NeurIPS 2023
Accepted: yes
Keywords: Operator-Regression Neural Networks

Neural Operators

Data-Driven Physics Modeling

Geometrical and Topological Shape Changes
Abstract: Fourier neural operators (FNOs) can learn highly nonlinear mappings between function spaces, and have recently become a popular tool for learning responses of complex physical systems. However, to achieve good accuracy and efficiency, FNOs rely on the Fast Fourier transform (FFT), which is restricted to modeling problems on rectangular domains. To lift such a restriction and permit FFT on irregular geometries as well as topology changes, we introduce domain agnostic Fourier neural operator (DAFNO), a novel neural operator architecture for learning surrogates with irregular geometries and evolving domains. The key idea is to incorporate a smoothed characteristic function in the integral layer architecture of FNOs, and leverage FFT to achieve rapid computations, in such a way that the geometric information is explicitly encoded in the architecture. In our empirical evaluation, DAFNO has achieved state-of-the-art accuracy as compared to baseline neural operator models on two benchmark datasets of material modeling and airfoil simulation. To further demonstrate the capability and generalizability of DAFNO in handling complex domains with topology changes, we consider a brittle material fracture evolution problem. With only one training crack simulation sample, DAFNO has achieved generalizability to unseen loading scenarios and substantially different crack patterns from the trained scenario. Our code and data accompanying this paper are available at https://github.com/ningliu-iga/DAFNO.

Domain Agnostic Fourier Neural Operators

Abstract

Fourier neural operators (FNOs) can learn highly nonlinear mappings between function spaces, and have recently become a popular tool for learning responses of complex physical systems. However, to achieve good accuracy and efficiency, FNOs rely on the Fast Fourier transform (FFT), which is restricted to modeling problems on rectangular domains. To lift such a restriction and permit FFT on irregular geometries as well as topology changes, we introduce domain agnostic Fourier neural operator (DAFNO), a novel neural operator architecture for learning surrogates with irregular geometries and evolving domains. The key idea is to incorporate a smoothed characteristic function in the integral layer architecture of FNOs, and leverage FFT to achieve rapid computations, in such a way that the geometric information is explicitly encoded in the architecture. In our empirical evaluation, DAFNO has achieved state-of-the-art accuracy as compared to baseline neural operator models on two benchmark datasets of material modeling and airfoil simulation. To further demonstrate the capability and generalizability of DAFNO in handling complex domains with topology changes, we consider a brittle material fracture evolution problem. With only one training crack simulation sample, DAFNOhas achieved generalizability to unseen loading scenarios and substantially different crack patterns from the trained scenario. Our code and data accompanying this paper are available at https://github.com/ningliu-iga/DAFNO .

1 Introduction

Deep learning surrogate models provide a useful data-driven paradigm to accelerate the PDE-solving and calibration process of scientific modeling and computing problems. Among others, a wide range of scientific computing applications entail the learning of solution operators, i.e., the learning of infinite dimensional function mappings between any parametric dependence to the solution field. A prototypical instance is the case of solving Navier-Stokes equations in fluid mechanics, where the initial input needs to be mapped to a temporal sequence of nonlinear parametric solutions. The demand for operator learning has sparked the development of neural operator based methods (Li et al., 2020a,b,c; You et al., 2022a,b,c; Goswami et al., 2022; Liu et al., 2022; Gupta et al., 2021; Lu et al., 2019; Cao, 2021; Hao et al., 2023; Li et al., 2022b; Yin et al., 2022c), with one of the most popular architectures being Fourier Neural Operators (FNOs) (Li et al., 2020c; You et al., 2022c).

The success of FNOs can be mostly attributed to its convolution-based integral kernel that learns in a resolution-invariant manner and the computationally efficient evaluation achieved via Fast Fourier Transform (FFT) (Brigham, 1988). While learning in the spectral domain is fast, the latter comes at a cost: the computational domain of the underlying problem needs to be rectangular with uniformly

meshed grids. This is often intractable as the domain of interest is, more often than not, irregular. An often taken trick for applying FNO to irregular domains is to embed the original domain into a larger rectangular domain and zero-pad or extrapolate on the redundant space (Lu et al., 2022). This poses two potential problems, one being the possible numerical errors and even instabilities due to the discontinuity at the original domain boundary (e.g., the Gibbs phenomenon (Gottlieb & Shu, 1997)) and the other, perhaps more importantly, being the fact that the padding/extrapolating techniques cannot handle domains with shallow gaps, as is the case in object contact and crack propagation problems. Meanwhile, another line of work emphasizes the learning of a diffeomorphic mapping between the original domain and a latent domain with uniform grids on which FNO can be applied (Li et al., 2022a). However, in this approach the changes on the boundary and the domain topology can only be informed via the learned diffeomorphism, which results in approximation errors when tested on a new domain geometry and possible failure when a change in topology is involved on the domain geometry.

In this work, we aim to design FNO architectures that explicitly embed the boundary information of irregular domains, which we coin Domain Agnostic Fourier Neural Operator (DAFNO). This is inspired by the recent work in convolution-based peridynamics (Jafarzadeh et al., 2022b) in which bounded domains of arbitrary shapes are explicitly encoded in the nonlocal integral formulation. We argue that, by explicitly embedding the domain boundary information into the model architecture, DAFNO is able to learn the underlying physics more accurately, and the learnt model is generalizable to changes on the domain geometry and topology. Concretely, we construct two practical DAFNO variants, namely, eDAFNO that inherits the explicit FNO architecture (Li et al., 2022a) and iDAFNO that is built upon the implicit FNO (IFNO) architecture characterizing layer-independent kernels (You et al., 2022c). Moreover, a boundary smoothening technique is also proposed to resolve the Gibbs phenomenon and retain the fidelity of the domain boundary. In summary, the primary contributions of the current work are as follows:

We propose DAFNO, a novel Fourier neural operator architecture that explicitly encodes the boundary information of irregular domains into the model architecture, so that the learned operator is aware of the domain boundary, and generalizable to different domains of complicated geometries and topologies.

By incorporating a (smoothened) domain characteristic function into the integral layer, our formulation resembles a nonlocal model, such that the layer update acts as collecting interactions between material points inside the domain and cuts the non-physical influence outside the domain. As such, the model preserves the fidelity of the domain boundary as well as the convolution form of the kernel that retains the computational efficiency of FFT.

We demonstrate the expressivity and generalizability of DAFNO across a wide range of scientific problems including constitutive modeling of hyperelastic materials, airfoil design, and crack propagation in brittle fracture, and show that our learned operator can handle not only irregular domains but also topology changes over the evolution of the solution.

2 Background and related work

The goal of this work is to construct a neural network architecture to learn common physical models on various domains. Formally, given D := { ( g i | Ω i , u i | Ω i ) } N i =1 , a labelled set of function pair observations both defined on the domain x ∈ Ω i ⊂ R s . We assume that the input { g i ( x ) } is a set of independent and identically distributed (i.i.d.) random fields from a known probability distribution µ on A ( R d g ) , a Banach space of functions taking values in R d g . u i ( x ) ∈ U ( R d u ) , possibly noisy, is the observed corresponding response taking values in R d u . Taking mechanical response modeling problem for example, Ω i is the shape of the object of interest, g i ( x ) may represent the boundary, initial, or loading conditions, and u i ( x ) can be the resulting velocity, pressure, or displacement field of the object. We assume that all observations can be modeled by a common and possibly unknown governing law, e.g., balance laws, and our goal is to construct a surrogate operator mapping, ˜ G , from A to U such that

Here, θ represents the (trainable) network parameter set.

In real-world applications, the domain Ω i can possess different topologies, e.g., in contact problems (Benson & Okazawa, 2004; Simo & Laursen, 1992) and material fragmentation problems (De Luycker et al., 2011; Agwai et al., 2011; Silling, 2003), and/or evolve with time, as is the case

in large-deformation problems (Shadden et al., 2010) and fluid-structure interaction applications (Kuhl et al., 2003; Kamensky et al., 2017). Hence, it is desired to develop an architecture with generalizability across various domains of complex shapes, so that the knowledge obtained from one geometry can be transferable to other geometries.

2.1 Learning solution operators of hidden physics

In real-world physical problems, predicting and monitoring complex system responses are ubiquitous in many applications. For these purposes, physics-based PDEs and tranditional numerical methods have been commonly employed. However, traditional numerical methods are solved for specific boundary, initial, and loading conditions g on a specific domain Ω . Hence, the solutions are not generalizable to other domains and operating conditions.

To provide a more efficient and flexible surrogate model for physical response prediction, there has been significant progress in the development of deep neural networks (NNs) and scientific machine learning models (Ghaboussi et al., 1998, 1991; Carleo et al., 2019; Karniadakis et al., 2021; Zhang et al., 2018; Cai et al., 2022; Pfau et al., 2020; He et al., 2021; Besnard et al., 2006). Among others, neural operators (Li et al., 2020a,b,c; You et al., 2022a; Ong et al., 2022; Gupta et al., 2021; Lu et al., 2019, 2021b; Goswami et al., 2022; Tripura & Chakraborty, 2022) show particular promise in learning physics of complex systems: compared with classical NNs, neural operators are resolution independent and generalizable to different input instances. Therefore, once the neural operator is trained, solving for a new instance of the boundary/initial/loading condition with a different discretization only requires a forward pass. These advantages make neural operators a useful tool to many physics modeling problems (Yin et al., 2022a; Goswami et al., 2022; Yin et al., 2022b; Li et al., 2020a,b,c; Lu et al., 2022, 2021a).

2.2 Neural operator learning

Here, we first introduce the basic architecture of the general integral neural operators (Li et al., 2020a,b,c; You et al., 2022a,c), which are comprised of three building blocks. First, the input function, g ( x ) ∈ A , is lifted to a higher-dimensional representation via h 0 ( x ) = P [ g ]( x ) := P [ x , g ( x )] T + p , where P ∈ R ( s + d g ) × d h and p ∈ R d h define an affine pointwise mapping. Then, the feature vector function h 0 ( x ) goes through an iterative layer block, where the layer update is defined via the sum of a local linear operator, a nonlocal integral kernel operator, and a bias function: h l +1 ( x ) = J l +1 [ h l ]( x ) . Here, h l ( x ) ∈ R d h , l = 0 , · · · , L , is a sequence of functions representing the values of the network at each hidden layer. J 1 , · · · , J L are the nonlinear operator layers defined by the particular choice of networks. Finally, the output u ( x ) ∈ U is obtained via a projection layer by mapping the last hidden layer representation h L ( x ) onto U as: u ( x ) = Q [ h L ]( x ) := Q 2 σ ( Q 1 h L ( x ) + q 1 ) + q 2 . Q 1 , Q 2 , q 1 and q 2 are the appropriately sized matrices and vectors that are part of the learnable parameter set, and σ is an activation function (e.g., ReLU (He et al., 2018) or GeLU).

Then, the system response can be learnt by constructing a surrogate operator of (1): ˜ G [ g ; θ ]( x ) := Q◦J 1 ◦ · · · ◦ J L ◦ P [ g ]( x ) ≈ u ( x ) , by solving the network parameter set θ via an optimization problem:

Here, C denotes a properly defined cost functional (e.g., the relative mean square error) on Ω i .

2.3 Fourier neural operators

The Fourier neural operator (FNO) is first proposed in Li et al. (2020c) with its iterative layer architecture given by a convolution operator:

where W l ∈ R d h × d h and c l ∈ R d h are learnable tensors at the l -th layer, and κ ∈ R d h × d h is a tensor kernel function with parameters v l . When a rectangular domain Ω with uniform meshes is considered, the above convolution operation can be converted to a multiplication operation through discrete Fourier transform:

n

T

where F and F -1 denote the Fourier transform and its inverse, respectively, which are computed using the FFT algorithm to each component of h separately. The FFT calculations greatly improve the computational efficiency due to their quasilinear time complexity, but they also restrict the vanilla FNO architecture to rectangular domains Ω (Lu et al., 2022).

To enhance the flexibility of FNO in modeling complex geometries, in Lu et al. (2022) the authors proposed to pad and/or extrapolate the input and output functions into a larger rectangular domain. However, such padding/extrapolating techniques are prone to numerical instabilities (Gottlieb & Shu, 1997), especially when the domain is concave and/or with complicated boundaries. As shown in Lu et al. (2022), the performance of dgFNO+ substantially deteriorates when handling complicated domains with notches and gaps. In Geo-FNO (Li et al., 2022a), an additional neural network is employed and trained from data, to continuously map irregular domains onto a latent space of rectangular domains. As a result, the vanilla FNO can be employed on the rectangular latent domain. However, this strategy relies on the continuous mapping from the physical domain to a rectangular domain, hence it is restricted to relatively simple geometries with no topology change.

3 Domain Agnostic Fourier Neural Operators

In this section, we introduce Domain Agnostic Fourier Neural Operator (DAFNO), which features the generalizability to new and unseen domains of arbitrary shapes and different topologies. The key idea is to explicitly encode the domain information in the design while retaining the convolutional architecture in the iterative layer of FNOs. In what follows, we present the eDAFNO architecture based on the standard/explicit FNO model (Li et al., 2020c), while the iDAFNO architecture based on the implicit FNO model (You et al., 2022c) is provided in Appendix A.

Concretely, we enclose the physical domain of interest, Ω , by a (slightly larger) periodic box T , as shown in Figure 1. Next, we define the following domain characteristic function:

which encodes the domain information of different geometries. Inspired by Jafarzadeh et al. (2022b), we incorporate the above-encoded domain information into the FNO architecture of (3), by multiplying the integrand in its convolution integral with χ ( x ) χ ( y ) :

Herein, we have followed the practice in You et al. (2022a) and reformulated (3) to a nonlocal Laplacian operator, which is found to improve training efficacy. By introducing the term χ ( x ) χ ( y ) , the integrand vanishes when either point x or y is positioned inside Ω and the other is positioned outside. This modification eliminates any undesired interaction between the regions inside and outside of Ω . As a result, it tailors the integral operator to act on Ω independently and is able to handle different domains and topologies. With this modification, the FFT remains applicable, since the convolutional structure of the integral is preserved and the domain of operation yet spans to the whole rectangular box T . In this context, (5) can be re-organized as:

Note that multiplying W l h ( x ) + c l with χ ( x ) does not alter the computational domain. Now that the integration region is a rectangular box T , the FFT algorithm and its inverse can be readily applied,

Input

function

Lifting

layer

Iterative layers

Projection

layer

Output

function

loop for L times

and hence we can further express the eDAFNO architecture as:

An illustration of the DAFNO atchitecture is provided in Figure 2. Note that this architectural modification is performed at the continuum level and therefore is independent of the discretization. Then, the box T can be discretized with structured grids (cf. Figure 1), as is the case in standard FNO.

Although the proposed DAFNO architecture in (6) can handle complex generalization in domains, it has a potential pitfall: since the characteristic function is not continuous on the domain boundary, its Fourier series cannot converge uniformly and the FFT result would present fictitious wiggling near the discontinuities (i.e., the Gibbs phenomenon (Day et al., 1965)). As a consequence, the introduction of χ can potentially jeopardize the computational accuracy. To improve the efficacy of DAFNO, we propose to replace the sharp characteristic function, χ , with a smoothed formulation:

Here, the hyperbolic tangent function tanh( z ) := exp( z ) -exp( -z ) exp( z )+exp( -z ) , dist ( x , ∂ Ω) denotes the (approximated) distance between x and the boundary of domain Ω , and β controls the level of smoothness, which is treated as a tunable hyperparameter. An illustration of the effect of the smoothed ˜ χ is displayed in Figure 3, with additional plots and prediction results with respect to different levels of smoothness, β , provided in Appendix B.1. In what follows, the ˜ · sign is neglected for brevity.

Remark: We point out that the proposed smoothed geometry encoding technique, although simple, is substantially different from existing function padding/extrapolation techniques proposed in Lu et al. (2022) who cannot handle singular boundaries (as in the airfoil tail of our example 2) nor notches/shallow gaps in the domain (as in the crack propagation of our example 3). Our proposed architecture in (5) is also more sophiscated than a trivial zero padding at each layer in that the characteristic function χ ( x ) is multiplied with the integrand, whereas the latter breaks the convolutional structure and hinders the application of FFTs.

4 Numerical examples

In this section, we demonstrate the accuracy and expressivity of DAFNO across a wide variety of scientific problems. We compare the performance of DAFNO against other relevant scientific machine learning models, including FNO (Li et al., 2020c), Geo-FNO (Li et al., 2022a), IFNO (You et al., 2022c), F-FNO (Tran et al., 2022), GNO (Li et al., 2020a), DeepONet (Lu et al., 2019), and UNet (Ronneberger et al., 2015). In particular, we carry out three experiments on irregular domains, namely, constitutive modeling of hyperelasticity in material science, airfoil design in fluid mechanics, and crack propagation with topology change in fracture mechanics. For fair comparison, the hyperparameters of each model are tuned to minimize the error on validation datasets, including initial learning rate, decay rate for every 100 epochs, smoothing parameter, and regularization parameter, while the total number of epochs is restricted to 500 for computational efficiency. The

Computational Domain 41x41

Boundary Encoding β = 10

Ground Truth 41x41

eDAFNO Prediction 41x41

eDAFNO Prediction 161x161

(a)

(b)

(c)

(d)

(e)

relative L2 error is reported as comparison metrics for both the training and test datasets. The experiments of each method are repeated on 5 trials with 5 different random seeds, and the mean and standard deviation of the errors are reported. Further details and additional results are provided in Appendix B.

4.1 Constitutive modeling of hyperelasticity

We start with a hyperelasticity problem in material science that models the fundamental principle of constitutive relations. The high-fidelity synthetic data in this benchmark is governed by

where ρ denotes the mass density, u and σ represent the corresponding displacement and stress fields, respectively. The computational domain is enclosed by a unit cell [0 , 1] 2 of which the center exists a randomly shaped void, as described by its radius r = 0 . 2 + 0 . 2 1+ exp (˜ r ) and ˜ r ∼ N (0 , 4 2 ( -∇ +3 2 ) -1 ) . The bottom edge is fixed and the top edge is subjected to a tensile traction of t = [0 , 100] . The underlying hyperelastic material is of the incompressible Rivlin-Saunders type. For training, we directly adopt the dataset in Li et al. (2022a), where a total of 1000, 200, 200 samples are selected for training, validation and testing, respectively. For this problem, the input is represented as point clouds and the target is the resulting stress field.

Ablation study We first carry out an ablation study by comparing (a) the proposed two DAFNO models with (b) the baseline FNO/IFNO with the sharp characteristic function as input (denoted as FNO/IFNO w/ mask), and (c) the original FNO/IFNO with our proposed smoothened boundary

characteristic function as input (denoted as FNO/IFNO w/ smooth χ ). In this study, scenarios (b) and (c) aim to investigate the effects of our proposed boundary smoothing technique, and by comparing scenarios (a) and (c) we verify the effectiveness of encoding the boundary information in eDAFNO architecture. In addition, both FNO- and IFNO-based models are tested, with the purpose to evaluate the model expressivity when using layer-independent parameters in iterative layers. Three training dataset sizes (i.e., 10, 100, and 1000) are employed to explore the effect of the proposed algorithms on small, medium, and large datasets, respectively. The number of trainable parameters are reported in Table 1. Following the common practice as in Li et al. (2022a), the hyperparameter choice of each model is selected by tuning the number of layers and the width (channel dimension) keeping the total number of parameters of the same magnitude.

The results of the ablation study are listed in Table 2. Firstly, by directly comparing the results of FNO (and IFNO) with mask and with smooth boundary encoding, one can tell that the boundary smoothing technique helps to reduce the error. This is supported by the observation that FNO and IFNO with smooth χ consistently outperform their counterparts with mask in all data regimes, especially when a sufficient amount of data becomes available where a huge boost in accuracy can be achieved (by over 300%). On the other hand, by encoding the geometry information into the iterative layer, the prediction accuracy is further improved, where eDAFNO and iDAFNO outperform FNO and IFNO with smoothed χ by 22.7% and 20.0% in large-data regime, respectively. Another interesting finding is that eDAFNO is 9.4% more accurate compared to iDAFNO in the large-data regime, although only a quarter of the total number of parameters is needed in iDAFNO due to its layer-independent parameter setting. This effect is less pronounced as we reduce the amount of available data for training, where the performance of iDAFNO is similar to that of eDAFNO in the small- and medium-data regimes. This is because iDAFNO has a much smaller number of trainable parameters and therefore is less likely to overfit with small datasets. Given that the performance of eDAFNO and iDAFNO is comparable, it is our opinion that both architectures are useful in different applications. In Figure 3, an example of the computational domain, the smoothened boundary encoding, the ground truth solution, and the eDAFNO prediction are demonstrated. To demonstrate the capability of prediction across resolutions, we train eDAFNO using data with 41 × 41 grids then apply the model to provide prediction on 161 × 161 grids-one can see that eDAFNO can generalize across different resolutions.

Comparison against additional baselines We further compare the performance of DAFNO against additional relevant baselines, including GNO, Geo-FNO, F-FNO, DeepONet, and UNet. Note that the results of GNO, DeepONet, and UNet are obtained using the same settings as in Li et al. (2022a). Overall, the two DAFNO variants are significantly superior to other baselines in accuracy, with eDAFNO outperforming GNO, DeepONet, UNet, Geo-FNO, and F-FNO by 1088.9%, 975.0%, 399.3%, 111.7%, and 191.9%, respectively, in large-data regime. DAFNO is also more memory efficient compared to Geo-FNO (the most accurate baseline), as it foregoes the need for additional coordinate deformation network. As shown in Table 1, when the layer-independent parameter setting in iDAFNO is taken into account, DAFNO surpasses Geo-FNO by 407.4% in memory saving.

4.2 Airfoil design

Computational Domain

Boundary Encoding (β = 5)

Ground Truth

eDAFNO Prediction

1.0

1.5

1.5

0.8

0.6

0.8

1.0

1.0

0.4

0.6

0.5

0.5

0.2

0.4

0.0

0.0

0.0

In this example, we investigate DAFNO's performance in learning transonic flow around various airfoil designs. Neglecting the effect of viscosity, the underlying physics can be described by the Euler equation:

with ρ , p and E being the fluid density, pressure and the total energy, respectively, and v denoting the corresponding velocity field. The applied boundary conditions are: ρ ∞ = 1 , Mach number M ∞ = 0 . 8 , and p ∞ = 1 on the far field, with no penetration enforced on the airfoil. The dataset used for training is directly taken from Li et al. (2022a), which consists of variations of the NACA-0012 airfoil and is divided into 1000, 100, 100 samples for training, validation and testing, respectively. For this problem, we aim to learn the resulting Mach number field based on a given mesh as input. An example of the computational domain, the smoothened boundary encoding, the ground truth, and the eDAFNO prediction is illustrated in Figure 4.

We report in Table 3 our experimental observations using eDAFNO and iDAFNO, along with the comparison against FNO, Geo-FNO, F-FNO and UNet, whose models are directly attained from Li et al. (2022a); Tran et al. (2022). We can see that the proposed eDAFNO achieves the lowest error on the test dataset, beating the best result of non-DAFNO baselines by 24.9% and Geo-FNO by 63.9%, respectively. Additionally, iDAFNO reaches a similar level of accuracy with only a quarter of the total number of parameters employed, which is consistent with the findings in the previous example. With the properly trained DAFNO models, efficient optimization and inverse design are made possible.

f

f

-1

eDAFNO

Physical space

Computational space

Prediction in physical space

eDAFNO prediction

Geo-FNO

eDAFNO

eDAFNO on irregular grids

a

b

0.099

0.088

0.077

0.066

0.055

0.044

0.033

0.022

0.011

0.000

Figure 5: eDAFNO applied to the airfoil dataset on irregular grids. (a): the highly irregular and adaptive grids in the physical space is firstly deformed (via either an analytical mapping f or a trainable neural network for grid deformation) to uniform grids in the computational space, on which eDAFNO is applied to learn the underlying physics. The learned prediction is then converted back to the physical space via the inverse mapping f -1 or with another trainable neural network. (b): an illustration of the absolute error distribution of predictions in Geo-FNO, eDAFNO trained on uniform grids, and eDAFNO trained on irregular grids.

Aside from the enhanced predictability, DAFNO can also be easily combined with the grid mapping technique in Geo-FNO to handle non-uniform grids. In particular, no modification on the model architecture is required, where one just needs to include an analytical or trainable mapping from

non-uniform/irregular mesh grids to uniform mesh grids. As a demonstration, we consider irregular grids of the airfoil dataset and use a pre-computed function to map irregular grids to regular grids. In the irregular grid set, we place more grid points near the airfoil to provide a better resolution near the important parts. The test error is provided in Table 3, where the test loss using the eDAFNO learned model is 0 . 659% ± 0 . 007% , which is similar to the DAFNO results on uniform grids. In Figure 5, we demonstrate the irregular mesh and the absolute error comparison across Geo-FNO, DAFNO on regular grids, and DAFNO on irregular grids. One can see that, while both DAFNOs substantially outperform Geo-FNO, the error contours from eDAFNO with an irregular mesh show a smaller miss-match region near the airfoil, illustrating the flexibility of DAFNO in meshing and its capability in resolving fine-grained features.

4.3 Crack propagation with topology change in domain

In this example, we aim to showcase DAFNO's capability in handling evolving domains by modeling crack propagation in brittle fracture. We emphasize that DAFNO represents the first neural operator that allows for learning with topology change. In the field of brittle fracture, a growing crack can be viewed as a change in topology, which corresponds to an evolving χ ( t ) in the DAFNO architecture. In particular, we define the following time-dependent characteristic function:

where Ω( t ) denotes the time-dependent domain/evolving topology. Employing time-dependent χ in (6) keeps the neural operator informed about the evolving topology. In general, the topology evolution rule that determines Ω( t ) can be obtained from a separate neural network or from physics as is the case in the current example. The high-fidelity synthetic data in this example is generated using 2D PeriFast software (Jafarzadeh et al., 2022a), which employs a peridynamics (PD) theory for modeling fracture (Bobaru et al., 2016). A demonstration of the evolving topology, as well as further details regarding the governing equations and data generation strategies, is provided in Appendix B.3.

In this context, we select eDAFNO as the surrogate model to learn the the internal force density operator. Specifically, let u 1 ( x , t ) , u 2 ( x , t ) denote the two components of the displacement field u at time t , and L 1 , L 2 be the two components of the internal force density L [ u ] . Given u 1 , u 2 and χ as input data, we train two separate eDAFNO models to predict L 1 and L 2 , respectively. Then, we substitute the trained surrogate models in the dynamic governing equation and adopt Velocity-Verlet time integration to update u for the next time step, whereafter Ω and χ are updated accordingly.

The problem of interest is defined on a 40 mm × 40 mmthin plate with a pre-crack of length 10 mm at one edge, which is subjected to sudden, uniform, and constant tractions on the top and bottom edges. Depending on the traction magnitude, crack grows at different speeds, may or may not bifurcate, and the final crack patterns can be of various shapes. Our training data is comprised of two parts, one consisting of 450 snapshots from a crack propagation simulation with a fixed traction magnitude of σ = 4 MPa, and the other consisting of randomized sinusoidal displacement fields and the corresponding L 1 , L 2 fields computed by the PD operator. The sinusoidal subset contains 4,096 instances without fracture and is used to diversify the training data and mitigate overfitting on the crack data. For testing, we use the trained models in two scenarios with traction magnitudes different from what is used in training, which allows us to evaluate the generalizability of eDAFNO. Note that these tests are highly challenging for the trained models, as the predictions of the previous time step are used as the models' input in the current step, which leads to error accumulation as the simulation marches forward.

Figure 6 displays the test results on the crack patterns under different traction magnitudes, where the low traction magnitude (left column) shows a slow straight crack, the mid-level traction results in a crack with a moderate speed and a bifurcation event (middle column), and the highest traction leads to a rapid crack growth and initiates a second crack from the other side of the plate (right column). The trained eDAFNO model is able to accurately predict the left and right examples while only seeing the middle one, indicating that it has correctly learned the true constitutive behavior of the material and generalized well to previously unseen loading scenarios and the correspondingly changed domain topology. To further verify eDAFNO's generalizability to unseen geometries, in Figure 7 we compare eDAFNO with the baseline FNO w/ smoothed mask model, and plot their relative errors in χ and u . As observed, the FNO predictions become unstable at a fairly early time in the two test scenarios,

eDAFNO Prediction

Ground Truth

σ = 2 MPa

σ = 4 MPa

σ = 6 MPa

1

0

1

2

3

4

Time (

s)

0

0.2

0.4

0.6

0.8

1

1.2

Relative error in geometry (mask)

eDAFNO,  =4MPa

eDAFNO,  =2MPa

eDAFNO,  =6MPa

FNO with mask,  =4MPa

FNO with mask,  =2MPa

FNO with mask,  =6MPa

0

1

2

3

4

Time (

s)

10

-1

10

0

10

Relative error in displacement

eDAFNO,  =4MPa

eDAFNO,  =2MPa

eDAFNO,  =6MPa

FNO with mask,  =4MPa

FNO with mask,  =2MPa

FNO with mask,  =6MPa

since it creates a mask which is out of training distribution, while DAFNO can handle different crack patterns by hard coding it in the architecture, so the results remain stable for a much longer time.

5 Conclusion

We introduce two DAFNO variants to enable FNOs on irregular domains for PDE solution operator learning. By incorporating the geometric information from a smoothed characteristic function in the iterative Fourier layer while retaining the convolutional form, DAFNO possesses the computational efficiency from FFT together with the flexibility to operate on different computational domains. As a result, DAFNO is not only highly efficient and flexible in solving problems involving arbitrary shapes, but it also manifests its generalizability on evolving domains with topology change. In two benchmark datasets and a real-world crack propagation dataset, we demonstrate the state-of-the-art performance of DAFNO. We find both architectures helpful in practice: eDAFNO is slightly more accurate while iDAFNO is more computationally efficient and less overfitting with limited data.

Limitation: Due to the requirement of the standard FFT package, in the current DAFNO we focus on changing domains with uniformly meshed grids. However, we point out that this limitation can be lifted by using nonuniform FFT (Greengard & Lee, 2004) or an additional mapping for grid deformation, as shown in the airfoil experiment. Additionally, in our applications, we have focused on applying the same types of boundary conditions to the changing domains (e.g., all Dirichlet or all Neumann). In this context, another limitation and possible future direction would be on the transferability to PDEs with different types of boundary conditions.

Acknowledgments and Disclosure of Funding

S. Jafarzadeh would like to acknowledge support by the AFOSR grant FA9550-22-1-0197, and Y. Yu would like to acknowledge support by the National Science Foundation under award DMS-1753031 and the AFOSR grant FA9550-22-1-0197. Portions of this research were conducted on Lehigh University's Research Computing infrastructure partially supported by NSF Award 2019035.

References

Agwai, A., Guven, I., and Madenci, E. Predicting crack propagation with peridynamics: a comparative study. International journal of fracture , 171(1):65, 2011.

Benson, D. J. and Okazawa, S. Contact in a multi-material Eulerian finite element formulation. Computer Methods in Applied Mechanics and Engineering , 193(39):4277-4298, 2004. ISSN 0045-7825.

Besnard, G., Hild, F., and Roux, S. 'finite-element' displacement fields analysis from digital images: application to portevin-le châtelier bands. Experimental mechanics , 46(6):789-803, 2006.

Bobaru, F., Foster, J. T., Geubelle, P. H., and Silling, S. A. Handbook of peridynamic modeling . CRC press, 2016.

Brigham, E. O. The fast Fourier transform and its applications . Prentice-Hall, Inc., 1988.

Cai, S., Mao, Z., Wang, Z., Yin, M., and Karniadakis, G. E. Physics-informed neural networks (PINNs) for fluid mechanics: A review. Acta Mechanica Sinica , pp. 1-12, 2022.

Cao, S. Choose a transformer: Fourier or galerkin. Advances in neural information processing systems , 34:24924-24940, 2021.

Carleo, G., Cirac, I., Cranmer, K., Daudet, L., Schuld, M., Tishby, N., Vogt-Maranto, L., and Zdeborová, L. Machine learning and the physical sciences. Reviews of Modern Physics , 91(4): 045002, 2019.

Day, S. J., Mullineux, N., and Reed, J. Developments in obtaining transient response using fourier transforms: Part i: Gibbs phenomena and fourier integrals. International Journal of Electrical Engineering Education , 3(4):501-506, 1965.

De Luycker, E., Benson, D. J., Belytschko, T., Bazilevs, Y., and Hsu, M.-C. X-FEM in isogeometric analysis for linear fracture mechanics. International Journal for Numerical Methods in Engineering , 87:541-565, 2011.

Ghaboussi, J., Garrett Jr, J., and Wu, X. Knowledge-based modeling of material behavior with neural networks. Journal of engineering mechanics , 117(1):132-153, 1991.

Ghaboussi, J., Pecknold, D. A., Zhang, M., and Haj-Ali, R. M. Autoprogressive training of neural network constitutive models. International Journal for Numerical Methods in Engineering , 42(1): 105-126, 1998.

Goswami, S., Bora, A., Yu, Y., and Karniadakis, G. E. Physics-informed neural operators. 2022 arXiv preprint arXiv:2207.05748 , 2022.

Gottlieb, D. and Shu, C.-W. On the gibbs phenomenon and its resolution. SIAM review , 39(4): 644-668, 1997.

Greengard, L. and Lee, J.-Y. Accelerating the nonuniform fast fourier transform. SIAM review , 46(3): 443-454, 2004.

Gupta, G., Xiao, X., and Bogdan, P. Multiwavelet-based operator learning for differential equations. Advances in Neural Information Processing Systems , 34:24048-24062, 2021.

Ha, Y. D. and Bobaru, F. Studies of dynamic crack propagation and crack branching with peridynamics. International Journal of Fracture , 162(1):229-244, 2010.

Hao, Z., Wang, Z., Su, H., Ying, C., Dong, Y., Liu, S., Cheng, Z., Song, J., and Zhu, J. Gnot: A general neural operator transformer for operator learning. In International Conference on Machine Learning , pp. 12556-12569. PMLR, 2023.

He, J., Li, L., Xu, J., and Zheng, C. ReLu deep neural networks and linear finite elements. arXiv:1807.03973, 2018.

He, Q., Laurence, D. W., Lee, C.-H., and Chen, J.-S. Manifold learning based data-driven modeling for soft biological tissues. Journal of Biomechanics , 117:110124, 2021.

Jafarzadeh, S., Mousavi, F., and Bobaru, F. Perifast/dynamics: a matlab code for explicit fast convolution-based peridynamic analysis of deformation and fracture. 2022a.

Jafarzadeh, S., Mousavi, F., Larios, A., and Bobaru, F. A general and fast convolution-based method for peridynamics: Applications to elasticity and brittle fracture. Computer Methods in Applied Mechanics and Engineering , 392:114666, 2022b.

Kamensky, D., Hsu, M.-C., Yu, Y., Evans, J. A., Sacks, M. S., and Hughes, T. J. Immersogeometric cardiovascular fluid-structure interaction analysis with divergence-conforming b-splines. Computer methods in applied mechanics and engineering , 314:408-472, 2017.

Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., and Yang, L. Physics-informed machine learning. Nature Reviews Physics , 3(6):422-440, 2021.

Kuhl, E., Hulshoff, S., and de Borst, R. An arbitrary Lagrangian Eulerian finite element approach for fluid-structure interaction phenomena. International Journal for Numerical Methods in Engineering , 57:117-142, 2003.

Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., and Anandkumar, A. Neural operator: Graph kernel network for partial differential equations. arXiv preprint arXiv:2003.03485 , 2020a.

Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Stuart, A., Bhattacharya, K., and Anandkumar, A. Multipole graph neural operator for parametric partial differential equations. Advances in Neural Information Processing Systems , 33:NeurIPS 2020, 2020b.

Li, Z., Kovachki, N. B., Azizzadenesheli, K., Bhattacharya, K., Stuart, A., and Anandkumar, A. Fourier Neural Operator for Parametric Partial Differential Equations. In International Conference on Learning Representations , 2020c.

Li, Z., Huang, D. Z., Liu, B., and Anandkumar, A. Fourier neural operator with learned deformations for PDEs on general geometries. arXiv preprint arXiv:2207.05209 , 2022a.

Li, Z., Meidani, K., and Farimani, A. B. Transformer for partial differential equations' operator learning. arXiv preprint arXiv:2205.13671 , 2022b.

Liu, N., Yu, Y., You, H., and Tatikola, N. INO: Invariant neural operators for learning complex physical systems with momentum conservation. arXiv preprint arXiv:2212.14365 , 2022.

Lu, L., Jin, P., and Karniadakis, G. E. Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators. arXiv preprint arXiv:1910.03193 , 2019.

Lu, L., He, H., Kasimbeg, P., Ranade, R., and Pathak, J. One-shot learning for solution operators of partial differential equations. arXiv preprint arXiv:2104.05512 , 2021a.

Lu, L., Jin, P., Pang, G., Zhang, Z., and Karniadakis, G. E. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence , 3(3): 218-229, 2021b.

Lu, L., Meng, X., Cai, S., Mao, Z., Goswami, S., Zhang, Z., and Karniadakis, G. E. A comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data. Computer Methods in Applied Mechanics and Engineering , 393:114778, 2022.

Ong, Y. Z., Shen, Z., and Yang, H. IAE-NET: Integral autoencoders for discretization-invariant learning. 03 2022. doi: 10.13140/RG.2.2.25120.87047/2.

Pfau, D., Spencer, J. S., Matthews, A. G., and Foulkes, W. M. C. Ab initio solution of the manyelectron schrödinger equation with deep neural networks. Physical Review Research , 2(3):033429, 2020.

Ronneberger, O., Fischer, P., and Brox, T. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention , pp. 234-241. Springer, 2015.

Shadden, S. C., Astorino, M., and Gerbeau, J.-F. Computational analysis of an aortic valve jet with Lagrangian coherent structures. Chaos , 20:017512, 2010.

Silling, S. A. Dynamic fracture modeling with a meshfree peridynamic code. Computational Fluid and Solid Mechanics , 2003:641-644, 2003.

Simo, J. C. and Laursen, T. A. An augmented Lagrangian treatment of contact problems involving friction. Computers & Structures , 42(1):97-116, 1992. ISSN 0045-7949.

Tran, A., Mathews, A., Xie, L., and Ong, C. S. Factorized fourier neural operators. In The Eleventh International Conference on Learning Representations , 2022.

Tripura, T. and Chakraborty, S. Wavelet neural operator: a neural operator for parametric partial differential equations. arXiv preprint arXiv:2205.02191 , 2022.

Yin, M., Ban, E., Rego, B. V., Zhang, E., Cavinato, C., Humphrey, J. D., and Em Karniadakis, G. Simulating progressive intramural damage leading to aortic dissection using DeepONet: an operator-regression neural network. Journal of the Royal Society Interface , 19(187):20210670, 2022a.

Yin, M., Zhang, E., Yu, Y., and Karniadakis, G. E. Interfacing finite elements with deep neural operators for fast multiscale modeling of mechanics problems. Computer Methods in Applied Mechanics and Engineering, in press , pp. 115027, 2022b.

Yin, Y., Kirchmeyer, M., Franceschi, J.-Y., Rakotomamonjy, A., et al. Continuous pde dynamics forecasting with implicit neural representations. In The Eleventh International Conference on Learning Representations , 2022c.

You, H., Yu, Y., D'Elia, M., Gao, T., and Silling, S. Nonlocal kernel network (NKN): A stable and resolution-independent deep neural network. Journal of Computational Physics , pp. arXiv preprint arXiv:2201.02217, 2022a.

You, H., Zhang, Q., Ross, C. J., Lee, C.-H., Hsu, M.-C., and Yu, Y. A physics-guided neural operator learning approach to model biological tissues from digital image correlation measurements. Journal of Biomechanical Engineering, accepted for publication , pp. arXiv preprint arXiv:2204.00205, 2022b.

You, H., Zhang, Q., Ross, C. J., Lee, C.-H., and Yu, Y . Learning deep implicit fourier neural operators (IFNOs) with applications to heterogeneous material modeling. Computer Methods in Applied Mechanics and Engineering , 398:115296, 2022c. doi: https://doi.org/10.1016/j.cma.2022.115296.

Zhang, L., Han, J., Wang, H., Car, R., and Weinan, E. Deep potential molecular dynamics: a scalable model with the accuracy of quantum mechanics. Physical Review Letters , 120(14):143001, 2018.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel method of extending Fourier Neural Operators (FNO) to irregular geometries using an indicator function to represent the shape of the geometric area. This area is then extended to a larger, regular area, thereby allowing FNO to handle irregular geometries. This is indeed an innovative approach, as the use of indicator functions in neural operators to handle irregular areas has not been done before. The methodology bears some similarities to the use of neural networks in  PINNs to learn hard constraints, ensuring regular equations comply with constraints on the geometric boundary. Experimental results indicate that the proposed method achieves commendable performance with fewer parameters.\n\nSTRENGTHS:\nThe paper's idea is innovative, using indicator functions in neural operators to handle irregular geometries is a novel approach. The authors show through experimental results that the proposed method performs well with fewer parameters, a significant advantage.\n\nWEAKNESSES:\n1. The review of related work seems insufficient. The recent development of methods represented by transformers [1,2,3] that handle irregular geometric areas well is not mentioned or cited at all. Moreover, last year's ICLR Factorized FNO [4] also dealt with several non-uniform geometric area problems.\n2. Regarding the technical aspect of the paper, I have a concern. The original irregular grid may be adaptive, implying that we may need a very high resolution when embedding it into a uniform grid. This might mean that the uniform grid's spacing needs to be smaller than the smallest spacing of the original grid, leading to considerable computation waste. The proposed method does not seem to avoid this waste.\n3. I suggest that for datasets like airfoil and hyperelasticity, which have been detailed in other works, there is no need for further extensive description in the main text. From a machine learning researcher's perspective, the dataset's attributes, sizes, and challenges are more important than detailed principles of its generation, especially when these have already been thoroughly discussed in other studies.\n\nConsidering the above points, the authors need first add a few key references, carefully consider the technical questions raised, and then heavily revise the paper. If the core technical challenges (question 2 ) can be addressed, I may consider increasing the score after the rebuttal. Otherwise, I would suggest that the authors continue to revise this work for submission to another conference or journal.\n\n1. Choose a Transformer: Fourier or Galerkin (https://arxiv.org/abs/2105.14995)\n2. Transformer for Partial Differential Equations' Operator Learning (https://arxiv.org/abs/2205.13671)\n3. GNOT: A General Neural Operator Transformer for Operator Learning (https://arxiv.org/abs/2302.14376)\n4. Factorized Fourier Neural Operators (https://arxiv.org/abs/2111.13802)\n\nQUESTIONS:\nNone.\n\nLIMITATIONS:\nNone.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-15"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper explores the extension of Fourier Neural Operators (FNOs) to irregular geometries and topology changes. To leverage the computational speed benefits of the fast Fourier transform (FFT) employed by FNOs, they enclose the physical domain with a period box. They then adapt the FNO kernel by multiplying the integrand with a smoothed characteristic function that encodes domain information. This encoding enables their method to generalize to new geometries while preserving the FFT computational efficiency. They outperform several FNO baselines on two benchmark datasets, and showcase the properties of their method on a real-world crack propagation dataset, which contains evolving topology.\n\nSTRENGTHS:\nThe paper is well written and clear. The different choices of architecture are well justified. \n- It tackles an interesting and useful problem for real-world application of FNO.\n- The idea of incorporating the domain information in the kernel is simple, but effective and grounded, and the smoothing of the domain characteristic function helps avoiding dicontinituty problems.\n- Experimental results on three datasets are strong.\n- The crack propagation dataset is well-considered and helps highlighting the advantages of DAFNO, especially its ability to adapt to evolving topologies.\n\nWEAKNESSES:\n- The comparison with several baselines is interesting, but given the recent improvements seen over FNO and Geo-FNO, having F-FNO [1] as a baseline seems important. In addition to neural operators, other grid-independent methods that can be used on irregular domains have been developed, for instance [2] implemented a continuous model using Implicit Neural Representations (INRs). It would have been interesting to see a comparison or at least a discussion about the advantage of DAFNO in the light of these recent improvements in the community (for e.g. computational cost?).\n\n- As explained in the limitations and clear from the methodology itself, the architecture struggles to handle non-uniformly meshed grids due to the inability to use FFT on such grids. This limitation significantly restricts the range of problems that can be efficiently tackled using this architecture. For example, if applied to real-world airfoil surrogate modeling problems like [3], it seems that DAFNO would not offer significant improvements over the original FNO.\n\n- While the idea behind DAFNO is interesting and novel, the architectural improvement upon the regular FNO architecture seems limited. The authors address this issue in a ‘remark’ paragraph, but I am not totally convinced that it is a sufficient improvement, especially since the architecture does not appear to efficiently handle non-uniformly meshed grids.\n\n- This is more a remark than a weakness. In Figure 3, the zero-shot super-resolution prediction from eDAFNO does not effectively highlights the resolution invariant properties, since there is no ground truth to compare it to. While some degree of \"deblurring\" is noticeable, it remains unclear if the network successfully captures higher frequency phenomena that may occur at higher resolutions. The appendix, specifically Figure 10, sheds light on this property for the crack propagation dataset. However, Figure 3 does not truly demonstrates this resolution invariant property.\n\n[1]: Tran, Alasdair, et al. \"Factorized Fourier Neural Operators.\" The Eleventh International Conference on Learning Representations. 2022.\n\n[2]: Yin, Yuan, et al. \"Continuous PDE Dynamics Forecasting with Implicit Neural Representations.\" The Eleventh International Conference on Learning Representations. 2023.\n\n[3]: Bonnet, Florent, et al. \"AirfRANS: High Fidelity Computational Fluid Dynamics Dataset for Approximating Reynolds-Averaged Navier–Stokes Solutions.\" Advances in Neural Information Processing Systems 35 (2022): 23463-23478.\n\nQUESTIONS:\n1. I wonder why Geo-FNO is not in the baselines for the Crack propagation dataset. It would have been an interesting baseline, especially to see if it could adapt to evolving geometries better than a masked FNO. It is claimed that Geo-FNO cannot handle topology change, but once a continuous mapping is learned, it can be applied. I agree that it should not adapt well, but is there a specific reason for its absence in these experimental results? \n\n2. Just to confirm my understanding, for the FNO baseline, the grid is interpolated as in Geo-FNO, and then a mask is given as an additional input?\n\n3. I don’t quite understand the link from equation 5 to the equation before 6. Why is $\\chi(x)$ in factor of $W^l h(x) + c^l$ ? Is it to ensure that DAFNO will not produce results for points outside of the domain? \n\n4. Given the link of the smoothing function to a signed distance function (SDF) with the $\\\\tanh(\\\\beta \\\\text{dist}(x, \\partial \\Omega))$ formulation, is there a reason for not directly using a SDF instead of the smoothing function ? It may have given more (or a different) information to the network, as in different previous works (for e.g. [4]). \n\n5. Out of curiosity, have you experimented with using other smoothing functions besides tanh (for e.g. a logistic function)?\n\n[4]: Guo, Xiaoxiao, Wei Li, and Francesco Iorio. \"Convolutional neural networks for steady flow approximation.\" Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016.\n\nLIMITATIONS:\nThe main limitations are already presented in the article.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe Fourier Neural Operator (FNO) is a model in the field of neural operators that has successfully interpreted various physical phenomena. However, one of the issues with FNO is its limitation to learn only on rectangular domains. In Geo-FNO, this problem is addressed by lifting irregular domains to the latent space of rectangular domains.\n\nThe authors propose a novel convolution integral kernel through the domain characteristic function X(x) and introduce DAFNO (Domazin-Adapted Fourier Neural Operator). DAFNO outperforms the existing baseline in problems involving airfoil and hyperelasticity.\n\nSTRENGTHS:\nConducting research on a domain-agnostic physics simulator is a crucial endeavor. In particular, the field of neural operators, which the authors have explored, offers a highly efficient approach zto interpreting recent physical information, thus holding significant potential.\n\nWEAKNESSES:\nCould you provide more detailed explanation about the mathematical motivation behind the domain characteristic function? It would be necessary to provide specific explanations for Eq(4) and Eq(5).\n\nDue to the varying model sizes between the baseline models and the proposed model in Table 1, it is difficult to claim that the comparison is fair. A more equitable comparison can be achieved when all model sizes are standardized to be the same.\n\nThe authors proposed a new neural operator model called DAFNO. However, besides modifying the internal structure of the iterative layers in Geo-FNO, specifically by splitting the structure of the iterative layers in Figure 2, it is challenging to identify any other distinct novelties.\n\nAdding more baselines in the field of neural operators could lead to better experiments and improved evaluations. Such as, MWT (Multiwavelet-based neural operator), WNO (Wavelet neural operator)\n\nQUESTIONS:\nIs there no experimental result for the benchmark dataset (Pipe) based on the Navier-Stokes equation in the Geo-FNO paper? I regard it as one of the significant experiments in this domain.\n\nIntuitively, it might be expected that DAFNO, having designed the Iterative layers in parallel, could have larger training and inference times compared to the baseline models like FNO and Geo-FNO. Can you provide any measurement results to demonstrate this?\n\nLIMITATIONS:\nThere is no limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors propose the Domain Agnostic Fourier Neural Operator (DAFNO), an FNO that can deal with irregular boundaries. While the classical FNO is limited by construction to irregular domains, DAFNO simply includes a smoothed function $I(\\cdot)$  of the characteristic function of the domain on which the data is defined. DAFNO is shown to outperform relevant baselines on 2-dimensional problems with irregular boundaries and one problem with topology changing over time.\n\nSTRENGTHS:\nThe paper is about a very simple yet effective idea: including a smoothed mask to model boundaries inside the integral operator. This approach is original to my knowledge and effective in terms of strong experimental evaluation relevant to practical applications, including being able to handle topology changes. The paper is clear in its description and well-placed in the current literature on neural operators and deep surrogate models for scientific ML.\n\nWEAKNESSES:\nIn my opinion, there is no major weakness in this paper. The main limitation could be that the approach is (particularly) simple, but I believe this does not need to be seen as a negative point. Sometimes, simpler is better. The experimental results are solid but do not encompass more complex settings such as 3D fluid dynamics. Another weakness is the lack of source code, which I believe should be provided, given the simplicity of the setting.\n\nQUESTIONS:\n- Can the proposed method be extended to modeling systems ≥ than 3 dimensions?\n- At line 262: “In general, the topology evolution rule that determines Ω(t) can be obtained from a separate neural network or from physics as is the case in the current example”, did you test out a separate neural network in the experiments as well?\n\nLIMITATIONS:\nThe main limitation (about uniformly meshed grids) is mentioned. Other limitations are covered in the “weaknesses” section.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nFourier Neural Operators (FNOs) are a popular method for modeling physical systems such as different types of PDEs. However, in order to use the FFT to make FNO efficient, the input needs to be a regular grid. The authors study the question of irregular grid inputs for FNO, as well as problems with changing topologies. They propose to add a smoothed characteristic function in the integral layer architecture. This directly encodes the topology into the architecture, and it allows the architecture to continue to use FFT while allowing irregular grid inputs. The authors apply this domain agnostic method to the regular FNO as well as implicit FNO. The authors run experiments on material modeling, airfoil simulation, and material fractures.\n\nSTRENGTHS:\n- This is the first neural operator that can handle learning with topology changes. This is a useful property to handle fracture problems, such as material fracture or earthquakes.\n- The technique is relatively simple: adding a characteristic function to the integral kernel and adding a smoothing term, but it is novel and elegant that the characteristic function encodes the topology directly in the architecture. This is how the authors’ technique can learn changing topologies, when, say Geo-FNO learns a fixed mapping and cannot learn changing topologies. \n- The authors experiment on three problems: hyperelasticity, airfoils, and crack propagation, and they also perform ablation studies to separate the characteristic function and the smoothness.\n\nWEAKNESSES:\n- My biggest concern is that the technique may not be able to handle highly irregular topologies or topologies with fine-grained features, for two separate reasons. As shown in Figure 1, the resulting grid from the authors’ technique is still uniform, but with a characteristic added, to tell what is inside or outside the topology. Since the grid is still uniform, this may not work well for a topology that is very irregular or has fine features. This is in contrast to Geo-FNO, which learns a mapping, allowing it to handle highly irregular topologies and also to focus more grid points on the “important” parts of the topology and fewer grid points on the coarse / less important parts.\nFurthermore, the smoothing function could magnify the above problem, making it even harder to model delicate parts of the topology. I think of this as a weakness because the authors say that handling irregular topologies is one of the main contributions of their work. \n- The technique is fairly simple: it boils down to adding a characteristic function with smoothing. But on the other hand, this technique is the first to handle dynamically changing topologies.\n- The authors did not release code or mention releasing code. If they authors released code (anonymously during submission) it would have a bigger impact.\n- Why not compare airfoil with the other baselines? Also, why do the 3rd and 4th panels of Figures 4 look like there is a vertical line discontinuity in the center of the image?\n- Why not compare the cracking to more baselines than just FNO, such as Geo-FNO?\n\nQUESTIONS:\nCan the authors comment on all of the weaknesses listed above?\n\nAlso, I wonder if Geo-FNO can be combined with DAFNO, to handle more complex topologies and also be able to handle changing topologies?\n\nDo you plan to release your code? It could even be released during the rebuttal with e.g. https://anonymous.4open.science/.\n\nI would raise my score if some or all of the points from the \"weaknesses\" section are addressed.\n\nLIMITATIONS:\nThe authors discuss limitations in Section 5: they only consider changing domains where the grid is uniform, and they only consider problems with the same boundary conditions as the domain changes. I agree that it would be good to add a mapping for grid deformation in future work.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-10"}