Title: MMGP: a Mesh Morphing Gaussian Process-based machine learning method for regression of physical problems under nonparametrized geometrical variability
Venue: NeurIPS 2023
Accepted: yes
Keywords: Gaussian process

mesh morphing

mesh parametrization

finite element interpolation

simulation

physics

predictive uncertainties

nonparametrized geometries
Abstract: When learning simulations for modeling physical phenomena in industrial designs, geometrical variabilities are of prime interest. While classical regression techniques prove effective for parameterized geometries, practical scenarios often involve the absence of shape parametrization during the inference stage, leaving us with only mesh discretizations as available data. Learning simulations from such mesh-based representations poses significant challenges, with recent advances relying heavily on deep graph neural networks to overcome the limitations of conventional machine learning approaches. Despite their promising results, graph neural networks exhibit certain drawbacks, including their dependency on extensive datasets and limitations in providing built-in predictive uncertainties or handling large meshes. In this work, we propose a machine learning method that do not rely on graph neural networks. Complex geometrical shapes and variations with fixed topology are dealt with using well-known mesh morphing onto a common support, combined with classical dimensionality reduction techniques and Gaussian processes. The proposed methodology can easily deal with large meshes without the need for explicit shape parameterization and provides crucial predictive uncertainties, which are essential for informed decision-making. In the considered numerical experiments, the proposed method is competitive with respect to existing graph neural networks, regarding training efficiency and accuracy of the predictions.

MMGP: a Mesh Morphing Gaussian Process-based machine learning method for regression of physical problems under non-parameterized geometrical variability

Abstract

When learning simulations for modeling physical phenomena in industrial designs, geometrical variabilities are of prime interest. While classical regression techniques prove effective for parameterized geometries, practical scenarios often involve the absence of shape parametrization during the inference stage, leaving us with only mesh discretizations as available data. Learning simulations from such mesh-based representations poses significant challenges, with recent advances relying heavily on deep graph neural networks to overcome the limitations of conventional machine learning approaches. Despite their promising results, graph neural networks exhibit certain drawbacks, including their dependency on extensive datasets and limitations in providing built-in predictive uncertainties or handling large meshes. In this work, we propose a machine learning method that do not rely on graph neural networks. Complex geometrical shapes and variations with fixed topology are dealt with using well-known mesh morphing onto a common support, combined with classical dimensionality reduction techniques and Gaussian processes. The proposed methodology can easily deal with large meshes without the need for explicit shape parameterization and provides crucial predictive uncertainties, which are essential for informed decision-making. In the considered numerical experiments, the proposed method is competitive with respect to existing graph neural networks, regarding training efficiency and accuracy of the predictions.

1 Introduction

Many problems in science and engineering require solving complex boundary value problems. Most of the time, we are interested in solving a partial differential equation (PDE) for multiple values of input parameters such as material properties, boundary conditions, initial conditions, or geometrical parameters. Traditional numerical methods such as the finite element method, finite volume method, and finite differences require fine discretization of time and space in order to be accurate. As a result, these methods are often computationally expensive, especially when the boundary value problem needs to be repeatedly solved for extensive exploration of the input parameters space. To overcome this issue, machine and deep learning have been leveraged for various tasks in computational physics, namely, solving and learning solutions to PDEs [37, 56, 65, 81], accelerating linear solvers [5, 34], reduced-order modeling [49], domain decomposition [41], closure modeling [52], and topology optimization [74], to name a few. As reported in the review papers [13, 17, 75], most of the recent advances have been relying on deep neural networks for their flexibility and expressiveness. In this work, we focus on learning simulations of physical phenomena, that are discretized on a non-

parameterized unstructured mesh. In this situation, traditional machine learning approaches cannot easily be leveraged as the inputs of the problem are given by graphs with different numbers of nodes and edges. In contrast, deep learning models such as graph neural networks (GNNs) [67] can easily overcome this limitation thanks to their ability to operate on meshes with different resolutions and topologies. While GNNs show promising results and their flexibility is highly appealing, they still suffer from a few shortcomings that prevent their deployement in engineering fields where decisions involve high stakes. Training GNNs usually requires large datasets and computational resources, and predicting their uncertainties is still an open and challenging problem of its own [32].

We propose a novel methodology, called Mesh Morphing Gaussian Process (MMGP), that relies on standard and well-known morphing strategies, dimensionality reduction techniques and finite element interpolation for learning solutions to PDEs with non-parameterized geometric variations. In contrast to deep learning methods, such as GNNs, the model can easily and efficiently be trained on CPU hardware and predictive uncertainties are readily available. Our method shares some limitations with any machine learning regressor for PDE systems: (i) within the predictive uncertainties, our method produces predictions with an accuracy lower than the reference simulations, (ii) unlike many methods used in reference simulators, like the finite element method, our method provides no guaranteed error bounds and (iii) our method requires a well sampled training dataset, which has a certain computational cost, so that the workflow becomes profitable only for many-query contexts where the inference is called a large number of times. Regarding (i), rough estimates may be sufficient in preproject phases, and accuracy can be recovered by using the prediction as an initialization in the reference simulator, or by allowing the designer to run the reference simulator on the identified configuration if the regressor is used in an optimization task.

We start by providing the background and assumptions of our method while mentioning some related works in Section 2. Then, the proposed methodology is detailed in Section 3. Three numerical experiments are presented in Section 4. Finally, a conclusion is given in Section 5.

2 Preliminaries and related works

Notations. Vectors and matrices are denoted with bold symbols. The entries i of a vector v and i, j of a matrix M are respectively denoted v i and M i,j . The i-th row of a matrix M is denoted by M i .

Background. Let U true : Ω → R d be a solution to a boundary value problem, where Ω ⊂ R d Ω denotes the physical domain of the geometry under consideration, and d Ω = 2 or 3 . The domain Ω is discretized into a conformal mesh M as M = ∪ N e e =1 Ω e . In traditional numerical approaches such as the finite element method [82], an approximation U of the solution U true is sought in the finite-dimensional space spanned by a family of trial functions, { φ I ( x ) } N I =1 , supported on the mesh M :

where N is the total number of nodes in the mesh M , U ∈ R d × N is the discretized solution (featuring d fields), and x ∈ R d Ω denotes the spatial coordinates. For simplicity of the presentation and without loss of generality, we consider the particular case of a Lagrange P 1 finite element basis, so that the solution is uniquely determined by its value at the nodes of M . In this setting, the basis { φ I } N I =1 spans the space { v ∈ C 0 ( M ) : v | Ω e ∈ P 1 , ∀ Ω e ∈ M} , and the discretized solution U is determined by solving the discretized weak formulation of the underyling boundary value problem. This problem also depends on some parameters µ ∈ R p , such as material properties and boundary conditions. It is assumed that there are scalar output quantities of interest w ∈ R q that depend on the discretized solution U , and possibly on M and µ . We restrict ourselves to stationary, time-independent, scalars and fields of interest, which still falls in the scope of many industrial problems of interest. The learning task that we consider herein consists in learning the mapping

For this purpose, it is assumed that we are given a training set of size n made of input pairs ( µ i , M i ) of parameters and meshes, and output pairs ( U i , w i ) of discretized fields and scalars. Each input mesh M i has a number of nodes denoted by N i , and corresponds to a finite element discretization of an input geometry Ω i . The associated discretized solution U i is a matrix of size ( d × N i ) . For

any i = 1 , . . . , n , the mesh M i can be represented as an undirected graph G i = ( V i , E i ) , where V i denotes the set of nodes and E i is the set of edges.

Assumptions and limitations. We assume that the observed input geometries, Ω 1 , . . . , Ω n , share a common topology. The parameterization that generates the input geometries is unknown, and we are left with the associated finite element meshes M 1 , . . . , M n . Being the discretization of physical domains involved in a boundary value problem, the input meshes inherit important features such as boundary conditions applied to subsets of nodes and elements. In finite element methods, error estimates strongly depend on the quality of the mesh [66]. Hence, in our context, it is assumed that the input meshes exhibit good quality in terms of elements aspect ratios and node densities, adapted to the regularity of the fields of interest. Our focus centers on the design optimization of industrial components with respect to specific physical phenomena. Consequently, we assume precise control over the geometry, free from any noise. Additionally, the employed geometrical transformations are constrained to avoid extreme distortions, as they are selected from sets of admissible designs that adhere to limitations on mass, volume, and mechanical resistance.

Related works. In recent years, there has been a substantial focus on advancing neural networks to emulate solutions to physical systems, either through the integration of domain-specific knowledge [44] or by devising efficient architectures for GNNs [63]. GNNs learn the mapping F by relying on the message passing framework introduced by Gilmer et al. [33] and extended by Battaglia et al. [11]. In the context of physical systems, only a few contributions address non-parameterized geometric variabilities. The early work of Baque et al. [10] explores the use of GNNs to emulate physics-based simulations in the presence of geometric variabilities by relying on geodesic convolutions [58, 61]. More recently, Pfaff et al. [63] develop the MeshGraphNets (MGNs) model, a GNN that updates nodes and edges features in order to learn time-dependent simulations. Most notably, the model can handle various physics, three-dimensional problems, and non-parameterized geometric variabilities. Fortunato et al. [28] introduce MultiScale MGNs that relies on two different mesh resolutions in order to overcome the computational cost of the message passing algorithm on dense meshes, and to increase the accuracy of MGNs. The efficiency of MGNs has been illustrated by Allen et al. [7] for inverse problems, and by Harsch et al. [39] for time-independent systems. There exist several variants of such GNNs for learning mesh-based solutions. A multi-scale GNN that learns from multiple mesh resolutions has been proposed in Lino et al. [53] and is illustrated on two dimensional PDEs with geometric variabilities. Lino et al. [54] also devise a multi-scale and rotation-equivariant GNN that extrapolates the time evolution of the fluid flow, and Cao et al. [19] propose a novel pooling strategy that prevents loss of connectivity and wrong connections in multi-level GNNs. Regarding morphing strategies, Gao et al. [30] and Li et al. [51] deform irregular meshes into a reference one in order to learn solution of PDEs, but rely on complex coordinate transformation to compute a physical residual-based loss in the reference domain, and on input meshes with equal number of nodes. It is worth emphasizing that while the aforementioned works show promising results, they do not provide predictive uncertainties. There exist several methods for quantifying the uncertainties of deep neural networks [32], but it remains an open problem to provide well calibrated uncertainty estimates at a reasonable computational cost.

3 MMGPmethodology

The proposed methodology is based on two main ingredients that allow us to leverage classical machine learning methods for regression tasks in the context of non-parameterized geometrical variability: (i) the data is pretreated by morphing each input mesh into a reference shape, and resorting to finite element interpolation to express all fields of interest on a common mesh of this reference shape, and (ii) a low-dimensional embedding of the geometries is built by considering the coordinates of the nodes as a continuous input field over the meshes. Formally, the proposed methodology consists in constructing a graph kernel by relying on three well chosen transformations such that the transformed inputs can be compared with any classical kernel functions defined over Euclidean spaces. Figure 1 illustrates the proposed strategy for a two-dimensional problem where we aim at predicting output fields of interest. The first transformation morphs the input mesh onto a chosen common shape. The second transformation performs a finite element interpolation on the chosen reference mesh of the common shape. Finally, a dimensionality reduction technique is applied to obtain low-dimensional embeddings of the inputs and outputs. These three steps are

all deterministic and described in the following subsections. The proposed kernel function can be plugged into any kernel method. Herein, we rely on Gaussian process regression in order to learn steady-state mesh-based simulations in computational fluid and solid mechanics.

Z

1

ℓ

{

˜

Z

i

ℓ

}

{

˜

U

i

k

}

U

1

k

U

n

k

U

n

k

U

1

k

GP

Z

1

ℓ

Z

n

ℓ

Z

n

ℓ

.

.

.

.

.

.

.

.

.

.

.

.

{

̂

Z

i

}

{

̂

U

i

k

}

morphing to

inverse morphing to

FE interpolation

FE interpolation

PCA

{

µ

i

}

inv. PCA

common shape

sample shape

to common mesh

to sample mesh

3.1 Deterministic preprocessings of the input meshes and fields of interest

In this section, we describe the methodology for building low-dimensional representations of the input meshes and output fields.

Mesh morphing into a reference shape Ω . Each input mesh M i , i = 1 , . . . , n , is morphed onto a mesh M i associated to a fixed reference shape Ω . The morphed mesh has the same number of nodes and same set of edges as the initial mesh M i , but their spatial coordinates differ. In this work, we consider two morphing algorithms, namely, Tutte's barycentric mapping [71] onto the unit disk, and the Radial Basis Function (RBF) morphing [9, 21] onto a chosen reference shape. Regardless of the morphing algorithm, physical features inherited from the boundary value problem are carefully taken into account. More precisely, points, lines and surfaces of importance in the definition of the physical problem are mapped onto their representant on the reference shape. Doing so, rigid body transformations that may occur in the database are corrected in the mesh morphing stage, and boundary conditions of same nature are matched together.

Common mesh M c of the reference shape Ω . At this stage, it should be noted that although the morphed meshes M i are associated to a common reference shape Ω , they do not share the same nodes and edges. This prevents us from measuring similarities between the input meshes and output fields with classical techniques. A strong advantage of the finite element method is that it provides accurate solution fields with a continuous description over the mesh, and a natural way to transfer fields from one mesh to another. This motivates us to introduce a common mesh M c of the reference shape Ω , as a common support for all the sample fields data. A possibility is to choose an input mesh in the training set, e.g. M 1 , and define M c as its morphing onto the chosen reference shape. The aim is twofold. First, it allows us to express the output fields on the common morphed mesh M c , leading to vector representations of same sizes. Second, the coordinates fields of the meshes M i are also transferred onto this common mesh in order to build a shape embedding. These procedures rely on classical finite element interpolation that is described in the rest of this section.

Transporting the fields of interest on the common mesh M c . The discretized solution U i , i = 1 , . . . , n , is first transferred on the morphed mesh M i as follows:

where { φ i I } N i I =1 is the finite element basis associated to the morphed mesh M i . The transported fields U 1 k , . . . , U n k share the same geometric support (the reference shape). This implies that they can be

interpolated onto the common mesh M c using the finite element interpolation operator P defined as:

where { φ c I } N i I =1 is the finite element basis associated to M c , x c J is the coordinates of the J -th node of M c and U i k ( x c J ) is evaluated using Equation (3). We are now in the much more favorable situation where all the fields of interest are expressed on a common mesh M c . More specifically, for each field of interest k and input mesh M i , we let ˜ U i k ∈ R N c be the transported output fields onto the common mesh, such that ˜ U i k,I = U i k ( x c I ) . In this setting, the vector representations of the output fields now have the same sizes N c . Notice that the derivation of the finite element interpolation is identical with higher-order Lagrange finite elements.

Transporting the coordinates fields on the common mesh M c . The same procedure can be applied to the coordinate fields of the input meshes in order to build a shape embedding of the input meshes. Let Z i ℓ be the ℓ -th component of the coordinate field over the mesh M i , ℓ = 1 , . . . , d Ω . Using the finite element basis associated to the mesh M i , the coordinates fields can be written as

where Z i ℓ,I denotes the ℓ -th of the coordinates of the node I in the mesh M i . Notice that the notation Z i ℓ,I is preferred to x i ℓ,I , since it denotes here the degrees of freedom of the coordinate fields defined over M i , whose continuity property is essential for the finite element interpolation stage. Then, in the same fashion as for the fields of interest, the coordinates fields are transferred on the morphed mesh M i and interpolated on the common mesh M c using the operator P given by Equation (4). For each coordinate ℓ = 1 , . . . , d Ω and input mesh M i , we have the common representations ˜ Z i ℓ ∈ R N c of the coordinate fields on the common mesh M c .

Dimensionality reduction. At this stage, the input coordinates fields of the meshes and the output fields are expressed on the same common mesh M c , and can be compared using standard machine learning techniques. We propose to build low-dimensional embeddings of these quantities using Principal Component Analysis (PCA). For each output field, PCA is applied to the set of observations { ˜ U i k } n i =1 , leading to the fields low-dimensional embeddings that we denote by { ̂ U i k } n i =1 . Similarly, PCA is applied to concatenated transported coordinate fields, { ( ˜ Z i 1 , . . . , ˜ Z i d Ω ) } n i =1 , leading to lowdimensional embeddings of the input geometries { ̂ Z i } n i =1 , that we refer to as the shape embeddings.

3.2 MMGPtraining

Once the operations of mesh morphing, finite element interpolation on a common mesh and dimensional reduction described in the previous subsection have been carried out, we are left with reduced-size objects of same dimension. Let { X i } n i =1 ∈ R l Z + p , where p the number of nongeometrical parameters and l Z is the size of shape embedding, be such that X i = ( ̂ Z i , µ i ) . Denoting l U k the size of the embedding of field U k , the machine learning task given by Equation (2) can be approximated by the following set of scalar and vector regression problems:

Gaussian processes can be trained in a classical fashion to address the regression problems (5a)-(5b).

MMGPforascalar output. Let D = { ( X i , w i m 0 ) } n i =1 be a training dataset for one of the problems given by Equation (5a), i.e. for the m 0 -th output scalar. It can be shown by standard conditioning [76] that the posterior mean and variance of the prediction on some given test input X ⋆ are given by

where w m 0 = { w i m 0 } n i =1 , and K is the Gram matrix such that K i,j = c ( X i , X j ) for 1 ≤ i, j ≤ n , the vector k ⋆ such that k ⋆j = c ( X ⋆ , X j ) , and the scalar K ⋆⋆ = c ( X ⋆ , X ⋆ ) , with c denoting the chosen kernel function which lengthscales are optimized, and σ denotes the optimized nugget parameter. This training procedure is repeated for the q scalar outputs.

MMGPfor an output field. Let D = { ( X i , ̂ U i k 0 ) } n i =1 be a training dataset for one of the problems given by Equation (5b), i.e. for output field k 0 . A multioutput GP is first trained to predict the output embeddings ̂ U k 0 . The predictions of the GP are then decoded with the inverse PCA mapping, and morphed back to the original input mesh M i . Due to this last nonlinear operation, the posterior distribution of the output field of interest U k 0 is no longer Gaussian. The predictive uncertainties are thus obtained through Monte Carlo simulations. This training procedure is repeated for each of the d output fields.

3.3 Properties of the methodology

The sequence of preprocessing operations, including mesh morphing, finite element interpolation, and PCA, leads to a non-linear dimensionality reduction. Leveraging these deterministic processes reduces the burden on the machine learning stage, potentially necessitating fewer training examples to achieve robust model performance on complex mesh-based data. In the numerical experiments presented in Section 4, the morphing technique is chosen a priori, with ongoing research focused on optimizing this morphing to minimize the number of PCA modes, which leads to a highly nonlinear dimensionality reduction stage that is finely tuned to the specific characteristics of the data.

Gaussian process regression between the input and output embeddings has several advantages. From a theoretical perspective, there exists conditions on the features of a continuous kernel so that it may approximate an arbitrary continuous target function [59]. Gaussian processes also come with built-in predictive uncertainties, that are marginally valid under the a priori Gaussian assumption. Nevertheless, the proposed methodology can be combined with any other regressor such as a deep neural network instead of the Gaussian process.

For clarity of the presentation, the MMGP methodology is illustrated with very simple, if not the simplest, morphing and dimensionality reduction techniques. Alternatives are possible for each algorithm brick. In particular, the fixed topology restriction may be lifted with other morphing algorithms, see Appendix B for more details.

4 Numerical experiments

Three regression problems are considered in order to assess the efficiency of the proposed methodology. The chosen datasets are first described in Section 4.1. The experimental setup is summarized in Section 4.2, and the results are discussed in Section 4.3.

4.1 Datasets

Three datasets in computational fluid and solid mechanics are considered, described below, and summarized in Table 1. All the considered datasets involve geometric variabilities and meshes with possibly different number of nodes and edges. Additional details can also be found in Appendix A.

Rotor37 dataset. We consider a 3D compressible steady-state Reynold-Averaged Navier-Stokes (RANS) simulation solved with elsA [18] using the finite volumes method. The inputs are given

by a mesh representing the surface of a 3D compressor blade [8], and two additional parameters that correspond to an input pressure and a rotation speed. The outputs of the problem are given by 4 scalars (massflow m , compression rate τ , isentropic efficiency η , polyentropic efficiency γ ), and 2 fields on the surface of the input mesh (temperature T , pressure P ).

Tensile2d dataset. The second dataset corresponds to a 2D quasi-static problem in solid mechanics. The geometrical support consists of a 2D square, with two half-circles that have been cut off in a symmetrical manner. The inputs are given by a mesh, a pressure applied on the upper boundary, and 5 material parameters modeling the nonlinear elastoviscoplastic law of the material [50]. The boundary value problem is solved with the finite element method and the Z-set software [60]. The outputs of the problem are given by 4 scalars ( p max , v max , σ max 22 , and σ max v ) and 6 fields of interest ( u , v , p , σ 11 , σ 12 , and σ 22 ). For the sake of brevity, the reader is referred to Appendix A for a description of these quantities.

AirfRANS dataset. The last dataset is made of 2D incompressible RANS around NACA profiles and taken from Bonnet et al. [14]. The inputs are given by a mesh of a NACA profile and two parameters that correspond to the inlet velocity and the angle of attack. The outputs are given by 2 scalars (drag C D and lift C L coefficients), and 3 fields (the two components of the fluid velocity u and v , and the pressure field p ). An additional version of this dataset is also considered, where the input meshes have been coarsened using the MMG remesher [1], and the output fields have been transferred to the coarsened meshes. The output scalars are unchanged. Illustrations of the input meshes can be found in the original paper [14].

4.2 Experimental setup

Morphings. The Tutte's barycentric mapping onto the unit disk is used for morphing the meshes in the Rotor37 and Tensile2d datasets. The input meshes in the AirfRANS dataset are morphed onto the first mesh using RBF.

PCA embeddings. Embeddings of sizes 32 and 64 are retained for respectively the spatial coordinates and output fields in the Rotor37 and AirfRANS datasets. Smaller embeddings of sizes 8 are considered for both the spatial coordinates and output fields in the Tensile2d dataset. Note that for the Tensile2d and AirfRANS , a more effective variant of PCA has been used, which can easily deal with very large meshes (see, e.g. , [24, 36]), with up to hundreds of millions of degrees of freedom. Details about this variant can be found in Appendix C.

Gaussian processes. Anisotropic Matern-5/2 kernels and zero mean functions are used for all the Gaussian processes priors. The lengthscales and nugget parameter are optimized by maximizing the marginal log-likelihood function with a L-BFGS algorithm and 10 random restarts using the GPy package [35].

Baselines. The performance of MMGP is compared with two baselines, namely, a graph convolutional neural network (GCNN) with a UNet-type architecture [29] and the GeneralConv [79], and MeshGraphNets (MGN) [63]. The hyperparameters of the GNNs are chosen by relying on a grid search. The GCNN and MGN models are implemented with PyTorch Geometric [25] and DGL [73], respectively. Additional details about the architectures and hyperparameters can be found in Appendix D. Due to the sizes of the input meshes in the AirfRANS dataset, the considered GNNbased baselines are prohibitively expensive. Similarly to the work of [28], the GNNs are trained using coarsened input meshes as described in Section 4.1. The output fields predicted on the coarse meshes are then transferred back on the original fine meshes thanks to finite element interpolation.

Evaluation metrics. Accuracy of the trained models is assessed by computing relative RMSE errors. Let { U i ref } n ⋆ i =1 and { U i pred } n ⋆ i =1 be respectively test observations and predictions of a given field of interest. The relative RMSE considered herein is defined as

where it is recalled that N i is the number of nodes in the mesh M i , and max( U i ref ) is the maximum entry in the vector U i ref . Similarly for scalar outputs, the following relative RMSE is computed:

Given that the input meshes may have different number of nodes, the coefficients of determination Q 2 between the target and predicted output fields are computed by concatenating all the fields together. For each of the considered regression problems, training is repeated 10 times in order to provide uncertainties over the relative RMSE and Q 2 scalar regression coefficients.

4.3 Results and discussion

Predictive performance. The relative RMSE and Q 2 scalar regression coefficients are reported in Table 2 for all the considered experiments. While the GNN-based baselines achieve good performance, the MMGP model consistently outperforms them with lower errors. It is worth emphasizing that the field p and scalar p max of the Tensile2d dataset are particularly challenging, see Appendix E.2 for more details. They represent the accumulated plasticity in the mechanical piece, which are non-zero for a small fraction of the training set. Figure 2 shows the graphs of the output scalars predictions versus the targets on test set for the Tensile2d case. Figure 3 also shows examples of fields prediction in the case of the AirfRANS problem. The MMGP model is able to accurately reproduce the output fields, with relative errors mostly located near the tips of the airfoils.

Uncertainty estimates. Once trained, the MMGP model provides access to predictive uncertainties for the output fields and scalars. Figure 4 shows an example of predicted pressure field for an

maxm(Von Mises)

maxm(p)

maxrtop(u2)

maxrtopg(22)

10-2

50

300

10-4

redictions

10-6

45

0

Q

200

P

GCNN

10-8

MMGP

10-3

40

200

300

10-7

10-4

10-3

10-2

40

45

50

Targets

Targets

Targets

Targets

UX_reference

UX_predicted

UX_relative_error

-5.0e-01

50

1.0e+02

-5.0e-01

50

1.0e+02

1.0e-05

0.001

1.0e-01

Min:1.1e-09 Max:2.8e-01

UY_reference

UY_predicted

UY_relative_error

-3.0e+010

5.0e+01

-3.0e+01

5.0e+01

1.0e-05

0.001

1.0e-01

Min:1.9e-08Max:6.9e-01

p_reference

p_predicted

p_relative_error

3.2e+03

3.2e+03

1.0e-05

0.001

1.0e-01

Min:1.3e-08 Max:6.7e-02

arbitrary test input mesh of the Rotor37 experiment, together with the predictive variance and the point-wise relative absolute error. High relative errors are localized where the pressure field exhibits a discontinuity, known as a shock in compressor aerodynamics. The predictive variance is also higher near this region, reflecting that the GP-based surrogate model is uncertain about its prediction of the shock position. Figure 5 shows graphs of two output scalars with respect to the input pressure in the Tensile2d problem. The predictive intervals of the MMGP model are discriminative: they get wider as the input pressure falls out of the support of the training distribution. In order to assess the validity of the prediction intervals, we compute the prediction interval coverage probability (PICP), i.e. the average of test targets that fall into the 95% prediction interval. For the AirfRANS dataset, PICPs of 93 . 05% and 93 . 5% for respectively the outputs C L and C D are obtained by averaging the individual PICPs of 10 independent MMGP models. The prediction intervals are slightly over-confident but this could be corrected by e.g. conformalizing the Gaussian process [69].

Prediction

Predictive variance

L1 rel.error

2.489e+05

3.936e-01

1.034e-01

1.9e+5

0.307

0.078

1.4e+5

0.22

0.052

80605

0.133

0.026

2.451e+04

4.593e-02

3.285e-07

Geometry 1 —←—Geometry 2

—←Geometry 3 —+Geometry 4

0.015

400

Mises)

0.010

300

(Von

200

0.005

Wxe

100

0.000

training interval

training interval

0

F

-70-60

-50

-40-30-20

-70

-60

-50

-40

）-30-20

Pressure

Pressure

Computational times. The MMGP model can easily be trained on CPU hardware and with much lower computational times, see Table 3.

5 Conclusion

In summary, our proposed method presents an innovative approach to approximating field and scalar quantities of interest within the context of solving complex physics problems for design optimization. Our work introduces two key contributions: firstly, the utilization of mesh morphing pretreatment in conjunction with finite element interpolation, and secondly, the incorporation of shape embedding through dimensional reduction of coordinates, treating them as continuous fields over the geometric support. These innovations alleviate the machine learning task from the challenges of handling variable-sized samples and the need to learn implicit shape embedding. By reducing the dimensionality of inputs and outputs, our approach allows for the application of efficient Gaussian process regression. Notably, our MMGP model exhibits several key advantages. It can seamlessly handle very large meshes, is amenable to efficient CPU training, is fairly interpretable, demonstrates high accuracy in our experimental results, and provides readily available predictive uncertainties.

Future works will explore the extension of our method to accommodate time-dependent quantities of interest and investigate the optimization of the morphing process to enhance data compression and overall performance. Our research opens exciting avenues for advancing the capabilities of machine learning in the realm of physics-based design optimization.

References

Mmg platform website. https://www.mmgtools.org/. Accessed: 2023-01-19.

MMGPdocumentation. https://mmgp.readthedocs.io/ . Accessed: 2023-10-16.

MMGPgithub repository. https://gitlab.com/drti/mmgp . Accessed: 2023-10-16.

Modulus package website. https://developer.nvidia.com/modulus. Accessed: 2023-10-05.

J. Ackmann, P.D. Düben, T.N. Palmer, and P.K. Smolarkiewicz. Machine-learned preconditioners for linear solvers in geophysical fluid flows. arXiv preprint arXiv:2010.02866 , 2020.

M. Alexa. Recent advances in mesh morphing. In Computer graphics forum , volume 21, pages 173-198. Wiley Online Library, 2002.

K.R. Allen, T. Lopez-Guevara, K. Stachenfeld, A. Sanchez-Gonzalez, P. Battaglia, J. Hamrick, and T. Pfaff. Physical design using differentiable learned simulators. arXiv preprint arXiv:2202.00728 , 2022.

A. Ameri. Nasa rotor 37 cfd code validation glenn-ht code. In 47th AIAA Aerospace Sciences Meeting including The New Horizons Forum and Aerospace Exposition , page 1060, 2009.

N. Arad, N. Dyn, D. Reisfeld, and Y. Yeshurun. Image warping by radial basis functions: Application to facial expressions. CVGIP: Graphical models and image processing , 56(2):161172, 1994.

P. Baque, E. Remelli, F. Fleuret, and P. Fua. Geodesic convolutional shape optimization. In International Conference on Machine Learning , pages 472-481. PMLR, 2018.

P.W. Battaglia, J.B. Hamrick, V. Bapst, A. Sanchez-Gonzalez, V. Zambaldi, M. Malinowski, A. Tacchetti, D. Raposo, A. Santoro, R. Faulkner, et al. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261 , 2018.

J. Besson, G. Cailletaud, J.-L. Chaboche, and S. Forest. Non-linear mechanics of materials , volume 167. Springer Science & Business Media, 2009.

F.E. Bock, R.C. Aydin, C.J. Cyron, N. Huber, S.R. Kalidindi, and B. Klusemann. A review of the application of machine learning and data mining approaches in continuum materials mechanics. Frontiers in Materials , 6:110, 2019.

F. Bonnet, J. Mazari, P. Cinnella, and P. Gallinari. Airfrans: High fidelity computational fluid dynamics dataset for approximating reynolds-averaged navier-stokes solutions. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems , volume 35, pages 23463-23478. Curran Associates, Inc., 2022.

M. Botsch and L. Kobbelt. Real-time shape editing using radial basis functions. In Computer graphics forum , volume 24, pages 611-621. Blackwell Publishing, Inc Oxford, UK and Boston, USA, 2005.

X. Bresson and T. Laurent. Residual gated graph convnets. arXiv preprint arXiv:1711.07553 , 2017.

S.L. Brunton, B.R. Noack, and P. Koumoutsakos. Machine learning for fluid mechanics. Annual Review of Fluid Mechanics , 52:477-508, 2020.

L. Cambier, M. Gazaix, S. Heib, S. Plot, M. Poinot, J.P. Veuillot, J.F. Boussuge, and M. Montagnac. An overview of the multi-purpose elsa flow solver. Aerospace Lab , (2):p-1, 2011.

Y. Cao, M. Chai, M. Li, and C. Jiang. Efficient learning of mesh-based physical simulation with bi-stride multi-scale graph neural network. 2023.

F. Casenave, N. Akkari, F. Bordeu, C. Rey, and D. Ryckelynck. A nonintrusive distributed reduced-order modeling framework for nonlinear structural mechanics-application to elastoviscoplastic computations. International Journal for Numerical Methods in Engineering , 121(1):32-53, 2020.

A. De Boer, M.S. Van der Schoot, and H. Bijl. Mesh deformation based on radial basis function interpolation. Computers & structures , 85(11-14):784-795, 2007.

C. Ding and H. Peng. Minimum redundancy feature selection from microarray gene expression data. Journal of bioinformatics and computational biology , 3(02):185-205, 2005.

R.P. Dwight. Robust mesh deformation using the linear elasticity equations. In Computational fluid dynamics 2006 , pages 401-406. Springer, 2009.

C. Farhat, T. Chapman, and P. Avery. Structure-preserving, stability, and accuracy properties of the energy-conserving sampling and weighting method for the hyper reduction of nonlinear finite element dynamic models. International Journal for Numerical Methods in Engineering , 102(5):1077-1110, 2015.

M. Fey and J.E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds , 2019.

M. Fey and J.E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds , 2019.

M. S. Floater. Parametrization and smooth approximation of surface triangulations. Computer aided geometric design , 14(3):231-250, 1997.

M. Fortunato, T. Pfaff, P. Wirnsberger, A. Pritzel, and P. Battaglia. Multiscale meshgraphnets. arXiv preprint arXiv:2210.00612 , 2022.

H. Gao and S. Ji. Graph u-nets. In international conference on machine learning , pages 2083-2092. PMLR, 2019.

H. Gao, L. Sun, and J.-X. Wang. Phygeonet: Physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state pdes on irregular domain. Journal of Computational Physics , 428:110079, 2021.

V. Garanzha, I. Kaporin, L. Kudryavtseva, F. Protais, N. Ray, and D. Sokolov. Foldover-free maps in 50 lines of code. ACM Transactions on Graphics (TOG) , 40(4):1-16, 2021.

J. Gawlikowski, C.R.N. Tassi, M. Ali, J. Lee, M. Humt, J. Feng, A. Kruspe, R. Triebel, P. Jung, R. Roscher, et al. A survey of uncertainty in deep neural networks. arXiv preprint arXiv:2107.03342 , 2021.

J. Gilmer, S.S. Schoenholz, P.F. Riley, O. Vinyals, and G.E. Dahl. Neural message passing for quantum chemistry. In International conference on machine learning , pages 1263-1272. PMLR, 2017.

M. Götz and H. Anzt. Machine learning-aided numerical linear algebra: Convolutional neural networks for the efficient preconditioner generation. In 2018 IEEE/ACM 9th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems (scalA) , pages 49-56. IEEE, 2018.

GPy. GPy: A Gaussian process framework in python. http://github.com/SheffieldML/ GPy , since 2012.

S. Grimberg, C. Farhat, R. Tezaur, and C. Bou-Mosleh. Mesh sampling and weighting for the hyperreduction of nonlinear petrov-galerkin reduced-order models with local reduced-order bases. International Journal for Numerical Methods in Engineering , 122(7):1846-1874, 2021.

X. Guo, W. Li, and F. Iorio. Convolutional neural networks for steady flow approximation. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining , pages 481-490, 2016.

W. Hamilton, Z. Ying, and J. Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems , 30, 2017.

L. Harsch and S. Riedelbauch. Direct prediction of steady-state flow fields in meshed domain with graph networks. arXiv preprint arXiv:2105.02575 , 2021.

K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770778, 2016.

A. Heinlein, A. Klawonn, M. Lanser, and J. Weber. Combining machine learning and domain decomposition methods-a review. 2020.

S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning , pages 448-456. PMLR, 2015.

V.R. Joseph, E. Gul, and S. Ba. Maximum projection designs for computer experiments. Biometrika , 102(2), 3 2015.

G.E. Karniadakis, I.G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang. Physics-informed machine learning. Nature Reviews Physics , 3(6):422-440, 2021.

D.P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

T.N. Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 , 2016.

B. Knyazev, G.W. Taylor, and M. Amer. Understanding attention and generalization in graph neural networks. Advances in neural information processing systems , 32, 2019.

M.A. Kramer. Nonlinear principal component analysis using autoassociative neural networks. AIChE Journal , 37(2):233-243, 1991.

K. Lee and K.T. Carlberg. Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders. Journal of Computational Physics , 404:108973, 2020.

J. Lemaitre and J.-L. Chaboche. Mechanics of solid materials . Cambridge university press, 1994.

Z. Li, D.Z. Huang, B. Liu, and A. Anandkumar. Fourier neural operator with learned deformations for pdes on general geometries. arXiv preprint arXiv:2207.05209 , 2022.

J. Ling, R. Jones, and J. Templeton. Machine learning strategies for systems with invariance properties. Journal of Computational Physics , 318:22-35, 2016.

M. Lino, C. Cantwell, A.A. Bharath, and S. Fotiadis. Simulating continuum mechanics with multi-scale graph neural networks. arXiv preprint arXiv:2106.04900 , 2021.

M. Lino, S. Fotiadis, A.A. Bharath, and C.D. Cantwell. Multi-scale rotation-equivariant graph neural networks for unsteady eulerian fluid dynamics. Physics of Fluids , 34(8), 2022.

I. Loshchilov and F. Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983 , 2016.

L. Lu, P. Jin, and G.E. Karniadakis. Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators. arXiv preprint arXiv:1910.03193 , 2019.

A.L. Maas, A.Y. Hannun, A.Y. Ng, et al. Rectifier nonlinearities improve neural network acoustic models. In Proc. icml , volume 30, page 3. Atlanta, Georgia, USA, 2013.

J. Masci, D. Boscaini, M. Bronstein, and P. Vandergheynst. Geodesic convolutional neural networks on riemannian manifolds. In Proceedings of the IEEE international conference on computer vision workshops , pages 37-45, 2015.

C.A. Micchelli, Y. Xu, and H. Zhang. Universal kernels. Journal of Machine Learning Research , 7(12), 2006.

Mines ParisTech and ONERA the French aerospace lab. Zset: nonlinear material & structure analysis suite. http://www.zset-software.com, 1981-present.

F. Monti, D. Boscaini, J. Masci, E. Rodola, J. Svoboda, and M.M. Bronstein. Geometric deep learning on graphs and manifolds using mixture model cnns. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 5115-5124, 2017.

H. Peng, F. Long, and C. Ding. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence , 27(8):1226-1238, 2005.

T. Pfaff, M. Fortunato, A. Sanchez-Gonzalez, and P.W. Battaglia. Learning mesh-based simulation with graph networks. In International Conference on Learning Representations , 2021.

C.R. Qi, H. Su, K. Mo, and L.J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 652-660, 2017.

M. Raissi, P. Perdikaris, and G.E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics , 378:686-707, 2019.

R. Rannacher and R. Scott. Some optimal error estimates for piecewise linear finite element approximations. Mathematics of computation , 38(158):437-445, 1982.

F. Scarselli, M. Gori, A.C. Tsoi, M. Hagenbuchner, and G. Monfardini. The graph neural network model. IEEE Transactions on Neural Networks , 20(1):61-80, 2009.

B. Schölkopf, A. Smola, and K.-R. Müller. Nonlinear component analysis as a kernel eigenvalue problem. Neural computation , 10(5):1299-1319, 1998.

G. Shafer and V. Vovk. A tutorial on conformal prediction. Journal of Machine Learning Research , 9(3), 2008.

M.L. Staten, S.J. Owen, S.M. Shontz, A.G. Salinger, and T.S. Coffey. A comparison of mesh morphing methods for 3d shape optimization. In Proceedings of the 20th international meshing roundtable , pages 293-311. Springer, 2012.

W.T. Tutte. How to draw a graph. Proceedings of the London Mathematical Society , 3(1):743767, 1963.

P. Veliˇ ckovi´ c, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio. Graph attention networks. arXiv preprint arXiv:1710.10903 , 2017.

M. Wang, D. Zheng, Z. Ye, Q. Gan, M. Li, X. Song, J. Zhou, C. Ma, L. Yu, Y. Gai, et al. Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv preprint arXiv:1909.01315 , 2019.

D.A. White, W.J. Arrighi, J. Kudo, and S.E. Watts. Multiscale topology optimization using neural network surrogate models. Computer Methods in Applied Mechanics and Engineering , 346:1118-1135, 2019.

J. Willard, X. Jia, S. Xu, M. Steinbach, and V. Kumar. Integrating physics-based modeling with machine learning: A survey. arXiv preprint arXiv:2003.04919 , 2020.

C.K.I. Williams and C.E. Rasmussen. Gaussian processes for machine learning , volume 2. MIT press Cambridge, MA, 2006.

F. Wu, A. Souza, T. Zhang, C. Fifty, T. Yu, and K. Weinberger. Simplifying graph convolutional networks. In International conference on machine learning , pages 6861-6871. PMLR, 2019.

S. Yoshizawa, A. Belyaev, and H.-P. Seidel. A fast and simple stretch-minimizing mesh parameterization. In Proceedings Shape Modeling Applications, 2004. , pages 200-208. IEEE, 2004.

J. You, R. Ying, and J. Leskovec. Design Space for Graph Neural Networks. November 2020.

A. Zaharescu, E. Boyer, and R. Horaud. Topology-adaptive mesh deformation for surface evolution, morphing, and multiview reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence , 33(4):823-837, 2010.

Y. Zhu and N. Zabaras. Bayesian deep convolutional encoder-decoder networks for surrogate modeling and uncertainty quantification. Journal of Computational Physics , 366:415-447, 2018.

O.C. Zienkiewicz and P.B. Morice. The finite element method in engineering science , volume 1977. McGraw-hill London, 1971.

Appendix

Available dataset and code The code corresponding to the two-dimensional solid mechanics case ( Tensile2d ) described in Section 4.1 is available at https://gitlab.com/drti/mmgp [3]. A documentation is available at https://mmgp.readthedocs.io/ [2], where details are provided on how to download the dataset Tensile2d and reproduce the corresponding numerical experiments.

Details regarding the datasets are provided in Appendix A. Morphing strategies and dimensionality reduction techniques are described in Appendices B and C. Details about the GNNs baselines are given in Appendix D. Finally, additional results about the considered experiments are gathered in Appendix E.

A Datasets

This section provides additional details regarding the synthetic datasets Tensile2d and Rotor37 . Regarding the AirfRANS dataset, the reader is referred to [14].

A.1 Rotor37 dataset

Examples of input geometries are shown in Figure 6 together with the associated output pressure fields. While the geometrical variabilities are moderate, it can be seen that they have a significant impact on the output pressure field. A design of experiment for the input parameters of this problem are generated with maximum projection LHS method [43]. For each input mesh and set of input parameters, a three-dimensional aerodynamics problem is solved with RANS, as illustrated in, e.g. [8]. The output scalars of the problem are obtained by post-processing the three-dimensional velocity.

A.2 Tensile2d dataset

Examples of input geometries are shown in Figure 7. A two-dimensional boundary value problem in solid mechanics is considered, under the assumption of small perturbations (see, e.g. [12]). The partial differential equation is supplemented with Dirichlet and Neumann boundary conditions: the displacement on the lower boundary is fixed, while a uniform pressure is applied at the top. The input parameters of the problem are chosen to be the magnitude of the applied pressure and 5 parameters involved in the elasto-visco-plastic constitutive law of the material [50]. The outputs of the problem are chosen as the components of the displacement field, u and v , the entries of the Cauchy stress tensor, σ 11 , σ 22 , σ 12 , and the cumulative plastic strain p . We also consider 4 output scalars obtained by post-processing the fields of interest: the maximum plastic strain p max accross the geometry, the maximum vertical displacement v max at the top of the geometry, and the maximum normal stress σ max 22 and Von Mises stress σ max v accross the geometry. It is worth emphasizing that the cumulative plastic strain p is challenging to predict, as illustrated in Section E.

B Morphing strategies

In this section, we briefly describe the Tutte's barycentric mapping [71] and the radial basis function morphing [9, 21] used in the considered experiments.

Tutte's barycentric mapping. For this method, we are limited to connected triangular surface meshes of fixed topology, either in a 2D or in a 3D ambient space. Tutte's barycentric mapping starts by setting the value of the displacement of the boundary points of the mesh (usually onto the unit disk), and solve for the value at all the remaining nodes of the mesh. The physical features available on the mesh, and inherited from the problem, are used in the specification of the displacement of the boundary nodes.

We recall that x I , I = 1 . . . N , denote the mesh nodes coordinates. We assume that the numbering of the nodes starts with the interior points of the mesh 1 . . . N int , and ends with the N b nodes on its boundary N int +1 . . . N . The morphed mesh node coordinates are denoted by x I , I = 1 . . . N . The coordinates of the boundary of the morphed mesh being known, we denote x b I = x I + N int , I = 1 . . . N b . Then the following sparse linear system is solved for the morphing of the interior points:

where N ( I ) and d ( I ) are respectively the neighbors and the number of neighbors of the node I in the graph (or in the mesh following its connectivity).

In the 2D solid mechanics case Tensile2d , we know the rank of the point separating the left and the bottom faces which we map onto the point (0 , 1) of the target unit disk. The linear density of nodes on the boundary of the target unit disk is chosen to be the same as the one of the mesh sample (relative to the length of the boundary), see Figure 8 for an illustration.

From [27, corollary 2], the morphing described above is called a parametrization, and defines an isomorphic deterministic transformation of the considered triangular surface mesh M , into a plane triangular mesh M of the unit disk. Notice that although these morphing techniques are called 'mesh parametrization', this do not mean that we need to know any parametrization of the shape: these are deterministic transformations of the meshes, requiring no other information than the nodes locations and the triangles connectivities.

This method is taken from the computer graphics community and has been improved over the years. In [78], a quality indicator called stretch metric is optimized during an iterative procedure, to obtain more regular morphed mesh. Recently in [31], a procedure was proposed to drastically improve mesh parametrization, even in difficult cases where some triangles are overlapping. It should be noted that such iterative procedures come with the additional cost of solving a series of sparse linear systems.

Radial Basis Function morphing. In the same fashion as Tutte's barycentric mapping, RBF morphing methods start by setting the value of the displacement at some particular nodes of the mesh (here the boundary points of the mesh, but interior points can be considered as well with RBF), and solve for the location at all the remaining nodes of the mesh. The physical features available on the mesh are also used in the specification of the displacement of the boundary points. RBF morphing methods are compatible with 2D and 3D structured and unstructured meshes, do not require any mesh connectivity information, and can be easily implemented in parallel for partitioned meshes.

We use the RBF morphing method as proposed in [21]. Once the mapping for the N b boundary points of ranks N int +1 . . . N is fixed, then the interior points 1 ≤ I ≤ N int are mapped such as

where ϕ is a radial basis function with compact support and α J are determined by the interpolation conditions. More precisely, we choose the radial basis function with compact support ϕ ( ξ ) = (1 -ξ ) 4 (4 ξ +1) and support radius equal to half of the mesh diameter, and interpolation conditions means that the morphing is known at the boundary points:

For the AirfRANS dataset, we make use of the physical properties of the boundary condition to morph each mesh onto the first geometry of the training set. Referring to Figure 9 (bottom), we know which nodes lie the external boundary (in black), airfoil extrado (in red), airfoil intrado (in blue) and which nodes define the leading and trailing points (green crosses). We choose to keep the points at the external boundary fixed (zero mapping), map the leading and trailing edge to the ones of the mesh of the first training sample, and map the points on the extrado and intrado along the ones of the mesh of the first training sample while conserving local node density (relative to the length of the boundary). A zoom of the RBF morphing close to the airfoil for test sample 787 is illustrated in Figure 10.

nV

nV

Notice that while Tutte's barycentric mapping requires solving a sparse linear system of rather large size N int , RBF morphing requires solving a dense linear system of smaller size N b . RBF morphing methods dealing with complex non-homogeneous domains have been proposed in [15].

Other methods. In [23], the morphing is computed by means of solving an elastic problem. See also [6, 70] for literature reviews on mesh morphing methods. Mesh deformation algorithms compatible with topology changes have been proposed [80].

C Dimensionality reduction

The principal component analysis can be replaced by more effective dimensionality reduction techniques such as the snapshot-POD. The latter is a variant where the underlying ℓ 2 -scalar product used to compute the coefficients of the empirical covariance matrices is replaced by the L 2 ( M c ) -inner product. Define the symmetric positive-definite matrix M ∈ R N c × N c , such that

In general, a quadrature formula, in the form of a weighted sum over function evaluations on the common mesh, is chosen such that the integral are computed exactly for functions in the span of the finite element basis. Then, the empirical covariance matrix is computed as

which corresponds to the continuous formula for the computation of the correlations of the fields of interest transported and interpolated on the common morphed mesh. Hence, the empirical covariance matrix can take into account any heterogeneity of the common morphed mesh, which may occur after morphing. The same construction can be made for the spatial coordinate field, while its derivation is more technical, because it involves vector fields instead of scalar fields. The computation of the empirical covariance matrix can be easily be parallelized on numerous computer nodes, provided that the common morphed mesh has been partitioned in subdomains, which enable efficient dimensionality reduction for meshes up to millions of degrees of freedom, see [20].

Other linear or nonlinear dimension reduction techniques can be considered, like mRMR feature selection [22, 62], kernel-PCA [68] or neural network-based autoencoders [48].

D Architectures and hyperparameters of GNN-based baselines

D.1 Graph convolutional neural network

A graph convolutional neural network (GCNN) [67] has been implemented using PyTorch Geometric [26] with the Graph U-Net [29] architecture and the following specifications: (i) top k pooling [29, 47] layers with a pooling ratio of 0 . 5 to progressively aggregate information over nodes of the graph, (ii) feature sizes progressively increased after each top k pooling, i.e. , 16 , 32 , 64 , 96 and 128 , (iii) between each pooling, residual convolution blocks [40] are added to combine two consecutive normalization-activation-convolution layers, (iv) BatchNorm [42] layers are introduced, and (v) LeakyReLU [57] activations are used with slope of 0 . 1 on negative values.

A weighted multi-loss L that combines scalars and fields is used, and defined as

where λ scalars and λ fields are two positive hyperparameters. For gradient descent, an Adam optimizer [45] is used with a cosine-annealing learning rate scheduler [55]. The following hyperparameters are optimized by grid search: (i) the learning rate, 13 values between 1 . 0 and 0 . 0001 , (ii) the weight λ field ∈ { 1 , 10 , 100 , 1000 } , and (iv) the type of convolution, chosen between GATConv [72], GeneralConv [79], ResGatedGraphConv [16] and SGConv [77]. There are many other hyperparameters that could be tuned, as the number of layers or the number of features on each layer. The chosen hyperparameters are summarized in Table 4 for each experiment. In the case of the Rotor37 problem,

the outwards normals to the surface of the compressor blade are added as input features to input graphs. Similarly, for the Tensile2d and AirfRANS problems, the signed distance function is added as an input feature.

D.2 MeshGraphNets

The MGN model [63] is taken from Nvidia's Modulus [4] package that implements various deep surrogate models for physics-based simulations. The same set of hyperparameters is used for all the considered regression problems, which is chosen after conducting a grid search over the learning rate, the number of hidden nodes and edges features, the number of processor steps. The learning rate is set to 0 . 001 , the numbers of hidden features hidden_dim_node_encoder , hidden_dim_edge_encoder , and hidden_dim_node_decoder are all set to 16 . The number of processor steps is chosen as 10 . The rest of the MGNs hyperparameters are left to the default values used in the Modulus package. The batch size is set to 1 , the activation is chosen as the LeakyReLU activation with a 0 . 05 slope, and 1 , 000 epochs are performed for training the network. For scalar outputs, a readout layer taken from [46] is added to the model. The input nodes features are given by the spatial coordinates of the nodes, and possible additional fields such as the signed distance function (for the Tensile2d and AirfRANS problems), or the outward normals (for the Rotor37 problem). Given two node coordinates x i and x j , the edges features are chosen as exp( -∥ x i -x j ∥ 2 2 / (2 h 2 )) , where h denotes the median of the edge lengths in the mesh.

For each considered regression problem, it is found that it is more effective to train two MGNs models, one dedicated to handling output fields and the other specialized for output scalars, respectively. Nevertheless, better hyperparameter tuning and more effective readout layers could lead to different conclusions regarding this matter.

D.3 Training on AirfRANS

As mentioned in Section 4.1, training GNNs on the AirfRANS dataset is computationally expensive due to the sizes of the input meshes. For this reason, the GNN-based baselines are trained on the AirfRANS-remeshed dataset (see Table 1) obtained by coarsening the input meshes (see Figure 11) and the associated output fields. Once trained, predictions on the initial fine meshes are obtained through finite element interpolation. It should be underlined that this strategy may hinder the performance of the GNN-based baselines, as the reconstructed fields are obtained by finite element interpolation.

E Additional results

This section gathers additional results about the experiments considered in Section 4.

E.1 Out-of-distribution inputs

Figures 12, 13, and 14 show histograms of the logarithm of the predictive variance for the output scalars of interest on different sets of samples. The aim is to empirically assess if the MMGP model is able to identify out-of-distribution (OOD) inputs by attributing higher predictive variances. In the case of the Rotor37 problem, three OODs samples are generated such that the support of the covariates ( µ 1 , µ 2 , and M ) are disjoint with the support of the training distributions. It can be seen that the variances of the OOD samples are higher than the ones of the in-distribution samples. Similar observations are made for the Tensile2d and AirfRANS problems. While such an analysis can help to identify OOD inputs, it should be underlined that the predictive uncertainties of Gaussian processes are only valid under the Gaussian a priori assumption, which may not be verified in practice. For instance, the ellipsoid geometry has a similar variance as the in-distribution samples in Figure 13.

E.2 Predicted output fields

Tensile2d dataset. For reproducibility matters, we mention that for the field p and the scalar p max , the denominators in the formulae RRMSE f and RRMSE s has been replaced by 1 when its value is below 1 e -6 for preventing division by zero, which corresponds to replacing the relative error by the absolute error for samples that do not feature plastic behaviors.

In Figures 15-18, we illustrate the MMGP prediction, variance and relative error for all the considered fields: u , v , p (evrcum), σ 11 , σ 12 and σ 22 , for respectively the first training inputs, first test inputs, and two out-of-distribution geometries (ellipsoid and wedge). In particular, the wedge cut-off geometry features stress concentrations that are not present in the training set. We notice that the predictions for selected train and test inputs (Figures 15 and 16) are accurate, with small relative errors and relatively small predictive variances, except for some small areas where the considered fields have larger magnitudes. As expected, the predictions for the ellipsoid and wedge cases (Figures 17 and 18) are less accurate than for in-distribution shapes, but the predictive variances are larger, which confirms that MMGP informs that, locally, the prediction cannot be trusted. This phenomenon is particularly strong for the wedge case, that largely differs from the training set shapes.

In Figures 19 and 20, we consider all the output interest fields. The 2D domain is visualized in 3D, in the form of three surfaces: a transparent blue for the 0.025-quantile, a white for the reference prediction and a transparent red for the 0.975-quantile. The point-wise 95% confidence interval is the distance (along the out-of-plane axis) between the transparent blue and red surfaces. We notice that the 95% confidence intervals are very small for the train and test inputs, larger for the ellipsoid case, and much larger for the wedge case (in particular σ 22 ). Not surprisingly, for the OOD shapes

Massflow m

Compression rate T

20

counts

10

-4

-3

-2

-6

-5

4

-3

Isentropic efficiency

Polyentropic efficiency

20

counts

10

0

-7

-6

-5

-7

-6

-5

-4

log(variance)

log(variance)

In distribution

OOD μ1

OOD μ2

OOD M

ellipsoid and wedge, some surfaces intersect, meaning that, locally, the reference solution is not inside the 95% confidence interval.

In Figure 21, we illustrate the finite element error occurring when predicting σ 11 with respect to the 95% confidence interval for samples taken from the training and testing sets. We notice that on the training set, the finite element error magnitude is comparable to the 95% confidence interval, which is very small on training samples. On the testing set, the 95% confidence interval is larger, and the finite element error magnitude can be neglected.

AirfRANS dataset. Figures 22 and 23 illustrate reference, MMGP prediction and relative errors for the fields of interest u , v and p on respectively test sample 430 and train sample 93 . In the first row, zooms are provided close to the trailing edge to illustrate the accuracy of the prediction in the thin boundary layer. Relative errors have larger magnitudes on spatially restricted areas. We notice that on train sample 93 , the areas with low relative error are larger than for test sample 430 .

In Table 5, we compare MMGP and our trained GCNN and MGN, as well as the four models trained in [14], for the scalars of interest drag coefficient C D and lift coefficient C L , computed by post-processing the predicted fields instead of directly predicting them as output scalars. This post-processing consists in integrating the reference and predicted wall shear stress (from the velocity) and pressure fields around the surface of the airfoil. The models from [14] are a MLP (a classical Multi-Layer Perceptron), a GraphSAGE [38], a PointNet [64] and a Graph U-Net [29] and the corresponding results are taken from [14, Table 19] ('full dataset' setting that we considered in this work). Refer to [14, appendix L] for a description of the used architecture. The limits of this comparison are (i) the mesh supporting the fields are not the same (they have been coarsen in [14] by process different from ours), (ii) the scalar integration routine are not identical (we integrate using

maxM(Von Mises)

15

counts

10

5

0

-1.0

-0.5

0.0

0.5

1.0

1.5

-13

12

-11

-10

-9

-8

-7

maxrtop

(u2)

maxrtop

（022)

15

counts

10

5

0

11

-10

-9

-8

-3

2

0

log(variance)

log(variance)

----

ellips.

P=-55

P=-45

P=-35

wedge

T

P=-60

P=-50

P=-40

P=-30

finite element representations). Within these limits, MMGP appears competitive with respects to the models of [14] and our trained GCNN and MGN.

20.0

CD

ICL

17.5

15.0

12.5

counts

10.0

7.5

5.0

2.5

0.0

-9

-8

-7

-6

-5

7

-6

-5

-4

3

log(variance)

log(variance)

iv=45

iv=160

aoa=-0.15

aoa=0.18

iv=1

iv=70

iv=240

aoa=-0.04

aoa=0.29

iv=10

iv=90

aoa=-0.26

aoa=0.07

aoa=0.4

U1_predicted

U1_variance

U1_relative_error

U2_predicted

U2_variance

U2_relative_error

-2.9e-03

02.2e-03

1.3e-40

5.7e-09

8.8e-18

35.4e-02

-5.7e-06 0.005 1.0e-02

1.6e-39

1.2e-07

7.5e-18

2.7e-02

evrcum_predicted

evrcum_variance

evrcum_relative_error

sig11_predicted

sig1l_variance

sig11_relative_error

-1.2e-06

1.8e-04

0.0e+00

1.1e-16

0.0e+00

6.6e-02

-2.5e+0202.2e+02

1.8e-0724.9e+00

3.9e-07

2.6e-02

sig12_predicted

sig12_variance

sig12_relative_error

sig22_predicted

sig22_variance

sig22_relative_error

-1.6e+0201.6e+02

1.3e-06

3.2e+01

2.5e-070.01

2.5e-02

-3.0e+00

2.5e+02

2.4e-01

6.8e+01

7.8e-08

1.9e-02

U1_predicted

U1_variance

U1_relative_error

U2_predicted

U2_variance

U2_relative_error

-8.0e-04

1.1e-04

5.4e-41

2.5e-09

2.3e-16

1.8e-02

-1.0e-06

1.7e-03

5.8e-40 2e-8 4.1e-08

2.5e-16

1.6e-02

evrcum_predicted

evrcum_variance

evrcum_relative_error

sig11_predicted

sig11_variance

sig11_relative_error

-4.7e-080

5.7e-08

0.0e+00

8.1e-10

0.0e+00

5.7e-08

-2.4e+010

3.3e+01

7.3e-07102.0e+01

1.1e-08 0.04 7.4e-02

sig12_predicted

sig12_variance

sig12_relative_error

sig22_predicted

sig22_variance

sig22_relative_error

-4.4e+0104.5e+01

5.7e-07 5

1.2e+01

2.1e-08 0.02 4.7e-02

-8.7e-01

1.5e+02

1.4e-01

3.7e+01

2.8e-08

1.6e-02

U1_predicted

U1_variance

U1_relative_error

U2_predicted

U2_variance

U2_relative_error

-9.8e-04

3.2e-04

1.5e-15

8.5e-08

1.2e-06

6.3e-02

-8.6e-08

2.5e-03

8.3e-18

1.3e-06

3.7e-07

9.6e-02

evrcum_predicted

evrcum_variance

evrcum_relative_error

sig11_predicted

sig11_variance

sig11_relative_error

-1.6e-05

1.9e-14

0.0e+00

4.6e-09

0.0e+00

1.6e-05

-6.0e+01

05.3e+01

4.9e-06 50

1.4e+02

6.3e-07 0.2 3.6e-01

sig12_predicted

sig12_variance

sig12_relative_error

sig22_predicted

sig22_variance

sig22_relative_error

-6.1e+0105.8e+01

1.9e-05

4.8e+02

7.9e-080.2 3.8e-01

-4.4e+00

1.5e+02

4.5e+00

1.2e+03

1.9e-06

2.4e-01

U1_predicted

U1_variance

U1_relative_error

U2_predicted

U2_variance

U2_relative_error

-2.1e-03

0 1.2e-03

2.3e-13

1.2e-05

3.1e-05

5.3e-01

-1.3e-05

8.5e-03

4.2e-17 5e-5 9.9e-05

1.2e-07 0.5 9.8e-01

evrcum_predicted

evrcum_variance

evrcum_relative_error

sigl1_predicted

sig11_variance

sig11_relative_error

-2.2e-05 5e-5 1.4e-04

0.0e+00

2.6e-07

0.0e+00

9.7e-01

-1.2e+0209.5e+01

1.3e-0420003.6e+03

5.8e-07 0.5 9.4e-01

sig12_predicted

sig12_variance

sig12_relative_error

sig22_predicted

sig22_variance

sig22_relative_error

-8.1e+0108.6e+01

5.6e-04

1.5e+04

4.7e-060.51.0e+00

-1.9e+01

1.8e+02

1.9e+02

5.1e+04

8.3e-07

6.7e-01

u1

人

u2

p

train0

test1

ellipsoid

wedge

MndM

011

012

022

train0

test1

ellipsoid

wedge

quantile 0.95-reference

quantile0.95-reference

finiteelement

interpolationerror

quantile0.05-reference

finiteelementinterpolationerror

finiteelement

-7.5e+00

0

4.1e+00

interpolation error

finiteelementinterpolationerror

-2.2e+00

0

1.9e+00

~quantile0.05-reference

UX_reference

UX_predicted

UX_relative_error

-3.4e+00

50

8.0e+01

-3.4e+00

50

8.0e+01

1.0e-05

0.001

1.0e-01

Min:7.2e-08 Max:1.6e-01

UY_reference

UY_predicted

UY_relative

error

-6.0e+01

0

3.0e+01

-6.0e+01

0

3.0e+01

1.0e-05

0.001

1.0e-01

p_reference

p_predicted

p_relative_error

-2.5e+03

0

1.7e+03

-2.5e+03

0

1.7e+03

1.0e-05

0.001

1.0e-01

Min:1.8e-07 Max:1.2e-01

UX_reference

UX_predicted

UX_relative_error

-1.7e+01

6.6e+01

-1.7e+01

6.6e+01

1.0e-05

0.001

1.0e-01

UY_reference

UY_predicted

UY_relative_error

-6.0e+00

6.4e+01

-6.0e+00

6.4e+01

1.0e-05

0.001

1.0e-01

Min:4.7e-09Max:1.4e-01

p_reference

p_predicted

p_relative_error

-1.9e+03

06.7e+02

-1.9e+03-1000

06.7e+02

1.0e-05

0.001

1.0e-01

Min:5.3e-10Max:1.1e-02

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nAuthors present a GP-based method for predicting outcomes of a physical simulation under variations in geometry and physical parameters. The method relies on deterministic pre-processing of meshes and a GP on a reduced dimensional space. The pre-processing puts meshes of different structure, e.g., different number of nodes, on a common mesh so that function values on the meshes can be compared. One crucial point that allows this is to see coordinates as functions on the meshes. All function values on the common mesh coming from different meshes are fed into a PCA to reduce their dimensions. GP runs on the reduced dimensional representations.\n\nSTRENGTHS:\n1. The proposed method – to the best of my knowledge – is quite novel. \n2. Leveraging mesh interpolation and morphing to help compare meshes of different structure is a great approach. Relying on deterministic operations reduce variability so that fewer examples are enough to learn powerful models. Combining these two is a great idea. \n3. Use of GP instead of DL is another great aspect. Here, one can quantify the uncertainty in the predictions with ease and potentially identify out-of-distribution geometries or parameters. This leads to identification of cases where the model predictions are no longer reliable. I suggest authors to discuss this aspect as well and perhaps even do some experiments. \n4. The article is very well written. This is surely a complicated method that requires background knowledge. Even in that setting, authors did a fantastic job in explaining the details.\n\nWEAKNESSES:\n1. ML-based methods for PDE systems currently only act as rough predictions. The proposed method is no exception. The predictions are not as accurate as simulations. For applications where accuracy is needed, current ML-based methods – including this work – can only act as a initialization. I believe this needs to be clearly stated. \n2. In the introduction authors menthion the computational expense of traditional methods and their need for discretization. There are two aspects that needs to be mentioned. \n    a) The proposed method also needs good discretization since it uses sample simulations, i.e., training set, in a GP framework. If those samples are bad, predictions will be bad. \n    b) Traditional methods have guarantees over error bounds. This is a priceless aspect of these works. ML-based methods – including this one – does not provide any guarantees. \n3. There are two parts that can be better explained: \n        a) How do authors deal with arbitrary rigid body transformations? The technique seems to assume there are no rigid body transformations between different meshes. \n        b) What is p on line 157? \n4. Authors do not take into account Jacobian of coordinate transformations when morphing to the common mesh while computing new values of interest on the nodes. Ideally, such transformations should be taken into account when computing U values in Equation 4. It is true that authors do a backward transformation for the final predictions. However, they also put together values coming from different meshes in the GP on the common mesh. I think the properties of the kernel is quite important at this point. Otherwise, the U values on the common mesh may not be comparable. This is a major assumption here that needs to be discussed. \n5. There is a huge underlying smoothness assumption due to the use of GP. I believe the method assumes that small changes in the geometry will yield small changes in the simulation results. This holds only for certain problems and does not hold in others. This assumption needs to be discussed. Let me also point out that DL-based works do not even mention this. They are roughly black-box methods that do not even allow one to examine such assumptions. The proposed method is quite in good in that respect. \n6. Figure 8 requires relative errors. Instead, absolute errors are given and they seem to be around 10% at certain points.\n\nQUESTIONS:\nPlease read the weaknesses for the questions.\n\nLIMITATIONS:\nLimitations are not completely addressed. I noted these in the weakness section. \nI do not believe there are potential negative societal impacts that should be explicitly mentioned here.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a method that offers advantages in terms of computational efficiency by employing modal reduction compared to full models and utilizing Gaussian processing.\n\nHowever, despite these strengths, the paper falls short in several aspects.\n- The writing lacks clarity and coherence, with inconsistent descriptions and key information missing.\n- The organization of the paper appears disjointed as if written by different individuals without synchronization.\n- Furthermore, recent related works are overlooked, outdated benchmarks are used, and there is insufficient theoretical support nor enough experiments for the proposed method.\n\nAs a result, based on these limitations, I recommend rejecting this paper in its current form. **If significant improving in writing is conducted, I will re-consider**\n\n## After rebuttal\n- The authors improved their references, presentation, and empirical evaluation; Hence I increased my score to 5\n\nSTRENGTHS:\n- Save computational requirements by modal reduction compared to full models\n- Save computational requirements by using Gaussian Processing\n\nWEAKNESSES:\n- The writing is very confusing and casual, lacking key information and consistent descriptions:\n  - Authors claim they can handle huge graphs, but their maximum mesh has about 30K nodes, which is not huge.\n  - They don't provide mesh size information for the solid mechanics case.\n  - Inconsistent usage of \"nodes\" and \"vertices\".\n  - The meaning of \"finite element interpolation\" (FEM shape functions?) is not defined or explained.\n  - The phrase \"(line 39)...high stakes\" and why it prevents deployment is unclear.\n  - Line 83: \"Physics-based machine learning\" should be corrected to \"machine learning for physics\".\n  - Lines 87-88: Inaccurate and unnecessary information regarding the significance of each node.\n  - Line 216: What are \"classical engineering curves\"?\n  - Caption for Figure 3: Schematic plots illustrating the appearance of the four different test case geometries are needed.\n  - Section 4.3: Unclear meaning of \"use case\".\n\n- The paper lacks organization and coherence in both writing and technical/experimental descriptions. It seems that different sections were written independently without synchronization:\n  - Multiple duplicate subsections such as \"Physical setting and dataset\", \"Training (Implementation)\", and \"Results\" for each dataset should be combined into one.\n  - Introduction of multi-scale GNNs is scattered across different sentences (lines 94, 97-98), lacking logical structure.\n\n- Recent related works are missing. Papers such as these utilize morphing [1,2], and there are other recent works addressing challenges in GNN performance by multi-scale [3,4,5] that should have been mentioned.\n\n- Outdated benchmarks are used. Although the authors mention various GNN family methods, they compare their approach with an outdated GCNN. Additional comparisons to more recent methods such as MeshGraphNet and MSGNN-Grid (mentioned in their paper but not compared) should be included.\n\n- Insufficient theoretical support is provided for the proposed method. While formal proof is not necessary, the authors fail to discuss any theoretical foundations supporting the effectiveness of their approach.\n\n[1] Han Gao, Luning Sun, Jian-Xun Wang, \"PhyGeoNet: Physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state PDEs on irregular domain\", Link: https://www.sciencedirect.com/science/article/pii/S0021999120308536\n\n[2] Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, Anima Anandkumar, \"Fourier Neural Operator with Learned Deformations for PDEs on General Geometries\", Link: https://arxiv.org/abs/2207.05209\n\n[3] Lino, Mario and Fotiadis, Stathi and Bharath, Anil A and Cantwell, Chris D. “Multi-scale rotation-equivariant graph neural networks for unsteady Eulerian fluid dynamics”. Link: https://pubs.aip.org/aip/pof/article/34/8/087110/2847850\n\n[4] Cao, Yadi, Menglei Chai, Minchen Li, and Chenfanfu Jiang. \"Efficient learning of mesh-based physical simulation with bi-stride multi-scale graph neural network.\". Link: https://openreview.net/forum?id=2Mbo7IEtZW\n\n[5] Chen, Runfa and Han, Jiaqi and Sun, Fuchun and Huang, Wenbing. \"Subequivariant Graph Reinforcement Learning in 3D Environments\". Link: https://arxiv.org/abs/2305.18951\n\nQUESTIONS:\nSee the writing part in **Weaknesses**\n\nLIMITATIONS:\n- Limited to geometries with the same topology but different transformations or deformations.\n- The use of PCA, known for encoding linear correlations, poses challenges when dealing with highly complex geometries. Even the preprocessing step becomes challenging in such cases.\n  - Specifically, the method is only suitable for geometries that can be compressed effectively using a cost-effective PCA approach. This contradicts the paper's claim of being applicable in real engineering use cases.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nCurrent work developed a novel framework combining finite element interpolation, Gaussian process and dimension reduction techniques. It doesn't require the use of a graph network and can be trained on a CPU. The result is demonstrated on three industrial design optimization problems and two of them can beat the baseline graph neural networks.\n\nSTRENGTHS:\nOriginality: The model is novel in case of combining the three well-known blocks of finite element analysis, Gaussian process algorithm and dimension reduction techniques without the need to use a graph neural network. \n\nClarity: The writing is overall clear, with minor typos.\n\nQuality: The result demonstrated for the first two cases can outperform baseline graph neural network models.\n\nSignificance: The method is applied to several industrial design cases.\n\nWEAKNESSES:\nQuality: The third case, airfoil design is not compared with any baseline models. The error plots indicate the maximum error is large. Moreover, for the first two cases, a more advanced baseline model, like MeshGraphNet, need to be included to demonstrate the performance. More details in the question part. \n\nSignificance: The author only presents the simplest combination of the three blocks. For example, P1 finite element, Gaussian process and PCA for dimension reduction and assume it only works on the steady problem, more details in questions part.\n\nQUESTIONS:\n1. The reviewer has two concerns about the baseline selection. The first one is for the third case, there is no baseline to compare with, and the maximum error is large. The second concern is for the first two cases, the baseline already works very well for most of the evaluated metrics. For example, in Table 1, the baseline is already rated over 0.99 for all the metrics. In Table 2, only two baseline metrics are lower than 0.95. Though the proposed work is more computationally efficient in the demonstrated problems, the baseline graph neural network is more flexible since it can also handle unsteady problems. Therefore, it is hard to say that the developed model is better in practical applications. For the first concern, the reviewer would suggest adding more advanced baselines, like MeshGraphNet in \"LEARNING MESH-BASED SIMULATION WITH GRAPH NETWORKS\" for all three cases. For the second concern, the author would need to justify why the developed model is generally better than graph neural networks, either using theory or experiment.\n\n2. The second problem is about the framework. The novel part is the view of an innovative combination of blocks of classical methods. However, inside each block, only some simple variants are chosen,  P1 finite element, vanilla Gaussian process and PCA. As the authors listed, there could be other variants for each block, but they are not tested. The review concerns that it is not a trivial extension to include other complex variants and scale the framework to more complex engineering design problems. For example, for the airfoil design, when the design condition is complex, like supersonic/hypersonic flow, the simple PCA analysis is not likely to work and requires more advanced reduction techniques, like hyper reduction listed in Ref [11]. Moreover, the P1 basis is usually not accurate enough for complex problems.\\\\\n\n3. Some typos: Line 292: \"the first raw\" should be \"the first row\".\n\nLIMITATIONS:\nThe author assumes the model work on the steady problem, fixed topology.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

comments: SUMMARY:
The paper proposes a framework for solving physical problems defined over discretized geometrical spaces, such as meshes. While prior methods like graph neural networks have been the go-to tool for solving such problems, they need large datasets to train and don't output predictive uncertainties out of the box. This paper, thus takes a mesh morphing approach (assuming fixed topology) to project various meshes to a common support followed by classical dimensionality reduction techniques and Gaussian processes. This allows for dealing with large meshes, without knowing any parameterization and yields predictive uncertainties.

STRENGTHS:
The major strength of this work lies in the fact that it uses classical techniques such as Gaussian processes, PCA and mesh morphing to solve a key problem in learning-based simulation research which is the need for large datasets and lack of generalization. The paper cleverly uses the mesh morphing idea to eliminate the need to learn shape characteristics and focus purely on the features required to solve the physical problem. It is also interesting to note the use of the finite element method to arrive at common support, which again provides a fair degree of inductive bias to this framework leading to a higher generalization without needing a large dataset to train with, as is evident via a significant agreement between GCNN and MMGP in Figure 1. The results presented in the paper are sufficient to demonstrate the utility of this method.

WEAKNESSES:
Perhaps the biggest limitation of this method arises from the fact that it assumes a fixed topology. Since most domains are not so simple and homogeneous, I wonder if this is going to cause issues when this work is applied more generally.

QUESTIONS:
NA

LIMITATIONS:
NA
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-09

comments: SUMMARY:
This paper presents a method for solving physics problems in design optimization, with scalar and non-parameterized mesh as inputs. One main contribution of the work is to use a mesh morphing pretreatment with finite element interpolation. The other idea is to leverage shape embedding by dimensional reduction of coordinates that are seen as continuous fields over the geometrical support. 
This work seems to be able to handle large meshes and is easily trained on CPU with good interpretability.

STRENGTHS:
- The idea of using mesh morphing pretreatment with finite element interpolation seems to be effective as shown in the experiments.
- The technique of using the shape embedding by dimensional reduction reduces the computational cost and enables efficient computation on CPU.
- The results look nice.

WEAKNESSES:
I'm not an expert in this field and not familiar with the latest advances in physical simulation.
One weakness I can think of maybe that dimensionality reduction using PCA seems to be pretty standard and commonly used in all kinds of applications. If this is the main contribution, the technical contribution may be limited.

QUESTIONS:
I'm not an expert in this field. Just curious, is it possible to use the particle-based method like MPM to solve the physics problems shown in this paper? If yes, what is the advantages of the proposed method?

LIMITATIONS:
No limitations are provided.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-26