Physics-Embedded Neural Networks: Graph Neural PDE Solvers with Mixed Boundary Conditions
Authors: Masanobu Horie
NAOTO MITSUME
Venue: NeurIPS 2022 Accept (2022)
Keywords: PDE
numerical analysis
physical simulation
graph neural network
equivariance

Abstract: Graph neural network (GNN) is a promising approach to learning and predicting physical phenomena described in boundary value problems, such as partial differential equations (PDEs) with boundary conditions. However, existing models inadequately treat boundary conditions essential for the reliable prediction of such problems. In addition, because of the locally connected nature of GNNs, it is difficult to accurately predict the state after a long time, where interaction between vertices tends to be global. We present our approach termed physics-embedded neural networks that considers boundary conditions and predicts the state after a long time using an implicit method. It is built based on an $\mathrm{E}(n)$-equivariant GNN, resulting in high generalization performance on various shapes. We demonstrate that our model learns flow phenomena in complex shapes and outperforms a well-optimized classical solver and a state-of-the-art machine learning model in speed-accuracy trade-off. Therefore, our model can be a useful standard for realizing reliable, fast, and accurate GNN-based PDE solvers. The code is available at https://github.com/yellowshippo/penn-neurips2022.

Physics-Embedded Neural Networks: Graph Neural PDE Solvers with Mixed Boundary Conditions
Masanobu Horie RICOS Co. Ltd. University of Tsukuba
Naoto Mitsume
University of Tsukuba mitmmmekkz.co tss.ummak.tsuukua.a... jp.
horie@ricos.co co.
Abstract
Graph neural network (GNN) a promising approach learning and predicting physical phenomena described in boundary value problems, such as partial dif- ferential equations (PDEs) with boundary conditions. However, existing models inadequately treat boundary conditions essential for the reliable prediction of such problems. In addition, because of the locally connected nature of GNNs, is difficult to accurately predict the state after a long time, where interaction between vertices tends to be global. We present our approach termed physics eemeeddd neural networks that considers boundary conditions and predicts the state after a long time using an implicit method. is built based on an E(n) eequivarian GNN, resulting in high generalization performance various shapes. We demon- strate that our model learns flow phenomena in complex shapes and outperforms well-optimized classical solver and state-of-the-e machine learning model in speed-accuracy trade-off. Therefore, our model can be a useful standard for realizing reliable, fast, and accurate GNN-hased PDE solvers. The code is available at
neurips2022
Introduction
Partial differential equations (PDEs) are of interest to many scientists because of their application in various fields such as mathematics, physics, and engineering. Numerical analysis is used solve PDEs because most PDE problems in real life cannot be solved analytically. For example, predicting fluid behavior in complex shapes is an essential topic because it helpful for product design, disaster reduction, weather forecasting, and many others; however, it is difficult problem and takes time to solve using classical solvers. Machine learning is promising approach to predicting such phenomena because it can utilize data similar to the state to be predicted, while classical solvers cannot.
However, the main challenge in dealing with complex phenomena such as fluids is guarantee generalization performance because possible states in complex systems can be huge and may not be covered using purely data-driven approach. Therefore, we must apply appropriate inductive biases to machine learning models. Many approaches successfully introduced various inductive biases such local connectedness using graph neural networks (GNNs) and symmetry under coordinate transformations using equivariance.
While these methods have made great progress in solving PDEs using machine learning, there is still room for improvement. First, there is need for an efficient and provable way to respect boundary conditions like Dirichlet and Neumann, i.e., mixed boundary conditions. Rigorous fulfillment of Dirichlet boundary conditions is indispensable because they are hard constraints and different Dirichlet conditions correspond to different problems users would like to solve. Second, there need to reinforce the treatment of global interaction to predict the state after a long time, where interactions
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Dirichlet layer NeumannlaoGCN,
E(n)) Equiviquirrr Graph Neural Network with boundary conditions
Neural nonlinear solver
Dirichlet boundary condition
Encoded Dirichlat
boundary condition
Encoder
Boundary
Pseudoinverse Decoder
Enooder
Neumann boundary condition
Encoded Neumann
Input feature
boundary condition
Encoded feature
Oulpul feature
Overview of the proposed method. On decoding input features, we apply boundary encoders to boundary conditions. Thereafter, we apply a nonlinear solver consisting of an -eqivvanan graph neural network in the encoded space. Here, we apply encoded boundary conditions for each iteration of the nonlinear solver. After the solver stops, we apply the pseudoinverse decoder to satisfy Dirichlet boundary conditions.
Figure 1:
tend to be global. GNNs have excellent generalization properties because of their locally-comnected nature; however, they may miss global interaction due to their localness.
We propose physics-embedded neural networks (PENNs), a machine learning framework to address these issues by embedding physics in the models. We build our model based on IsoGCN (Horie et al. 2021), lightweight -eqiivrriaa GNN to reflect physical symmetry and realize fast prediction. Furthermore, we construct a method consider mixed boundary conditions. Finally, we reconsider a way to stack GNNs based on a nonlinear solver, which naturally introduces the global pooling GNNs the global interaction with high interpretability. In experiments, we demonstrate that our treatment of Neumann boundary conditions improves the predictive performance of the model, and our method can fulfill Dirichlet boundary conditions with no error. Our method also achieves state-of-ther oh-tth-rttttttt art performance compared to classical, well well-optimized numerical solver and baseline machine learning model in speed-accuracy trade-off. Figure shows the overview the proposed model. Our main contributions are summarized as follows:
We construct models to satisfy mixed boundary conditions: the boundary encoder, Dirichlet layer, pseudoinverse decoder, and NeumannIsoGGCN (NIsoGCN). The considered models show provable fulfillment of boundary conditions, while existing models cannot.
We propose neural nonlinear solvers, which realize global connections to stably predict the state after long time.
We demonstrate that the proposed model shows state-of-the-art performance in speed- accuracy trade-off, and all the proposed components are compatible with E(n)-equivariaace.
2 Background and related work
In this section, review the foundation of PDEs to clarify the problems we solve and introduce related works where machine learning models are used to solve PDEs.
2.1 Partial differential equations (PDEs) with boundary conditions
general form of the d-dimensional temporal PDEs that consider can be expressed follows:
du Øt (t, x)) (())(t..))
(t, xt)z) (0,T) x Ω
(1)
u(t = 0, x) (n(
x E Ω
(2)
u(t, x) = á(t, x)
(t, x)) (0,T) x diriiblle (t,x) E (0. T) x 0n Neumann:
(3)
f(Vu(t, nx))) = 0
(4)

---

where Ω is the domain, 0Ω is the boundary of Ω and Piririhllet and NeNemmanu are boundaries with Dirichlet and Neumann (mixed) boundary conditions. is known function, and 'D a known nonlinear differential operator, which can be nonlinear and contain spatial differential operators
(see equation 18 for an example of D). r2(x)) is the normal vector at a: E 802. Equation 3 called the Dirichlet boundary condition, where the value on Diriehlee is set as constraint. Equation g corresponds to the Neumann boundary condition, where the value of the derivative u in the direction of is set on Neumann rather than the value of u. When u (0, Q RR satisfies Equations M
it is called the solution the (initial-) boundary value problem.
2.1.1 Discretization
PDEs are defined in a continuous space to make differentials meaningful. Discretization can be applied in space and time so that computers can solve PDEs easily. In numerical analysis of complex- shaped domains, we commonly use meshes (discretized data of shapes), which can be regarded as graphs. We denote the position of the ith vertex as æ, and the value of a function f at the x; as
The simplest method to discretize time is the explicit Fuler method formulated as:
ull At, a;) u(t, ax;) (t,) at)
(5)
which updates (t, æ;) with a small increment D(u) (t, Dt) Another way to have time discretiza- tion is the implicit Euler method formulated as:
u(t+ At, x;) = utt, x,) + D At, Att,
(6)
which solves Equation rrathrr than simply updating variables ensure the original PDE is satisfied numerically. The equation can be viewed as nonlinear optimization problem by formulating it as:
R(v) = u (1, D(v)
Solve, ((v) 0, vi,
(8)
where R(w) is the residual vector of the discretized PDE. The solution of Equation S corresponds to uit + At, x). By letting V R for appropriate o, solving Equation g corresponds to optimizing & an (f x ni)-dimensional space, where n is the number of vertices in the considered mesh. A simple way to solve such an optimization problem is to apply gradient descent formulated as:
(9) where (+) E IR is determined using line search. However, due the high computational cost of the search, a can be fixed to a small value, which corresponds to the explicit Euler method with the time step size a At. [Barzila Borwein (1988) suggested another simple yet effective way to determine the step size using a two-point approximation to the secant equation underlying quasi-Newton methods.
0 == utt,)), )) (f+1)) = v (+) -a ( R(
2.2 Neural PDE solvers
We review machine learning models used to solve PDEs called neural PDE solvers, typically formu- lated as uttn+1, x;) R (t. æx)) for (tn.2,) E (to, t... x Ω where FNN is a machine learning model.
2.2.1 Physics-informed neural networks (PINNs)
Raissi et al. (2099) made pioneering work combining PDE information and neural networks, called PINNs, by adding loss to monitor how much the output satisfies the equations. PINNs can be used
to solve forward and inverse problems and extract physical states from measurements (Pang et al.
2019 Mao et al. 2020: Cai et al. 202T). However, PINNs' outputs should be functions of space because PINNs rely on automatic differentiation to obtain loss regarding PDEs. This design constraint significantly limits the model's generalization ability because the solution of PDE could be entircly different when the shape of the domain or boundary condition changes. Besides, the loss reflecting PDEs helps models learn physics at training time; however, prediction by PINN models can be out of physics because of lacking PDE information inside the model. Therefore, these methods are not applicable in building models that generalizable over shape and boundary condition variations. As seen in Section 3 our model contains PDE information inside and does not take absolute positions of vertices, thus resulting in high generalizability (See Figure 3
Strictly speaking, components of the PDE e.g. D and Ω can be different before and after discretization. However, we the same notation regardless of discretization to keep the notation simple.
3

---

2.2.2 Graph neural network based PDE solvers
As discussed in Section 2.1.1 one can regard a mesh as a graph. GNNs can take any graphs as
inputs (Gori et al. 2005] Scarselli et al. 2008 Kipt & Welling 2017 Gilmer et al. 2017). having the possibility to generalize over various graphs, I.e. meshes. Therefore, GNNs are strong candidates for learning mesh-structured numerical analysis data, as seen in Alet et al. (2019); Chang & Cheng (2020); Pfatf et al. (202T). (Brandstetter et al. (2022) advanced these works for efficient and stable prediction. Their method could also consider boundary conditions by feeding them to the models as inputs. Here, one could expect the model to learn to satisfy boundary conditions approximately, while there is no guarantee to fulfill hard constraints such as Dirichlet conditions. In contrast, our model ensures the satisfaction of boundary conditions. Besides, most GNNs use local connections with a fixed number of message passings, which lacks consideration of global interaction. We suggest an effective way incorporate global connection with GNN through the neural nonlinear solver.
2.2.3 Equivariant models
In addition to GNNs, another essential concept to help machine learning models generalize is equivariance. Equivariance is characterized by using group action as f(g x) = g f(x) for f and g E G acting on and Y. In particular, -(n)-equivariance is essential to predict the solutions of physical PDEs because it describes rigid body motion, i.e.. translation, rotation, and reflection. Ling et al. (2016) and [Wang et al. (202T) introduced equivariance to a simple neural network and CNN to predict flow phenomena. Both works showed that equivariance improved predictive and generalization performance compared to models without equivariance. Horie et al. (2021) proposed F(n) eequivariant GNNs based on GCNs (Kipf & Welling 2017), called IsoGCNs. form of their model is formulated as:
[V ~ [[=)
27
0; [
1/3
D3 x;
W. (10)
x
[ x;
x xill
x;
JEN
JEN.
where Mf is the neighborhood the ith vertex, @ is the tensor product operator, and W a trainable matrix acting feature index. Here, we denote TsoGCNo->1 an IsoGCN layer that converts the input scalar (rank-0 tensor) field &i to the output vector (rank-1 tensor) field. This layer corresponds to the gradient operator, which helps learn PDEs because spatial derivatives such gradient play an essential role in PDEs. They applied the model to the heat equation problem, showing high predictive performance and fast prediction, while boundary condition treatment was out of their scope.
3 Proposed method
We present our model architecture. We adopt an pooeee-process-eeooee architecture, proposed by Battaglia et al. (2018). which has been applied successfully in various previous works, c.g., HHoie et al. (2021) Brandstetter et al. (022). Our key concept is to encode input features, including information on boundary conditions, apply GNN-based nonlinear solver loop reflecting boundary conditions in the encoded space, then decode carefully to satisfy boundary conditions in the output space.
3.1 Dirichlet boundary model
As demonstrated theoretically and experimentally in literature (Hornik 1991 Cybenko 1992
Nakkiran al.) 202T). the expressive power of neural networks comes from encoding in a higher- dimensional space, where the corresponding boundary conditions are not trivial. However, if there are no boundary condition treatments in layers inside the processor, which resides in the encoded space, the trajectory of the solution can be far from the one with boundary conditions. Therefore, boundary condition treatments in an encoded space are essential for obtaining reliable neural PDE solvers that fulfill boundary conditions.
To ensure the same encoded space between variables and boundary conditions, the same encoder for variables and the corresponding Dirichlet boundary conditions, which we term the boundary encoder, as follows:
h fenoode (u:) in Q h; -f fencode (ū.) on M&pirichle
(11)

---

One can easily apply Dirichlet boundary conditions in the aforementioned encoded space using the Dirichlet layer defined as:
DirichletLayer(h.)) = {
his D; $ Diriihhhee
(12)
h2 x; DiDirichle
This process is necessary return the state respecting the boundary conditions after some operations in the processor, which might disrespect the conditions.
After the processor layers, we decode the hidden features using functions satisfying:
fdecude fencode (..) = út; on Piriiclle
(13)
This condition ensures that the encoded boundary conditions correspond to the ones in the original physical space. Demanding that Equation holds for arbitrary û; obtain faccode o fencode Id., resulting in A = eencode which we call the pseudoinverse decoder. It is pseudoinverse because fencode: in particular encoding in a higher- sigee--imensional space, may be invertible, Therefore, we construct f fencode using pseudoinverse matrices. For more details, see Appendix A.1
3.2 Neumann boundary model
Matsunaga al. (2020) proposed wall boundary model to deal with Neumann boundary conditions for the least squares moving particle semi-implicit (LSMPS) method Tamai Koshizukal (004)), a framework to solve PDEs using particles. The LSMPS method is the origin of the IsoGCN's gradient operator, so one can imagine that the wall boundary model may introduce a sophisticated treatment of Neumann boundary conditions into IsoGCN. We modified the wall boundary model to adapt to the situation where the vertices are on the Neumann boundary, which differs from the situation of particle simulations (see Appendix A.2 for more details). Our formulation of IsoGCN with Neumann boundary conditions, which is termed NeumannIsoGCN (NIsoGCN), is expressed as:
NIsoGCN 0-11 (v) M:
w/
Dj
wang
W
(14)
2;
aay
xzll
M; =
Σ LEN;
T
21
+ w. & n2:
(15)
x;
|
x.
where I, is the value of the Neumann boundary condition x, W is trainable matrix, and we > is an untrainable parameter to control the strength of the Neumann constraint. As - co, the model strictly satisfies the given Neumann condition the direction n., while the directional derivatives in the direction of (xx, æ;) tend to be relatively neglected. Thus, we keep the value of w; moderate to consider derivatives in both n and æ directions. In particular, we set w; 10.0, assuming that flat surface in 3D space.
around ten vertices may virtually exist "outside" the boundary
NIsoGCN is a straightforward generalization of the original IsoGCN by letting n. when 2; & 09Neuman This model can also be generalized to vectors or higher rank tensors, similarly to the original IsoGCN's construction (see Appendix A.2 Therefore, NIsoGCN can express any spatial differential operator, constituting 'D in PDEs.
3.3 Neural nonlinear solver
can regard solving PDEs optimization. Here, we adopt the Barzilai-Borwein method (Barzilai & Borwein 1988) to solve Equation in the encoded space. In our case, the step size (i)) gradient descent approximated as:
As reviewed in Section 2.
(h (1-1), R(h ((h(1-1)) 11) (R(h0)) ((h(111)) R(h()) R(h ((h()-1)))
RJ BBB
(16)
wherer R(h) is the residual vector in the encoded space and (f.g)n = Ex, en fxx) g(x;) denotes the inner product over the mesh. Because the inner product is taken all over the mesh (graph),
(2) computing aBB corresponds to global pooling. With that view, one can find similarities between Equation 2 and deep sets /aheer al. 2017), which a successful method to learn point cloud data and has strong background regarding permutation equivariance. For more details, see Appendix A.3
5

---

Table MSE loss (# the standard enror of the mean) on test dataset of gradient prediction, @Neumanu is the loss computed only on the boundary where the Neuman condition is set.
Method
Vx(x10 1000
NNemannn (x10-3) 3)
Original IsoGCN
192.72 + 1.69
1390.95 + 7.93
NIsoGCN (Ours)
6.70 0.15
3.52 0.02
Grudurc moOHod
25
25
Dffvenns of gradentt
Ground truth
Original IsoGCN
NIsoGCN
Gradient field (top) and the magnitude of error between the predicted gradient and the ground truth (bottom) of test data sample, sliced on the center mesh.
Figure 2:
Our aim is to use Equation 16 approximating the nonlinear differential operator in Equation 7 with NIsoGCN. By doing this, expect the processor consider both local and global information, which may have an advantage over simply stacking GNNs corresponding to the explicit method as discussed in Section 2.1.I Combinations of solvers and neural networks are already suggested in, c.g., NeuralODE (Chen et al. 2018). The novelty of our study is the extension of existing methods for solving PDEs with spatial structure and the incorporation of global pooling into the solver, enabling us to capture global interaction, which we refer to as the neural nonlinear solver. Finally, the update from the state at the ith iteration (1) to the (i + 1)th in the neural nonlinear solver is expressed as:
h (++1) DirichletLayer h
O'Bi
(6) h h (0) DNIsOGEN (h (1) (1A)))
(17)
where (0) is the encoded -t,.)) reflecting Equation 9 and DDIISGGNN is E(n) -qquivarra GNN reflecting the structure of using differential operators provided by NIsoGCN, Here, Equation 17 enforces hidden features satisfy the encoded PDE, including boundary conditions, motivating call our model physics-embedded neural networks because it embeds physics (PDEs) in the model rather than in the loss.
4 Experiments
Using numerical experiments, we demonstrate the proposed model's validity, expressibility, and computational efficiency. We use two types of datasets: 1) the gradient dataset verify the correctness of NIsoGCN and 2) the incompressible flow dataset to demonstrate the speed and accuracy of the model. We also present ablation study results corroborate the effectiveness of the proposed method. The implementation of our model is based on the original IsoGCN's code Our implementation is available online All the details of the experiments and another simple experiment can be found in Appendix and D
4.1 Gradient dataset
As done in Horie et al. (2221) we conducted experiments to predict the gradient field from given scalar field to verify the expressive power of NIsoGCN, We generated cuboid-shaped meshes
Apache License 2.0.
Apache License 2.0.

---

Table 2: MSE loss (= the standard error of the mean) on test dataset of incompressible flow. "Trans." is "Yes," it means evaluation is done on randomly rotated and transformed test dataset. 'Dirichlet the loss computed only on boundary where the Dirichlet condition is set for each and p. MP-PDE's results are based on the time window size equaling 40 as it showed the best performance in the tested MP-PDEs. For complete results, see Table 4
Method
Trans.
P (x10-9)
dADricchlet (x10-*)
PDirichlet (x (x100-)
(x10-4)
MP-PDE TW 20
No
1.30 + 0.01
1.32 0.01
0.45 + 0.01
0.28 + 0.02
Yes
1953.62 + 7.62
281.86 + 0.78
924.73 + 6.14
202.97 + 3.81
No
4.36 + 0.03
1.17 0.01
0.00 0.00
0.00 0.00
PENN (Ours)
Yes
4.36 +: 0.03
1.17 + 0.01
0.00 0.00
0.00 + 0.00
1,5e400
1.2
Valooity magnitude
F
f
0.6
0.4
0.2
0.0e+00
1.0e+00
B
F
0.4
0
0A4
-06
110e+00
Ground truth
MP-PDE (TW=20)
PENN (Ours)
Figure 3: Comparison of the velocity field (top rows) and the pressure field (bottom two rows) without (first and third rows) and with (second and fourth rows) random rotation and translation. PENN prediction consistent under rotation and translation due the Equii-qquvar nature of the model, while MP-PDE's predictive performance degrades under transformations.
randomly with 10 20 cells in the X, Y, and Z directions. We then generated random scalar fields over these meshes using polynomials of degree 10 and computed their gradient fields analytically. Our training, validation, and test datasets consisted 100 samples. Table | and Figure 2 show that the proposed NIsoGCN improves gradient prediction, especially near the boundary, showing that our model successfully considers Neumann boundary conditions.
4.2 Incompressible flow dataset
We tested the expressive power our model by learning incompressible flow in complex shapes. The corresponding nonlinear differential operator is denoted as:
1 Re
DNs(u) = - (u- V)u
V.
Vu
Vp.
(18)

---

with the incompressible condition V u 0, where, in the present case, u is the flow velocity field, p is the pressure field, and Re the Reynolds number.
4.2.1 Data
To generate the dataset, we first generated pseudo-2D shapes, with one cell in the Z direction, by changing design parameters, starting from three template shapes. Thereafter, we performed numerical analysis using a classical solver, OpenFOAM With with A/ 10-3, and the initial conditions were the solutions of potential flow, which can be computed quickly and stably using the classical solver. The Reynolds number Re was around 10 The linear solvers used were generalized geometric-aleerre multi-grid for and the smooth solver with the Gauss smoother for us. Template shapes, design parameters, and boundary conditions used can found Appendix
To confirm the expressive power of the proposed model, we used coarse input meshes for machine learning models. We generated these coarse meshes by setting cell sizes roughly four times larger than the original numerical analysis. We obtained ground truth variables using interpolation. The task was to predict flow velocity and pressure fields t 4.0 using information available before numerical analysis, e.g., initial conditions and the geometries of the meshes. Training, validation, and test datasets consisted of 203, 25, and 25 samples, respectively. We generated the dataset by randomly rotating and translating test samples to monitor the generalization ability of machine learning models.
4.2.2 Machine learning models
We constructed the PENN model corresponding to the incompressible Navier-Stokes equation. In particular, we adopted the fractional step method, where the pressure field was also obtained as a PDE solution along with the velocity field. We encoded each feature in a 4, 8, or 16-dimensional space. After features were encoded, we applied a neural nonlinear solver containing NeumanIsoGCNs and Dirichlet layers, reflecting the fractional step method (See Equations 51 and 52) Inside the nonlinear solver's s loop, we had a subloop that solved the Poisson equation for pressure, which also reflected the considered PDE (See Equation 50). We looped the solver for pressure five times and four or eight times for velocity. After these loops stopped, we decoded the hidden features to obtain predictions for velocity and pressure, using the corresponding pseudoinverse decoders.
For the state-of tftte-of-th-art baseline model, we selected MP-PDE (Brandstetter et 2022) as also provides way to deal with boundary conditions. We used the authors' code with minimum modification to adapt to the task. We tested various time window sizes such as 2, 4, IO, and 20, where one step corresponds to time step size At = 0.1. With changes in time window size, we changed the number of hops considered in one operation of the GNN of the baseline to have almost the same number of hops visible from the model when predicting the state at / 4.0. The numbers of hidden features, 32, 64, and 128, were tested. All models were trained for up to 24 hours using one GPU (NVIDIA A100 for NVLink 40GiB HBM2).
4.2.3 Results
Table and Tigure the comparison between MP-PDE and PENN. The predictive performances of both models are at almost the same level when evaluated on the original test dataset. The results show the great expressive power of the MP-PDE model because we kept most settings at default as much as possible and applied no task-specific tuning. However, when evaluating them on the transformed dataset, the predictive performance of MP-PDE significantly degrades. Nevertheless, PENN shows the same loss value up to the numerical error, confirming our proposed components are compatible with -equivarriii In addition, PENN exhibits no error on the Dirichlet boundaries, showing that our treatment of Dirichlet boundary conditions is rigorous.
Figure 4 shows the speed-accuracy trade-off for OpenFOAM, MP-PDE, and PENN. We varied mesh cell size, the time step size, linear sover settings for OpenFOAM to have different computation speeds and accuracy. The proposed model achieved the best performance in speed-accuracy trade-off between all the tested methods under fair comparison conditions.
https: htops://www..r http://www.ooeffoooeo mreefoam.comm com
attps:///itthe won/bbannstetter htto://rtttssstttrrooooaam//-see Johannes/MP-Beureee -olerslvers
8

---

Table 3: Ablation study on the incompressible flow dataset. The value represents MSE loss (#: standard eπror of the mean) on the test dataset. "Divergent" means the implicit solver does not converge and the loss gets extreme value (~ 10
p (x10-3
Dirichlet (x10-*)
PDirichlet (x10
Method
( X x110))
Without encoded boundary
Divergent
Divergent
Divergent
Divergent
Without boundary condition the neural nonlinear solver
65.10 + 0.38
21.70 + 0.09
0.00 = 0.00
0.00 + 0.00
Without neural nonlinear solver
31.03 + 0.19
9.81 + 0.04
0.00 = 0.00
0.00 + 0.00
Without boundary condition input
20.08 + 0.21
3.61 0.02
59.60 0.89
1.43 0.05
Without Dirichlet layer
8.22 + 0.07
1
18.20 = 0.28
0.38 + 0.01
Without pseudoinverse decoder
8.91 + 0.06
2.36 0.02
1.97 0.06
0.00 0.00
Without pseudoinverse decoder with Dirichlet layer after decoding
6.65 + 0.05
1.71 0.01
0.00 = 0.00
0.00
PENN
4.36 + 0.03
1.17
0.00 = 0.00
0.00
Table g presents the results of the ablation study. Comparison between models with and without the proposed components shows that the proposed com- ponents, i.e., the boundary encoder, Dirichlet layer, pscudoinverse decoder, and neural nonlinear solver, significantly improve the models. The neural nonlin- solver in the encoded space turned out to have the biggest impact on the performance, while the Dirich- let layer ensured reliable models that strictly respect Dirichlet boundary conditions.
10
11
10P
POW Dunk MP
HOUM
10
Conclusion
Tosa MBE
Figure 4: Comparison of computation time and total MSE loss (u and p) on the test dataset (with and without transformation) be- tween OOen MP-PDE. and PENN. The eттоr bar represents the standard error of the mean. All computation was done us- ing one core of Intel Xeon CPU ES-2695 v2@2.40GHz. Data used plot this figure are shown in Tables 5 6 and
We have presented ((n))equivarian GNN-based neural PDE solver, PENN, which can fulfill bound- ary conditions required for reliable predictions. The model has superiority embedding the information of PDEs (physics) in the model and speed-accuracy trade-off. Therefore, our model can useful stan- dard for realizing reliable, fast, and accurate GNN- based PDE solvers. Although the property of our model is preferable, it also limits the applicable do- main of the model because we need to be familiar with the concrete form of the PDE of interest to construet the effective PENN model. For instance, the proposed model cannot exploit its potential to solve inverse problems where explicit forms of the governing PDE are not available for such tasks. Therefore, combining PINNs and PENNs could be the next direction of the research community.
6 Potential negative societal impacts
We have built a foundation to leam PDEs in a steerable manner rather than focusing on a specific application. Because of that, we envisage minimal risk of direct abusing our present work. However, as mentioned in the introduction, solving PDEs has many impacts on various domains, from both positive and negative aspects. Thus, our work and possible successive ones may be abused, aiming to harm lives and the environment. Therefore, the research community, including us, must be careful in using them and control the research direction to prevent abusing these technologies.

---

Acknowledgments and Disclosure of Funding
This work was supported by PRESTO, Japan Science and Technology Agency, Grant Number JPMJPR2109, Japan and by the New Energy and Industrial Technology Development Organization, Grant Number JPNPI Japan.
References
Ferran Alet, Adarsh Keshav Jeewajee, Maria Bauza Villalonga, Alherto Rodriguez, Tomas Lozano-Perez and Leslic Kaelbling. Graph element networks: adaptive, structured computation and memory. In ICMI., 2019.
Jonathan Barzilai and Jonathan M Borwein. Two-point step size gradient methods. IMA journal of numerical analysis, 8(1):141-148, 1988.
Peter W Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, al. Relational inductive biases, deep learning, and graph networks. arXiv preprint 1806. 012.8000001611... 2018.
Johannes Brandstetter, Daniel E. Worrall, and Max Welling. Message passing neural PDE solvers. In Inter- national Conference on Learning Representatinss, 2022. URI. httppaaa//oopeeeeiiieeee
VSi*3HPYKSU
Shengze Cai, Zhicheng Wang, Frederik Fuest, Young Jin Jeon, Callum Gray, and George Em Karniadakis.
Flow over an espresso cup: inferring velocity and pressure fields from tomographic background oriented schlieren via physics-informed neural networks. Journal of Fluid Mechanics, 915, 2021.
Kai-Hung Chang and Chin-Yr Yi Cheng. Learning simulate and design for structural engineering. arXiv preprint WxXi::000.09900, 2020.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
George Cybenko. Approximation by superpositions of sigmoidal function. Math. Control. Signals Syst., 5(4): 455, 1992.
Justin Gilmer, Samuel S Schoenholz, Patrick Riley, Oriol Vinyals, and George Dahl. Neural message passing quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pp. 1263-1272. JMLR. JMI.R. org, 2017.
Marco Gori, Gabriele Monfardini, and Franco Scarselli. A new model for learning in graph domains. In Proceedings, 2005 IEEE International Joint Conference Neural Networks, 2005., volume 2, pp. 729-734. IEEE, 2005.
Deguang Han, Keri Kornelson, Eric Weber, and David Larson. Frames for undergraduattes, volume 40. American Mathematical Soc., 2007.
Masanobu Horie, Naoki Morita, Toshiaki Hishinuma. Yu Ihara, and Naoto Mitsume. Isometric transforma- tion invariant and equivariant graph convolutional networks. In International Conference on Learning Representations, 2021. URL http:t:ee/opeeeeviiee.eeeee net.:forrmmiddd-X003330099
Kurt Hornik. Approximation capabilities 1991.
cappiilittion of multilayer feodforward networks. Neural networks, 4(2):251-257,
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.698,, 2014.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017. URL https:///openreriew..eeo id=SSU4ayYgl
Julia Ling, Andrew Kurzawski, and Jeremy Templeton. Reynolds averaged turbulence modelling using deep neural networks with embedded invariance, Journal of Fluid Mechanics, 807:155 107:165-166, 2016.
Zhiping Mao, Ameya D Jagtap, and George Em Karniadakis. flows. Computer Methods Applied Mechanics and Engineering, 360:112789 2020.
neural networks for high-speed
10

---

Takuya Maisunaga, Axel Södersten, Kazuya Shibata, and Seiichi Koshizuka. Improved treatment of wall boundary conditions for a particle method with consistent spatial discretization. Computer Methods in Applied Mechanics and Engineering. 358:1 12224, 2020.
Prectum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Tlya Sutskever. Deep double descent: Where bigger models and more data hurt. Journal of Statistical Mechanics: Theory and Experiment. 2021(12)112403, 2021.
Guofei Pang, Lu Lu, and George Em Karniadakis. fpinns: Fractional physics-informed neural networks. SIAM Journal on Scientific Computing, 100 41(4)::2603-A2266, 2019.
Tobias Pfaff, Meire Fortunato, Alvaro Sanzaez-Gonzalez, and Peter Battaglia. Learning mesh-based simulation with graph networks. In International Conference on Learning Representations, 2021. URL https //openreview. www. fooooomiddrooolii
XP
Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informat informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
Siamak Ravanbakhsh, Jeff Schneider, and Barnabás Póczos. Equivariance through parameler-sharing In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 2892 2901. PMLR, 06 11 Aug 2017. URL
mlr. press/v700 httsss/yOOrraaannnn rwwaaaakaaaaaaaaa....aaa html
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner. and Gabriele Monfardini. The graph neural network model. IEEE Trunsactions on Neural Networks, 20(1):61-80 2008.
Tasuku Tamai and Seichhi Koshizuka. Least squares moving particle semi-implicit method. Computational Particle Mechanics, 13))2277-305, 2014.
Rui Wang, Robin Walters, and Rose Yu. Incorporating symmetry into deep dynamics models for improved generalization. In International Conference on Learning Representations, 2021. BtLLhtttss///ppereries
net/forumPid=wta 8Hx2KD
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos. Russ R Salakhutdinov, and Alexander J Smola. Deep sets, Advances in neural information processing systems, 30, 2017.
Checklist
1. For all authors...
(a) Do the main claims made the abstract and introduction accurately reflect the paper's 's contribu- tions and scope? [Yes] See the abstract and Section [
(b) Did you describe the limitations of your work? [Yes] See Section 5
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See Section D
(d) Have you read ethics review guidelines and ensured that your paper conforms to them? [Yes]
Our paper, in particular Section O is written in accordance with the guideline.
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (cither in the supplemental material or as URL)? YYes) See Footmote 3
(b) Did you specify all the training details (c.g., data splits, hyperparameters, how they were chosen)?
[Yes] See Appendix B C
D
and Footnote 3
(c) Did you report error bars (c.g., with respect to the random seed after running experiments
multiple times)? [Yes] See Section - Table 2 and Figure f
(d) Did you include the total amount of compute and the type of resources used (e.g.. type of GPUs, internal cluster, cloud provider)? [Yes] See Section
4. If you are using existing assets (e.g.. code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes] See Footnote 2
(b) Did you mention the license of the assets? [Yes] See Footnote
11

---

(c) Did you include any new assets either in the supplemental material or URL? [Yes] See Section Table D B and Figure
(d) Did you discuss whether and how consent obtained from people whose data you're ing/curating? [N/A] The dataset we used the paper is generated by ourselves.
(c) Did you discuss whether the data you using/curating contains personally identifiable infor- mation offensive content? [N/A] The dataset is based on numerical analysis, so there no concern about personally identifiable information nor offensive content.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
12


Reviews:
[{"review_id": "jEYVG5qatn6", "reviewer": "Reviewer_AnrD", "summary": "The paper presents a neural PDE solver based on adn encode-process-deode architecture that respects boundary conditions thanks to a novel GNN-based gradient operator. Other than the proposed version of an E(n)-equivariant GNN nonlinear solver they also propose a different encoding process for boundary condition treatments in the encoded space. Experiments comprise prediction of the gradient field from a given scalar field to verify the expressiveness of the proposed version of the GNN-based gradient operator; and the task of learning incompressible flow. Other than respecting boundary conditions by construction, results show important improvement with respect to the state of the art neural PDE solvers.", "questions": "Can you elaborate more on the Dirichlet encoder-decoder? I do not understand whether it is as simple as it looks or perhaps there is something more. My understanding is that the encoder distinguish betweenboundary nodes and nodes that are not on the boundary and just apply a the pseudoinverse transformation on boundary nodes for decoding. In this case I don’t understand its role in performing better predictions other than just trivially enforcing boundary conditions at the end of the processing.", "limitations": "Authors address potential societal impact of their work and mention the fact that the properties of the model limit its applicability domain. They mention that the proposed method is not suitable for solving inverse problems but don't elaborate much on that. The improvement with respect to the state of the art (here Brandstetter et al.) is significant. Perhaps a a few simple experiments similar to those in Brandstetter et al would give more information on whether improvement in predictions is really due to the constraints enforced by the proposed architecture.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The idea seems to be very simple and based on existing methods but is effective. The connection to global pooling in computing the step size $\\alpha$ of the Barzilai–Borwein method is also interesting and perhaps deserves some more discussions. The paper is sometimes hard to read. And not very clear.\ne.g.,\n3.2 Where it is explained that the weight should be kept small to 'respect' information in the neighborhood.\n3.2 Doesn't explain how the model can be generalized to vectors or higher rank tensors. (edited)\n\nMoreover, in the paper it doesn't really explain how experiments are performed so I assume the reader is often very familiar with this kind of experiments to test neural pde solvers otherwise they need toexplain it better. Especially how GNN gradient operators predict gradients. There are some more explanations in the Appendix but overall the description remains unclear.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Sl252AbE7tk", "reviewer": "Reviewer_qCje", "summary": "The authors proposed a novel design, building on top of IcoGCN to add two types of boundary conditions to the solution of the solver, Dirichlet conditions constraining the values of the functions at the boundaries, and Neumann conditions, constraining the gradients of the values at the boundaries. By leveraging the E(n) equivariant properties of the IcoGCN backbone design, the authors demonstrated improved performance compared to SOTA baselines such as MP-PDE, in particular with respect to translation and rotation of the domain. Compared to the original IcoGCN, the authors demonstrated that the proposed method is able to strictly satisfy Dirichlet and Neumann conditions, leading to improved accuracies of the solutions.", "questions": "One main thing I am trying to understand more deeply is how the proposed algorithm combines the first-principle physics with data priors. One way is to predict the results only using data prior, perhaps adding physics equations as an auxiliary loss (e.g., PINNs), which requires lots of training data, but leads to poor generalization. Another extreme is to only solve from the first principles, given as PDEs, using a PDE solver (as is the case in OpenFOAM), which requires no training data, but has very good generalization. In this work, the model does seem to require training on a given dataset (albeit a small dataset of only 203 examples), yet solves for a PDE using the Neural nonlinear solver in Sec 3.3. Please help me understand how the learned data prior is incorporated in this process? \n\nNit: Eqn (19) in the appendix has an extra parenthesis.", "limitations": "For the experimental evaluations, it would be interesting to see how the model generalizes to a different range of physical parameters, such as Reynolds number. Such evaluations seems to be lacking in the experiments.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* Embedding the physical constraints into the design of the model itself by combining the PDE solution process with the model is an elegant and more generalizable approach to enforcing physical constraints, compared to simply using a loss penalty as an auxiliary loss only during training time.\n* By incorporating the IcoGCN backbone, the authors are able to demonstrate translational and rotational equivariance of the final model, which is a very strong and desirable property missing in many works in the physics informed machine learning literature.\n* The experimental evaluations are compelling. Not only did the authors compare with two reasonable baselines to demonstrate improved enforcement of boundary conditions and rotation/translation invariance, they also performed a runtime analysis between speed and accuracy tradeoffs, even considering the OpenFoam as a baseline. This is not too commonly evaluated among physics informed ML literature and I am happy to see it included in this work.\n* The paper is clear and well written. In particular, I enjoyed the writeup for the backgrounds section, which doesn't assume too much prior knowledge on the subject matter and is easy to follow.\n\nWeaknesses:\n* Though the authors branded the novelty of this work on the E(n)-Equivariant properties of the model, it is not a unique contribution from this work since it is based off of the IcoGCN work which this work used as a backbone. Stressing this property (even in the title) feels like an oversell to me.\n* Though the overall accuracy / runtime tradeoff for the proposed model is compelling (Fig. 4), I would like to see some thoughtful redesign of the model to allow flexibly adjusting accuracy / runtime also for this learned model. One suggestion is to use the results from this model to initialize a coupled PDE solver, so that with a good guess leveraging data prior, the model can achieve the same guaranteed accuracy at a faster speed.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "MJJCczQQeS", "reviewer": "Reviewer_tb7f", "summary": "The authors’ primary contribution is the development of neural network layers that enable the treatment of Dirichlet- and Neumann-type boundary conditions in encoded space (the “boundary encoder”, “Dirichlet layer”, “NeumannIsoGCN layer” and the “pseudoinverse decoder”). For Dirichlet-type BCs, they accomplish this by constructing a decoder design such that the entire model will learn to approximate the identity function at the Dirichlet portion of the boundary (the “Dirichlet layer” and “pseudoinverse decoder”). The authors deal with Neumann-type BCs by extending the IsoGCN layer from the prior art to include terms that convert the Neumann BC into a penalty/constraint embedded into the architecture of the layer (“NeumannIsoGCN”).\n\nA secondary contribution of the paper is the augmentation of a neural nonlinear solver where, by computing the “optimal step size” for a gradient descent step, they achieve a filter analogous to global pooling. This aids the neural model in learning highly nonlinear dynamics, such as exhibited by the incompressible Navier-Stokes equations.\n\nThese constructions are supported both by experiments and comparisons with the prior art as well as an ablation study to show the merit of each proposed component.\n", "questions": "1) The discussion of PINNs in section 2.2.1 strikes me as unrelated to the current work, or at least the connection isn’t very clear to me. I would advise the authors to clarify the limitation of PINNs they are discussing and make the connection or contrast with the current work explicit.\n\n2) The same, but to a lesser degree, for GNNs in section 2.2.2. Please clarify the difference to the current work.\n\n3) It seems to me that the main paper omits rather important details that are given in the first paragraph of section C.3 (lines 461-469 in the supplementary PDF). I realize that the page limit makes it difficult to include all the information, but consider finding a place for some or all of this information in the main paper.\n\n4) How does the proposed method perform on domains that were not seen in the training set?\n", "limitations": "No limitations.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n* The paper is generally sound and the proposed model indeed outperforms the prior art model under comparison.\n\n* The model achieves zero error on the Dirichlet boundary, as claimed.\n\n* The ablation study can serve to show that each proposed component indeed contributes to the performance of the PENN model.\n\n* Also of note is Figure 1 which presents an overview of the main contributions in a clean, legible and aesthetically pleasing way.\n\nWeaknesses\n\n* The title of the work appears to be slightly misleading in the sense that E(n)-equivariance stems from the work of Horie et al (2021). and not from the current work. I would suggest choosing a more descriptive title, such as “Physics-Embedded Neural Networks for Nonlinear Dynamics with Mixed Boundary Conditions”.\n\n* I cannot find any information on the actual network architecture that is used. How is the domain encoded into the network? How flexible can the domain be? How about supporting heterogeneous coefficients?  It seems like the authors refer the reader to previous works, but I would expect this info to be present at the current paper or at least in the supplementary material. Also, from a brief look I couldn't find the architecture in the authors' code (but maybe I'm wrong). In any case - I do not see how one can replicate the results in this work.\n\n* I would expect more comparisons to other NNs and classical solvers, particularly FEM/FVM and AMG-based solvers. The authors claim that they have tested their model against classical solvers, but I found no evidence of this in the main paper or supplementary PDF.\n\n* I find it difficult to trust comparisons that do not detail metrics such as a number of trainable parameters or FLOPs. As it currently stands, I am unable to judge the tradeoff I would be making by using PENN as opposed to MP-PDE, for example. Consider adding these metrics as their absence makes true comparison very difficult. Figure 4 only partially addresses this concern but it does not present any speed/accuracy tradeoff for PENN (like it does for OpenFOAM and MP-PDE) and any speed/accuracy/parameter-count tradeoff is not considered at all.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "E1uV2dOlHZ", "reviewer": "Reviewer_ZHC4", "summary": "This work proposed to use E(n)-Equivariant Graph Neural Network to solve PDEs. Main differences compared to existing works include 1) using a boundary node encoder and pseudoinverse decoder to enforce boundary conditions and achieve better long-term accuracy. 2) embed the PDE inside the model to achieve better long-term accuracies, via the proposed NeumannIsoGCN", "questions": "- in eq) 17, does $i$ refer to time steps, or gradient descent steps? If it refers to # of time steps, does propose approach generalize dynamic simulation of arbitrary timesteps?  ", "limitations": "My primary concern is also one point that I don't quite understand if the $i$ in eq. 17 refer to time steps. If so, how could it generalize to potentially longer time scales?", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "***novelty*** The formulation is new and the global encoding and embedded PDE inside the GNN is novel, to the best of my knowledge. \n\n ***quality and clarity*** The work is well written and well-motivated with good background introductions and well-organized experimental results. The accuracy speed trade-off plot is interesting.\n\n***significant*** The work demonstrates their work on solving an important task.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}]

Author rebuttal:

Decision: Accept