PhyMPGN: Physics-encoded Message Passing Graph Network for spatiotemporal PDE systems
Authors: Bocheng Zeng
Qi Wang
Mengtao Yan
Yang Liu
Ruizhi Chengze
Yi Zhang
Hongsheng Liu
Zidong Wang
Hao Sun
Venue: ICLR 2025 Spotlight (2025)
Keywords: Physics-encoded; Spatiotemporal PDEs; Graph Network; Deep Learning;
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: Presented a Physics-encoded Message Passing Graph Network for simulation of spatiotemporal PDE systems.

Abstract: Solving partial differential equations (PDEs) serves as a cornerstone for modeling complex dynamical systems. Recent progresses have demonstrated grand benefits of data-driven neural-based models for predicting spatiotemporal dynamics (e.g., tremendous speedup gain compared with classical numerical methods). However, most existing neural models rely on rich training data, have limited extrapolation and generalization abilities, and suffer to produce precise or reliable physical prediction under intricate conditions (e.g., irregular mesh or geometry, complex boundary conditions, diverse PDE parameters, etc.). To this end, we propose a new graph learning approach, namely, Physics-encoded Message Passing Graph Network (PhyMPGN), to model spatiotemporal PDE systems on irregular meshes given small training datasets. Specifically, we incorporate a GNN into a numerical integrator to approximate the temporal marching of spatiotemporal dynamics for a given PDE system. Considering that many physical phenomena are governed by diffusion processes, we further design a learnable Laplace block, which encodes the discrete Laplace-Beltrami operator, to aid and guide the GNN learning in a physically feasible solution space. A boundary condition padding strategy is also designed to improve the model convergence and accuracy. Extensive experiments demonstrate that PhyMPGN is capable of accurately predicting various types of spatiotemporal dynamics on coarse unstructured meshes, consistently achieves the state-of-the-art results, and outperforms other baselines with considerable gains.

Published as a conference paper at ICLR 2025
PHYMPGN: PHYSICS-ENCODED MESSAGE PASSING GRAPH NETWORK FOR SPATIOTEMPORAL PDE SYS- TEMS
Bocheng Zeng', Qi Wang', Mengtao Yan', Yang Liu", Ruizhi Chengze", Yi Zhang", Hongsheng Liu², Zidong Wang?, Hao Sun
"Gaoling School of Artificial Intelligence. Renmin University of China, Beijing, China
"Scoool of Engineering Science, University of Chinese Academy of Sciences, Beijing, China Huuawii Technologies, Shenzhen, China
Emails: zengbochenge www.hhennnrrrrrree.mm edu.e..cc (B.7.); w.o.hhaaunnuuc.du..ooo (H.S.)
ABSTRACT
Solving partial differential equations (PDEs) serves as a comnestonee for modeling complex dynamical systems. Recent progresses have demonstrated grand benefits of data-driven neural-based models for predicting spatiotemporal dynamics (e.g.. tremendous speedup gain compared with classical numerical methods). However, most existing neural models rely rich training data, have limited extrapola- tion and generalization abilities, and suffer to produce precise or reliable physical prediction under intricate conditions (e.g., irregular mesh geometry, complex boundary conditions, diverse PDE parameters, etc.). To this end, we propose a new graph learning approach, namely, Physics-encoded Message Passing Graph Network (PhyMPGN), to model spatiotemporal PDE systems on irregular meshes given small training datasets. Specifically, we incorporate a GNN into numerical integrator to approximate the temporal marching of spatiotemporal dynamics for a given PDE system. Considering that many physical phenomena are governed by diffusion processes, further design a learnable Laplace block, which encodes the discrete Laplace-Beltre Beltrami operator, to aid and guide the GNN learning in a physically feasible solution space. A boundary condition padding strategy is also designed to improve the model convergence and accuracy. Extensive experiments demonstrate that PhyMPGN is capable of accurately predicting various types of spatiotemporal dynamics on coarse unstructured meshes, consistently achieves the state-of-the-art results, and outperforms other baselines with considerable gains.
INTRODUCTION
Complex dynamical systems governed by partial differential equations (PDEs) exist in a wide va- riety of disciplines, such computational fluid dynamics, weather prediction, chemical reaction simulation, quantum mechanisms, etc. Solving PDEs is of great significance for understanding and further discovering the underlying physical laws.
Many traditional numerical methods have been developed to solve PDEs (Ames) 2014 Anderson 1995), such as finite difference methods, finite element methods, finite volume methods, spectral methods, etc. However, to achieve the targeted accuracy. the computational cost of these methods for large simulations prohibitively high, since fine meshes and small time stepping are required (typically relaxed for implicit methods). In recent years, deep learning methods have achieved great success in various domains such as image recognition (He et al., 2016). natural language process- ing (Vaswani et al.) 2017) and information retrieval (Devlin et al. 2019). A increasing number of neural-based methods have been proposed to learn the solution PDEs.
Physical-informed neural networks (PINNs) Raissi et al. 2019) and its variants have shown promise performance modeling various physical systems, e.g., fluid dynamics (Gu et al. 2024 Raissi et al,
"Corresponding author

---

Published as a conference paper at ICLR 2025
2020), rarefied-gas -as dynamics (De Florio et al. 2021). equation discovery (Chen al. 202 D, and
engineering problems (Rezaei et al. 2022 Haghighat et al. 2021; I: Rao et al. 2021: Eshkofti & Hos-
seimi 2023). However, the family of PINN methods easily encounter scalability and generalizabiliity
issues when dealing with complex systems, due to their fully connected neural network architecture and the incorporation of soft physical constraints via loss regularizer. Neural operators (Lu al. 2021
Li et al.] [2021; 2023b Wu et al. (202), the variants of Transformer (Geneva & Zabaras HAN al. 2022; Li al.) 2023a), graph neural networks (GNN) (Battaglia al, 2018 Seo et al. 2020; lakovlev et al. |022]] Plaff et al. 2021, Brandstetter et al., (022), and pretrained diffusion models (Li et al. (024c) overcome the above issues some extent; however, they require substantial training datasets to learn the mapping between infinite function spaces and temporal evo- lution, due to their "black-box" data-driven learning manner. The physics-encoded learning, e.g.. PeRCNN (Rao et al.) [2222] 2023), has the ability to learn spatiotemporal dynamics based on lim- ited low-resolution and noisy mcasurement data. However, this method is inapplicable to irregular domains. Therefore, achieving accurate predictions of spatiotemporal dynamics based on limited low-resolution training data within irregular computational domains remains a challenge.
2022
To this end, we propose graph learning approach, namely, eensded Message Passing Graph Network (PhyMPGNN, to model spatiotemporal PDE systems on irregular meshes given small training datasets. Specifically, our contributions be summarized as follows:
We develop a physics-encoded graph learning model with the message-passing mecha- nism (Gilmer et al.) (017) to model spatiotemporal dynamics coarse (low-resolution) unstructured meshes, where the temporal marching is realized via a second-order numeri- cal integrator (e.g. Kunge-Kutta scheme).
Considering the universality of diffusion processes in physical phenomena, we design learnable Laplace block, which encodes the discrete Laplace-Beltrami Beltrami operator, to aid and guide the GNN learning in a physically feasible solution space.
The boundary conditions (BCs) are taken as the a priori physics knowledge. We propose a novel padding strategy to encode different types of BCs into the learning model to improve the solution accuracy.
Extensive experiments show that PhyMPGN is capable of accurately predicting various types of spatiotemporal dynamics on coarse unstructured meshes with complex BCs and outperforms other baselines with considerable margin, e.g., exceeding 50% gains.
RELATED WORK
Traditional numerical methods solve PDEs via spatiotemporal discretization and approximation. However, these methods require fine meshes, small time stepping, and fully known PDEs with predefined initial conditions (ICs) and BCs. Recently, given the advance of deep learning, numerous data-driven neural-based models have been introduced to learn PDE systems with speedup inference.
Physics-informed learning: PINNS (Raissi et al. 2019) pioneer the foundation for the paradigm of ahnsice-informed learning, where automatic differentiation are used determine the PDE resid- ual as soft regularizer in the loss function. Further, several variants YYu et al.) 2022 Eshkofti & Hosseini 2023] Miao & Li| (2023) have been proposed to improve their accuracy and enhance the capability to handle complex geometries. In addition to automatic differentiation, other methods can be employed to compute the PDE residual, such as finite difference (Ren et al. [2022] Rao al.,
2023). finite element (Rezaei et al. 2024 (ahli Costabal et al. 2024), and finite volume methods (Li ctt al. 2024a b). The family PINN methods (Raissi 2018 Seo & Liu 2019 Yang Foster 2022
Ren et al. 2023 2024) perform well in solving various forward and inverse problems given limited
training data or even no labeled data. However, the explicit PDEs needs to be supplied.
Neural Operators learning: Neural operators learn a mapping between two function spaces from finite input-output data pairs, such as the ICs, BCs, and PDE parameters. Finiteddimensional opera- tor methods (Bhatnagar et al. 2019 KHOO et al., 2020 Zhu & Zabaras 2018 lakovlev et al. 2021 are mesh-dependent and cannot obtain solutions unseen nodes in the geometry. DeepONet (Lu
et al. (02T), as a general neural operator, has demonstrated with generalizability to diw-dimensional
systems. FNO (Li et al. 2021 and its variants (George al. 2024 Tran et al. 2023 Guibas al.
2022 al (023b) learn an integral kernel directly in the Fourier domain. Moreover, U-NO (Rah-
man et al. 2023) and U-FNO (Wen et al. 2022) incorporate UNet. (Ronneberger et al. 2015 and
2

---

Published as a conference paper at ICLR 2025
FNO for deeper architecture and data efficiency. LSM (Wu et al. 2023 designs neural spectral block to learn multiple basis operators in the latent space. Generally, neural operator methods do not require any knowledge about the underlying PDEs, but demand large amount of training data.
Autoregressive learning: Autoregressive learning methods model spatiotemporal dynamics iter- atively. Generally, such methods employ neural network extract spatial patterns and recurrent block to model temporal evolution, such as RNN (Hochreiter & Schmidhuber 1997 Cho et al. 2014). Transformers (Vaswani et al., 2017). or a numerical integrator. PA-DGN (Seo et al.) 2020) combines the spatial difference layer with a recurrent GNN to learn the underlying dynamics. Phy- CRNet (Ren et al., 2022) employs Convs LSMM (Shi et al. 2013) to extract spatial features and evolve over time. LED (Vlachas et al. 2022) deploys RNNs with gating mechanisms to approxi- mate the evolution of the coarse-scale dynamics and utilizes auto-encoders to transfer the informa- tion across fine and coarse scales. The Transformer architecture combined with different embedding techniques (Geneva & Zabaras 2022 HAN al. 2022) for physical systems has also been explored.
Besides, there are also lots of neural models (Rao al. 2023 Brandstetter et al. 2022 Pfaff al.
2021
Choi et al. 2023
Mi et al. 2024) employing numerical integrators for temporal marching.
Graph diffusion processes:
Diffusion processes on graphs (Freidlin Wentzell 1993
Freidlin
& Sheu 2000) have a various applications, such as image processing (Lozes et al. 2014 Gilboa &
Osher 2009) and data analysis (Coifman al. 2005 Belkin & Niyogi (003). Recently, studies
exploring the connection between these processes and GNNs have been Cecreasing.mmplemees.
diffusion operations on graphs enhances the representational capacity of graph learning (Atwood &
Towsley 2016 Liao et al. 2019. Gasteiger et al. 2019)). The notion of the diffusion PDE graphs 2022).
also inspires the understanding and design of GNNs (Chamberlain al. 2021: Thorpe et al.
METHODS
3.1
PROBLEM SETUP
Let us consider complex physical systems, governed by spatiotemporal PDEs in the general form:
ú(x,t) Ftt, x, u, Vu, Au....)
where t( E IR' is the vector of state variable with m components we are interested in, such as velocity, temperature or pressure, defined over the spatiotemporal domain {x, t} E Ω x [0, 7]. Here, denotes the derivative with respect to time and F nonlinear operator that depends on the current state u and its spatial derivatives.
We focus on a spatial domain Q with non ooniurnfoy and sparsely observed nodes fao.... ..... NN (e.g., on an unstructured mesh), presenting a more challenging scenario compared with struc- tured grids. Observations (U())).... (IT-1)) are collected at time points lg,... (T-1, where U44)= {u(20, (:),... u 44) denote the physical quantities. Considering that many phys- ical phenomena involve diffusion processes, we assume the diffusion term in the PDE is known as a priori knowledge. Our goal is to develop a graph learning model with small training datasets capable of accurately predicting various spatiotemporal dynamics on coarse unstructured meshes, handling different types of BCs, and producing the trajectory of dynamics for arbitrarily given IC.
Before further discussion, we provide some notations. Let graph G = (V,E), with node i E V denoting the observed node in the domain and undirected edge (i.j) E denoting the connection between two nodes. We apply Delaunay triangulation to the discrete nodes to construct the non- uniform mesh which forms the edges of the graph.
3.2 MODEL ARCHITECTURE
According to the method of lines (MOL) to discretize the spatial derivatives in F at these discrete nodes, Eq. [ can be rewritten as system of ordinary differential equations (ODEs) by numerical discretization (Schiesser 2012: lakovlev et al., 2021). The ODE at each node can be described by (tt) u0)) 1 dd) Numerous ODE solvers be applied to solve it, e.g., the second-order Runge- (RK2) scheme:
u I+ = " + (91 + g2) 8t; g1 = F(th, x. u .u...)): g2 = F(th+t), (ichtt,a0 xx," u² + :8tg.....) (2)
3

---

Published as a conference paper at ICLR 2025
Laplace Block
MLP
MPNNS
MLP
Mesh Laplace
Au
GNN Block
MN 1000k
82
MPNN Layer
MPNN Layer
h'
h
Figure I: (a) Model with the second-order Runge-Kuue Kutta scheme. (b) NN block consists of two parts: GNN block followed the rection architecture. Mesh Laplace in Laplace block denotes the discrete Laplace-Belnn Belttrmm operator in geometric mathematics. The other three modules, MLP MLP3, and MPNNs, constitutes the lightweight learnable network for correction. (c) The MPNNs module GNN block consists of L identical MPNN layers. Residual connection and padding in latent space applied in the first layers, excluding the last layer.
framework and Laplace block with the cor-
where w is the state variable at time t² and ôt denotes the time interval between 1 and gk+1
According the recursive scheme in Eq. 2 we develop a GNN to learn the nonlinear operator F. Once the ICs are given, all subsequent states be quickly obtained. In addition to the RK2 scheme, other numerical methods such as the Euler forward scheme and the fourth-order Runge- Kutta scheme (RK4) also be applied, offering trade-off between computing resource and curacy (more details found in Appendix A}. Notably, all experimental results presented in this paper obtained using the RK2 scheme.
Figure Ta shows the architecture of our model with the RK2 scheme. The NN block aims to learn the nonlinear function F and consists of two parts (Figure [)): GNN block followed the Encode- Process- Decode module (Battaglia et al. 2018) and learnable Laplace block. Due to the univer- sality of diffusion processes in physical phenomena, we design the learnable Laplace block, which encodes the discrete Laplace-Beltran Beltrami operator, to learn the increment caused by the diffusion term Au in the PDE, while the GNN block is responsible learn the increment induced by other un- known mechanisms or sources.
3.2. GNN BLOCK
We utilize message passing GNN Gilmer et al. 2017) with the Paccode-rrocess--e frame- work (Battaglia et al. 2018) as a pivotal component of our model, referred to as the GNN block. comprises three modules (Figure c): encoder, processor, and decoder.
Within the GNN block, there are two encoders (MLPs), one for computing node embed- dings and the other for edge embeddings. For each node the node encoder maps the state variable uf spatial coordinate æ; and one-hot node type C (normal, ghost, ... details shown in Section 3.3) to node embedding h NodeEnc((uf)) I., C.)). For each edge (i, the edge encoder maps the relative offset u u u² displacement 2ij æ, I; and distance di ||>اح| I|2 to an edge embedding Gjj EdgeEne((uu)) Tij diy) 1) aiming capture local features. The concatenated features at each node and edge are encoded to high-dimensional vectors, respectively.
Encoder:
The processor consists L message passing neural network (MPNN) layers, each with its set of learnable parameters. Consistent with the intuition of the overall model,
Processor:

---

Published as a conference paper at ICLR 2025
expect each MPNN layer update node embeddings in incremental manner. Therefore, introduce residual connections (He et al., (016) in the first L layers, excluding the last layer which produces the aggregated increment desire. The updating procedure is given follows
edge (i,j) message:
mlj = Φ (h', h h/
eig). os1<l
node update:
h++ = 7 h
mis)
+ h 051<
jEN
h = 7 h ----_ m
JEN;
where h denotes the embedding of node output by the I E L) layer, and Φ and y are implemented using MLPs.
Decoder: After L MPNN layers, the decoder, which is also MLP like the encoder, transforms the node embedding h- in the high-dimensional latent space to the quantity physical dimensions.
3.2.2 LAPLACE BLOCK
Motivated by PeRCNN (Rao et al. 2023), which uses physics-based finite difference convolutional layer to encode prior physics knowledge of PDEs, we design a Laplace block to encode the discrete Suplace-Beltrami operators for the diffusion term Au commonly seen PDEs. This aids and guides the GNN learning in physically feasible solution space.
Using the finite difference method compute the Laplacian works well on regular grids, but it becomes ineffective on unstructured meshes. Therefore, we em- ploy the discrete Laplace-Beltrant operators (Reuter ct al.] 2009) to compute the discrete geometric Lapla- cian non-uniform mesh domain (Pinkall & Polth- jer 1996 Meyer et al. 2003), which are usually defined
b
Ay C q
C2
P
Figure (a) Four discrete nodes (i. j. p. and q) and five edges connecting them. The two angles opposite to the edge (i, are aij and 8ig. (b) The blue region de- notes the area the polygon formed by connecting the circumeenters (e.g., C1, C2. etc.) of the triangles around node i.
Af. = - Σ d
1
(4)
JEN.
where N₁ denotes the neighbors of node i and f; de- notes the value of continuous function node cotl / (Pinkall Polthier 1996), where Osy and 3, are the two angles opposite to the edge (i.j). The mass is defined d; denotes the area of the polygon formed by connecting the circumcenters of the triangles around node i (see Figure
The weights read wi
(i) (Meyer al. 2003), where av
2 more details found in Appendix B
Note that Eq. referred to as the Mesh Laplace module in Figure Db. exhibits good accuracy on dense meshes but yields unsatisfactory results on coarse meshes. Therefore, employ lightweight neural network to rectify the Laplacian estimation on coarse meshes, which also lever- framework. Consequently, Eq. is updated as follows:
ages the Soadee-rooees-Deeee
Af.
(++ +
21
Σ
wi (f))
(5)
de
JEA.
where is the output of the lightweight network, as shown in Figure lp. where MLP, and MLP, are employed as the encoder and the decoder, respectively, and MPNNs serve as the processor consisting of L' message passing layers. The MPNNs in the Laplace block differ slightly from those in the GNN block (in particular, residual connections are applied to every layer, not just the first L' layers). This correction architecture of the Laplace block greatly improves the accuracy of Laplacian predictions on coarse unstructured meshes.
In summary, we approximate the nonlinear operator F by
F(th, xx, th V Au A ss NN.block(z, u GNN.block(ze u²) caapacc_blocc((( )²)
(6)
5

---

Published as a conference paper at ICLR 2025
3.3 ENCODING BOUNDARY CONDITIONS
Given the governing PDE, the solution depends on both ICs and BCs. Inspired by the padding methods for BCs on regular grids in PeRCNN (Rao et al., 2023). we propose novel padding strategy to encode four types of BCs on irregular domains. Specifically, we perform BC padding in both the physical and latent space, PENN (Horie& MITSUME 2022) also constructs their model to satisfy Dirichlet and Neumman BCs by designing special neural layers and modules while we apply the padding strategy directly to the features. More details of the comparison between these methods are presented in Appendix [
Darchhet BC
Neumann / Robin BC
Pertodic
Before constructing the unstructured mesh by Delaunay triangulation, we first apply the padding strategy to the
Type
nodes
discrete nodes in the physical space (c.g., )²), as shown in Figure 3 We consider four type of BCs: Dirich- let, Neumann, Robin, and Periodic. Nodes on the Dirichlet boundary will
Chost nodes
Figure 3: Diagram of boundary condition (BC) padding.
be directly assigned specific values. For Neumann/Robin BCs, ghost nodes are created symmetri- cally with respect to the nodes near the boundary (along the normal direction) in the physical space, and their padded values depend on derivatives in the normal direction. The goal is to ensure the true nodes, the boundary, and the ghost nodes satisfy the BCs in manner similar to the central difference method. For periodic BCs, the nodes near the houndary are flipped and placed near the other corresponding boundary, achieving a cyclic effect in message passing. Once the padding is completed, Delaunay triangulation is applied to construct the mesh, which serves as the graph input for the model. Apart from this, we also apply padding to the prediction (e.g., +*) of the model at each time step to ensure that it satisfies BCs before being fed into the model for next-step prediction.
As for the latent space (e.g.. h in the GNN block), padding is also applied after each MPNN layer except the last layer. Details of encoding different types of BCs are provided in Appendix C
Encoding BCs into the model indeed Ieads to a well-posed optimization problem. By incorporating these BCs into the learning process, the model is guided produce solutions that adhere to the physical constraints, resulting in more accurate and physically meaningful predictions.
3.4 NETWORK TRAINING
The pushforward trick and the use of training noise as demonstrated in previous approaches (Brand-
stetter et al.) 2022 Plaff et al. 2001: Sttachenfeld et al.) 2022 Gonzaez- et al., 2020) have effectively improved the stability and robustness of the model. We leverage these insights by syner- gistically integrating them to train our network for long-term predictions.
Due GPU memory limitations, directly feeding the entire long time series (T steps, 2n ...., u --) into the model and performing backpropagation across all steps is impractical. To address this we segment the time series into multiple shorter time sequences of M steps (u*" "u...... u -1 T M). With the input u" the model rolls out for M - 1 times to generate predictions (ū" u "M 1), but backpropagation is only applied to the first and last predictions. Furthermore, we introduce a small noise to the first frame u of each segment during training. Both techniques aim to alleviate overfitting and error accumulation. Therefore, the loss function of each time segment is defined as
L = MSE(u"), " + MSE (u 1, 7. 1)
(7)
where denotes the ground truth of state variable, á denotes the prediction from our model, and the superscript of u, à denotes their time steps in the segment.
Evaluation metries:
We choose the mean square error (MSE), relative L2 norm error (RNE), and Pearson correlation coefficient between the prediction i and ground truth u as the evaluation met- rics, where RNE is defined as i uu/l//|/u||2 and correlation is defined as cov(u, ú
4 EXPERIMENTS
Here, we present comprehensive evaluation of our methods. Firstly, we validate the feasibility and efficacy of the Laplace block to compute the Laplacian coarse non-uniform meshes. Subse-

---

Published as a conference paper at ICLR 2025
Table 1:
The MSE and RNE of four methods to approximate Laplacian on coarse domain.
Table 2: Datasets description.
Burgers
FN
GS
Model
Parameters
MSE
RNE
train/test sets
10/10
3/10
6/10
Laplace block
5.6k
151
0.033
time steps
1600
2000
2000
Mesh Laplace
2574
0.138
öt 0.001
0.004
0.5
SDL
21.5k
1210
0.095
nodes
503
983
4225
SDL-padding 21.5k
408
0.055
BC
periodic
periodic
periodic
quently, the entire model trained and evaluated on various PDE datasets and compared with sev-
eral baseline models. Finally, we perform ablation studies to demonstrate the positive contribution of each individual component within our model and the impact of numerical integrator. The source code is publicly available in the Pytorch and Mindspore repositoriie.
4.1 LAPLACE BLOCK VALIDATION
We create a synthetic dataset validate the effectiveness of the Laplace block. First, we generate 10,000 samples of random continuous functions on high-resolution (129 x 129) regular grid by cumulatively adding sine and cosine functions, satisfying periodic BCs. By employing the 5-point Laplacian operator of the finite difference method, the Laplacian of the functions at each node is computed as the ground truth. Then the dataset is down-sampled from the high-resolution regular grids to set of coarse non-uniform mesh points yielding only 983 nodes. Note that 70% of the dataset is used as the training set, 20% as the validation set, and 10% as the test set. More details of the synthetic dataset are shown in Appendix D.I
We compare the performances four methods: (1) Laplace block with BC padding; (2) the discrete Laplace-Beltrame operator with BC padding, referred to Mesh Laplace; (3) the spatial difference layer for Laplacian, referred to as SDL (Seo et al. 2020); (4) SDL with BC padding, referred to as SDL-padding. The MSE and RNE of all experiments poorly compared to other learnable methods, while our Laplace block with the fewest parameters greatly outperforms other methods. SDL-padding exhibits an obvious performance improvement, indicating the effectiveness and importance of our padding strategy. A snapshot of random func- tion, ground truth of its Laplacian, and prediction are shown in Appendix Figure A.4
shown in Table [
Mesh Laplace performs
4.2 MODELING PDE SYSTEMS
Solving PDEs serves as a cornerstone for modeling complex dynamical systems. However, in fields such climate forecasting, reactions of chemical matters, and social networks, the underlying governing PDEs are either completely unknown or only partially known. Hence, there is a great need for data-driven modeling and simulation. Based on the assumption, we train and evaluate our model on various physical systems, including Burgers' equation, FiizHugh-Nagumo (FN) equation, Gray-Scott (GS) equation, and flows past a cylinder, particularly in small training data regimes. We also compare our model against several representative baseline models, including DecpONet (Lu al. 202T), PA-DGN (Seo et al., 2020). MGN (Plaff al. 202I), and MP-PDE (Brandstetter et al. 2022). The numbers of learnable parameters and training iterations for all models are kept as consistent as possible to ensure fair comparison. All the data are generated using COMSOL with fine meshes and small time stepping, which are further spatiotemporally down-sampled to establish the training and testing datasets. More details of the datasets are discussed in the Appendix D.2
We firstly test the generalizability of the model over different ran- dom ICs. For the Burgers' equation, we generate 20 trajectories of simulation data given different ICs (10/10 for training/testing). For the FN and GS equations, the number of trajectories used for training are 3 and 6, respectively. More descriptions about the datasets of these three equations are provided in Table 2). We consider periodic BCs in these PDEs. Figure 4 depicts the prediction re- sult of each training model for the testing ICs, including the error distribution across all time steps, correlation curves and predicted snapshots at the last time step. The generalization prediction by
Generalization test over ICs:
htttp://giituu..o//mtttll//////s http://www..c.ccoooooo///ooottee mecaaa
http://wt.eetaaooooocommmmoococn mechanism fusion/phymppn

---

Published as a conference paper at ICLR 2025
Rot
Qurs
DeepiONet
PA-DON
MGN
101
10
MP POE
Correatioo
0.8
M80
10
MON
MP RDE
0.
PA-322
DeupChes
modets
0.4
0.8 (9)
12
1.0
Snagshot at 1.6%
Rof.
Durs
DeepONet
PA-DON
MGN
MP-PDE
10a
10
DeepONer
Condatio
0.5
Out
Ours
MEN
10
MP-PCE
0.4
RA DGN
DeepONer
PA-DOK
10
models
(()
Snopshot
Ref.
Ours
DeepONet
PA-DEN
MGN
MP-PDE
10
1.0
Plu DIGN
10-4
10a
Cura
DeepONIT
MGN
Coneeelion
0.6
Duts MGN
10"
0.2
DospONer
10-4
400
(55
Snepshhot al 1000
Figure 4: The error distribution (left) over time steps, correlation curves (medium), and predicted snapshots (right) at the last time step. (a) Burgers' equation. (b) FN equation. (c) GS equation. All systems of these PDEs have periodic BCs.
our model agrees well with the ground truth for all the three datasets. In the contrast, the baseline models yield predictions with obvious deviations from the ground truth, although MGN performs slightly better and produces reasonable prediction. To explore the dataset size scaling behavior of model, using Burgers' equation trajectories: 5, 10, 20, and 50. The numerical results are presented Table A.6 showing that as the dataset size increases, the test error decreases.
example, train model with different numbers of
Scenario A
Scenario
validate our model's ability to encode different types of BCs, we use Burgers' equation an example and con- sider three additional scenarios with irregular domain and hybrid BCs, resulting in total four scenarios, as illustrated in Figure 5 (1) Scenario A, a square domain with periodic BCs, as mentioned in the previous paragraph; (2) Scenario B, square domain with a circular cutout, where periodic and Dirichlet BCs are applied; (3) Scenario C, a square domain with a dolphin-shaped cutout, where periodic and Dirichlet BCs are ap- plied; and (4) Scenario D, a square domain with semicircular cutouts on both the left and right sides, where periodic and Neumann BCs are ap- plied. Similarly, 10/10 training and testing trajectories are generated and down-sampled with nearly N 600 observed nodes in each domain. Figure [ depicts the error distribution over all time steps, correlation curves, and predicted snapshots at the last time step, for the last three scenarios (results of Scenario A are shown in Figure 44). It is seen that our model significantly outperforms baseline models and shows the efficacy of handling PDE systems with different BCs
Handling different BCs:
Scenario
Scanarto
Periodic
Dirichiet
Neumann
Figure 5: Four scenarios with different domains and hybrid BCs.
Generalization test over the Reynolds number: We also evaluate the ability of our model to generalize over the inlet velocities for the cylinder flow, governed by the Naiver-Stokes (NS) equa- tions. Here, the Reynolds number is defined as Re = pÝ... D/ D/u, where p is the fluid density, D the cylinder diameter, and μ the fluid viscosity. With these parameters fixed, generalizing over the inlet velocities Um, also means to generalize across different Re values. By changing the inlet velocity Un. we generate 4 trajectories with Re = [160, 240, 320, 400] for training and 9 trajectories for testing, with only about N 1, 600 observed nodes in the domain. The testing sets divided into three groups based on Re: 3 trajectories with small Re = 200, 280, 360j, trajectories with medium Re- = [440, 480, 520), and 3 trajectories with large Re [600, 800, 1000). During the eval-
8

---

Published as a conference paper at ICLR 2025
DeepONet
10
MP FDe
Consation
0.8
06
30
Q4
RM-DGN
models
0.4
0.8
(5)
1.2
1.0
Snapshot 1.6
Rat.
Ours
DeepONit
PA-DGN
MGN
MP-PDE
1.0
Dusptrrii
NP-FDD
Comdation
0.8
PA-OtoR DespOhee
M
MSE
30
MGN
RA-DON
044
models
0.4
0.5 (8)
1.2
1.8
Snapsioot 1.6.
Ret.
Durs
DeepONet
PALDGN
MGN
MP-FDE
30
1.0
10
MSB
DreppONet
MGN MP-PDE
10
MGN
24
30-3
&
DerpOhler
30
PAUDENN
models
Snapshot
Figure 6: The error distribution (left) over time steps, correlation curves (medium), and predicted snapshots (right) at the last time step, for the Burgers' equation. (a) Periodic and Dirichlet BCs. (b) Periodic and Dirichlet BCs. (c) Periodic and Neumann BCs.
Ref.
DeepONet
PA-DGN
MGN
MP-PDE
Snepshot t 100
1.0
10-1
DespChhar
Conroctton
Comelation
0.6
0.6
10--
DON
0.4
Ours MGN
NP POE DeepOria
0.4
Ra-300
Rc:280
Ro-500
10
MON
02
0.2
Rec440
Rec
Ours
PA-DON
10-7
60
76
100
modets
number shed
Figure 7: (a)-(b) Predicted snapshots at the last time step, enror distribution all time steps, and correlation curves for the cylinder flow problem (Re 480). (e) Correlation curves of our model on generalization over Re. The number of vortex sheds within & seconds is defined as t/Tnes Re. where Tre the vortex shedding period.
uation, it is observed that the baselines struggle generalize Re values unseen during training (e.g., the larger the Re, the greater the prediction error). For example, the predicted snapshots at t 100 along with the error distributions and correlation curves for Re 480 are shown in Fig- ure b... Our model consistently maintains high correlation between the prediction and ground truth over time, while the performance of the baselines rapidly declines. is interesting that MGN's and MP-PDE's correlation initially declines but then recovers, suggesting that they capture the cyclic dynamics but fail to generalize to bigger Re values, leading to mis-prediction of the vortex shedding period. Besides, we evaluate our model's generalizability over all other Re values. Considering that the larger the Re, the shorter vortex shedding period Tiee and the more vortex sheds t/TRe in the same time span t, we track the prediction performance against the number of vortex sheds. Fig- 7 shows the generalization result our model across different Re values, where the correlation decays faster only for Re 1000. denser set of observed nodes could help mitigate this issue.

---

Published as a conference paper at ICLR 2025
To evaluate the extrapolation capability, model trained over the time range 0 100s is tested on the 200-second trajectory with Re = 480. We observe a slow decline of our model's correlation in The correlation is 0.95 at 100s and decreases to 0.85 at t 200s.
Appendix Figure A.3
The computational cost of the Laplace block and the padding strategy minimal. The computation of Mesh Laplace in the Laplace block includes only matrix multipli- cation, while the cost of the padding strategy involves merely copying the corresponding nodes in the domain and is small. A detailed analysis and comparison with MGN and MP-PDE as well as the classical numerical method (COMSOL) are presented in Appendix [ In all experiments, our model outperforms the baselines with considerable margins, c.g., exceeding 50% gains (see the specific nu- merical results reported in Appendix Tables A.5 A.7 and A.8) The predicted trajectory snapshots
Computational efficiency:
of our model and ground truth are provided in Appendix Figure A.6
a 10¹
b 1.0
4.3 ABLATION STUDY
Model C
10
Comdation
0.8
We study the effectiveness and contribu- tion of two key components in our model, namely, the learnable Laplace block and the BC padding strategy. Specifically, we conducted ablation studies on the cylinder flow problem, comparing the performance of four models: (1) the full model, referred to as Ours; (2) the model only without the Laplace block, referred to Model A; (3) the model only without the padding strategy, referred to as Model B; (4) the model without both the Laplace block and the padding strategy, referred to as Model C. Notably, the trainable parameters of the Laplace block comprise only about 1 1% of the entire model, and the padding strategy itself introduces no additional trainable parameters. In other words, the number of trainable parameters in these four models almost the same. We train these models on the same cylinder flow dataset described previously and evaluate them on the testing datasets with Re = [440, 480, 520). Figure 8 shows the error distribution and correlation curves, averaged over the testing datasets. The corresponding numerical MSEs of these four models are listed in Table[A.9] seen that the ablated models exhibit a deteriorated performance (Model C performs the worst). Despite their minimal contribution to the overall trainable parameter count, the two key components greatly enhance the model's 's capacity for long-term predictions.
MSB
10
Model A
0.8
0.7
10
Model
Mocil
Model
10-
0.5
Model
models
(6)
Figure 8: Results of Models A, B, C. and Ours.
Additionally, we investigate the impact of three types of numerical integrators employed in our model on performance: Euler forward, RK2, and RK4 scheme. The MSEs of these models are presented in Table A. showing that the higher the order of the integrator, the better the model's prediction accuracy. However, higher-order integrators also come with greater memory and com- putational costs. Moreover, empirical evidence suggests that RK4 may increase the difficulty of training, including issues such as training instability. Therefore, we choose RK2 to strike a balance between computational resources and accuracy. Notably, even with Euler scheme, PhyMPGN's s curacy remains superior to baseline models with a big margin. We also explore the impact of the time segment size M on results and computational requirements. Details are presented in Appendix B
CONCLUSION
We present graph learning approach, namely, PhyMPGN, for predicting spatiotemporal PDE systems on coarse unstructured meshes given small training datasets. Specifically, we develop a physics-encoded
cesssag--passing GNN model, where the temporal marching is realized via a second-order numerical integrator (e.g. Runge-Kutte scheme). The priori physics knowledge is embedded to aid and guide the GNN learning in a physically feasible solution space with improved solution accuracy, via introducing (1) learnable Laplace block that encodes the discrete Laplace- Beltrami operator, and (2) a novel padding strategy to encode different types of BCs. Extensive experiments demonstrate that PhyMPGN outperforms other bascline models with considerable mar- gins, e.g., exceeding 50% gains across diverse spatiotemporal dynamics, given small and sparse training datasets. However, several challenges remain to be addressed: (1) how to effectively en- code Neumann/Robin BCs in latent space; (2) the approach cannot be applied to other nonlinear or high-order diffusion terms (e.g., 4 (3) extending the model to 3-dimensional scenarios. These challenges highlight key directions for our future research.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
The work supported by the National Natural Science Foundation of China (No. 62276269 and No. 922701 18), the Beijing Natural Science Foundation (No. 1232009), and the Fundamental Research Funds for the Central Universities (No. 202230265).
REFERENCES
W.F. Ames. Numerical Methods for Partial Differential Equations, Computer Science and Scientific Computing. Elsevier Science, 2014. ISBN 9780080571300 URL htLppt:s://ook google.
co. jp/books?id-km 1B0ASss BOAAQBAJ]
J.D. Anderson. Computational Fluid Dynamics: The Basics with Applications. McGraw-Hii Inter-
national Editions: Mechanical Engineering. McGraw-Hill, 1995. ISBN 9780071132107. URL.
https::/bbole www.goooooooooooo co. http://ww....ooo.eooooopoo/ooada 2agpasaaddddpppooaaaAAA..... QgAACAAJ
James Atwood and Don Towsley. Diffusion-coovolutionaal neural networks. Advances in neural information processing systems, 29, 2016.
Peter Battaglia, Jessica Blake Chandler Hamrick, Victor Bapst, Alvaro Sanchez, Vinicius Zam- baldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Caglar Gulcehre, Francis Song. Andy Ballard, Justin Gilmer, George E. Dahl, Ashish Vaswani,
Kelsey Allen, Charles Nash, Victoria Jayne Langston, Chris Dyer, Nicolas Heess, Daan Wier- stra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, and Razvan Pascanu. Re- lational inductive biases, deep learning, and graph networks. arXiv, 2018. URL https:
Mikhail Belkin and Partha Niyogi. Laplacian eigenmaps for dimensionality reduction and data rep- resentation. Neural Computation, 15(6))1373-1399 2003. doi: 10.1 10.1162/0899766033222777
Saakaar Bhatnagar, Yaser Afshar, Shaowu Pan, Karthik Duraisamy, and Shailendra Kaushik. Predic- tion of aerodynamic flow fields using convolutional neural networks. Computaaionall Mechanics, 64, 08 2019. doi: 10. 1111000/0446-011--00777
Johannes Brandstetter, Daniel Worrall, and Max Welling. Message passing neural PDE solvers. In International Conference Learning Representations, 2022. URL httpt:ttt//ppeeee/////
net Tust/fosee Eorumm www.:iidddiixxxxXxX
Ben Chamberlain, James Rowbottom, Maria I Gorinova, Michael Bronstein, Stefan Webb, and Emanuele Rossi, Grand: Graph neural diffusion. In International conference on machine learn- ing, pp. 1418-1418. PMLR, 2021.
Zhao Chen, Yang Liu, and Hao Sun. Mhysics-informat learning of governing equations from scarce data. Nature communications, 12(1):6136, 2021.
Kyunghyun Cho, Bart Merrienboer, Caglar Gulcehre, Fethi Bougares, Holger Schwenk, and Y. Ben- gio. Learning phrase representations using RNN encoder-decoder for statistical machine transla- tion. 06 2014. doi:
Hoyun Choi, Sungyeop Lee, B. Kahng, and Junghyo Jo. GNRK: Graph neural Runge- Kutta method for solving partial differential equations, 2023.
R. R. Coifman, S. Lafon, A. B. Lee, M. Maggioni, B. Nadler, Warner, and S. W. Zucker. Geometric diffusions as tool for harmonic analysis and structure definition of data: Diffu- sion maps. Proceedings of the National Academy of Sciences, 102611)7426-7431, 2005. doi: 10.1073/pnas 12.1773/pas.055003440009 0500334102. URL https: www. pnas org wttp///........ able///abs/ii 0500334102
Mario De Florio, Enrico Schiassi, Barry D. Ganapol, and Roberto Furfaro. Physics-informer neural networks for rarefied-gas dynamics: Thermal creep flow in the Bhatnagar- Gross-Krook approxi- mation. Physics of Fluids, 33(4):047110; 10, April 2021. doi: 10.15 0046181.

---

Published as a conference paper at ICLR 2025
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), pp. 4171-4186, Minneapolis, Minnesota, June 2019. Association for Com- putational Linguistics. doi: 10.18653/1//11--11 1423, URL NN9-1423
org//
N19- 1423
Katayoun Eshkofti and Seyed Mahmoud Hosseini. A gradient-enhancee thyyics--infomne neural network (gPINN) scheme for the coupled non non-fickian/non-furree
analysis: novel gPINN structure. Engineering Applications of Artificial Intelli-
gence, 126: 126:106908, 2023. ISSN 0952-1976. doi: http:///dbboorg/1000000///.o
1023.106908, URL https://wwws wtt.://wwweeeeee....oooeeeeeeeea 509521976230114
M. I. Freidlin and Shuenn-Jyi Sheu. Diffusion processes on graphs: stochastic differential equations, large deviation principle. Probability Theory and Related Fields, 116:181 -200, 2000. URL
wttgCorrpsII.........o.. 119873771
Mark I. Freidlin and Alexander D. Wentzell. Diffusion Processes on Graphs and the Averaging Principle. The Annals of Probability, 21(4)222 2245, 1993. doi: 101721990/1177768901118
URL
Johannes Gasteiger, Stefan WeiB enberger, and Stephan Güinnemann. Diffusion improves graph learning. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Bud E. Fox, and R. Gamett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https::/prrocedd neur wttppppp..c.ppppprrrrre....pp... files/ paper/201 9/ fi /ellel//3333222 (aee77762555ala. 2aaaa22c5556770666332.333
Nicholas Geneva and Nicholas Zabaras. Transformers for modeling physical systems. Neu- ral Networks, 146:272-289, 2022.
ISSN 0893-6080.
doi: http:///bb.orr//1.0001//..mmmm
2021.11.022. URL https:: /www sci. www.sscieneediiom rect www.://eeeeeecctcecoocooooeaaeee /tt.ett..oo..cccceeeeaaarailllla
50893608021004500
Robert Joseph George, Jiawei Zhao, Jean Kossaifi, Zongyi Li, and Anima Anandkumar: Incremental spatial and spectral learning of neural operators for solving large-scale pdes, 2024.
Guy Gilboa and Stanley Osher. Nonlocal operators with applications to image processing. Multi- scale Modeling & Simulation, 1005:1005 2009. doi: 10.1 10.1137/07099859.. URL https:
10. 1tt.g/001111377766995553333
Justin Gilmer, Samuel S. Schoenholz, Patrick Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning Volume 70, ICML' pp. 1263-1272. IMLR.org, 2017.
Linyan Gu, Shanlin Qin, Lei Xu, and Rongliang Chen. Physics-informed neural networks with domain decomposition for the incompressible navier-stokes equations. Physics of Fluids, 36(2), 2024.
John Guibas, Morteza Mardani, Zongyi Li, Andrew Tao, Anima Anandkumar, and Bryan Catanzaro. Efficient token mixing for transformers via adaptive Fourier neural operators. In International Conference on Learning Representations, 2022. /RL petttp:://ppereviiee net forum?
LddEEXH-20 A3 131
Ehsan Haghighat, Maziar Raissi, Adrian Moure, Hector Gomez, and Ruben Juanes. A physics- informed deep learning framework for inversion and surrogate modeling in solid mechanics. Com- puter Methods in Applied Mechanics and Engineering, 379: 2021. ISSN 0045-7825. doi: https://doi.org/10.. http:00016.....2011111.3.3...111
URL https: www. www.sseiienccrrctccc.cmmm [ttttpsss///woooceeeeeeeerrrrrr com/
sci
http:////pp11000000222220 02007773
XU HAN, Han Gao, Tobias Pfaff, Jian-Xun Wang, and Liping Liu. Predicting physics in mesh- reduced space with temporal attention. In International Conference on Learning Representations, 2022. URL. htt.tttttt///eeeeeeee
net. http://foommmmmmmmmcmmmmm
12

---

Published as a conference paper at ICLR 2025
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9:1735- 80, 12 1997. doi: 10.1162/neco.1 1997.9.99 1733..
Masanobu Horie and NAOTO MITSUME. Physics-embedded neural networks: Graph neural PDE solvers with mixed boundary conditions. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https: http:::///ppeeeeviiee
net htte///ormmmmmddddem
YCtzo,
Valerii lakovlev, Markus Heinonen, and Harri Lähdesmäki. Learning continuous-time PDEs from sparse data with graph neural networks. International Conference on Learning Representa- tions, 2021. URL
YUEHAW KHOO, JIANFENG LU, and LEXING YING. Solving parametric PDE problems with artificial neural networks. European Journal of Applied Mathematics, 32(3):421-43, July 2020. ISSN 1469-4425.
doi: 10. 001100776095772222000000 10. 00.1077//055772220000011
URL http:/ttt/// //xx doi oorg/
Tianyu Li, Shufan Zou, Xinghua Chang, Laiping Zhang, and Xiaogang Deng. Predicting unsteady incompressible fluid dynamics with finite volume informed neural network. Physics of Fluids, 36 (4):043601, 04 2024a. ISSN 1070-6631, doi: 10. 10.60//..1199455. URL (ttpss/ts: .org/ 0197425
Tianyu Li, Yiye Zou, Shufan Zou, Xinghua Chang, Laiping Zhang, and Xiaogang Deng. A fully differentiable gnn-based pde solver: With applications to poisson and navier-stokes equations. arXiv preprint 240i:::2000004466,, 2024b.
Zeyu Li, Wang Han, Yue Zhang, Qingfei Fu, Jingxuan Li, Lizi Qin, Ruoyu Dong, Hao Sun, Yue Deng, and Lijun Yang. Learning spatiotemporal dynamics with pretrained generative model. Nature Machine Intelligence, 6(12):1566-155) 2024c.
Zijie Li, Kazem Meidani, and Amir Barati Farimani. Transformer for partial differential equations' operator learning. Transactions on Machine Learning Research, 2023a. ISSN 2835-8856. URL
net htt////ormmmmmmmme 3uERT
Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Burigede liu, Kaushik Bhat- tacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations, 2021. URL
https://openreviiet net/ Fo NNOVinnC
Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar. Fourier neural operator with learned deformations for PDEs on general geometries. Journal of Machine Learning Re- search, 24(388):1 -26, 2023b. URL /ttp:/ //it::://mmmlr:r wttp://ppppp///////-//////////
html
Renjie Liao, Zhizhen Zhao, Raquel Urtasun, and Richard Zemel. Lanezosnet: Multi-scale deep graph convolutional networks. In International Conference on Learning Representations, 2019.
URL http:httss///peeeeviieee
François Lozes, Abderrahim Elmoataz, and Olivier Lézoray. Partial difference operators on
weighted graphs for image processing on surfaces and point clouds. IEEE Transactions on Image Processing, 23(9):38963399 -909,, 2014. doi: 10.1 2000///..011.2336666
Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3):18-223) 229, March 2021. ISSN 2522-5839. doi: 10.1038 1038 01111/10000000022220000000100000 02300002-5
042256-021-003022 URL StRLhttp::/// httpt:::p//xddoo..oooo wttp::/doo.oooo/o..oooooooo
Mark Meyer, Mathieu Desbrun, Peter Schröder, and Alan H. Barr. Discrete differential differential-gcomettyy operators for triangulated 2-annfoldds In Hans- Christiann Hege and Konrad Polthier (eds.), Visu- alization and Mathematics III. pp. 35-57, Berlin, Heidelberg, 2003, Springer Berlin Heidelberg. ISBN 978-3-662-05105-4,
13

---

Published as a conference paper at ICLR 2025
Yuan Mi, Pu Ren, Hongteng Xu, Hongsheng Liu, Zidong Wang, Yike Guo, Ji-Rong Wen, Hao Sun, and Yang Liu. conservation-infommee graph learning for spatiotemporal dynamics predic- tion. CoRR, abs/2412.20962,, 2024. URL
20962
Yuyang Miao and Haolin Li. GPINN: Physics-informed neural network with graph embedding, 2023.
Tobias Pfaff, Meire Fortunato, Alvaro Sanchez- and Peter Battaglia. Learning mesh-based simulation with graph networks. In International Conference on Learning Representations, 2021. URL
ntt/ it id=LON XP
Ulrich Pinkall and Konrad Polthier. Computing discrete minimal surfaces and their conjugates.
Experimental Mathematics, 2, 1996. doi: 10.1080/10586458 1993.
Md Ashiqur Rahman, Zachary Ross, and Kamyar Azizzadenesheli. U-NO: U-shaped neural operators. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https:
//openreview.. wwt. www.:ntee//oomotoee f 9coJd
Maziar Raissi. Deep hidden physics models: deep learning of nonlinear partial differential equa- tions. J. Mach. Learn. Res., 19(1):932 955222955,1111 jan 2018. ISSN 1532-4435.
Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-infomme neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
Maziar Raissi, Alireza Yazdani, and George Em Karniadakis. Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations. Science, 367(6481), 1 2020. ISSN 0036- 8075. doi: 10. http:/cience.aaw47... URL https::::: //wp:///wwwwwww ost htt.g//obbbbbb/
Chengping Rao, Hao Sun, and Yang Liu. Physics-informed deep learning for computational elasto- dynamics without labeled data. Journal of Engineering Mechanics, 147(8):04020433, 2021. doi: 10. 10.106)/ASCEE)MM. 1989-7889 00009990009177
http:::tt//tt..... / wtw.aceeilrrrrrr//o/....
1061 0A011//288ACCE... 9EM. 1943- 11133-7788 7889. 0001947
Chengping Rao, Pu Ren, Yang Liu, and Hao Sun. Discovering nonlinear PDEs from scarce data with physics-encoded learning. In International Conference on Learning Representations, 2022.
Chengping Rao, Pu Ren, Qi Wang, Oral Buyukozturk, Hao Sun, and Yang Liu. Encoding physics to learn reaction-diffusiae diffusion processes. Nature Machine Intelligence, 5(7):65- Jul 2023. ISSN 2522 doi: 10. (0.10338/44225-023--0000 8:2256-023-00685--
URL
Pu Ren, Chengping Rao, Yang Liu, Jian-Xun Wang, and Hao Sun. PhyCRNet: Physics-informed convolutional-recuree network for solving spatiotemporal PDEs. Computer Methods Applied Mechanics and Engineering, 389:1 14399, 2022. ISSN 0045-7825. doi: https://doi.or//1..0011
j.cma.0211.113999 14399.. URL httos: www wwtp::/wwwwwwwwwwwcccccc....e com/ hamm//eeenn//rttclee
pii 011/800045782221005577
Pu Ren, Chengping Rao, Yang Liu, Zihan Ma, Qi Wang, Jian-Xun Wang, and Hao Sun. PhySR: Physics-informed deep super-resolution for spatiotemporal data. Journal of Computational Physics, 492:1 12438. 2023.
Pu Ren, Chengping Rao, Su Chen, Jian-Xun Wang, Hao Sun, and Yang Liu. SeismicNet: Physics- informed neural networks for seismic wave modeling in semi-infinite domain. Computer Physics Communications. 295: 109010 2024.
Martin Reuter, Silvia Biasotti, Daniela Giorgi, Giuseppe Patanè, and Michela Spagnuolo. Discrete Laplace-Beltrami operators for shape analysis and segmentation. Computers & Graphics, 33(3):381-333 2009. ISSN 0097-8493. doi: http://ddo..or///0.011//// 2009.03.005. URL https:/ /wwww/www/./////////////w/ www.ssseeeeedrret...mmmmm com/ wrom//cience/aaa art htt.aacccccppppll/ SSC97849309000272 2009.
IEEE International Conference on Shape Modelling and Applications
14

---

Published as a conference paper at ICLR 2025
Shahed Rezaci, Ali Harandi, Ahmad Moeineddin, Bai Xiang Xu, and Stefanie Reese. A mixed for- mulation for physics-informed informed neural networks as a potential solver for engineering problems in heterogeneous domains: Comparison with finite element method. Computer Methods in Applied Mechamics and Engineering, 401:115616, 2022. ISSN 0045-7825. doi: https://dbi.or//10..
j.cma.202..11660 115616 URL https: www. sci encedi aciendedi rect com/ feam///ciennc//rrtiill...
p11/804458822200055 011/8004578252200330
Shahed Rezaci, Ahmad Mocineddin, Michael Kaliske, and Markus Apel. Integration of physics- informed operator learning and finite element method for parametric learning of partial differential equations, 2024. URL httpt:://///..... /attppi org abs 2401 02363
Olaf Ronneherger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional networks for biomed- ical image segmentation. volume 9351, pp. 234-241, 10 2015. ISBN 978-3-319-24573-7. doi:
Francisco Sahli Costabal, Simone Pezzuto, and Paris Perdikaris. A-PINNs: Physics-informed neu- ral networks on complex geometries. Eng. Appl. Artif. Intell., 127(PB), February 2024. ISSN 1 2ngappai.2023.10733. URL //doi https:///oi..r//110 1016//
0952-1976. doi: 101
engappai. 2023 100222...032
Alvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter W. Battaglia. Learning to simulate complex physics with graph networks. In Proceedings of the 37th International Conference Machine Learning, ICML'20. JMLR.org, 2020.
W.E. Schiesser. The Numerical Method of Lines: Integration of Partial Differential Equations. Elsevier Science, 2012. ISBN 9780128015513. URL books?1G=2YDICqAAO0..
google..oooooecccooooooo jp.p
Sungyong Seo and Yan Liu. Differentiable physics-informed graph networks, 2019.
Sungyong Seo, Chuizheng Meng, and Yan Liu. Physics-aware difference graph networks for
sparsely-observed dynamics. In International Conference on Learning Representations, 2020.
URL https:://ppenreview www.eeettttiideeoooomo...m....ae Eorum?id
Xingjian Shi, Zhourong Chen, Hao Wang. Dit- Yan Yeung, Wai-Kin Wong, and Wang-chun Woo. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. Ad- vances neural information processing systems, 28, 2015.
Kim Stachenfeld, Drummond Buschman Fielding, Dmitrii Kochkov, Miles Cranmer, Tobias Pfaff, Jonathan Godwin, Can Cui, Shirley Ho, Peter Battaglia, and Alvaro Manchez-Gonzill Learned coarse models for efficient turbulence simulation. In International Conference on Learning Rep- resentations, 2022. URL /ttttp:::/peereeeeeeeeee
net ft htttt//ffrmmmmmmmsssaa -2--
Matthew Thorpe, Tan Minh Nguyen, Hedi Xia, Thomas Strohmer, Andrea Bertozzi, Stanley Osher, and Bao Wang. GRAND++: Graph neural diffusion with source term. International Confer- ence on Learning Representations, 2022. URL
net ftt./oorrmmmim
(zxuu
Alasdair Tran, Alexander Mathews, Lexing Xie, and Cheng Soon Ong. Factorized Fourier neural operators. In The Eleventh International Conference on Learning Representations, 2023. URL
http:::://ppnneveeee...o http://wwweeee.rrrueeeee forum? pewrtmmmid=mIimm.m 14IPa
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gamett (eds.), Ad- vances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL F1113355e223477ee ngs
pdf
Pantelis Vlachas, Georgios Arampatzis, Caroline Uhler, and Petros Koumoutsakos. Multiscale sim- ulations of complex systems by learning their effective dynamics. Nature Machine Intelligence, 03 2022. doi:
13431.
15

---

Published as a conference paper at ICLR 2025
Gege Wen, Zongyi Li, Kamyar Azizzadenesheli, Anima Anandkumar, and Sally M. Benson. U- FNO-An enhanced Fourier neural operator-based deep-learning model for multiphase flow. Advances in Water Resources, 163:104180, 2022. ISSN 0309-1708. doi: https///oi.org/100
atbbjjaadwattes 104180. URL https: /ttp:://www... /wwp///ww///////////////////w/ sci encedi rect. www.cooc/cc.... ence/ httpc////pp1///00011012200000000
Haixu Wu, Tengge Hu, Huakun Luo, Jianmin Wang, and Mingsheng Long. Solving high- dimensional PDEs with latent spectral models. In Proceedings of the 40th International Con- ference on Machine Learning, ICML'23. Jor...rgg 2023.
Mingyuan Yang and John Foster. Multi-output physics-informed neural networks for for- ward and inverse PDE problems with uncertainties. Computer Methods in Applied Mechan- ics and Engineering, 402:1 15041, 2022. ISSN 0045-7825, doi: 2022. 1 115041. URL https:/ /www. sci encedi rect. wtt.ccoccciii...oo ccmm//ccience//rtticleee//e Special Issue Honor of the Lifetime Achievements of Tins-
S0045782222002662
ley Oden.
Jeremy Yu, Lu Lu, Xuhui Meng, and George Em Karniadakis. Gradient-enbanced physics-informed neural networks for forward and inverse PDE problems. Computer Methods in Applied Mechan- and Engineering, 393:1 14823, 2022. ISSN 0045-7825. doi: http:///ooi.rg//100011////
2022.114823. URL https: www. www.wsscieeediicccccc.c... rect. wtt coom/scleence//rtiilll//....
50045792522014488
Yinhao Zhu and Nicholas Zabaras. Bayesian deep convolutional encoder-decoder networks for surrogate modeling and uncertainty quantification. Journal of Computational Physics, 366: 415-447, August 2018. ISSN 0021-9991.
doi: 10.1016/f.epp22018.44.....
URL http:
// 1ttp://0...0111111
04.018
16

---

Published as a conference paper at ICLR 2025
APPENDIX
A NUMERICAL INTEGRATOR
Let us consider complex physical systems, governed by spatiotemporal PDEs of the general form: ( 1) F(1, Dz, u, Vu, Au.....
(A.1)
where t( E RR is the vector of state variable with m components we are interested in, such as velocity, temperature or pressure, defined over the spatiotemporal domain {æ, t} 52 x [0, 7]. Here, ú denotes the derivative with respect to time and F is nonlinear operator that depends on the current state u and its spatial derivatives.
According to the method of lines (MOL), Eq. A. can be rewritten as system of ordinary differ- ential equations (ODEs) by numerical discretization. And the ODE each node can be described by
u(t) = =()) + L. ü(F)dr
(A.2)
Numerous ODE solvers can be applied to solve it, such as Euler forward scheme
*+ ++ =x. + 91 öt
(A.3)
gi F(,, x ......)
where 2 is the state variable time th and öt denotes the time interval between th and + While Euler forward scheme is a first-order precision method, other numerical methods with higher precision such as Kunge-Kutta schemes can also be applied, offering trade-off between computing resource and accuracy. And the second-order Runge-Kuu Kutta (RK2) scheme can be described by
u k++ u²
8t(g1 + +22),
(A.4)
2
where
91 = F(1h, 20 u ...,,
(A.5)
g2 F(ctt1 x. u + blg)....)
and the fourth-order Runge-Kutta (RK4) scheme is followed
" k+1
u²
M0 2g2 2gz + 94)
(A.6)
where
91 F(th x., u ....).
F(th + 51, x, u + 81 91
2
(A.7)
93 F(t x. a + 8t 92
2
F(t + öt, x, r + Stg......
B DISCRETE
OPERATORS
Using the finite difference method to compute the Laplacian works well on regular grids, but it becomes ineffective on unstructured meshes. Therefore, we employ the discrete saplace-Beltram operators to compute the discrete geometric Laplacian non-uniform mesh domain, which are usually defined as
Af:
Σ JEN.
Wis (Fr fi)
(A.8)
d,
17

---

Published as a conference paper at ICLR 2025
c
q
Bij q
C2
C2
p 4.0%
C2
p
C1
j
P
j
C1
Figure A.I: :: (a) Illustration of four discrete nodes (i, j. p, and q) and five edges connecting them. The two angles opposite the edge (i.j) di and Bij. (b) The blue region denotes the area of the polygon formed by connecting the circumcenters (e.g., C1. C22 and etc.) of the triangles around obtuse-angled triangle (i.p.j) adjacent to node i with the blue region extending beyond the boundaries of these triangles. (d) The circumcenter of the obtuse- angled triangle (i. j) is replaced by the midpoint of edge (p, resulting in new polygon known as the mixed Voronoi region.
node i. (c) The scenario involves
where Nr denotes the neighbors of node and f; denotes the value of continuous function f node i. The weights can be described as
cot(ar;) + (or(
Waj
(A.9)
2
where ay and Bi are the two angles opposite to the edge (i. The mass can be defined as d, (i), where (i) denotes the area of the polygon formed by connecting the circumeenters of the triangles around node (i.e., the Voronoi region, shown in PigureA.Tl, is noteworthy that if there is an obtuse-angle triangle p:j) adjacent to node i, its circumcenter will extend beyond itself. In this scenario, the circumeenter of the obtuse-angled triangle (i.p. )) is replaced by the midpoint of edge (p, j). resulting in a new polygon known as the mixed Voronoi region.
C ENCODING BOUNDARY CONDITIONS
Given the governing PDE, the solution depends on both ICs and BCs. Inspired by the padding methods for BCs on regular grids in PeRCNN, we propose novel padding strategy to encode different types of BCs on irregular domains. Specifically, we perform BC padding in both the physical space and the latent space.
Before constructing the unstructured mesh by Delaunay triangulation, we first apply the padding strategy to the discrete nodes in the physical space (i.e., u²), shown in Figure A.2 We consider four types of BCs: Dirichlet, Neumann, Robin, and periodic. All nodes on the Dirichlet boundary will be directly assigned with specified values. For Neumann/Robin BCs, ghost nodes created symmetrically with respect to the nodes near the boundary (along the normal direction the bound- ary) in the physical space, and their padded values depends derivatives in the normal direction. The goal is to ensure the true nodes, the boundary, and the ghost nodes satisfy the BCs in a man- ner similar to the central difference method. For periodic BCs, the nodes near the boundary I'm, are flipped and placed near the corresponding boundary 1'2,, achieving a cyclic effect in message passing (detailed formulations of various BCs are provided in Table Once the padding is com- pleted, Delaunay triangulation is applied to construct the mesh which serves as the graph input for the model. Apart from this, we also apply padding to the prediction (i.e., u (++1 of the model each time step to ensure that it satisfies the BCs before being fed into the model for next-step prediction, as shown in Figure A.3
As for the latent space (i.e., h in GNN block), padding is also applied after each MPNN layer except the last layer. For Dirichlet BCs, the embedding features of nodes on the boundary from the node encoder will be stored as the specified features for padding. For Neumann BCs with zero flux, ghost nodes are created to be symmetric with the nodes near the boundary in both physical and latent spaces. For other Neumann challenge, which expected to be addressed the future. And for periodic BCs, nodes near the boundary flipped to the other side, along with their embedding features.
and Robin BCs, padding in the latent space remains unsolved
18

---

Published as a conference paper at ICLR 2025
Dirichlet BC
Neumann Robin BC
Periodic BC
Node Type
Internal nodes
Edge nodes
Ghost nodes
Figure A.2: Diagram of boundary condition (BC) padding
pad
pad
pad'
u
Model
k++
u 6+1
Figure A.3: Padding during the model rollout
PENN and method both encode BCs. However, there are several key differences between the two approaches, namely,
Methods: PENN designs the special neural layers and modules for the model to satisfy BCs while we develop padding strategy directly applied the features.
Types of BCs: PENN proposes Dirichletl.ay Layer and pseudoinverse decoder for Dirichlet BCs, and NeumannIsoGCN for Neumann BCs. contrast, our padding strategy can be applied to four types of BCs (Dirichlet, Neumman, Robin, periodic), which can be seam- lessly integrated into any GNN model and thus has a better generality.
Accuracy: Both PENN and padding strategy are designed to completely satisfy Dirich- let BCs without any emor. However, neither method can enforce other BCs without error.
During training, the additional for PENN satisfy BCs involves con- structing the ineudo-inverse decoder after the parameters are updated. The efficiency of this process depends on the number of parameters the encoder. In contrast, the cost of our padding strategy involves mere copying the corresponding nodes in the domain during both training and testing, which is small.
Efficiency:
To quantify the time overhead of our padding strategy, taking Burgers' trajectories with 1600 time steps and 503 nodes in the domain and without the padding strategy. As shown in Table A.I the results demonstrate that the overhead introduced by the padding strategy is small.
example, compare the inference time of our models with
D DATASET DETAILS
D.1 SYNTHETIC DATASET FOR LAPLACE BLOCK
We create synthetic dataset to validate the effectiveness of the Laplace block. First, generator 10,000 samples of random continuous functions on a high-resolution (129 x 129) regular grid by cumulatively adding sine and cosine functions, satisfying periodic BCs. The formula of the random
19

---

Published as a conference paper at ICLR 2025
Table A.I: The inference time of models with and without the padding strategy.
Inference time (s)
PhyMPGN (padding)
10.71
PhyMPGN (without padding)
10.02
Ref.
Pred.
0.8
1000
1000
0.4
500
500
0.
0.
0.4
500
500
0.8
1000
1000
Snapshot of a random function f. ground truth of its Laplacian and prediction by our Laplace block.
Figure A.4:
continuous functions is as followed
f(z,v) = Σ Atysin
2n
((i
i-
N
+
(
N 2
2
4==
Biycos 2T ((:
N
N
)
2)
+
j
(A.10)
f(r,y) v)
f(z,v)
max f(x, (Ixz)
where N = 12, Aij, Bi ~ NO 1), and 1-] denotes the floor function in mathematics. The function f(z, y) exhibits a period of 1, satisfying periodic BCs on the computational domain [0, 1] x [0, 1].
By setting various random seeds, generated set of 10,000 samples of the random continuous function. Then, we compute the Laplacian at each node for all samples as the ground truth by employing the 5-point Laplacian operator of the finite difference method:
0
0
-1
0
0
16
0
LA = 12(8x)²
1
16
-
60
16
-1
(A.11)
0
0
16
0
0
0
-1
Note that 70% of the dataset is used as the training set, 20% the validation set, and 10% as the test set. To achieve spatial downsampling, we generate much coarser mesh by COMSOL and obtain the low-resolution simulation by choosing the closest nodes between the high-resolution and coarser mesh points. This approach is consistently applied throughout the paper for all downsampling tasks. Figure A.4 shows the snapshot random function, ground truth of its Laplacican and prediction by Laplace block with N 983 observed nodes.
D.2 SIMULATION DATASET OF PDE SYSTEMS
We train and evaluate PhyMPGN's performance various physical systems, including Burgers' equation, FitzHugn-Nagumo (FN) equation, Gray-Scott (GS) equation and cylinder flow. All the simulation data for PDE systems generated with fine meshes and small time stepping using COMSOL, a multiphysics simulation software based on the advanced numerical methods. Subse- quently, the simulation data is downsampled on both time and space before being fed into the model for training and evaluation. All the simulation data details for PDE systems are summarized in 'Table and Table A.4
20

---

Published as a conference paper at ICLR 2025
Table A.2: S2 physical domain, Ia Ta n, r, are Dirichlet, Neumann, and Robin boundaries, respectively And pl: T22 are each other's 's periodic boundaries Ba, Bus Br, Bp,, B, represent the set of nodes on the boundary of Dirichlet, Neumann, Robin, and Periodic, respectively. For Neumann and Robin BC, the node i.j have been indicated Figure A.2 The node k is located at the intersection of the segment ij and the boundary. If such node does not exist on the discrete point cloud, we can obtain it by interpolation or other means. For Periodic BC, nodes near the boundary I'p1 are flipped to the other boundary p22 achieving cyclic effect message passing.
(g) - a044)) +
My = + 26 fs
16 = Wgg Up = 14
20
Padding Value
0 = 1t
B
14 =
Location (Nodes)
Padding
Edge
Ghost
Ghost
Ghost
Ghost Nodes
No
Yes
Yes
Yes
au + " - = 9k. k B.
14 = u E By2
fk,k E B..
14 ū. Ba
26x
Discrete Form
25x
((x1)) = ((x2)))) 2y pl, 22 E T
= ( x ET. T.
((x) = ū x Td
and (x) + 0 Ou(a:)
8n
Continuous
Form
(u(
òn
Dirichlet
Neumann
Periodic
Name
Robin
21

---

Published as a conference paper at ICLR 2025
Burgers' equation Burgers' equation is a fundamental nonlinear PDE to model diffusive wave phenomena, whose formulation is described by
vsu Vu
(A.12)
where the diffusion coefficient is set to 10-3
There are four different scenarios of Burgers' equation: (1) Scenario A, a square domain with periodic BCs: (2) Scenario B, a square domain with circular cutout, where periodic and Dirichlet BCs are applied: (3) Scenario C, a square domain with a dolphin holphin-shaped cutout, where periodic and Dirichlet BCs are applied; and (4) Scenario D, square domain with semicircular cutouts on both the left and right sides, where periodic and Neumann BCs are applied. The diagram of irregular domain and hybrid BCs for the four scenarios are illustrated Figure ICs for Burgers' equation are generated similarly to the synthetic dataset for the Laplace block (Appendix D.I). We generated 20 trajectories with different ICs for each scenario, using 10 for training and TO for testing. To explore the dataset size scaling behavior of our model, generated 50 trajectories with different ICs in total for training. Each trajectory consists of 1600 time steps with a time interval of = 0.001 for the model.
FN equation FizzHughNNaguno (FN) equation is an important practiondiiffssin equation, de- scribed by
ú P Au + - v è = Av + (u - B
(A.13)
where the diffusion coefficients fy and fou are respectively and 10, and the reaction coefficients a and | are respectively 0.01 and 0.25.
The system, governed by FN equation and defined on a square computational domain with periodic BCs, begins with random Gaussian noise for warmup. After period of evolution, time trajectories are extracted to form the dataset. We generated 13 trajectories with different ICs, using 3 for training and 10 for testing. Each trajectory consists of 2000 time steps with time interval of öt 0.004 for the model.
GS equation Gray-Scott (GS) equation is a mathematical model used to describe changes in sub- stance concentration in reaction-diffusion systems, which is governed by
à Hru Au un + F(1 - u)
(A.14)
Mk Au u (F + k)v
where the diffusion coefficients pa and μ4 are respectively 2.0 10-5 and 5.0 10-6, while the reaction coefficients JF and k are respectively 0.04 and 0.06.
The system, governed by GS equation and defined on a square computational domain with periodic BCs, starts the reaction from random positions. The time trajectories of the reaction' process are extracted form the dataset. We generated 16 trajectories with different ICs, using 6 for training and 10 for testing, Each trajectories consists of 2000 time steps with time interval of öt 0.5 for the model.
Cylinder flow The dynamical system of two-dimensional cylinder flow is governed by Navier- Stokes equation
u - - Vu 1 Vp / Au
(A.15)
where the fluid density p is 1, the fluid viscosity is 5.0 x 10-3 and the external force f is O. We are focusing on to generalize over the inflow velocities Um of fluids while keeping the fluid density p. the cylinder diameter D = 2, and the fluid viscosity μ constant. According to formula Re = pl D/uu ju, generalizing over the inlet velocities Um also means to generalize across different Reynolds numbers.
The upper and lower boundaries of the cylinder flow system are symmetric, while the cylinder surface has no-slip boundary condition. The left boundary serves as the inlet, and the right bound- ary as the outlet. We generated 13 trajectories with different Re values, using trajectories with Re [160, 240, 320, 400) for training and 9 trajectories for testing. The testing sets are divided
22

---

Published as a conference paper at ICLR 2025
Table A.3: Details of simulation data of four PDE systems.
Burgers
FN
GS
Cylinder Flow
Domain
[0, 1]2
[0, 128)
00. 1]
[0, 16] [0, 8]
Original nodes
12659
15772
37249
36178
Down-sampled nodes
503
983
4225
1598
Original öf
0.0005
0.002
0.25
0.025
Down-sampled öt
0.001
0.004
0.5
0.05
Time steps
1600
2000
2000
2000
Train / Test sets
10/ 10
3/
6/ 10
4/9
Table A.4: Details of simulation data of four Burgers' scenarios.
Scenario A
Scenario B
Scenario C
Scenario D
Domain
[0, 1|2
[0, 1]
[0, 1]2
[0, _]
Original nodes
12659
14030
14207
15529
Down-sampled nodes
503
609
603
634
Original öt
0.0005
0.0005
0.0005
0.0005
Down-sampled 5z
0.001
0.001
0.001
0.001
Time steps
1600
1600
1600
1600
Train / Test sets
10/ 10
10/ 10
10/ 10
10/ 10
Table A.5: The MSEs of the three PDE systems.
Burgers
FN
GS
DeepONet
1.68e-2
9,47e-2
4.04e-2
PA-DGN
8.66e-1
7.19e+2
2.93e+0
MGN MP-PDE
9.19e-4
5.54e-2
4.44e-3
5.88e-3
2.88e-2
2.82e+0
PhyMPGN (Ours)
2.99e-4
2.82e-3
4.05e-4
Lead f
67.5%
90.2%
90.9%
Table A.6: The MSEs of model on different dataset size effect (Burgers).
No. trajectories
5
10
20
50
PhyMPGN (ours)
6.35e-4
2.99e-4
1.60e-4
4.43e-5
into three groups based Re; 3 trajectories with small Re 200, 280, 360), trajectories with medium Re = [440, 480, 520). and 3 trajectories with large Re [600, 800, 1000]. Each trajectory consists of 2000 time steps with time interval of = 0.05 for the model, except for the trajectories with larger Reynolds numbers (Re [600, 800, 1000]), which have a time interval of öt 0.005.
E MODELING PDE SYSTEMS
We train and evaluate PhyMPGN's performance various physical systems, including Burgers' equation, FN equation, GS equation and cylinder flow. Leveraging four NVIDIA RTX 4090 GPUs, the training process is completed within range of 4 to 15 hours. And the inference time is under one minute on a single GPU. We also compare the performance of PhyMPGN against existing approaches, such DeepONet, PA-DGN, MGN, MP-PDE.
23

---

Published as a conference paper at ICLR 2025
Table A.7: The MSEs of Burgers' equation with various boundary conditions.
Scenario A
Scenario B
Scenario C
Scenario D
DeepONet
1.68e-2
1.37e-2
1.16e-2
2.22e-2
PA-DGN
8.66e-1
1.10e+0
1.03e+0
Nan
MGN
9.19c-4
1.68e-3
1.89e-3
2.70e-3
MP-PDE
5.88e-3
6.59e-3
5.11e-2
1.63e-2
PhyMPGN (Ours)
2.99e-4
2.89e-4
3.06e-4
9.30e-4
Lead f
67.5%
82.8%
83.8%
65.6%
Table A.8: The MSEs of cylinder flow for three groups of Re values.
Re
2200, (800,60)
[440, 480, 520]
[600, 800, 1000]
DeepONet
9.07e-2
3.10e-1
1.53e+0
PA-DGN
7.58e+3
Nan
Nan
MGN
1.73e-1
2.17e-1
8.88e-1
MP-PDE
7.23e-3
2.77e-1
5.96c-1
PhyMPGN (Ours)
2.14c-4
3.56e-2
1.25e-1
Lead
97.0%
83.4%
79.0%
Generalization test over ICs: To test the models' generalizability over different random ICs, we train PhyMPGN and baseline models the three physical systems: Burgers' equation, FN equation and GS equation. All these PDE systems have a square domain with periodic BCs. The MSEs of PhyMPGN and baseline models are listed in Table A.5 The value of "Lead" in the tables is defined as (MSE, MSE&)/MSE, where MSE, denotes the best results among all models and MSE, denotes the second- best result. To explore the dataset size scaling behavior of our model, using Burgers' equation as an example, we train our model with different numbers of trajectories: 5, 10, 20, and 50. The numerical results are presented Table A.6
To validate our model's ability encode different types of BCs, we use Burgers' equation as an example and consider three additional scenarios with irregular domain and hybrid BCs, resulting in a total of four scenarios, illustrated in Figure [5] (1) Scenario A, square domain with periodic BCs, as mentioned in the previous paragraph; (2) Scenario B, square domain with a circular cutout, where periodic and Dirichlet BCs are applied; (3) Scenario C, square domain with dolphin-shaped cutout, where periodic and Dirichlet BCs are applied; and (4) Scenario D, a square domain with semicireular cutouts on both the left and right sides, where periodic and Neumann BCs are applied. The MSEs of PhyMPGN and baseline models are listed in Table A.7
Handling different BCs:
Generalization test over the Reynolds number: We also evaluate the ability of our model to generalize over the Reynolds number for the cylinder flow, governed by the Naiver-Stokes (NS) cquations. We generate 4 trajectories with Re = [160, 240, 320, 400] for training, and trajectories for testing. The testing sets are divided into three groups based on Re: 3 trajectories with small Re = |200, 280, 360). 3 trajectories with medium Re = [440, 480, 520], and 3 trajectories with large Re |600, 800, 1000). The MSEs for the three groups of PhyMPGN and baseline models listed Table A.8 To evaluate the extrapolation capability, our model trained over the time range 0 100s is tested the 200-second trajectory with Re 480. We observe slow decline of model's correlation in Appendix Figure A.5 The correlation is 0.95 at = 100s and decreases to 0.85 at 200s.
Computational efffcciecy: The computational cost of the Laplace block and the padding strategy is minimal. The computation of Mesh Laplace in Laplace block includes only matrix multiplication f. where are the node features and T.g. is a weight matrix related mesh geometric that can
24

---

Published as a conference paper at ICLR 2025
100s
1.0
0.8
Correlation
0.6
Ours
MGN
0.4
MP-PDE
0.2
Mmm
PA-DGN
DeepONet
100
150
200
Figure A.5: The correlation of our model trained on the 100-second trajectories but tested on the 200-second trajectory with Re = 480.
Table A.9: The MSEs averaged on the testing sets with Re [440, 480, 520] of the four models in ablation study: Ours, Model A, B, and C.
Model
MSE
PhyMPGN (Ours)
3.56e-2
Model A
1.57e-1
Model B
1.06e-
Model C
2.50c-1
Table A.10: The inference time (seconds) and MSE of four methods for Burgers' equation.
Total time (10 trajectories)
Time per trajectory
MSE
MGN MP-PDE
4.01
5.19
9.19c-4
0.33
1.00
5.88e-3
COMSOL
19.5
195
4.91e-5
PhyMPGN 10.71
13.64
2.99e-4
Table A.11:
The MSEs of our models employing different numerical integrators.
Model
Re 200,, 280, 360]
Re [440, 480, 520]
Re [600, 800, 1000]
PhyMPGN (Euler)
1.95e-3
1.07e-1
1.41e-1
PhyMPGN (RK2)
2.14e-4
3.56e-2
1.25e-1
PhyMPGN (RK4)
1.21e-4
1.97e-2
9.26e-2
be calculated offline. The cost of the padding strategy involves merely copying the corresponding nodes in the domain and is small.
For comparative analysis of computational efficiency, we select MGN and MP-PDE as baselines from neural-based methods, and COMSOL as a baseline from classical numerical methods. Taking Burgers' trajectories with 1600 time steps and 503 nodes in the domain an example, compare the inference times between these three baselines and our model (PhyMPGN). The inference time and MSE of these four methods NVIDIA RTX 4090 GPU and an 13900 CPU. Additionally. the ground truth is generated with high resolution in both time and space using COMSOL with the implicit scheme.
shown in Table A. All evalutions conducted on a single
25

---

Published as a conference paper at ICLR 2025
Table A.12:
The training and inference time of models employing different types of numerical integrators for cylinder flow problems.
Inference time
Inference time
Training time
per trajectory
(5 trajectories)
PhyMPPN (Euler)
5.2 h
8.98
13.47 s
PhyMPGN (RK2)
14.4 h
17.57
27.08 s
PhyMPGN (RK4)
39.3 h
34.99
53.49 s
Table A.13: The MSE and the training time with different segment sizes M.
M
Re
280,,280,
360]
Re
(440, 480, 520]
Re [600, 800, 1000]
Training time
10
2.43e-3
1.48e-1 1
2.42e-1
8.3 h
15
9.34e-4
3.05e-2
1.34e-1
15.0 h
20
2.14c-4
3.56e-2
1.25c-1
14.4
MGN predicts one step given one step as input, similar to PhyMPGN. However, due the addi- tional Laplace block, padding strategy, and RK2 scheme, PhyMPGN has more inference time than MGN. On the other hand, MP-PDE predicts Tw steps (with Tie = 20) single forward pass using Tw steps as input, resulting in the least inference time among neural-based models. Overall, these neural-based models can greatly improve efficiency by simultancously inferring multiple trajecto- ries. The results from COMSOL, a multiphysics simulation software based on advanced numerical methods, presented in the comparison, are obtained using an implicit scheme.
In summary, PhyMPGN achieves the lowest eπror among neural-based models with reasonable time overhead as a trade-off, while COMSOL delivers the least error among these four methods but the cost of significantly higher time overhead. Notably, simulate the PDE system, COMSOL requires the complete governing PDE formulas, whereas the three neural-based methods do not, as they learn the dynamics from data.
We study the effectiveness and contribution two key components in our model, namely, the learnable Laplace block and the BC padding strategy. Specifically, we conducted ab- lation studies on the cylinder flow problem, comparing the performance of four models: (1) the full model, referred to as Ours; (2) the model only without the Laplace block, referred to as Model A; (3) the model only without the padding strategy, referred to as Model B; (4) the model without both the Laplace block and the padding strategy, referred to as Model C. We train these models on the same cylinder flow dataset described previously and evaluate them on the same testing sets. The error distribution and correlation curves of these four models averaged on the testing sets with Re = [440, 480, 520]
Ablation study:
are shown in Figure The corresponding MSEs are listed in Table A.9
Investigation on numerical integrators: Additionally, using the same datasets as before, we in- vestigate the impact of three types of numerical integrators employed in model on performance: the Fuler forward scheme, the RK2 scheme, and the RK4 scheme. The MSEs our models em- ploying these three numerical integrators are presented in Table [A.I We trained our model for 1600 epochs using four NVIDIA RTX 4090 GPUs and performed inference for 2000 time steps per trajectory on single GPU. Their training and inference time are listed in Table A. All inference times are under one minute. Thanks to the parallel capabilities of GPUs, the inference speed can be further accelerated by processing batches simultaneously.
The impact the time segment size M: Many previous methods (Seo et al. 2020 HAN et al.,
2022 Geneva & Zabaras 2022 Ren et al. 2022 Rao et al. 2023) unroll models to predict multiple steps within short time segment then backpropagate to learn the dynamics of physics systems. In- tuitively, longer time segments can be advantageous for capturing long-term dependencies, thereby improving the model's term- prediction capabilities. However, excessively long segments may introduce challenges during training, such as increased memory consumption and difficulties in
26

---

Published as a conference paper at ICLR 2025
Table A.14: The number of trainable parameters of all models.
Burgers
FN
GS
Cylinder Flow
DeepONer
219k
295k
422k
948k
PA-DGN
194k
194k
260k
1025k
MGN
175k
175k
267k
1059k
MP-PDE
193k
193k
253k
1069k
PhyMPGN (Ours)
157k
161k
252k
950k
achieving convergence. To investigate the impact of the segment size M on the results and com- putational requirements, we conduct experiments using the cylinder flow datasets in the ablation study. We train our model with segment sizes M 10. 15, 20, and the MSEs on the testing sets and the training time are presented in Table [A.13] which shows that the bigger the segment size M, better the model's prediction accuracy. Additionally,, since we need to adjust the batch size to fit the GPU memory as the segment size M varies, the training time does not scale linearly with M. Specifically, the batch sizes for M = 15 and AM 20 are the same, while the batch size for M 10 is double that of the other two configurations. Therefore, the training time differences also reasonable.
In all experiments, the numbers of trainable parameters and training iterations for all models kept as consistent as possible to ensure a fair comparison, as listed in the Table [A.14] Snapshots of ground truths and predictions of component from our model in all experiments shown in Figure A.6
27

---

Published as a conference paper at ICLR 2025
Ref.
Ref.
Pred.
Pred.
t=
1500
500
1500
d
Ref.
Ref.
Pred.
Pred.
= 500
t=1000
1500
t=
t= 500
t= 1000
1500
Ref.
Ref.
Pred.
Pred.
t=
= 600
1200
1800
t= 600
I= 1200
1800
Pred.
t=
600
t= 1200
Figure A.6: Snapshots of component of ground truths in several simulations and predictions from our model. (a, b, c, d) Burgers' equation in various domains. (e) FN equation. (f) GS equation. (g) Cylinder flow with Re 440.
28


Reviews:
[{"review_id": "XynLI72QZm", "reviewer": "Reviewer_QKXd", "summary": "The work focuses on solving PDE with low-resolution unstructured meshes. In particular, the authors incorporate the discrete Laplace–Beltrami operator to express the diffusion process and a specific padding method to satisfy boundary conditions.\n\nThe experimental evaluation demonstrates that the introduced Laplace block (GNN + Laplace Beltrami operator) is strongly expressible to approximate Laplacian compared to the raw numerical scheme and machine learning models considered in the evaluation. In addition, the method has a stable prediction for long-time series prediction with generalization regarding initial condition, boundary condition, and Reynolds number to some extent.", "questions": "* The method seems quite simple yet powerful, as seen in the numerical experiments. However, the reviewer wonders where the expressibility and generalizability come from. Could the authors explain the reason for high performance, connecting with the method proposed in the paper?\n* Is the method generalizable regarding mesh resolution? For instance, learning on coarse meshes and predicting on finer meshes or vice versa would be interesting to see, while that would be out-of-scope of the work.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The numerical experiments are thorough, and the proposed method demonstrates its high performance in various tasks and generalization scenarios. In particular, the method can extrapolate the Reynolds number to some extent, which is impressive.\n* The method is simple and easy to understand. Due to its simplicity, the reviewer expects the method to be compatible with other GNN-based methods, which would be solid knowledge to the community.", "weaknesses": "* The novelty of the proposed method is limited. The connection between GNN and the Laplace (–Beltrami) operator is not new (e.g., baselines in Section 4.1 and [Li et al. AAAI 2018 https://arxiv.org/abs/1801.07606 ]). Since the performance is better than the baseline, there might be a big improvement for the community, but it is unclear what it is. The author could have elaborated on what were the problems of existing methods and how they overcame the difficulties.\n* The method to treat boundary conditions in unstructured meshes is also not new and proposed in [Horie et al. NeurIPS 2022 https://openreview.net/forum?id=B3TOg-YCtzo ]. At least the author should mention existing works and possibly compare the proposed method with them to discuss the pros and cons.\n* For time-series training, the motivation to incorporate the first step in the loss (Eq 7). As mentioned in the paper, [Brandstetter et al. 2022] proposed a method called the pushforward trick, which computes the loss using only the last time step, demonstrating the trick performs better than adding noise. In the present work, Eq 7 seems to contradict the previous works. Thus, the authors should clarify the reason why they have the first step with noise and demonstrate the proposed one is better.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MiQcYuGTnW", "reviewer": "Reviewer_cFxe", "summary": "The paper is related to PDE systems solution in two-dimensional space on irregular domains. The problem is addressed with the mix of GNNs and classical numerical schemes. The authors propose several novel elements, such as trainable Laplace block, second order Runge-Kutta scheme for simulating temporal dynamics and a padding strategy to incorporate boundary conditions of different types. These improvements allowed the authors to achieve new quality of simulation. The obtained model has good generalization capabilities over initial conditions and over the unseen types of flow (such as another Reynolds number)", "questions": "* Why in eq. (7) for loss function you use only first and last step? What will change if we will compare full rollouts?\n* How the padding strategies will work for boundary conditions on irregular shapes?\n* What do you mean when you say that boundary conditions are used as a priori physics knowledge?\n* Will the Laplace block still be useful for PDEs without the Laplace operator?\n\nalso look for the suggestions arising from weaknesses", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The authors have shown that spatiotemporal dynamics can be modeled by GNN in spatial domain together with 2nd order numerical scheme in temporal domain\n* The authors proposed and tested a combination of reasonable improvements to the previous works with similar techniques (Pfaff et al. 2021). There key additions are Laplace block and padding strategy for different types of boundary conditions\n* The proposed improvements allowed authors to outperform the competitors in terms of quality for chosen test problems\n* Generalization and ablation studies substantiate the authors contributions", "weaknesses": "The limitations of the work were not addressed properly. In particular:\n* applicability to the PDE systems without the Laplace operator\n* simulation for the big number of time steps (for example this occurs for weather simulations)\n* boundary conditions padding for the domain with non-straight boundary – it’s not clear how to work with them\n\nAlso the experiments are insufficient:\n* It’s important to investigate the computational requirements of the proposed approach for training and inference in dependence on the important parameters (number of time steps, size of the training dataset, etc.) in order to assess its applicability in practice\n* Also it would be useful to understand how the segment size M in 3.4 influences the results and computational requirements\n\nAlso the paper is hard to read and it needs some restructurization. Probably  the introduction can be shortened a little bit together with the beginning of the abstract and authors could explain in more details how the model is trained and how different types of boundary conditions are introduced for curved boundary", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EGRdRVFyGT", "reviewer": "Reviewer_LbRf", "summary": "The manuscript introduces a way to approximate a non-linear PDE operator with a GNN block completed with a Laplace operator block.\nThe examples include a varity of linear and linear PDEs.", "questions": "1.\tCould you give mathematical and physical interpretation of $z_i$, Eq. (5)? What activation functions are used to parametrize $z_i$? How do you formally differentiate between coarse and fine meshes?\n2.\tSection 3.4 implies that over all there are $T/M$ loss functions. Are these loss values combined together somehow? What is ground truth data in this setting?\n3.\tCould you include experiments with $F = \\text{div} (D \\nabla u)$ with either $D(x,y)=1 $ or $D(x,y)=1+4x+5y$ (or other varying from 1 to 10)? It will give to the reader a good understanding of approximation properties of Eq.(6).\n4.\tThe discussion in the manuscript covers only 2D spatial data. Does the proposed method admit a generalization to the 3D case? How the Laplace block will like in this case?\n5.\tThe RK2 and RK4 schemes are known to be applicable to non-stiff problems only due to stability limitations. What scheme would you suggest for a stiff PDE of form Eq. (1) and how would you implement such a scheme?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The way to approximate a non-linear PDE operator with a GNN block completed with a Laplace operator block looks fairly original.", "weaknesses": "1.\tThe ultimate goal of the manuscript is to learn the nonlinear operator $F$.  However, it is not explained why it is needed while $F$ is known and given.\n2.\tThe way how the authors approximate the nonlinear operator F, Eq. (6), evidently has some limitations. For example, it won’t work well when $F = (\\Delta u)^2 $ and so on.  Consider being more specific in defing the operator F in Eq. (1).\n3.\tSection 4 is difficult to follow since the actual PDEs and their coefficients are not specified.\n4.\tIt is not really explained why paddling is needed to incorporate boundary conditions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5JVHmg36ri", "reviewer": "Reviewer_m5AJ", "summary": "I love the paper for its outperforming MGN; it has a solid baseline by DeepMind! It introduces PhyMPGN, a Physics-encoded Message Passing Graph Network that effectively models spatiotemporal PDE systems on irregular meshes using limited data. By integrating physics through a learnable Laplace-Beltrami operator and a novel boundary condition padding strategy, the approach ensures physically accurate predictions. PhyMPGN significantly outperforms existing methods, achieving over 50% performance gains and demonstrating strong generalization across various PDEs and conditions. The model's efficiency and robustness make it a valuable advancement for scientific simulations where data is sparse or complex geometries are involved.", "questions": "I have several questions to help me understand the method thoroughly. Firstly, how will the model behave if we delete (or do not use) the Mesh Laplace? Will it still be better than MGN? Secondly, why second-order RK for MOL? What about RK4 or forward Euler? Do these different choices matter? Thirdly, I understand graph Laplace preserves some diffusion physics, which is excellent and universal enough. Still, for examples of inviscid first order PDE such as wave/ advection equation, or inviscid Burger equation,  Euler equations, will add graph Laplacian make the traveling feature blur?", "limitations": "", "rating": 10, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "Outperform a solid baseline; writing is good, presentation is good. The method is clear and new. The strength of this paper lies in its innovative integration of physics-based knowledge into graph neural networks, which enables accurate and efficient modeling of complex spatiotemporal PDE systems on irregular meshes with limited data. By employing a learnable Laplace block and a novel boundary condition padding strategy, PhyMPGN ensures solutions remain physically consistent and precise, overcoming limitations of traditional and purely data-driven methods. Additionally, the model's demonstrated ability to generalize well to different PDEs, geometries, and conditions, along with its significant performance gains over existing techniques, highlights its robustness and versatility for real-world scientific applications", "weaknesses": "There is some need for some clarification when it is for convection-dominant problem.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bJsUdoTmuJ", "reviewer": "Reviewer_zATM", "summary": "The paper introduces a graph learning approach called Physics-encoded Message Passing Graph Network (PhyMPGN) designed to model spatiotemporal PDE systems on coarse unstructured meshes using small training datasets. Authors incorporate a learnable Laplace block that encodes the discrete Laplace-Beltrami operator to constrain the GNN learning within a physically feasible solution space. Additionally, a boundary condition padding strategy is proposed to handle different types of boundary conditions, enhancing model convergence and accuracy. Extensive experiments demonstrate that PhyMPGN accurately predicts various spatiotemporal dynamics on coarse unstructured meshes with complex BCs, outperforming other baseline models with considerable gains.", "questions": "1.\tCould the authors clarify their statement regarding the limitations of traditional numerical methods? How does their approach compare with implicit numerical schemes in terms of stability and time-stepping requirements?\n2.\tCan the authors provide insights into the computational performance of their method? \n3.\tWhat are the main obstacles in generalizing PhyMPGN to three-dimensional problems? How might the current architecture need to adapt to handle the increased complexity?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1.\tThe integration of physics-encoded components within a message-passing GNN addresses the challenges of modeling spatiotemporal dynamics on unstructured meshes.\n2.\tThe learnable Laplace block that encodes the discrete Laplace-Beltrami operator ensures solutions remain within a physically feasible space.\n3.\tThe experiments are thorough, showcasing the model's ability to generalize across various spatiotemporal dynamics. The ablation studies further strengthen the validity of the proposed methods.\n4.\tThe model demonstrates strong generalization capabilities despite being trained on small datasets.", "weaknesses": "1.\tThe authors state that traditional numerical methods require fine meshes and small time stepping. This overlooks implicit schemes, which can handle larger time steps while maintaining stability.\n2.\tThe paper does not provide a comparative analysis of the computational speed or efficiency of PhyMPGN.\n3.\tThe rationale behind focusing solely on coarse meshes is not explained.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)