Bi-Stride Multi-Scale Graph Neural Network for Mesh-Based Physical Simulation
Authors: Yadi Cao
Menglei Chai
Minchen Li
Chenfanfu Jiang
Venue: Submitted to ICLR 2023 (2023)
Keywords: GNN
physics-based simulation

Abstract: Learning physical systems on unstructured meshes by flat Graph neural networks (GNNs) faces the challenge of modeling the long-range interactions due to the scaling complexity w.r.t. the number of nodes, limiting the generalization under mesh refinement. On regular grids, the convolutional neural networks (CNNs) with a U-net structure can resolve this challenge by efficient stride, pooling, and upsampling operations. Nonetheless, these tools are much less developed for graph neural networks (GNNs), especially when GNNs are employed for learning large-scale mesh-based physics. The challenges arise from the highly irregular meshes and the lack of effective ways to construct the multi-level structure without losing connectivity. Inspired by the bipartite graph determination algorithm, we introduce Bi-Stride Multi-Scale Graph Neural Network (BSMS-GNN) by proposing \textit{bi-stride} as a simple pooling strategy for building the multi-level GNN. \textit{Bi-stride} pools nodes by striding every other BFS frontier; it 1) works robustly on any challenging mesh in the wild, 2) avoids using a mesh generator at coarser levels, 3) avoids the spatial proximity for building coarser levels, and 4) uses non-parametrized aggregating/returning instead of MLPs during pooling and unpooling. Experiments show that our framework significantly outperforms the state-of-the-art method's computational efficiency in representative physics-based simulation cases.

Under review as a conference paper at ICLR 2023
BI-STRIDE MULTI-SCALE GRAPH NEURAL NETWORK FOR MESH-BASED PHYSICAL SIMULATION
Anonymous authors Paper under double-blind review
ABSTRACT
Learning physical systems on unstructured meshes by flat Graph neural networks (GNNs) faces the challenge of modeling the long-range interactions due the scaling complexity w.r.t. the number of nodes, limiting the generalization under mesh refinement. On regular grids, the convolutional neural networks (CNNs) with U-net structure can resolve this challenge by efficient stride, pooling, and upsampling operations. Nonetheless, these tools are much less developed for graph neural networks (GNNs), especially when GNNs are employed for learn- ing large-scale mesh- mesh-based physics. The challenges arise from the highly irreg- ular meshes and the lack of effective ways construct the multi-level structure without losing connectivity. Inspired by the bipartite graph determination algo- rithm, introduce Bi-Stride Multi-Scale Graph Neural Network (BSMS-GNN) by proposing bi-stride as a simple pooling strategy for building the multi-level GNN. Bi-stride pools nodes by striding every other Srradtt-Fiirt-Searee (BFS) frontier; it 1) works robustly on any challenging mesh in the wild, 2) avoids us- ing a mesh generator at coarser levels, 3) avoids the spatial proximity for build- ing coarser levels, and 4) uses con-parametrized aggregating/returing instead of MLPs during pooling and unpooling. Experiments show that our framework sig- nificantly outperforms the state-of-the-art method's computational efficiency in representative physics-bare based simulation cases.
INTRODUCTION
Simulating physical systems through numerically solving partial differential equations (PDEs) plays key role in various science and engineering applications, ranging from particle-based (Tiagg et al.) 2016) and mesh-based (Li et al. 2020a) solid mechanics to grid-based fluid (Bridson) 2015) and aero (Cao et al., 2022) dynamics. Despite the extensive successes in improving their stability, accu- racy, and efficiency, numerical solvers are often computationally expensive for time-sensitive appli- cations, especially iterative design optimization where fast online inferring is desired.
Recently, machine learning approaches have demonstrated impressive potential in improving the efficiency of inferring physical states with competitive accuracy. Representative methods include end-to-end frameworks (Obiols-Sales et af. (2020) and those with Aonysics-informed neural networks (PINNs) (Raissi al., [2019] Karniadakis et al. 2022 Sun et al. 20220] Gao al. 2021. Many
existing works apply convolutional neural networks (CNNs) (Fukushima & Miyake 1982) to learn
physical systems on two- or three-dimensional structured grids (Kim al.
2019] Fotiadis et al.
2020 Gao et al. 2021: Guo et al.) 2016 Tompson et al., 2017). It is generally recognized that CNNs exhibit strong performance on handling local information with convolution and global infor- mation with pooling/upsampling. However, the strict dependency on regular domain shapes makes non-trivial to be applied on unstructured meshes. Although possible to deform the domains to rectangular shapes to apply CNNs (Gao et al.) (022]) other models, such as NeuralOpera- torNets (Li et al. 2022), the challenge remains for domains with complex topologies, which are common in practice.
On the other hand, graph neural networks (GNNs) have been considered as a natural choice for physics-based based simulation unstructured meshes (Battaglia al. 2018 Salbute-Peree et al. 2020 Gao et al., 2022; Harsch & Riedelbauchi 2021 [Paff et al.) 22020] Sanchez-Gonzalez et al. 2018 2020). However, all the above methods the flat GNN that faces two challenges when the graph

---

Under review as a conference paper at ICLR 2023
size increases: (1) Oversmoothing: the graph convolution be seen low-pass filter that suppresses the signal with higher frequency than a certain value (Chen et al. 2020 Li et al. 2020b).
Multiple passes of graph convolution then become iterative projection onto the eigenspace of the graph where all higher frequency signals are smoothed out, which also makes training harder. (2) Complexity: Under mesh refinement, not only that more nodes are there be processed, but the message passing (MP) iterations also grow linearly to propagate information to the same physical distance (Fortunato et al. 2022). As a result, a quadratic complexity becomes inevitable for both the running time and the memory to store the computational graph.
To mitigate these limitations, researchers recently start investigating multi-scale GNNs (MS- INNG) for physics-basee based simulation (Fortunato et al. 2022; Li et al., 2020b; Lino et al.) 2021 Liu et al.
2021 Lino et al. 2022a b). The multi-scale approach is appealing as tackles the oversmoothing
issue by building sub-level graphs on coarser resolutions, which lead to longer range interaction and naturally fewer MP times. However, pooling and adjacency building should be conducted carefully
to avoid introducing partitions into the coarser levels (Gao & Jil 2019). which stops information exchange across the separated clusters. Existing solutions inc utilizing the spatial proximity for
building the connections the coarser levels (Lino et al. [202 Liu al. 2021 Lino et al. [022a b),
or generating coarser meshes for the original geometry (Fortunato et al. 2022 Liu et al. (0211). and
randomly pooling nodes then applying Nyström approximation for the original adjacency matrix (Li
ett al.][ 2020b). However, all of them suffer from limitations: the spatial proximity can result in wrong connections across the geometry boundaries; the mesh generation is laboring and often unavailable for unseen meshes; and the random pooling may introduce partitions in the coarser levels.
We observe that all the aforementioned limitations originate from pooling and building connections at coarser levels. To the best of our knowledge, no existing work can systematically generate multi- scale GNNs with arbitrary levels for an arbitrary geometry in the wild while completely avoiding cutting or wrong connections across the boundaries. To this end. in this work, we introduce simple yet robust and effective pooling strategy, bi-stride. Bi-stride is inspired by the bi-partition determi- nation in DAG (directed acyelic graph). It pools all nodes on every other BFS frontiers, such that a 2nd -oweeed adjacency enhancement conserves all the connectivity. We also accompany bi-stride with non-parameterized aggregating/returiing method to handle the transi- tion between adjacent levels to decrease the model complexity. Our framework, namely Bi-Stride Multi-Scale Graph Neural Network (BSMS- (NN), is tested on three benchmarks (CYLINDERFLOW AIRFOIL, and DEFORMINGPLATE) from GraphMeshNets and INFLATINGFONT, a new dataset of inflating elastic surfaces with many self-contacts. In all cases, BSMS-GNN shows dominant ad- vantage in memory footprint and required training and inference time compared to alternatives.
2 BACKGROUND AND RELATED WORKS
GNNs for Physics-Basse Simulation GNNs are first applied to physical simulation to learn the
behaviors of particle systems, deformable solids, and Lagrangian fluids (Battaglia et al. al, [
et al. 2016 Mrowca et al. 20188 (aanchez-Gonzalez et al.] (020).. Notably, the generalized Message Passing (Sanchez-Gonzalez et al. (2018) is broadly accepted for information propagation. Based on that, GraphMeshNNess (Plaff al., 2020) sets a milestone for learning mesh-based simulation. Fol- lowing GraphMeshNNets, which predicts a single forward timestep, there have been several variants, including 1) solving forward and inverse problems by combining GNNs with PINNs (Gao et al.) 2022]. 2) predicting long-term system states combined with GraphAutoEncodee (GAE) and Trans- former (Han et al., 2022). 3) predicting steady states with multi-layer readouts (Harsch & Riedel- bauch 2021), and 4) up-sampling from coarser meshes with differentiable simulation (Belbute-Pers ctt al. 2020). Yet still, with flat GNNs, the quadratic computation complexity on finer meshes poses great challenges. We claim that adopting multi-level structure is an effective solution.
Multi-Scale GNNs It is common to apply GNNs with multi-level structures in various graph- related tasks, such as graph classification (w et al. 2020. Mesquita et al., 2020 Zhang et all] 2019). GraphUNet (GUN) (Gao & Ji 2019) first introduces the UNet structures into GNN with trainable scoring module for pooling, it also has a 2nd -po-eeed adjacency enhancement to reduce the chance of losing connectivity. A few works have investigated multi-scale GNNs (MS-GNNs) for paysics-based simulation. Specifically, Fortunato et al. (2022) and Liu et al. (202T) define two- and multi-level GNNs, respectively, for physics-based simulation, but both of them rely pre-
2

---

Under review as a conference paper at ICLR 2023
9x
9e
9043
A
Gr11
(a) learnallc pooling
(b) pooling by rasterization
(c) pooling by spatial proximity
Figure 1: Issues of existing multi-level GNNs. (a) A learnable pooling (Gao & Jil 2019) may lead loss of connectivity even after 1 19-order enhancement. (b) A pooling by rasterization (Lino al.,
2021 2022a b) and (c) by spatial proximity (Liu et al. 2021 Fortunato et al. 2022) can lead to wrong connections across the boundaries the coarser level.
generated coarse meshes. Lino et al. (2021 2022a b) use the original mesh at the first level and project it to regular grids (MS- (MS-GNN- GRID) at the coarser levels. [Li et al. (2020b) adopt multi-level matrix factorization to generate the kernels at arbitrary levels without requiring mesh generators or K-nearest ueighbor (K-NN) interpolations. Concerning building the connections and hicrarchics on point clouds with radius samplers, there are representative works such GNS Sanchez- Sanchez-Gonzalezz et al. 2020), PointNet Oi et al. (2017a), PointNet++ Qi et al. (2017b), and GeodesicCony Masci al. 2015). Still, these methods by construction suit better cases without meshes, such particle fluid simulations.
Motivations of Our Method We present an overview of representative GNN architectures with U-net structure in Fig. [ Two major disadvantages we observed are: 1) easy loss of connectivity by pooling, even with 2d -powered adjacency enhancement; and 2) lack of direct connections between pooled and unpooled nodes, leading to additional edges built by the spatial proximity for transition between levels.
For more clear illustration, we start with a few definitions. We first define the adjacency enhance- ment by the 1tth-oder matrix power as A +- AK, where A is the adjacency matrix of the graph. Geometrically, A(i,)) 1 means the edge (i,j) exists, and A A (i,j) = means that node j is connected to node via at most K hops. Given pooling strategy P and the selected pooled nodes Sy, we define a K'th-rrder outlier set as OK, where the nodes in O k are not connected to any pooled nodes even after ordde adjacency enhancement: A (i.)j)
vi Sp, Vj Ok-
We further define that a pooling strategy is -od-order connection conservative (K-CC) if OK is empty. We argue that larger K in -oddr adjacency enhancement harmful to distinguish the node features. As K increases, A K (i, approaches a matrix with all its entries equal to 1, repre- senting a fully connected graph, where a single step of convolution will average all node features and make them indistinguishable. The most favorable and possible, i.e. the smallest, K we should seek Gao & 2019 uses the 2nd order enhancement help conserve the connectivity. Nonetheless, there is no theoretical guarantee that learnable pooling module is consistently 2-CC for any graph (a counter example shown in Fig. aa)). There are two alternative solutions to the matrix power enhancement that ensure conservation of the connectivity at coarser layers: 1) Lino et al.) (2022a) 2021 2022b) build the coarser graph by projecting the finer nodes to the nearby background grids (Fig. Ib)); 2) Liu et al. (2021); Fortunato et al. (2022) create coarser meshes for the same do- main (Fig. [.). However, both methods need spatial proximity to build additional connections for the transition between levels, which may produce wrong connections across the boundary. These limitations motivate us to create a consistent 2-CC pooling strategy, as described in Sec. 3.2
An additional overhead is the learnable transition modules which have the same network architecture as the message passing. This overhead in model size and computational complexity grows linearly w.r.t. the number of levels of U-net. As result, they often end up with relatively shallow level at
3

---

Under review as a conference paper at ICLR 2023
Bi-stride Multi-scale GNN
Bi-stride Multi-scale GNN
Bi Bi-siride Multi-ccle GNN
p
update
encoding
decoding
9
f
91
1xMP
1xMP
downsample
upsample
g
G
g
1XMP
1xMP
downsample
upsample
Ga
Go
Figure 2: BSMS-GNN pipeline uses encode-proces--eecod trained with one-step supervision. 91.92. Ga represents the graph at different levels (finest to coarsest). The encoder/decoder only connects the input/output fields with the latent fields at G1. The latent nodal fields are updated by one MP (message passing) each level. The bi-stride pooling selects the pooled nodes for the adjacent coarser level, and the transition is conducted in a non non-parameterized way.
or 3. We claim that none- parameterized transition is performance -ise crucial for deeper multi-layer GNNs, and propose the first non- con-parameteriee transition method in Sec. 3.3
Overall, our method adopts similar message passing layer as in GraphMeshNets (Plaff et al. 2020))
Compared to Liu et al. (2021): Fortunato et al. (2022), our advantage that no mesh generators is needed for the coarser-level graphs. Compared to Lino et al. (2021 2022a b), our advantage is
that no spatial proximity is necessary. Together, we eliminate the need for building connections via
spatial proximity nor using learnable MLP for aggregation and returning. Note that the work of Li et al. (2020b) shares similar advantages to some extent, but it focuses on generalization with PDE parameters, while ours focuses systematic pooling strategy for arbitrary complex geometries.
METHODOLOGY
3.1 DEFINITIONS
Figure. prresents the overall structure of GNN. We consider the evolution of a physics-based system discretized on a mesh which converted to an undirected graph 91 (V1, E,). Here, with subscript 1, V and & label the nodal fields and the connectivities at the finest level (the input mesh), respectively. Specifically for edges, we define & = (E).... & where El is the edge set directly copied from the input mesh, and {E11S are optionally the additional problem-dependent edge sets involved. For example, both DEFORMING benchmarks have second edge set E? for the nearby colliding vertices. We use {p.g). stacked vectors of {p., 9: of all nodes i E V1. to denote the input and output nodal fields, respectively. Given input field yd previous time Lj. one pass of our BSMS-GNN returns the output field gi+l at time 1-+1 t, + At, where At is the fixed time step size. The output g contain more physical fields than the input p and must be able to derive the input for the next pass. The rollout refers to iteratively conducting BSMS-GNT from the initial state pd q¹ p² + - q" and producing the temporal sequence output {a """} within the time range of (to, to + nt., where n is the total number of evaluations.
(Fig. 5(c) and INFLATINGFONT (Fig. d))
Message Passing In general, we follow the encode procesd-cooessdeeodee fashion in GraphMeshNets, where encoding and decoding only appear at the beginning and the end of the finest level 9 map- ping the nodal input p and output q to/from the latent feature v, respectively (see Table A. for the -opaii- information). As for the process part, unlike GraphMeshNets where multiple message passings (MPs) are needed, we observe that single MP each level is sufficient for all experiments. Therefore, it becomes unnecessary to keep updating the latent edge information across multiple MPs. To include the directional information of an edge (æ,, my), simply prepend its
4

---

Under review as a conference paper at ICLR 2023
positional offset Az 2; 2; to the stacked sender/receiver latent imput to calculate the information flow. For problem involving S edge sets, an MP pass at level / formulated as:
ei + 17 (At Vi,i. vLss),
= == S
v + v1, + f Vl,i. Eelee Σ o..5)
where is MLP function, is the latent information flow through edge, and is the latent node feature. Please refer to Sec. A.2 for the detailed architecture of the model.
Cross-Level Transition We handle information transition between adjacent levels with down- sampling and upsampling modules. Here we define downsampling as the sequence of pooling (se- lecting pooled nodes) and then aggregating the information from the neighbors the coarser level, and we define upsampling as the sequence of unpooling and then returning the information of the pooled nodes their neighbors at the finer level. Please refer Sec. 3.3 for details.
3.2 BI-STRIDE POOLING AND ADIACENCY ENHANCEMENT
As aforementioned, two challenges of building a multi-level GNN for learning physical simulation, especially on wild geometries, are 1) not introducing partitions that break the connectivity, and 2) not introducing wrong edges by spatial proximity. We tackle these chal- lenges by improving the pooling phase. Specifically, is there pool- ing strategy that consistently 2nd connection conservative (2-CC) for any imput graph so that an efficient 2rdder enhance- ment is sufficient to conserve the connectivity? We draw the initial inspiration from the bi-partition determination algorithm (Asratian et al. 1998) in a directed acyclic graph (DAG). As show in the inset figure (a), after topological sorting, pooling every other depth (yellow and green) generates bi-partition. To resemble the bi- partition determination on a mesh, which is not bi-partite due cycles, we can conduct a breadth- first search (BFS) to compute the geodesic distances from an initial seed to all other nodes, and then stride and pool all nodes at every other BFS frontiers (bi-stride). the inset figure (b), where the number in each vertex represents the distance to the seed (node 1 in red circle) by BFS. This pooling is 2-CC by construction and conserves direct connections between pooled nodes and unpooled nodes. As result, avoid building edges by spatial proximity or handling the cumbersome corner cases such as cross-boundary connections.
(a) Bi-partition of DAG
(b) Bi-stride of a mesh
bi-stride example shown in
Seeding Heuristics We claim that there should exist some freedom long the seeding is bal- anced to a certain degree. The time complexity for searching seeds is tolerable because of the one- pass preprocess. For training datasets, choose two deterministic seeding heuristics: 1) closest to the center of cluster (CloseCenter) for INFLATINGFONT, and 2) the minimum average distance (MinAve) for all other cases, and we preprocess the multi-level building in pass. One sider the cheaper heuristic CloseCenter during the online inferring phase if an unseen geometry is encountered. The details of the algorithms can be found in Sec A.6
Auxiliary Edges For multi-physics problems, such as DEFORMINGPLATE. (Fig. INGFONT (Fig. ((d)). the auxiliary edges (such as contact edges AC) should be built dynamically by spatial proximity to exchange the interfacial information between different systems. The enhance- ment of these edges should be handled properly for multi-layered GNN, which, to the best of our knowledge, has not been addressed yet. At level I, given two adjacent matrices A, and A for the mesh edges from the input graph G and the contact edges, respectively, we apply the enhancement followed by per-cluster bi-stride pooling for A, with selected node indices I:
(Fig.5)c) ((c) and INFL.AT-
A'm1 + A,Ar, A4t1 < A [II]. 1 AAPA,
(2)
A AC III].
This enhancement can be geometrically interpreted as such: auxiliary edge (i.j) should exist if j is reachable from in 2 hops and one of which is an auxiliary edge at the finer level. We prove in Sec. A.5 that our pooling conserves all the contact edges under this enhancement.
5

---

Under review as a conference paper at ICLR 2023
3.3 TRANSITION BETWEEN LEVELS
We propose unified non-parameterized method to reduce the overhead of the learnable transition modules between every pair of adjacent levels.
12K
15
Big A /,, Asj
Cij des/ E, Das
Dj Σ, is Ci
D. C
(a) weight convolution
(b) edge contribution
(c) aggregation
(d) returning
Figure 3: Schematic plot of the transition steps between adjacent levels.
Downsampling We treat the latent information as a conserved variable and project it to the pooled nodes. We define A as the unweighted adjacency matrix where its row and column indices represent the sender and the receiver, respectively. We further represent the nodal mass or importance as a nodal weight field w, which is initialized the finest level to ones for near near-uniform meshes the volume/mass field for highly irregular meshes. With the receiver vertex j and its sender vertices i, the formal procedure is formulated (Fig. 3):
Normalize by row as in a standard graph convolution A.g + Au/ E, Agg and then convolve the weight once ühj w, (Fig. 3(a)::
Calculate edge weights Ciyy f uuz/ E, Way. where C Can be viewed as contribution table with Ciş as the share of weights in the receiver j contributed by the sender i (Fig. bb);
Convolve the latent information by the contribution table v, + Σ, vn Cij. which is equiv- alent to equally splitting and sending the weighted information neighbors, and then ob- taining the weighted average (Fig. ((c)..
Upsampling After unpooling, all nodes except the pooled ones have zero information. A returning process, resembling the transposed convolution in CNNs, can help distinguish the receivers, With the contribution table C' recording the edge weights, natural choice is v; + Uj C (Fig. (()).
4 EXPERIMENTS
4.1 EXPERIMENT SETUP
Datasets We adopt three representative public datasets from GraphMeshNets (Plaff et al.) 2020
1) CYLINDERFLOW: incompressible fluid around cylinder where the mass conservation has be enforced globally. 2) AIRFOIL: compressible flow around an airfoil where an auxiliary prediction, the pressure, is included, and 3) DEFORMINGPLAATE: deforming an elastic plate with an actuator where simple contact is included. In addition, to illustrate the ease of extending our method to multiset problems, we further create a new dataset, INFLATINGFONT, featuring the inflation of enclosed elastic surfaces with massive self-contacts (Fang et al. 2021).
Baselines On all datasets, we compare computational complexity, training/inference time and memory footprint of BSMS-GNN to baselines: GRAPHMESHNETS (Pfaff et al.) 2020 the single-level GNN architecture of GraphMeshNNts, 2) MS-GNN-GIA -GRID (Lino et al. 2021 2022a b): a representative work for those building the hierarchy with spatial proximity (i.e. using the distance between nodes), and 3) GRAPHUNET (Gao & Jil ) representative work for those using learn- able modules for pooling. The detailed reimplementation of these works can be found in Sec. A.2 We note again that methods such as Liu et al. (202I); Fortunato al. (2022) are not practical be- cause they require pre-drawing meshes at multiple levels. For all cases reported this work, this means manually drawing 20, 000 meshes using CAE meshing software.

---

Under review as a conference paper at ICLR 2023
Measurements
Case
Our's
Lino et al. 2021
Plaff et al.
2020
Gao & Ji 2019
Cylinder
10.14
15.36
19.29
16,20
Training time/step
Airfoil
18.82
25.26
36.72
55.08
[ms]
Plate
15.58
49.65
49.15
31.88
IDP
45.96
107.16
117.48
1,833.37
Cylinder
6.75
6.18
14.50
24.30
Infer time/step
Airfoil
8.64
20.40
24.20
33.60
[ms]
Plate
14.01
18.12
15.70
16,20
IDP
33.33
41.66
82.35
629.33
Cylinder
21,41, 19
35,84, 21
64,30, 30
76.15,
Training cost
Airfoil
122.33, 39
176.82,
275.40, 45
206.55, 37
[hrs].
Plate
56,07, 27
125.78, 19
176.94, 27
127.50, 30
Final epoch
IDP
2.68E+01, 21
5.66E+01, 19
6,20E+01, 19
NA
Cylinder
2.04F-01
2.201:-11
2.2613-01
8.09
RMSE-1 [Ie-2]
Airfoil
2.888+01
2.68E+01
4.35B+01
2.93E+01
Plate
2.87E-02
2.20E-02
1.98--02
2.03E-02
IDP
1.77E.412
1887-00
1,95E-02
NA
Cylinder
2.42
2.74
4,39
1.87++01
RMSE-50 [Ie-2]
Airfoil
1.10+003
1.22++03
166E+03
1,17+003
Plate
3.18E-02
2.78E-02
2.88E-02
5.19E-02
IDP
LOSE-01
3.24E-01
1.78E-01
NA
Cylinder
8.37
8.49
L07E+01
165E+02
RMSE-all [Ic-2]
Airfoil
4.21E+04
5,56E+04
6.95E+04
6.11E+04
Plate
1.60E-01
1.48E-01
1,51E-01
5.46E-01
IDP
2.20E-001
3.78E-01
3.65E-01
NA
Table 1: Detailed measurements of our method, MS-GNN- GRAPHMESHNETS, and GRA-
PHUNET. All measurements are conducted using a single Nvidia RTX 3090. BSMS- -NNN consis- tently generates stable and competive global rollouts with the smallest training cost. BSMS-GNI also lightweight and has the fastest inference time. also free from the large RMSE due poor pooling on unseen geometries where the learnable pooling module of GRAPHUNET suffers.
Implementation We implement BSMS GNN framework with Py Torch Paszke et al. 2019) and PyG (PyTorch Geometric) (Fey & Lenssen 2019). We train the entire model by supervising the single-step Ly loss between the ground truth and the nodal field output of the decoding module. For more detailed information, such as the statistics of the mesh the number of layers, the multi-edge sets, and the hyperparameters of the MLP network, please refer Sec. A. and A.2 Our datasets and code are publicly available at
4open. www.pppw....incee///ieeeeeceeeee BSMS MANN-ICLR-20.33
We also conduct the ablation study for the specific choice of our transition method in Sec. A.3 and include the scaling test on INFLATINGFONT in Sec. A.4 Another ablation study can be performed on whether or not to use learnable pooling module. But we already covered this aspect by comparing to GRAPHUNET in the full experiments (details in Sec. 4.2).
MISCs
4.2 RESULTS AND DISCUSSIONS
We evaluate BSMS- GNN on all described benchmarks and compare it with the baselines (Sec. In general, our method builds multi-level graphs without the loss of connectivity; it is free from spatial proximity and therefore avoids wrong edges across the boundary for complex geometries (the generated multi-level graphs each example are plotted in Fig. 5), leading to high-quality rollouts on all tasks. Compared to all basclines, method shows dominant advantages in significantly less memory footprint and training time to reach the desired accuracy, as plotted in Table. A.
Disadvantages of Learnable Pooling Compared other methods, GRAPHUNET has similar er- ror only in AIRFOTL where the mesh is consistent across instances, but has significantly larger error (2 4x 1-step RMSE and 20 rollout RMSE) in CYLINDERFLOW and DEFORMINGPLATE where instances have varying meshes. Empirically, this difference indicates that Iearnable modules infer poor pooling results for unseen geometries and harms information passing at coarser levels. Another concern is the overhead by adjacency enhancement. Though the learnable pooling module (Linear + Top-K) itself does not take long, GRAPHUNET needs to enhance adjacency by matrix multiplication in the forward pass. These multiplications, although reimplemented with sparse op- erations, result in 2 ~ 40x unit training time and 4 ~ 20x unit infer time (except for DEFORMING- PLATE). In INFLATINGFONT, a single epoch takes unaffordable 50 hours, making it impossible for the full experiment. Due to these issues, we conclude that GRAPHUNET is not suitable for large-scale or cases with varying meshes, hence exclude it in the further discussions.

---

Under review as a conference paper at ICLR 2023
Case
Method
Train with different Batch
Infer
2
4
16
32
64
Our's
2.41
2.92
4.37
6.06
11.4
22.27
1.92
Cylinder
Lino al.
2021
2.79
3.60
5.31
8.56
15.10
1.97
Plaft
(2022)
3.25
4.46
6.91
11.84
21.60
1.94
Gao Ji
2019
23.33
2.18
Our's 5
3,66
5,46
8.88
15,70
2.02
Aifroil
Lino et al.
2021
4.18
6.25
10.65
19.25
2.02
Platf al.
20:20
5.53
8,90
16.08
2.06
2019
2.67
Our
2.36
2.87
3.85
5.78
9.28
16.85
1.95
Lino et al.
2021
3,41
4.81
7.75
13,20
2.00
Plate
Platt ct al.
20:20
3.10
4.29
6.59
11.49
20.80
1.93
Gao J1
2019
2.18
Our
6,28
10.80
Lino al.
2021
10.87
19.79
2.45
Font
Platf al.
20120
12:48
23.39
2.28
Gao & J1
2019
4.51
Table 2: Memory footprint under multi-batches, BSMS-GNN consistently cuts RAM con- sumption by approximately half in all cases in the training stage, and also has the smallest (except for DEFORMINGPLATED inference RAM.
We evaluate the time complexity with unit training time per step. Compared to GRAPHMESHNETS, ing time for Eulerian systems (CyLINDERFLOW and AIRFOIL) and Lagrangian systems with con- tacts (DEFORMINGPLAT)
Unit Training/Inference Time
only takes 51% and 31% 39% of the unit train-
and INFLATINGFONT), respectively. The main source of the speedup is the reduction of total MP times. In GRAPHMESHNETS,, 15 MP passes are conducted on the finest level of the mesh. While in our method, 2 x levels + 3 MPs are conducted, and only of them happen on the finest level. As for MS-GNN-GRID they share the similar advantage of performing more MPs on smaller subsets at coarser levels, but 4 x levels + 6 total MPs are required, while of which happen at the finest level; they also have the overhead of learnable aggregation/returiing modules. When applied to Eulerian systems, their unit training/infer time lies between our method and GRAPHMESNNEESS For Lagrangian systems with contacts, the contact edge sets bring in addi- tional overhead and degrade the unit training time to the same level as GRAPHMESHNETS.
Regarding inference time, the performances for DEFORMINGPLATERR (~ 1K) are very similar. Our method and DE-GNN-GGRDhaan similar performance CYLINDER- (moh size 1.5K) as well, and both outperformed GRAPHMESUNETS, As mesh size grows (5K ~ 15K), BSMS-GNN boost the inference time gradually up to 2.5x compared to MS-GNN- GRID, and 2.9x compared to GRAPHMESHNETS.
the smallest mesh size
Training time to reach desired rollout accuracy Since rolling out the ultimate purpose for predicting physical systems, we define the training cost (in time) the earliest wall time to obtain the converged global rollout RMSE. The global rollout error is reduced by feeding the model with noisy inputs but correct outputs at each epoch, so that it can learn to better correct noises generated during inference (Pfaff et al., 2020). The essential point is epoch number, i.c. the number of random noise patterns seen. In observation, all methods reach the desired global rollout RMSE with similar amount of epochs, which leads to our superiority due to much faster unit training time.
Accuracy We plot the detailed RMSES with different rollout steps (1, 50, or until the end) for different methods. Our method has the smallest global rollout RMSE for all cases except DEFORM- INGPLATE, where the error is slightly higher than the alternatives. For INN,ATINGFONT, with the most complicated contact connectivities, our method cuts about 55% of the training time while also reducing 40% of the global rollout error.
Memory Footprint
The memory footprint affects both the training and the inference stage. In the training stage, the higher RAM consumption sets the lower cap batch number and results in more data transfer from CPU to GPU and a larger overhead to finish epoch. In our observation, we can achieve at most ~ 3x acceleration by simply increasing the batch number. In the inference stage, the RAM consumption is closely related to the deployment in production. We measure the memory footprint of all methods under varying batch sizes (Table. B) Compared to GRAPHMESHNETS, BSMS- GNN consistently reduces memory consumption by approximately half in all cases. As for
8

---

Under review as a conference paper at ICLR 2023
Thiniy Set
Raid lus So.
Temperioo
Tempe aet
Taggel
Teapeeatie
Tutinn
Dan i
CatE
BIMS-ONK
Posuloa
Postion
PostsOn
Posttion
Figure 4: Failure cases for MS-GNN-GRID. Left: the configuration of the simplest failure case for multi-level GNNs by spatial proximity: the steady-state 1-D heat transfer. Right, leading two columns: two tests showing that even if trained to convergence, the erroneous edge across the bound- ary can still result in wrong inference. Right, last two columns: the erroneous edge coincidentally does not affect the results due to the symmetry of the solution and that no heat will diffuse between two nodes with the same temperature.
MS-GNN-GREE we observe a similar phenomenon as the unit training time: their advantage only stands for the Eulerian systems. For the Lagrangian systems with additional contact edge sets, they consume similar or even higher memory than GRAPHMESINETSS. Overall, our method consumes 17% 57% less memory than MS- MSS-GNN-GE GNN-GNIN Our method also has the smallest inference RAM, except for DEFORMINGPLATE where ours is slightly higher (20MBs) than GRAPHMESHNETS.
The Failure Case for Spatial Proximity To illustrate the adversarial impact of wrong edges built by spatial proximity, we design a simple 1-D steady-state heat transfer on sticks (Figure. 4 left). The training set contains two mirrored instances, where one end of the stick is fixed at a specific temperature, and the other has the fixed heat flux. The result of such a configuration is the linear temperature distribution. the test set, we simply align two sticks in a head-to-tail fashion but leave some space between them so no heat diffuses across the boundary. We choose MS-GNN-GRID as an example of those utilizing spatial proximity. The training for BSMS-GNN and MS-GNN- -GRID can converge quickly under few hundred iterations. However, in the test phase, the errоneous connection by proximity transfers the information between two isolated sticks and can yield wrong results (Figure. rigght, leading two columns). We also note that although preprocessing (separately inferring for two sticks) can help resolve the issue in this simple example, it is not doable for a single but wild geometry. The simplest counter example the fluid dynamics in U-shaped channel where the two ends of the channel are close spatially but far away geodesically.
5 CONCLUSION, LIMITATIONS, AND FUTURE WORK
Bi-Stride Multi-Scale GNN features a simple and robust pooling strategy that systematically gen- erates an arbitrary-depth multi-level graph neural network given geometry in the wild as the sole input. It does not rely on mesh generators or projecting to regular grids. Bi-Stride guarantees direct connections between the pooled and unpooled nodes, while free from any redundant connections by spatial proximity. This further helps replace the MLPs for the transition between adjacent levels with a unified non-parametrized transition scheme. BSMS-GNN eliminates the necessity of multi- ple MPs and the latent edge embedding. Combined, it significantly reducing computational costs. With moderate tailoring, BSMS-GNN can be easily extended to multi-edge-set problems involving different dynamical behaviors.
In summary, we believe that the non- panaaaametteizeed Bi-Stride strategy will conceptually complete the methodology path created by GraphMeshNets,, just like what striding and up-sampling by interpo- lation are for CNN. Following our non- non-parametrized strategy, there are interesting ideas to explore. For example, although any general multi-level GNN can reduce the time complexity to linear, it still need to load the whole graph initially. Combining the multi-level GNN and batch training is crucial for huge-scale graphs. Second, as stated in Li et al. (2020b), the transition from fine to coarse levels is equivalent to the transition from sparse, high-rank kernels to dense, low-rank kernels. Although the dense or fully connected graphs only appear near the bottom layers with minimum nodes in practice, there is no theoretical guarantees. Whether strategies like edge pruning is needed to avoid dense graphs at coarser levels becomes an interesting question. addition, since all the nodal fea- tures will be smoothed without the saip-layers, how to migrate our strategy to GAE+ Transformer (Han et al. 2022) is also a meaningful direction.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Armen S Asratian, Tristan MJ Denley, and Roland Häggkvist. Bipartite graphs and their applica- tions, volume 131. Cambridge university press, 1998.
Peter Battaglia, Razvan Pascanu, Matthew Lai, Danilo Jimenez Rezende, et al. Interaction networks for learning about objects, relations and physics. Advances in neural information processing systems, 29, 2016.
Peter W Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Sanzhez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.000 2018.
Filipe De Avila Belbute-Peres, Thomas Economon, and Zico Kolter. Combining differentiable pde solvers and graph neural networks for fluid flow prediction. In international conference on ma- chine learning, pp. 2402-2411, PMLR, 2020.
Robert Bridson. Fluid simulation for computer graphics. AK Peters/CRC Press, 2015.
Yadi Cao, Yunuo Chen, Minchen Li, Yin Yang, Xinxin Zhang, Mridul Aanjaneya, and Chenfanfu Jiang. An efficient b-spline lugrangian/eulerian method for compressible flow, shock waves, and fracturing solids. ACM Transactions on Graphics (TOG), 41(5):1-13, 2022.
Michael B Chang, Tomer Ullman, Antonio Torralba, and Joshua B Tenenbaum. A compositional object-based approach learning physical dynamics. arXiv preprint urXiv:1612.0034 2016.
Deli Chen, Yankai Lin, Wei Li, Peng Li, Jie Zhou, and Xu Sun. Measuring and relieving the over- smoothing problem for graph neural networks from the topological view. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 3438-3445, 2020.
Yu Fang, Minchen Li, Chenfanfu Jiang, and Danny Kaufman. Guaranteed globally injective 3d deformation processing. ACM Trans. Graph., 40(4):75-1, 2021.
Matthias Fey and Jan E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.
Meire Fortunato, Tobias Pfaff, Peter Wirnsbrrger, Alexander Pritzel, and Peter Battaglia. Multiscale meshgraphnets. In ICML 2022 2nd Al for Science Workshop, 2022.
Stathi Fotiadis, Eduardo Pignatelli, Mario Lino Valencia, Chris Cantwell, Amos Storkey, and Anil A Bharath. Comparing recurrent and convolutional neural networks for predicting wave propaga- tion. arXiv preprint arXiv:2002.0898) 2020.
Kunihiko Fukushima and Sei Miyake. Neocognitron: A self-organizing neural network model for a mechanism of visual pattern recognition. In Competition and cooperation in neural nets, pp. 267-285. Springer, 1982.
Han Gao, Luning Sun, and Jian-X Wang. Phygeonet: Physics-informed geometry-adaptive convo- lutional neural networks for solving parameterized steady-state pdes on irregular domain. Journal of Computational Physics, 428: 10079, 2021.
Han Gao, Matthew J Zahr, and Jian-Xun Wang. Physics-informed graph neural galerkin networks: A unified framework for solving pde-governed forward and inverse problems. Computer Methods in Applied Mechanics and Engineering, 390: 14502, 2022.
Hongyang Gao and Shuiwang Ji. Graph u-nets. In international conference machine learning, pp. 2088-20092 PMLR, 2019.
Xiaoxiao Guo. Wei Li, and Francesco Iorio. Convolutional neural networks for steady flow ap- proximation. In Proceedings of the 22nd ACM SIGKDD) international conference on knowledge discovery and data mining, pp. 481-490, 2016.
Xu Han, Han Gao, Tobias Pffaf, Jian-Xun Wang, and Li-Ping Liu. Predicting physics in mesh- reduced space with temporal attention. arXiv preprint arXiv:2201.091 13, 2022.
10

---

Under review as a conference paper at ICLR 2023
Lukas Harsch and Stefan Riedelbauch. Direct prediction of steady-state flow fields in meshed do- main with graph networks. arXiv preprint arXiv:2105.02577 2021.
Chenfanfu Jiang, Craig Schroeder, Joseph Teran, Alexey Stomakhin, and Andrew Selle. The ma- terial point method for simulating continuum materials. ACM SIGGRAPH 2016 Courses, pp. 1-52. 2016.
George Em Karniadakis, loannis Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physics-informed machine learning. Nature Reviews Physics, ((6):422-44)) 2021.
Byungsoo Kim, Vinicius Azevedo, Nils Thuerey, Theodore Kim, Markus Gross, and Barbara Solenthaler. Deep fluids: A generative network for parameterized fluid simulations. In Computer graphics forum, volume 38, pp. 59-70. Wiley Online Library, 2019.
Minchen Li, Zachary Ferguson, Teseo Schneider, Timothy R Langlois. Denis Zorin, Daniele Panozzo, Chenfanfu Jiang, and Danny M Kaufman. Incremental potential contact: intersection- and inversion-free aage-deformation dynamics. ACM Trans. Graph., 39(4):49, 2020a.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Andrew Stuart, Kaushik Bhat- tacharya, and Anima Anandkumar. Multipole graph neural operator for parametric partial differ- ential equations. Advances in Neural Information Processing Systems, 33:6755-6766, 2020b.
Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar. Fourier neural operator with learned deformations for pdes on general geometries. arXiv preprint arXiv:2207.05209, 2022.
Mario Lino, Chris Cantwell, Anil Bharath, and Stathi Fotiadis. Simulating continuum mechanics with multi-scale graph neural networks. arXiv preprint arXiv:2106.04900 2021.
Mario Lino, Stathi Fotiadis, Anil A Bharath, and Chris Cantwell, Towards fast simulation of environ- mental fluid mechanics with multi-scale graph neural networks. arXiv preprint arXiv:2205.02637, 2022a.
Mario Lino, Stathi Fotiadis, Anil A Bharath, and Chris D Cantwell. Multi-scale rotation-equivvariant graph neural networks for unsteady eulcrian fluid dynamics. Physics of Fluids, 34(8):087110, 2022b.
Wenzhuo Liu, Mouadh Yagoubi, and Mare Schoenauer. Multi-resolution graph neural networks for pde approximation. In International Conference on Artificial Neural Networks, pp. 151-163. Springer, 2021.
Jonathan Masci, Davide Boscaini, Michael Bronstein, and Pierre Vandergheynst. Geodesic con- volutional neural networks on riemannian manifolds. In Proceedings of the IEEE international conference on computer vision workshops, pp. 37-45, 2015.
Diego Mesquita, Amauri Souza, and Samuel Kaski. Rethinking pooling in graph neural networks. Advances in Neural Information Processing Systems, 33:2220-2231, 2020.
Damian Mrowca, Chengxu Zhuang, Elias Wang, Nick Haber, Li F Fei-Fei, Josh Tenenbaum, and Daniel L Yamins. Flexible neural representation for physics prediction. Advances in neural information processing systems, 31, 2018
Octavi Obiols-Sales, Abhinav Vishnu, Nicholas Malaya, and Aparna Chandramowliswharan Cíd- net: A deep learning-based accelerator for fluid simulations. In Proceedings of the 34th ACM international conference supercomputing, pp. 1-12, 2020.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary De Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy. Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performmace deep learning library. In Advances in Neural Information Processing Systems 32, pp. 8024-8034 Curran Associates, Inc., 2019.
11

---

Under review as a conference paper at ICLR 2023
Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzaler and Peter W Battaglia. Learning mesh- based simulation with graph networks. arXiv preprint arXiv:2010.03409, 2020.
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652-660, 2017a.
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical fea- ture learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017b.
Maziar Raissi, Paris Perdikaris, and George E Kariiadaki. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707,, 2019.
Alvaro Sanchez-Gonzalez, Nicolas Heess, Jost Tobias Springenberg, Josh Merel, Martin Riedmiller, Raia Hadsell, and Peter Battaglia. Graph networks as learnable physics engines for inference and control. In International Conference on Machine Learning, pp. 4470 4479. PMLR, 2018.
Alvaro Gonzaez- Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovee, and Peter Battaglia. Learning simulate complex physics with graph networks. In International Confer- ence on Machine Learning, pp. 8459-8468. PMLR, 2020.
Luning Sun, Han Gao, Shaowu Pan, and Jian-Xun Wang. Surrogate modeling for fluid flows based on physics-constraaned deep learning without simulation data. Computer Methods in Applied Mechanics and Engineering, 361:112732, 2020.
Jonathan Tompson, Kristofer Schlachter, Pablo Sprechmann, and Ken Perlin. Accelerating eulerian fluid simulation with convolutional networks. International Conference on Machine Learning. pp. 3424-344444 3433. PMLR, 2017.
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, 32(1):4-24 24, 2020.
Zhen Zhang, Jiajun Bu, Martin Ester, Jianfeng Zhang, Chengwei Yao, Zhi Yu, and Can Wang. Hierarchical graph pooling with structure learning. arXiv preprint 1rXiv:191..05954, 2019.
12

---

Under review as a conference paper at ICLR 2023
APPENDIX
A
(b) AIRFOIL
(c) DETORMINGPLATE
(a) CYLINDERFLOW
(d) INFLATINGPONT
Figure 5: Example plots of the multi-level graphs produced by bi-stride pooling. Our dataset contains both Eulerian and Lagrangian systems. Many meshes are highly irregular and contain massive self-contact, which poses strong challenges for building the coarser level connection by spatial proximity. The bi-stride strategy only relies on topological information and has proven to be robust and reliable arbitrary kinds of geometry.
A.1 DATASET DETAILS
We adopt three existing test cases: Cylinder (Flow), Airfoil, and (Deforming) Plate from GRAPUMESINETS. The Cylinder includes the transient incompressible flow field around a fixed cylinder at varying locations. The Airfoil includes the transient compressible flow field varying Mach numbers around the airfoil with varying angles of attack (AOA). The Plate includes hyperelas- tic plates squeezed by moving obstacles. In addition these three cases, our Fom(LNPLATINNFONT case involves the quasi-static inflation of enclosed elastic surfaces (3I) surface mesh) possibly with self-contact. We create the INFLATIIGFONTCaseR using the open-soure simulator (Fang et al.) 2021), with the same material properties and inflation speed. The input geometries for INFLATING- FONTare 1, 400 2 2-character matrices in Chinese. All the datasets are split into 1000 training. 200 validation, and 200 testing instances. In the following table, the second entries with superscript" in the average edge number column are for the contact edges:
Case
Ave # nodes
Ave # edges
Mesh type
Seed method
# Levels
# Steps
Cylinder
1886
5424
triangle, 2D
MinAve
7
600
Airfoil
5233
15449
triangle, 2D
MinAve
600
Plate
1271
4611, 94"
tetrahedron, 3D
MinAve
6
400
Font
13177
39481, 6716
triangle, 3D
CloseCenter
6
100
Below we list the model configurations: 1) the offset inputs prepend before the material edge processor .. and W and 2) nodes Pi. as well as the nodal outputs q from the decoder for each experiment cases, where X and æ stand for the material-space and world-space positions, is the velocity, o is the density, is the absolute pressure, and the dot a++1 ag stands for temporal change for a variable All the variables involved normalized to zero-mean and unit variance via pre-processing.
As for time integration, Cylinder, Airfoil, and Plate inherited the first-order integration from GRAPHMESHNETS. For INFLATINGFONT, the first-order quasi-static integration (Fang et al.) 202T used in the solver. Hence, also adopt the first-order integration for HNT.ATING
13

---

Under review as a conference paper at ICLR 2023
Case
Type
Offiet inputs
Offise inputs
W
Inputs ps
Outputs qi
Cylinder
Bulerian
Xij-
NA
Wey fl
12;
Airfoil
Eulerian
Xij.
X
NA
Piy Me my
0i, Pi, P:
Plate
Lagrangian
X
Dkj
Fif. Fij Tif.
By ngg.
Font
Lagrangian
X ig
Zijs
Dij
n4
A.2
ADDITIONAL MODEL DETAILS
A.2.1 BASIC MODULES AND ARCHITECTURES
The MLPs for the nodal encoder, the processor, and the nodal decoder are ReLU-activated two- hidden-laye MLPs with the hidden-layer and output size 128, except for the nodal decoder whose output size matches the prediction q. All MLPs have residual connection. A LayerNorm normal- izes all MI.P outputs except for the nodal decoder.
A.2.2 BASELINE: GRAPILMESHNNETS
Our GrAPHMESHHEErSiipeeeennaata
uses the same MLPs above but with additional mod- ule: the edge encoder. Also, the edge latent is updated and carried over throughout the end of multiple MPs. We use times MP for all cases keep it consistent with GRAPIMESINETTS.
A.2.3 BASELINE: MS-GNN-GRID
Our re-implementation of MS GIN-GNINDDIDDD the same MLPs above but with four additional modules: the edge encoder at the finest level, the aggregation modules for nodes and edges at every level for the transitions, and the returning modules for nodes at every level. This method also requires assigning the regular grid nodes for each level. We assign these grid nodes by defining an initial grid resolution and inflation rate between levels. As for the MP times each level, follow Lino et al. 2022a) to use four at the top and bottom levels and two for the others.
Case
# Levels
Initial grid dx
dx inflation
Level-wise # MPs
Cylinder
4
[5e-2, 5e-2]
2
[4, 2, 2, 4]
Airfoil
[4.5, 4.5] [4e-3, 4e-3, 4e-3]
2
[4, 2, 2,
Plate
4
2
[4. 2, 2,
Font
[1.5e-2, 1.5e-2, le-3]
2
[4, 2, 2,
A.2.4 BASELINE: GRAPHUNET
Our re-implementation of GRAPHUNET the same number of levels those of BSMS- Likewise, we make the following modifications to the original GRAPHUNET: (1) We change the information passing from GCN to our message passing module for consistency and translational invariance. (2) GraphUNet was intended for tiny graphs (100 nodes) and used dense matrix multi- plications. This design is not scalable it break the memory limit and slow down the training to take more than 30 days per epoch in our graph size (1500 to 15000 nodes). We thus optimize the operations such as matrix multiplication and aggregation with sparse implementations.
A.2.5 NOISE AND BATCH NUMBER
For all three methods, enhance the datasets by shuffling noise into them the model resist the noise produced by single-step predictions. Each method's batch number has been tuned to achieve good convergence rate under smaller subsets.
Case
Batch size
Noise scale
Ours
Pfaff et al.
2020
Limo et al.
2022a
Gao & Ji 2019
Cylinder
32
16
16
velocity: 2c-2
Airfoil
8
4
velocity: 2e-2, density: lel pos: 3e-3
Plate
8
2
2
Font
2
pos: [5e-3, 5e-3, 3.33e-
14

---

Under review as a conference paper at ICLR 2023
GT
Ours
None
Graph cowv
Ours None
stop
Graph Conv
Werade
step 20
(a) Training loss
step 50
stop 600
(b) Global rollout
Visual comparison
step
5p
stap
step 600
None
Oura
Global rollout distribution
Figure 6: (a) All three transition methods can reach the target training RMSE given 200 iterations. (b) However, weighted graph aggregration+rrturiing has the strongest resistance to the noise during the rollout. (c) The visual comparisons show that no transition produces mosaic-like pat- terns, while the graph convolution transition smeared out the information and ceased propagating downstream. (d) The global rollout error distribution of no transition (Left) shows the edge of the mosaic patterns look similar the simulation mesh; The error of our transition (Right) travels with the generated vortices downstream and leaves the domain after step 200, which explains the RMSE drop (b).
A.3 ABLATION STUDY
A.3.1 TRANSITION METHOD
While exploring the non-parametric transition solutions, started with transition because method is adopted directly from GUN (Gao & Jil 2019). The no-transition strategy produces low enough 1-step RMSE and visually correct rollouts for ANNG However, in the global
15

---

Under review as a conference paper at ICLR 2023
rollouts of CYLInDERFLOwand AIRFOILcases, observed stripe patterns (Figure. 6 (c). column None) where the stripes are aligned with the edges at the coarser levels (Figure. (d). We suspect that this error results from the fact that the unpooled nodes all have zero information before MP, making them indistinguishable for the processor modules and exaggerating the difference between pooled and unpooled nodes over rollouts.
The no-transition strategy resembles no interpolation during the super-resolution phase of CNN+UNet. Naturally, we then tried single step of graph convolution (without activation) to resemble the interpolation in regular grids. However, this turns out to over-smooth the features (Fig- ure. (e). column Graph Conv), and the information propagation was smeared except for the area near the generator (in this case, near the cylinder).
We believe the over-smoothing issue arises from the ignorance of the irregularity of the mesh. Un- like CNN, where the fine nodes regularly lie at the center of coarser grids, irregular meshes have varying topology and element sizes. The element sizes are almost always smaller near the inter- face for higher precision in simulations; hence an unweighted graph convolution can smear the finer information near the cylinder and their adjacent neighbors during returning. The natural choice to account for the irregularity is to include reasonable nodal weights (such as the size). In the end, we arrive at the solution proposed in Sec. 3.3 by utilizing the nodal weights during aggregation and recording the shares of contribution for later returning. Our transition method works consistently for all experiment cases and produces the lowest RMSE for global rollouts (Figure. (b)).
Comparing to alternative transition methods Additionally, we compare our transition methods two alternatives extracted from previous works: (1) calculating the edge weights (kernel) for the information flow using the inverse of its length (node position offset), which we refer to as Pos-Kernel (Liu et al. 202T); and (2) the level-wise learnable transition modules implemented by
additional MP, which we refer to as Learnable (Fortunato et al. 2022
Measurements
Ours
None
Graph-Cony
Pos-Kernel
Learnable
Training time/step [ms]
10.14
9.30
10.07
10.06
17.75
Infer time/step [ms]
6.75
5.70
6.46
6.90
11.28
Training RAM [GBs]
11.041
11.041
11.041
11.041
18.033
Infer RAM [GBs]
1.923
1.923
1.923
1.923
1.931
RMSE-1 [le-2]
2.85E-01
1.49E-01
3,41E-01
6.38E-01
4.70B-01
RMSE-50 [le-2]
1.43E+01
2.05E+02
2.40E+02
1.77E+01
1.35E+01
RMSE-all [le-2]
1.68E+01
2.59E+02
5.51E+02
2.01E+01
1.57E+01
Table 3: Detailed measurements of different transition methods. Ours and Pos-Kernel are the only two non-parametric transitions which are light-weighted and procude reliable rollouts compred to the expensive Learnable transition.
In addition to the high RMSE of None and Graph-Con shown in Figure. we can also observe
that: (1) the training/infer time and RAM consumption for all non-parametric transitions (including None) are similar, which supports the statement that our transition method is light-weighted. (2) Learnable transition can reach slightly higher accuracy but the price of 70% more training/infer time and RAM. As mentioned in Sec. 4.2 higher training RAM can limit the batch number and increase the frequency of data communication between CPU and GPU, slowing down the training process even further when the scale goes up. (3) Pos-Kernel results in slightly higher RMSE compared to our method, making it competitive alternative choice production.
A.4 SCALING ANALYSIS
We train and evaluate three different methods INFLATINGFONT with varying resolutions 15K,15,,00K and 45K) for the sealing analysis.
Adjustments for datasets and models We generate the downscale and the upscale version of IN- FLATINGFONT with different average node numbers for the initial geometry, and then use the same settings to simulate the sequence. As reported in Fortunato et al. (2022), the low-resolution model suffers from converging very small RMSE; hence loosen the termination criteria by enlarging
16

---

Under review as a conference paper at ICLR 2023
the target RMSE relative to the average edge length to prevent convergence failures. Similarly, the noise injection is also adjusted to be relative to the average edge length. Moreover, with a smaller number of nodes, the number of levels required achieve the same bottom resolution also reduces. We make the corresponding adjustments to the levels of our model dj and that of the MS-GNN- GRIDd2. The adjustments are plotted below.
# Nodes
dj
d2
Initial grid dx
Target RMSE
Noise in pos
5k
4
2
[6e-2 6e-2 4c-3]
1.73e-4
[8.5e-3, 8,5e-3, 5. 5.7e-4]
15k
6
4
[1.5e-2 1.5e-2 le-3]
le-4
[5e-3, 5e-3, 3.33e-4]
30k
7
5
[7.5e-3 7.5e-3 5e-4]
le-4
[3.5e-3, 3.5e-3, 2.4e-4] [2.9e-3, 2.9e-3, 1.9e-4]
45k
7
5
[7.5e-3 7.5e-3 5c-4]
le-4
The results in Figure. show that both BSMS-GNE GNNand MS-GNN-GRIDSeNN up well, preserving near-linear scale-up rate, in contrast to GRAPHMESINETS Still, our method is lighter weighted and more efficient than fewer level-wise MP.
Results
of the non-parametric transitions and
Pfaff et al. (2020)
Infer time/stee (me)
200
Lino et al. (2022)
Training time [hrs]
40
Our's
150
30
100
20
50
10
5k
15k
30k
45k
5k
15k
30k
45k
Nodes
# Nodes
Figure 7: Scale analysis. With the growing size of INFLATINGFONT, BSMS-GNN Nshhows an obvi- ous trend of growing advantage over GRAPHMESHNETS.
A.5 THE PROOF OF CONSERVATION OF CONTACT EDGES
With Bi-stride pooling, our pooling conserves all the contact edges under the enhancement in Eq We assume the graph is undirected and unweighted, such that the adjacent matrix is a boolean matrix.
Formally speaking, given any contact edge (i,j) at level (i.e. [i, = 1) and a Bi-stride pooling P which pools nodes I, there exists contact edge (i')") that remains in the coarser level (i.e. A A'H+] ['' = 1, j' E Z) and j/j'' are connected (i.e. A i'] A: j'] = 1). There are only four scenarios concerning the pooling nodes I and the contact edge nodes i, j, under which the assertion always holds:
1. Both i.j are pooled, i.c. i,j E I. Obviously A
by letting i.j
2. Only i is pooled, E I,j & I. Since use Bi-stride pooling, level 0 (Bi-stride can select either even or odd levels) that directly connects to all nodes at level 1, or must have least one direct connection from the previous level. Le, at least one neighbor of j in the adjacent level is pooled, we let it be jr: Arjj, j'l = 1, E I. Then
either be the seed
A A 2 Af fl J' 1, and (A A.) J'1 Z A i, (A A')
1. Let = i, then A [",'= =1
3. Only j is pooled, $ I.j e I. Similarly we have least one /' such that: A, = 1,
Z. Then A, [1
J] 2 Aiii'
[ 1, and (A:4F) (i, [.]] 2 (AAA))(
A[jij] 1. Let = j. then A'C "1441'', 1.
4. None of i.j is pooled, i.j & I. Then, we select one direct pooled neighbor for i,
respectively, such that 1,l i] Alj.j' = 1, I. Then A, [F . 2 A A
[i = and (A.49 A j'' (A,A)) o Abiji 1
17

---

Under review as a conference paper at ICLR 2023
A.6 ALGORITHMS FOR THE SEEDING HEURISTICS
Here we elaborate our two seeding heuristics for the bi-stride pooling at every levels: picking the seed that is closest the center of cluster (CloseCenter), and with the minimum average geodesic distance to its neighbors (MinAve). The complexity for MinAve is O(N)) as we need to conduct BFS for every nodes to find the one with the minimum average distance neighbors. In our experiments, the quadratic cost of MinAve is tolerable for all cases but INFLATINGFONT.
Algorithm 1: MinAve: seeding by minimum average geodesic distance to neighbors
Input: Unweighted, Bi-directional graph, G (N, F) Output: List of seeds in each clusters I.
Le DetermineCluster(())
La 0
BFS(s) returns the list distances to all other neighbors from +/
/* unreachable, the distance is set to infinity
D < (BFS(s) for in N)
for idx T.e do
D. + Dlidx, idx]
D. + average (D,, dim 1)
idx (ar/margmin( [D.)] a..appedd(s)
return L,
For INFLATINGFONT, the largest mesh has around 47K nodes, and the time for pre-processing with MinAve becomes intolerable. We switch to CloseCenter with the linear complexity.
Algorithm 2: CloseCenter: seeding by minimum distance the center of cluster
Input: Unweighted, Bi-directional graph, G = (N, E); Positions of the nodes, X Output: List of seeds in each clusters L,
The DetermineCluster((
Tig <0
for idx in Le do
X + average X [idx], dim = 0)
AX -
D AX l
+ idx (argmin( ap.app) (s)
return T.g
For both heuristics, we conduct the search in a per-cluster fashion to avoid the information from other clusters that could pollute the search result. For example, when determining the center of an isolated part of the input geometry, the positions of nodes from other clusters could pollute this process. The determination of clusters given a graph is elaborated below.
18

---

Under review as a conference paper at ICLR 2023
Algorithm 3: DetermineCluster
Input: Unweighted, Bi-directional graph, G (N, E) Output: List of clusters Le
/* R stands for remaining nodes that are not inside any cluster */
R N
The
while R + 0 do 8 - R.pop() if [t| then
Lc appenddd((f)))
else
D - (sS C+
R" <- 0
11
for n in do
if [[n] then
R' appppdddd
else Capppedd(n)
appapp))
17
R R
18 return Le
19


Reviews:
[{"review_id": "t5Rrn2McGy", "reviewer": "Reviewer_fbaN", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper tackles the problem of pooling at different scales in the context of irregular geometries with a different level of granularity. The challenge is to learn / design the transition matrix between two successive scales while preserving the topological structure of the physics of the underlying systems. This strategy could be beneficial to study efficiently physical systems which are described by graph-meshes. The latter are highly irregular, especially at the boundary layer.\n", "summary_of_the_review": "The paper is good but several points need to be clarified and adrressed. I can increase my score depending on the responses and the updates. ", "strength_and_weaknesses": "The strength of the paper is in the novel design of the inter-scale pooling strategy that preserves the complex geometrical structure of a mesh.  It is not an easy problem in physics tasks because the downsampling / upsampling operations need to respect the mesh topology including the faces and cells. This is also related to the design of mesh and type of triangulations (tetraether, triangle, rectangle, etc) at the different parts of the physical domain. The developed strategy is extensively compared with state-of-the-art methods and shows an important gain in the training complexity, inference time, and memory footprint.\n\nHowever, a set of points are not discussed :\n\n1/ Absence of quantitative results (in a table) and comparison with state-of-the-art methods including some ablation studies on pooling strategy. We need to assess how the model performs. This point is of high importance. \n\n2/ A baseline comparison with deterministic pooling is missing. The idea is to see to what extent is important to have a learnable pooling rather than setting à priori the different adjacency matrices at different scales and the transition matrices between two successive scales given the physical knowledge we have on the task and that the finest-grained mesh is known. In other terms,  One can predefine the structure of pooling in a deterministic manner given the physical domain boundary and geometry. Local connectivity is easily achieved and doesn't let the downsampling make connected-components (avoid isolated nodes).\n\n3/ Why do you think you need to keep the graph mesh structure? What about training a model on point clouds such as PointNet, PointNet++, and Geodesic convolution where the graphs at different scales are constructed using a radius graph sampler? These models have shown good results in neighboring tasks and can help to get rid of the graph/mesh. Especially, in physics, there is no unique graph for a downstream task. Graph/mesh construction is an ill-posed problem. To confirm / infirm that a comparison between a cloud of points methods VS Graph methods is needed. This point is a follow-up of 1/ and 2/. \n\n4/ Is your model impacted by the over-smoothness, and-oversquashing problems encountered in GNNs? \n\n5/ In section 2 \"background and related works\", challenges in GNNs are not discussed. It could be helpful to add a paragraph to discuss the current challenges in GNNs including over-smoothness, and over-squashing w.r.t to spatial and spectral methods while making a link with the bi-stride neural network.\n\n6/ In figure one,  you said that it can lead to a wrong connection ? one simple way is to use PointNet and iterate with a neural network sufficiently deep to propagate the information from all the points to all the points in the coarse level. It ensures the global representation of information by local propagation in a hierarchical way which can help to keep the physical structure of data. It is linked to 3/\n\n7/ You mentioned in the motivation that a loss of connectivity is observed even with a powered adjacency matrix (higher-order). From an over-smoothness standpoint, higher-order adjacency matrices lead to a drop in performance. It is related to the fact that to ensure that the information has reached all the points, the depth of the neural network needs to be equivalent to the diameter of the graph (which is not tractable in practice). Moreover, higher-order adjacency matrices as the depth increase lead to A=1 at every entry. As a consequence the nodes become indistinguishable. \n\n8/ What is wrong if pivot and non-pivot nodes are not connected? Can you elaborate on that and add more information to the paper?\n\n9/ The claim of 3.2 Bi-stride pooling and adjacency enhancement: 2) not introducing wrong edges by spatial proximity. Why not?\nExtra edges beyond the local neighboring system could help to propagate information to further nodes which could not be accessible due to the over-smoothness of GNN architectures. It is related to 7/ \n\n10/ Can you support your claim? \"This enhancement can be geometrically interpreted as such: an auxiliary edge (i, j) should exist if j is reachable from i in 2 hops and one of which is an auxiliary edge at the finer level. \"\n\n11/ In A.4 scaling analysis, what is the graph-conv used in figure 7 ? and why not compared with GraphMeshNet and other related methods?\n\n\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "kt9uY6ArKx", "reviewer": "Reviewer_YsGt", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper is concerned with how to pool irregularly spaced meshes for multi-scale estimation in physical simulations. The idea proposed herein is to use so-called bi-strides which \"pools nodes by striding every other BFS frontier\". The importance lies in the fact that many meshes of important problem exhibit complex topologies so naive pooling strategies may fail.", "summary_of_the_review": "see above", "strength_and_weaknesses": "* The problem considered in this paper seems to be very relevant.\n\n* The experimental evaluation seems compelling as the authors can demonstrate results superior to SOTA. The caveat here is that the algorithms are compared on simulated data only although that is common practice in that particular field.\n\n* \"works on any challenging mesh in the wild\": Considering that the authors evaluate on simulated data, this is a bold claim and should be removed.\n\n* My objections against this paper stem from the description of the algorithm. Generally, there are so many inaccuracies (each one of them minor by itself) that I have difficulties understanding the proposed solution.\nIn details:\n2 \n\"Motivations of Our Method\": What is $A$? The definition (adjacency matrix) only comes in 3.3. This makes reading unnecessary speculative.\nWhat are \"pivot nodes\"?\nWhat does connection conservatism mean? Why is this a good metric to optimize?\nFig 2: What is the update step? What is  an MP?\n3\nedge set: Are edges discrete or continuous?\nEq (1): Where does $v_0,i$ come from? \n3.2\nWhat is 2nd order enhancement?\nWhat is a BFS frontier?\nWhat does \"stride and pool all nodes at every other BFS frontiers\" mean? Why is that a bi-stride?\nSeeding heursitics: What is that and why is it relevant for this problem?\nWhat is \"interfacial information\"?\nHow can \"two adjacent matrices $A_l$ and $A_l^C$ be determined? What is $C$? Is it related to the edge weights in $C$ in \"Downsampling\" in 3.3?\n3.3\nThe authors claim to \"reduce the overhead of the learnable transition modules\" but do not provide insight as to why that is the case.\nWhat is a \"conserved variable\"?\nWhat is the nodal mass?\nWhat is a near uniform mesh?\nWhat are the volume/mass fields for irregular meshes? How can that be determined?\nUpsampling: Why do all nodes except the pivots have zero information after unpooling?\n\nIt could be that all those minor issues are clear to an expert. Nonetheless, the paper needs a thorough re-write to avoid guesswork and speculation.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "k9__gyr4NeY", "reviewer": "Reviewer_4tsk", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors provide a new pooling strategy for GNN on learning physical systems on unstructured meshes along with a down sampling and up sampling method to reduce overhead between levels. Current GNN method to infer physical states suffers from two limitations. The computation complexity is high and over-smoothing problems will happen.  The author claim that such limitations is caused by pooling and building connection at coarser levels. So, the authors provide a new pooling strategy to solve this problem.  \n\nThe motivation is to create a pooling strategy that creates a 2-nd enhancement to preserve the connectivity on any input graph. So, they topologically sort the input graph and find that pooling on every depth will create a bipartition. To resemble the bipartition in mesh, they implement the BFS to compute the distance from a seed to other nodes. Then, they will do the pooling on the BFS frontiers. Also, building auxiliary edges can benefit from the bi-stride pooling.  \n\nAlso, the propose a method to reduce the overhead of transition modules between adjacent levels. There is a down sampling process to project the latent information to pivot nodes. After un-pooling, there will be an up-sampling process to return message back. \n\n## update after reading rebuttal.\nThe authors resolved my concerns and I would like to keep my score.", "summary_of_the_review": "This paper proposes a new pooling strategy for GNN in physics-based learning. They try to tackle the problem from a different aspect compared to previous research work. They borrow the idea of bi-partition, which is novel in this area. And the training speed and memory usage is decreasing, which means the computation is decreased.", "strength_and_weaknesses": "The authors propose that the current limitation of GNN method for physics-based learning is pooling and building connections at coarser level. This is novel since not too many people is focused on this area. The idea is simple and easy to follow. The result shows a noticeable decreasing in training and inference time since the overhead is reduced. \n\nThere are still a few things that the author can do to improve data. First, it is better to involve some ablation study to show how each part contribute to the increase of training speed. Also, I am wondering if there are erroneous connections existed in graph produced by your method and what causes these wrong connections. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "a1U3Gsr9jk", "reviewer": "Reviewer_iDNU", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Long-range spatial information can be passed with a low computational cost through pooling and unpooling operations. On an irregular mesh, there is a challenge of how to select pivot nodes and assign non-pivot nodes to a pivot node for pooling so that all assignments are at most 2 hops away. Previous methods have used spatial information for assignment, but this can result in assignments of nodes greater than 2 hops away or even for which a path does not exist. \n\nThis paper makes two contributions:\n\n1. A pooling strategy for pivot node selection and non-pivot node assignment that guarantees all non-pivots are assigned to a pivot at most 2 hops away. \n\n2. A deterministic, parameter-free pooling/ unpooling algorithm\n\nBSMS-GNN is compared to 2 competitors: one is a flat GNN, and the other is a hierarchical GNN that uses spatial information for pooling. The comparison is made on 4 datasets for physical simulation. BSMS-GNN is shown to significantly improve number of training epochs to reach a target test error, time per epoch, inference time, and memory requirements on all datasets. It's also shown on a small toy dataset that the hierarchical competitor adds incorrect edges, while BSMS-GNN does not. \n", "summary_of_the_review": "This contribution is marginally above the acceptance threshold. It provides an elegant solution to an important problem with the bi-stride pooling strategy and proposes a parameter-free downsampling technique that allows for deeper hierarchies. This contribution could be strengthened with more thorough experiments.", "strength_and_weaknesses": "------------------------------------------------------\n\nStrengths \n\nThe paper is well written and addresses a relevant problem. Previous methods that attempt to solve this problem are well-documented. The solution is elegant and is shown to be quite effective at significantly improving efficiency relative to the considered competitors. \n\n------------------------------------------------------\n\nWeaknesses \n\nThe main weakness are in the experiments:\n\n1. Not enough competitors are considered. Several previous methods for graph pooling are mentioned: \n\n- Gao & Ji (2019) \n- Lino et al. (2022a; 2021; 2022b) \n- Liu et al. (2021)\n- Fortunato et al. (2022) \n\nHowever, only the work of Lino, et al. is compared in the experiments. It is important that the test error and efficiency for all methods be compared.\n\n2. For the considered methods, there is no report of the test error from the converged models. Although BSMS-GNN is shown to achieve the same test error as GraphMeshNets as reported by Pfaff et al., Lino et al. (Simulating Continuum Mechanics with Multi-Scale Graph Neural Networks) report greater performance for MS-GNN-Grid than than GraphMeshNets. Thus, it is not clear which method produces the best test error. \n\n3. Related to points 1 and 2, it is not shown experimentally that connections that violate the boundary necessarily result in worse performance.\n\n4. It is claimed that BSMS-GNN will work well on the cloth benchmarks. This claim is too strong to be made without experimental evidence since the dynamics of the cloth appear to be very different from those considered.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject