Learning Flexible Body Collision Dynamics with Hierarchical Contact Mesh Transformer
Authors: Youn-Yeol Yu
Jeongwhan Choi
Woojin Cho
Kookjin Lee
Nayong Kim
Kiseok Chang
ChangSeung Woo
ILHO KIM
SeokWoo Lee
Joon Young Yang
SOOYOUNG YOON
Noseong Park
Venue: ICLR 2024 poster (2024)
Keywords: graph transformer
physics-based simulation
mesh
collision
flexible dynamics
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Recently, many mesh-based graph neural network (GNN) models have been proposed for modeling complex high-dimensional physical systems. Remarkable achievements have been made in significantly reducing the solving time compared to traditional numerical solvers. These methods are typically designed to i) reduce the computational cost in solving physical dynamics and/or ii) propose techniques to enhance the solution accuracy in fluid and rigid body dynamics. However, it remains under-explored whether they are effective in addressing the challenges of flexible body dynamics, where instantaneous collisions occur within a very short timeframe. In this paper, we present Hierarchical Contact Mesh Transformer (HCMT), which uses hierarchical mesh structures and can learn long-range dependencies (occurred by collisions) among spatially distant positions of a body --- two close positions in a higher-level mesh correspond to two distant positions in a lower-level mesh. HCMT enables long-range interactions, and the hierarchical mesh structure quickly propagates collision effects to faraway positions. To this end, it consists of a contact mesh Transformer and a hierarchical mesh Transformer (CMT and HMT, respectively). Lastly, we propose a flexible body dynamics dataset,  consisting of trajectories that reflect experimental settings frequently used in the display industry for product designs. We also compare the performance of several baselines using well-known benchmark datasets. Our results show that HCMT provides significant performance improvements over existing methods. Our code is available at https://github.com/yuyudeep/hcmt.

Published as a conference paper at ICLR 2024
LEARNING FLEXIBLE BODY COLLISION DYNAM- ICS WITH HIERARCHICAL CONTACT MESH TRANS- FORMER
Youn-Yeol Yu', Jeongwhan Choi', Woojin Cho', Kookjin Lee", Nayong Kim Kiseok Chang Chang-Seung Woo Ihoo Kim
Seok-Woo Lee². Joon- Joon-Young Yang'. Sooyoung Yoon Noseong Park 1+ Yonsei University AArizona State University LG Display Co., Ltd. *KAIST
(yyyou, jeongwhan. choi, nosecngekaist.ac.krr
kockjin, www.:eleeesseeeeeeeeeeee fnayong, kim, kschang, wcs0916, 6, i1ho. kim, ardorsu, masteryang, syyoon) eagoonnelldiin @lgdisplayyll..l...o......l.llpp com
ABSTRACT
Recently, many mesh-based graph neural network (GNN) models have been pro- posed for modeling complex high- nigh-dimensional physical systems. Remarkable achievements have been made significantly reducing the solving time compared to traditional numerical solvers. These methods are typically designed to i) reduce the computational cost in solving physical dynamics and/or ii) propose techniques enhance the solution accuracy in fluid and rigid body dynamics. However, it remains under-explored whether they are effective in addressing the challenges flexible body dynamics, where instantaneous collisions occur within very short timeframe. In this paper, we present Hierarchical Contact Mesh Transformer (HCMT), which uses hierarchical mesh structures and can learn long-range de- pendencies (occurred by collisions) among spatially distant positions of body two close positions in a higher-level mesh correspond to two distant positions in a lower-level mesh. HCMT enables long-range interactions, and the hierarchi- cal mesh structure quickly propagates collision effects to faraway positions. To this end, it consists of contact mesh Transformer and hierarchical mesh Trans- former (CMT and HMT, respectively). Lastly, we propose a flexible body dynam- ics dataset, consisting of trajectories that reflect experimental settings frequently used in the display industry for product designs. We also compare the performance of several baselines using well-known benchmark datasets. Our results show that HCMT provides significant performance improvements over existing methods. Our code is available at htttts://gtthtii com httpyyyuudddppeeeee /homt
INTRODUCTION
There is escalating interest in accelerating or replacing expensive traditional numerical methods with
learning- simulators (Li et al.) 2020. 2021; Raissi et al., 2019: Cho et al.) 2023). Learning- based simulators have delivered promising outcomes across various domains, e.g., molecular (Noč) (ct al. 2020), acro (Bhatnagar et al. 2019), fluid (Kochkov et al. 2021| Stachenfeld et al. 2021), and rigid body dynamics (Byravan & Fox 2017). In particular, graph neural networks (GNNs) with a mesh have demonstrated their strength and adaptability on these topics
et al.)
2020 Pfaff et al.] 2020). These methods capable of i) directly operating simulation meshes and ii) modeling systems with intricate domain boundaries. However, solving flexible dynamics with contacts remains under-explored.
Fig. and Table [ represent the complexity of various physical systems. Flexible dynamics must consider factors such as mass, damping, and stiffness, and due to its high non-linearity, it presents challenging problems (see Appendix A for the importance of flexible dynamics with contacts). Therefore, solving collision problems in flexible dynamics within a very short timeframe, is
"Corresponding Author.

---

Published as a conference paper at ICLR 2024
Non-
linearity
Flexible Dynamic
Level
5
Rigid Dynamic
Explicit
Level
Static
Implicit
Level
Level
Velosly
An illustration of the contact propa- gation via one- propagation in various levels. Level is the i-th level mesh (after pooling nodes from the previous level) in our hierarchical mesh structure. After node pooling, our Transformer accelerates contact propagation, which is a key in simulating flexible body collision dynamics.
Figure 2:
Figure I: Relationship between velocity and non-linearity in various physical systems. Typi- cally, implicit methods are used in static system, while explicit methods (Chang (007)) are em- ployed in dynamic domain. Flexible dynamics solve problems with highly non-linear charac- teristics and occur quickly in a very short time.
Table 1: A comparison table of various systems. The biggest difference between rigid and flexible dynamics is that stress cannot be calculated. M(x), D(x), and T(x) represent mass, damping, and stiffness matrices, respectively. In Table, drop the dependency external force vector in governing equations. x and a are acceleration and stress, respectively.
for simplicity. f denotes
Behavior System
Governing Equation
Output Solver
Dataset
Flexible Static
Tx f Rigid Dynamic MX(() + Dx(()= ((1)
%. a
Implicit Deforming, Deformable Plate
Explicit Explicit
Sphere Simple
Flexible Dynamic MN(() + (x + Tx(() = (())
X, a
Impact Plate
essential verify whether mesh-based GNNs are effective in handling the phenomenon, where the impact of collisions is propagated far from the point of collision between two objects. GNN models typically perform local message passing (Wu et al.) 2020), which means that they cannot quickly propagate the influence of collision over long distances. To achieve such long-range propagation, multiple GNN layers are required, which increases the training/inference time. Therefore, GNN- based simulators have trade-off relation between accuracy and training time (Bojchevski et al. 2020 Zhang et al. 2022).
Recent studies (Janny al. 2023 Han et al. 2022 in fluid dynamics and mesh domains introduce Transformer to apply global self-attention. However, Transformers require a computational com- plexity of ((N2) to generate predictions (Kreuzer al., 2021), where N is the number of nodes. Additionally, flcxible dynamics require an additional instantaneous contact edge for two colliding objects represent collision, which slows down training. For this reason, when there are large number of nodes, the naïve adoption of Transformers for modeling collisions still incurs high costs.
To address these challenges, propose a novel method called Hierarchical Contact Mesh Graph Transformer (HCMT). The multi-level hierarchical structure of HCMT i) quickly propagates col- lisions and ii) decreases training time due to the reduced number of nodes in Level i, i.c. the i-th mesh, resulting from pooling nodes from previous level (cf. Fig. 2b. HCMT with higher-level meshes focuses on long-range dynamic interactions. In addition, HCMT has two Transformers: one for con- tact dynamics, and the other for flexible dynamics. Finally, introduce novel benchmark dataset named Impact Plate. Impact Plate replicates simulations conducted in the display industry for mo- bile phone display rigidity assessments. We evaluate model three publicly available datasets (Sphere Simple (Praff et al. 2020). Deforming Plate (Pfaff et al., 2020). Deformable Plate (Link- erhägner et al.] 2023)) and the novel Impact Plate, and our model achieves consistently the best performance.
The main contributions of this paper be summarized as follows:
2

---

Published as a conference paper at ICLR 2024
We propose a Hierarchical Contact Mesh Transformer (HCMT), which, to the best of our knowledge, incorporates collisions into flexible body dynamics for the first time.
We efficiently use two Transformers with different roles for flexible and contact dynamics.
We provide an Impact Plate benchmark dataset based on explicit methods where collisions occur in a very short timeframe. Various design parameters have been applied to make suitable for use in manufacturing.
HCMT outperforms baseline models in static, rigid, and flexible dynamics systems.
2
RELATED WORK
2.1
GNN-BASED MODELS AND HIERARCHICAL MODELS IN PHYSICAL. SYSTEMS
The prediction of complex physical systems using GNNs is active research area in scientific machine learning (Belbute-Pene Perues et al., 2020 Rubanova et al., 2021 Mrowca et al. 2018 Li et al. 2019 2018 The most representative work is MGN (Praft et al. 2020), which uses message passing network to learn the dynamics of physical systems. It is applied to various systems such as Lagrangian and Eulerian. Due to local processing nature of MGN, signal tends to propagate slowly through the mesh (graph). MGN has local processing that requires one layer for one propagation step. Therefore, MGN requires many layers for long-distance propagation.
Recently, several hierarchical models have been introduced to increase the propagation radius (Gao
& Ji 2019; Fortunato et al., 2022] Han et al.) 2022]] Janny et al. 2023 Cao al. 2022] Grigorev
et al. 2023). Hierarchical models in these studies are divided into two types: The first type is
level structure: i) Teii--rrafffomr--GMMe
(Han al., 2022) uses a pooling method to
select pivotal nodes through uniform sampling, ii) EAGLE Transforrrrn (Janny et al. (003) proposes a clustering-based pooling method and shows promising performance in fluid dynamics, and iii) MS-MGN (Fortunato et al., 2022) proposes a dual-layer framework that passes messages on two different resolutions (fine and coarse resolutions) for mesh-based simulation learning. The second type has multi-level structure: i) Cao et al. (2022) analyzes the limitations of existing pooling strategies and proposes bi-stride pooling, which uses breadth-first search (BFS) to select nodes, ii) HOOD (Grigorev al. 2023) leverages multi-level message passing with unsupervised training to predict clothing dynamics in real-time for arbitrary types of garments and body shapes.
2.2 CONTACT AND COLLISION MODELS
The field dealing with contact/collision problems is primarily used in computer games (De Jong jtt al.] 2014), animation (Hahn) 1988 Erleben 2004). and robotics (Posa et al.] 2014). method has been proposed to GNN for quick motion planning in path-finding the time required for object collision detection is significant Yu & Gao (021). In mesh-based GNN models, collisions edge-to-vertes interactions close to each other (Zhul it al. 2022). Additionally, as the used node-to-node contact/collision recognition method assumes that contacts always occur at nodes, FIGNET (Allen al., (022b) has been proposed intro- duce face-to-face (or edge-to-edge) recognition and utilize face features for training. Due to the simulation-to- gap, there are works demonstrating the learning and prediction of contact discon- tinuities by GNNs using both real and simulation datasets (Allen et al. (023). Previous studies have mainly focused on addressing contact recognition issues or improving accuracy in rigid body dy- namics. However, while it is possible to determine the motion of an object in rigid body dynamics, it is not applicable in flexible dynamics because the stress distribution is unknown.
determined by recognizing edge-to-edge
3 METHODLOGY
3.1 PROBLEM DEFINITION
Our model extends the picoder-processor-dee 2020) and MGN (Paff et al., (020) for solving flexible dynamics. The encoder and decoder in HCMT follow those in GNS and MGN, but enhance the processor by designing hierarchical
framework of GNS (Sanchez-Gonzaee et al.
3

---

Published as a conference paper at ICLR 2024
HCMT
Encoder
CMT
HMT
Decoder
M
Go
Go
M M M+
Figure 3: Overview of Hierarchical Contact Mesh Transformer (HCMT) with four layers: encoder, CMT. HMT, and decoder. The light blue graph in Go corresponds to a ball in the Impact Plate dataset, while the green graph represents plate.
G
G
Go=
HMT Block
Pooling
HMT Block
Pooling
HMT Block
6
HMT Block
Unpooliig
HMT Block
Unpooling
G
Figure 4: Hierarchical architecture of HMT layer in Fig. 3 constructed by repetitively stacking HMT blocks and pooling. highly effective in reducing computational complexity.
The overall process of HMT layer is Utilizing this hierarchical structure is
Transformer that consists of i) contact mesh Transformer (CMT) for propagating contact messages and ii) hierarchical mesh Transformer (HMT) for handling long-range interactions.
At time t, we represent the system's state using current mesh M (V, E), which includes node coordinates, density Pi and Young's modulus Y; within each node, as well as connecting edges myj- The objective of icMT to predict the next mesh M++1, utilizing both the current mesh M and the previous meshes M generated through the simulator iteratively based on the previous prediction; M M ...., M where k is the length of total steps.
A where p history size. Finally, The rollout trajectory can be
3.2 OVERALL ARCHITECTURE
Fig. 3 shows the overall architecture HCMT which consists an encoder, a contact mesh Trans- former (CMT), hierarchical mesh Transformer (HMT), and decoder. The overall workflow is as follows - for simplicity but without loss of generality, we discuss predicting Mr++1 from M only:
The mesh M is transformed into a graph Go = (Vo, Eo, C') via encoders: the features of the i-th node, the (i,j) mesh edge. and the (i, contact edge, f. mij. and Cig in M are transformed to hidden vectors, 2; E Vo, Cij E Eg. and S;g EC, by their respective encoders.
2. CMT layer captures the contact dynamics from those transformed hidden vectors. CMT propagates contact messages through the contact edges between two colliding objects (c.g., ball and a plate).
3. HMT layer follows CMT layer by taking the output of CMT as its input. HMT module uses hierarchical graph with nodes properly pooled into the mesh structure enable long-range propagation. We only consider propagation over mesh edges by contact-then- propagate strategy, i.e., contact edges are excluded this process.
4. The decoder predicts the next velocity of each node by using the last hidden representa- tion produced by HMT layer and utilizes an updater to calculate the node positions of the objects.
4

---

Published as a conference paper at ICLR 2024
2 4+
h
Eq.(3)
H Heads
concat
Heads
Feed Forward
Feed Forward
Mesh Atlention
Contact Atlention
Norm
A46
xL
B.46
xL
(5,
Dual-Branch Self-Attention
a,3
bas
bi
Mesh Self-Attention
B2p2
Norm
aij from (q.
big from (2)
Norm
Big
hf
(a) CMT block (a CMT layer has Le CMT blocks)
(b) HMT block (an HMT layer has LH HMT blocks)
Figure 5: (a) depicts our proposed CMT block and dual self-attention blocks. The mesh attention on the left only computes attention weights a, for each of the edges of two colliding objects nodes. The contact attention on the right only considers the weights big of contact edges. (b) describes our proposed HMT module. The mesh self-attention in HMT is the same as mesh attention CMT.
3.3 LEARNING FLEXIBLE BODY DYNAMICS
Features and Encoder Layer The mesh M is transformed into a graph Go = (Vo, Eo. C). Mesh nodes become graph nodes Vo, mesh edges become bidirectional mesh-edges Eg in the graph. Contact edges are connections between two different objects' nodes or self-contacts within an object, resulting in C in the graph. For every node, find all neighhoring nodes within radius and connect with them via contact edges, excluding previously connected mesh edges. The subscript "O' in Go denotes the initial level of our hierarchical mesh structure.
It is imperative that our model can learn collision behaviors in scenario where its node positions not fixed, and the mesh shape undergoes changes. Therefore, we structure the state of the system using two edge features: i) mesh edge feature m.J. where (i.j) E Eo, contains information about connections within objects, and ii) contact edge feature Cig. where (i, g) C, rerpreeent connections between two objects within the collision range. Notably, the contact edge feature is novel addition not found in fluid models. Both features are defined based on relative displacements between nodes. For detailed information of features used in each domain, see Table in Appendix C
Next, the features are transformed into hidden vector at every node and edge. The hidden vec- tors for nodes are denoted as 2, and for mesh and contact edges as e" and Sig. respectively. The transformation is carried out by the encoder MLP éaode, Emesh. and ceontact- The encoder emesh shares weight parameters across levels because edge connections are lost when the level changes and new edge features need to be generated.
Contact Mesh Transformer (CMT) Layer CMT layer follows the encoder and utilizes both mesh and contact edge encoded features (see Fig. a))) We use dual-branch self-attention method with two different edges in the collision dynamics to capture the effects of the collision, resulting in a better node representation. As shown in Fig. a). dual-branch self-attention branches into two self-attentions. In this case, both edge features are used to take into account the dynamically changing positions the nodes and the length information of the edges the dynamics system when computing the self-attention weights, For the self-attention weight aj the mesh edge fcature ei, can capture the relative distance between two nodes affected by collisions and used in the calculation of the self-attention weight ij self-attention weight, b is calculated by including the contact edge feature Sig (see Equation 2
k.f' for mesh edges in an object,
&. as shown in Equation [ Another
The two attention weights for the dual-branch self-attention of the E-th block and the k-th head
5

---

Published as a conference paper at ICLR 2024
defined as follows:
Q LL(() K. L + LN(())
k softmax, a (j
clip
(I)
vdz
(L((((
K. (L(())
b R. softmax, iq
clip
LN((g))
LN(iL)()))))
(2)
Vag
z concat
|===
K.l (V (L()))
Σ biy (VHLN(z)))) GET:
(3)
JEN;
where N₁ is the set of the neighbors of node i, excluding contact edges between other objects and k. Ti is the set of the neighbors for contact edges between different objects. a big are the attention scores for mesh and contact edges, respectively. Q k V.k. E. Wh. E Rd,xd, are train- able parameters. & = to H denotes the number of attention heads, and || denotes concatenation. We use clamping for numerical stability (see Appendix [..). dg is the dimension of node hidden vector. We adopt Pre-Layer Norm architecture (Xiong et al. 2020), which is denoted as LN(-).
To move on to the next block, the output x f+1 is projected and passed to the Feed Forward Network (FFN) and represented by the residual update method as follows:
++1 2 + O प + (LN + OZz))),
(4)
where O IR2 x2d, is the learned output projection matrix. Notably, in our model, the edge network remains unaltered and does not undergo updates.
Hierarchical Mesh Transformer (HMT) Layer Our proposed HMT layer propagates messages through mesh edges in an object. HMT layer begins after the initial collision dynamics that have been captured by CMT layer. By using only mesh edge features, HMT layer uses only the mesh self-attention, as shown in Fig. (b). Note that the behavior is identical to the mesh attention in CMT layer. HMT layer uses node pooling to construct hierarchical graphs so that the propagation of nodes can reach a long-range of nodes. The updated Go from CMT layer is pooled up to level From Gg. We apply the node pooling operation to construct [G1, G2. ... Gx) (with decreasing numbers of nodes) and the propagation by the mesh self-attention occurs at each level G₁ (see Fig. 44.. The reduced number of mesh edges due to pooling can reduce the computational complexity of the model, i.e., the training speed, and at the same time consider longer range of interactions. HMT layer can achieve the effect of shortcut message passing, as shown Fig. 2 and further extend the impact of collisions over long-range distances. One block of HMT layer is defined as follows:
2% LLN())) K LL((E)) E (
k a
softmax,
clip
W (5)
vdh
6
H
afj k.l
LV()))
(6)
JEN.
het hf + O FEN (LN(H) + O,1())
(7)
The (E - 1) edge feature uses the (LH - -) edge feature after reaching the final level X. Note that h/, the input to the first block of HMT, is the same z Lcc
Pooling Method Our pooling method consists of two steps: i) sampling nodes and ii) remeshing. In the first step, For node sampling, we follow the node selection method using the breadth-fistt -irst-- search (BFS) proposed by [Ca et al. (2022). As the level increases, the number of nodes is reduced by almost half compared to the previous level. A pooling operation is performed for each individual object.
In the second step, we regenerate mesh edges using Delaunay triangulation (Lee & Schachter 1980, Lei et al. 2023) based on the selected nodes at each level. Delaunay triangulation produces well- shaped mesh because triangles with large internal angles are selected over triangles with small in- ternal angles when satisfying the Delaunay criterion By performing the remeshing procedure, the overall quality of the mesh is improved. For detailed results, see Tables and 3lin Appendix
"This criterion is known the empty circumcircle property.

---

Published as a conference paper at ICLR 2024
Decoder and Updater We describe the decoder and updater based Impact Plate dataset. Ac- cording to the MGN approach, the model predicts one more output features o; such velocity X and next stress of by employing an MLP decoder. Finally, X used to calculate the next position x through the updater, which performs a first-order integration (&. x x().
Training Loss We use the one-step MSE loss as a training objective. Since stresses are included, the position and stress MSE in flexible dynamics are calculated as follows:
Vol
Vol
C
-ottt--ott7 0.t1)2
(8)
Vol
Vo
3.4 DISCUSSION
In this subsection, we discuss the importance of contact edge features in the design of HCMT and compare HCMT with existing graph Transformers in the mesh domain.
Why Mesh and Contact Edge Features are Important? The contact edge represents crucial information that depicts the collision phenomenon between two objects. is defined as the relative displacement between nodes where contact occurs within specific radius, which can distinguish the strength or weakness of a collision. When an object is deformed after a collision, the shape of the mesh (cell) also changes, so mesh edges also contain important information. Hence, collision models place considerable emphasis on considering the impacts of collisions as the decrease or increase of edge lengths. Edge consists of both mesh edge, which encompasses mesh connections within objects, and contact edge, which considers the effects of collisions for flexible dynamics.
Comparison with Transformer-based Models EAGLE (Janny et al. 2023 3, which learns fluid dynamics, does fluid dynamics, node positions and edge lengths are fixed, making edge functions less useful. In flexible dynamics where contact occurs, however, contact edges carry important messages. addi- tion, because the object is severely deformed, mesh edge features are also greatly affected. GMR- Transformer-GMe (Han et al., (022) is representative example of applying a Transformer in the mesh domain. model uses two Transformers with edge features. Graph Transformer (Dwivedi Bresson 2020) (GT) has been extended to incorporate edge representations, which can work with information asso- ciated with edges. In comparison with them, we utilize both mesh and contact edge features to infer accurately.
edge features in its Transformer but its last GNN. In Fulerian-based
TeR-Transforme--Cinn GMUS uses one Transformer without edge features, while our
EXPERIMENTS
4.1 EXPERIMENTAL SETTINGS
Datasets We use 4 datasets of 3 different types: Impact Plate dataset for flexible dynamics, ii) Deforming Plate and Deformable Plate for static dynamics, and iii) Sphere Simple for rigid dynamics. Impact Plate is created with traditional solver with ANSYS (Stolarkki et al. 2018 to validate the efficacy of our model in flexible dynamics (see Table 6 in Appendix
Baselines, Setups and Hyperparameters As baselines, we use MGN (Pfaff et al. 2020). a state- of-the-art art model in the field of complex physics, and GT (Dwivedi & Bresson, 2020) with edge features. A detailed description for all baselines provided in Appendix D The number of blocks
L Le + Ln is set to 15. For further details on hyperparameters, Table 8 Appendix E
4.2 EXPERIMENTAL RESULTS
Table shows the results of comparing our HCMT to two baselines. HCMT outperforms in all cases (e.g., 49% improvement position RMSE for Impact Plate) and has the lowest standard deviations. We also report empirical complexity in Appendix F

---

Published as a conference paper at ICLR 2024
Table 2: RMSE (rollout-all, x x10%) for our model and the baselines. Improv. means the percentage improvement over the runner-up and bold denotes the best performance.
Impact Plate
Deforming Plate
Sphere Simple Deformable Plate
Model
Position
Stress
Position
Stress
Position
Position
19.48544
29,291
11.3441028
9, 1,168,28822 164,941
243 14:
13.74
MGN
40.73:2294
15,871
7.83-10.16
4,644.483-1422500
33.26:16.33
10.78.10.54
HCMT
20.71:10577
14,742:5502
7.49:10.7
4,535,956:1499p8
30.41:1.71
7.67
Improv.
49.2%
58.9%
4.3%
2.3%
8.6%
28.9%
(a) Level
(b) Level 2 1/l Vol
0.26) (c) Level 4
V4 Val
0.07) (d) Level 6
Vil Vol
0.02)
Figure 6: Level-wise mesh visualization of Impact Plate. See more visualizations Appendix
700
400
100
(a) Ground Truth
(b) HCMT
(c) MGN
(d) GT
Figure 7: 2D cross-sectional contour of the stress field in Impact Plate. In the red bounding box, HCMT is the most similar to the node positions in the ground truth. Brighter colors mean higher stress. Rollout images of other datasets can be found in Appendix L
Result HCMT shows superior performance in terms of long-term prediction across all domains compared MGN and GT. In particular, we can that HCMT outperforms other methods in Impact Plate, where strong impacts occur, and that our hierarchical architecture effectively propa- gates collisions. Fig. shows that as the level increases, shortcut message passing possible in our method. Fig. shows the stress on the falling ball and plate after collision, visually demonstrating the superiority of HCMT. We also report generalization abilities of HCMT in Appendix G
Visualize Attention Map We visualize the dual-branch self-attention maps of CMT layer on Impact Plate in Fig.8 -ee note that the self- attention is naturally sparse in Gg- The contact and mesh self-attention show non-overlapping attention maps and the varying importance of contact and mesh edges. The bounding box in Fig. 8 (a) represents the importance of con- tact edges between the ball and the plate, and blue box in Fig. 8 (b) signifies the self-attention among plate nodes, and the yellow box rep- resents interactions through edges among ball nodes (see Appendix J for self-attention map of HMT and results in different datasets).
Contact Self-Attention (b) Mesh Self-Attentinn
Figure 8: Self-attention maps in CMT (see fig. 13 in Appendix for map of HMT).
8

---

Published as a conference paper at ICLR 2024
Table 3:
The results of ablation studies.
Impact Plate
Deforming Plate
Sphere Simple
Deformable Plate
Model
Position
Stress
Position
Stress
Position
Position
HCMT
20.34
14,447
7.37
4,475,616
28.30
7.67
Aate-Contact
42.90
22,874
7.74
4.991.179
38.81
7.97
Only HMT
46.27
31,932
22.32
33,773,408
144.73
24.69
Only CMT
55.63
28,764
7.96
4,662,353
30.15
HCMT+LPE
21.11
14,920
7.52
4,618,027
30.16
7.87
12
35
Rollout RMSE
Ro lout RMSE
Roll hollout RMSE
20
30
Bollout
25
5
2
3
2
20
2
3
Level
Level
(a) Impact Plate
(b) Deformable Plate
(c) Deforming Plate
(d) Sphere Simple
Figure 9; Sensitivity to X. The red dashed lines represent RMSE (rollout-all, x x10*) of MGN.
4.3 ABLATION AND SENSITIVITY STUDIES
This subsection describes the ablation and sensitivity studies for HCMT. Additional studies and analyses are in Appendix[]
Ablation Studies We define the following models for ablation studies: TLate-Contaee where the order of CMT layer is followed by HMT layer, ii) "Only CMT", which uses only CMT layer, iii) "Only HMT", which uses only HMT layer, and iv) "HCMT+LPE", which adds Laplacian po- sitional encoding (LPE) (Dwivedi & Bresson 2020). In Table "LatteCCottct" performs strategy, where CMT layer is
than HCMT. This shows that in HCMT, the centtacttteen-roppeee followed by HMT layer, is appropriate for propagating the forces generated by the interaction and reaction two objects. In all cases, "Only CMT" performs better than "Only HMT" Finally, "HCMT+LPE" performs the best among the ablation models, but shows slight performance drop compared to our HCMT. We conjecture that this is because our feature definitions already include node positions and therefore. additional positional encodings are needed for nodes.
Sensitivity to the Number of Level Fig. shows the position RMSE varying the number of level X. For Impact Plate, the RMSE tends to decrease as the level increases due to short time intervals and substantial impacts. For Deformable Plate, due to small number of nodes, there is tendency for the RMSE to decrease as the level decreases. For Deforming Plate, levels 2 to 4 show the best RMSE because a plate is affected by object pushing it up about half the total plate size. For Sphere Simple, the RMSE gets worse as the level increases, but it shows the best result at level 4.
5 CONCLUSION AND FUTURE WORK
We presented HCMT, novel mesh Transformer that can effectively learn flexible body dynamics with collisions. HCMT uses hierarchical structure and two different Transformers to enable long- range interactions and quickly propagate collision effects. We show that HCMT outperforms other baselines variety of systems, including flexible dynamics. We believe that HCMT has potential to be used in a variety of applications, such as product design and manufacturing. Future work topics include learnable pooling methods and combining mesh-based simulators with high-quality mesh generation models (Lei et al. 2023 Nash et al. 2020) from complex geometries for design optimization (Allen et al. 2022a).

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
This work was supported by the LG Display and an ITTP grant funded by the Korean government (MSIT) (No.2020-0-01361, Artificial Intelligence Graduate School Program (Yonsei University)).
ETHICS STATEMENT
There is no ethical problem with our experimental settings and results.
REFERENCES
Kelsey Allen, Tatiana
Kimberly Stachenfeld, Alvaro Sanchez-Gonzalez Peter Battaglia, Jessica Hamrick, and Tobias Pfaff. Physical design using differentiable learned simu- lators. arXiv preprint arXiv:2202.0072: 007288 2022a.
Kelsey R Allen, Yulia Rubanova, Tatiana Lopez-Guevans William Whitney, Alvaro Sanchez- Gonzalez, Peter Battaglia, and Tobias Pfaff. Learning rigid dynamies with face interaction graph networks. arXiv preprinl arXiv: 2022b..
Kelsey R Allen, Tatiana Lopez Guevara, Yulia Rubanova, Kim Stachenfeld, Alvaro Sanchez- Gonzalez, Peter Battaglia, and Tobias Pfaff. Graph network simulators can learn discontinuous, rigid contact dynamics. In Conference Robot Learning, pp. 1157-1167. PMLR, 2023.
Filipe De Avila Belbute- belbute-Peres, Thomas Economon, and Zico Kolter. Combining differentiable pde solvers and graph neural networks for fluid flow prediction. In international conference ma- chine learning, pp. 2402-242222222 1. PMLR, 2020.
Saakaar Bhatnagar, Yaser Afshar, Shaowu Pan, Karthik Duraisamy, and Shailendra Kaushik. Predic- tion of aerodynamic flow fields using convolutional neural networks. Computational Mechanics, 64:525-545, 2019.
Aleksandar Bojchevski, Johannes Gasteiger, Bryan Perozzi, Amol Kapoor, Martin Blais, Benedek Rózemberczki, Michal Lukasik, and Stephan Günnemann. Scaling graph neural networks with approximate pagerank. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2464-2473, 2020.
Arunkumar Byravan and Dieter Fox. Se3-nets: Learning rigid body motion using deep neural networks. 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 173- 180. IEEE, 2017.
Yadi Cao, Menglei Chai, Minchen Li, and Chenfanfu Jiang. Bi-stride multi-scale graph neural network for mesh-based physical simulation. arXiv preprini urXiv:2210002573. 2022.
Shuenn-Yit Yih Chang. Improved explicit method for structural dynamics. Journal of Engineering Mechanics, 133(7):748-760, 2007.
Woojin Cho, Kookjin Lee, Donsub Rim, and Noseong Park. Hypernetwork- based meta-learning for low-rank physics-ifformed neural networks. arXiv preprint arXiv:2310.09528 2023.
JH De Jong, Kjetil Wormnes, and Paolo Tiso. Simulating rigid-bodies, strings and nets for engineer- ing applications using gaming industry physies simulators. In i-SAIRAS: International Symposium on Artificial Intelligence, Robotics and Automation in Space, Montreal, June, pp. 17-19, 2014.
Vijay Prakash Dwivedi and Xavier Bresson. A generalization of transformer networks to graphs. arXiv preprint arXiv:2012.09699 2020.
Kenny Erleben. Stable, robust, and versatile multibody dynamics animation. Unpublished Ph. D. Thesis, University of Copenhagen, Copenhagen, pp. 1-241, 2004.
Meire Fortunato, Tobias Pfaff, Peter Wirnsberger Alexander Pritzel, and Peter Battaglia. Multiscale meshgraphnets. arXiv preprint arXiv:2210.006/12 2022.
10

---

Published as a conference paper at ICLR 2024
Hongyang Gao and Shuiwang Ji. Graph u-nets. In international conference machine learning, pp. 209822002.. PMLR, 2019.
Floris Goerlandt and Pentti Kujala. Traffic simulation based ship collision probability modeling. Reliability Engineering & System Safety, 96(1):11107, 2011.
Artur Grigorev, Michael J Black, and Otmar Hilliges. Hood: Hierarchical graphs for generalized modelling of clothing dynamics. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition, pp. 16965-16974, 2023.
Chunyang Gui, Jiantao Bai, and Wenjie Zuo. Simplified crashworthiness method of automotive Thin-Walled
frame for conceptual design. Thin- Walled Structures, 131:324-333 2018.
James K Hahn. Realistic animation of rigid bodies. ACM Siggraph computer graphics, 22(4): 299-308, 1988.
Xu Han, Han Gao, Tobias Pfaff, Jian-Xun Wang, and Li-Ping Liu. Predicting physics in mesh- reduced space with temporal attention. arXiv preprint arXiv:2201.091 2022.
Steeven Janny, Aurélien Beneteau, Nicolas Thome, Madiha Nadri, Julie Digne, and Christian Wolf. Eagle: Large-scale learning of turbulent fluid dynamics with mesh transformers. arXiv preprint arXiv:2302. 10803, 2023.
Suwat Jirathearanat, Christoph Hartl, and Taylan Altan. Hydroforming of y-shapes-produus and process design using fea simulation and experiments. Journal of Materials Processing Technol- ogy, 146(1):124 129, 2004.
Dmitri Kochkov, Jamie A Smith, Ayya Alieva, Qing Wang, Michael P Brenner, and Stephan computational fluid dynamics. Proceedings of the National Academy of Sciences, 118(21) (18(21):e2100) 1e22112011111111111 2021.
Hoyer. Machine
Devin Kreuzer, Dominique Beaini, Will Hamilton, Vincent Létourneau, and Prudencio Tossou. Re- thinking graph transformers with spectral attention. Advances in Neural Information Processing Systems, 34:21618 226299, 2021.
Der-Tsai Lee and Bruce J Schachter. Two algorithms for constructing delaunay triangulation. International Journal of Computer Information Sciences, 93))219--22, 1980.
Na Lei, Zezeng Li, Zebin Xu, Ying Li, and Xianfeng Gu. What's the situation with intelligent mesh generation: A survey and perspectives. IEEE Transactions on Visualization and Computer Graphics, 2023.
Yunzhu Li, Jiajun Wu, Russ Tedrake, Joshua B Tenenbaum, and Antonio Torralba. Learning par- ticle dynamics for manipulating rigid bodies, deformable objects, and fluids. arXiv preprint arXiv: 1810.01 566,, 2018
Yunzhu Li, Jiajun Wu, Jun-Yan Zhu, Joshua B Tenenbaum, Antonio Torralba, and Russ Tedrake. Propagation networks for model-based control under partial observation. In 2019 International Conference on Robotics and Automation (ICRA). pp. 1205-1211. IEEE. 2019.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, An- drew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895, 2020.
Zongyi Li, Hongkai Zheng, Nikola Kovachki, David Jin, Haoxuan Chen, Burigede Liu, Kamyar Azizzadenesheli, and Anima Anandkumar. Physics-ifformed neural operator for learning partial differential equations. arXiv preprint arXiv:2111.03794 2021.
Jonas Linkerhägner, Niklas Freymuth, Paul Maria Scheikl, Franziska Mathis-Ullrich, and Ger- hard Neumann. Grounding graph network simulators using physical sensor observations. arXiv preprint ar arXiv:2302. / 2023.
Hu Liu, Mohd Hasrizam Che Man, and Kin Huat Low. Uav airbome collision to manned aireraft engine: Damage of fan blades and resultant thrust loss. Aerospace Science and Technology. 113: 106645, 2021.
11

---

Published as a conference paper at ICLR 2024
D Moxey, D Ekelschot, Ü Keskin, SJ Sherwin, and J Peiró. A thermo-elastic analogy for high- high-order curvilinear meshing with control of mesh validity and quality. Procedia engineering, 82:127-135, 2014.
Damian Mrowca, Chengxu Zhuang, Elias Wang, Nick Haber, Li F Fei-Fei, Josh Tenenbaum, and Daniel Yamins. Flexible neural representation for physics prediction. Advances neural information processing systems, 31, 2018.
Charlie Nash, Yaroslav Ganin, SM Ali Eslami, and Peter Battaglia. Polygen: An autoregressive generative model of 3d meshes, In International conference on machine learning, pp. 7220-7229. PMLR, 2020.
Frank Noé, Alexandre Tkatchenko, Klaus-Robert Müller, and Cecilia Clementi. Machine learning for molecular simulation. Annual review of physical chemistry, 71:361-390, 2020.
EJ Pavlina and CJ Van Tyne. Correlation of yield strength and tensile strength with hardness for steels. Journal of materials engineering and performance, 17:888 893, 2008.
Tobias Pfaff, Meire Fortunato, Alvaro Manchez-Gonzalee and Peter W Battaglia. Learning mesh- based simulation with graph networks. arXiv preprint WxXiv:2010.0340% 2020.
Michael Posa, Cecilia Cantu, and Russ Tedrake. A direct method for trajectory optimization of rigid bodies through contact. The International Journal of Robotics Research, 33(1):69-81, 2014.
Maziar Raissi, Paris Perdikaris, and George Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
Yulia Rubanova, Alvaro Lanchez-Gonzalez Tobias Pfaff, and Peter Battaglia. Constraint-baa 2021.
graph network simulator. arXiv preprint
Alvaro Sanchez-Gonzaler, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter Battaglia. Learning to simulate complex physics with graph networks. In International conference machine learning, pp. 4469. PMLR, 2020.
Kimberly Stachenfeld, Drummond B Fielding, Dmitrii Kochkov, Miles Cranmer, Tobias Pfaff, Jonathan Godwin, Can Cui, Shirley Ho, Peter Battaglia, and Alvaro Gonzalez. Learned coarse models for efficient turbulence simulation. arXiv preprint 12. 2021.
Tadeusz Stolarski, Yuji Nakasone, and Shigeka Yoshimoto. Engineering analysis with ANSYS soft-
ware. Henerwoort--einemaant 2018
ML Williams. Stress singularities resulting from various boundary conditions in angular corners of plates in extension. 1952.
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A comprehensive survey graph neural networks. IEEE transactions on neural networks and learning systems, 32(1):4-25 2020.
Ruibin Xiong, Yunchang Yang, Di He, Kai Zheng, Shuxin Zheng, Chen Xing, Huishuai Zhang, Yanyan Lan, Liwei Wang, and Tieyan Liu. On layer normalization in the transformer architecture. In International Conference on Machine Learning, pp. 10524-10533, PMLR, 2020.
Liang Xue, Claire R Coble, Hohyung Lee, Da Yu, Satish Chaparala, and Seunghae Park. Dy- namic analysis of thin glass under ball drop impact with new metrics. In International Electronic Packaging Technical Conference and Exhibition, volume 55751, pp. V00IT03A006, American Society of Mechanical Engineers, 2013.
Chenning Yu and Sicun Gao. Reducing collision checking for sampling-based motion planning using graph neural networks. Advances in Neural Information Processing Systems, 34:4274 2021.
Wentao Zhang, Yu Shen, Zheyu Lin, Yang Li, Xiaosen Li, Wen Ouyang, Yangyu Tao, Zhi Yang, and Bin Cui. Pasca: A graph neural architecture search system under the scalable paradigm. In Proceedings of the ACM Web Conference 2022, pp. 1817-1828, 2022.
12

---

Published as a conference paper at ICLR 2024
Xin Zhu, Yinling Qian, Qiong Wang, Ziliang Feng. and Pheng-Ann Heng. Collision-avee aware inter- active simulation using graph neural networks. Visual Computing for Industry, Biomedicine, and Art, 5(1):1113, 2022. 2022.
13

---

Published as a conference paper at ICLR 2024
A
WHY IS A STUDY ON FLEXIBLE DYNAMICS NECESSARY?
Collision simulation is widely utilized methodology in various industries such shipping (Go- erlandt & Kujala, 2011), automotive (Gui et al. 2018), aviation (Liu et al. 2021). etc. In collision dynamics, there are rigid body dynamics where the shape of the mesh remains unchanged and ii) flexible body dynamics where it can change. Rigid body dynamics forms meshes of objects where collisions can occur, allowing them detect collisions and predict the movement of the objects by calculating their accelerations. However, the goal of flexible body dynamics is to determine the maximum stress value and the part of interest on an object after collision has occurred. Exceeding the yield strength (Pavlina & Van Tyne, 2008) results in product damages, making it possible to pro- duce robust products through simulations for detecting vulnerable areas and changing their designs. For instance, when evaluating the rigidity of display panel to prevent damages caused by external forces, a cover is attached to the back of the panel (Xue et al. 2013). The reliability varies based on design elements such as the material, shape, thickness, and material of the cover. Therefore, before prototype production, a robust design (Jirathearanat et al. 2004) can be derived through flexible body dynamics.
Among previous studies, the hierarchical GNN model, BSMS-GNI (Cao et al. 2022). ). despite per- forming long-range interactions, exhibits lower accuracy in the collision domain compared to MGN. GNN-based hierarchical models, MS-GNN (Fortunato et al. (022) and HOOD (Grigorev et al., 2023), focus on studying the fluid domain or rigid body dynamies. Therefore, we need research flexible body dynamics and verification of the effectiveness of the hierarchical structure.
B POOLING METHOD DETAILS
Our pooling strategy proceeds in two steps. In the first step, nodes selected using breadth-fire firstt search (BFS), and in the second step. Delaunay triangulation applied to regenerate the mesh in high-quality shape for HMT. Delaunay triangulation divides plane into triangles by connecting points on the plane such a way that the minimum angle of the resulting triangles is maximized. Table represents the number of nodes at each level after BFS pooling. In each level the number of nodes decreases by half compared to its previous level. Table shows the mesh quality before and after remeshing. The quality of the meshes (cells) is evaluated using the scaled Jacobian met- ric (Moxey et al., 2014), and a scaled Jacobian value closer to indicates a mesh with a shape closer an equilateral triangle, signifying higher mesh quality. Angle refers the interior angles within the triangular cell, and as the minimum angle decreases and the maximum angle increases, the shape of the mesh appears more distorted. The angle and the scaled Jacobian values are the average values across all levels within the training data. Fig. 10 shows the results of remeshing.
Table 4: The number of nodes for each level.
Datasets
Level
Level Level Level Level Level Level
Impact Plate
2208
1108
559
285
148
Deforming Plate
1276
658
344
183
98
55
32
Sphere Simple
1731
898
523
330
224
170
138
Deformable Plate
138
77
48
32
24
20
18
Table The quality of the cells is evaluated using the scaled Jacobian, and scaled Jacobian closer to indicates mesh with shape closer to an equilateral triangle, i.e., high mesh quality.
Before remeshing
After remeshing
Datasct
Max Angle ())
Min Angle (*) Jacobian
Max Angle (*) Min Angle (*) Jacobian
Impact Plate
120.4
16.7
0.32
99.4
27.2
0.51
Deforming Plate
111.4
21.4
0.41
98.2
28.3
0.53
Sphere Simple
110.0
20.4
0.39
104.6
23.9
0.45
Deformable Plate
114,8
21.6
0.41
104.9
28.4
0.53
14

---

Published as a conference paper at ICLR 2024
(a) Before at Level
(b) After at Level
(c) Before at Level
(d) After at Level
(c) Before at Level
(f) After at Level 3
(g) Before at Level
(h) After at Level 4
ii Before Level 5
After Level
(k) Before Level
After Level
Figure 10: Images show before and after remeshing from the Impact Plate for each level. The blue dots represent the pooled nodes, and when Delaunay triangulation is applicd, the mesh quality improves.
C ADDITONAL DATASET DETAILS
Table 6 represents that our datasets are created with various solvers and different physies systems. Specifically, Sphere Simple involves self-contacts, while Deforming Plate, Deformable Plate, and Impact Plate do not exhibit self-contacts. In dynamics system, there simulation time step Al. The length of steps x is determined from the time at which the simulation ends.
Table represees that the input and output o; features defined for each domain. Node features f, contain physical properties. Node types n; correspond to boundary conditions. For Impact Plate, required properties such as density P: and Young's modulus Y, added node features for flexible dynamics. Young's modulus characterizes material's 's ability to stretch and deform, and is defined as the ratio of tensile stress tensile strain.
Mesh edges are defined by relative displacement vector Ujj and its norm |u:yl in mesh space and relative displacement vector X,y and its norm |x,y in world space, which are used for object propagation. For every node, we find all neighboring nodes within radius and connect with them via contact edges, excluding previously connected mesh edges. (Dx; x,1 < 7). the mesh space coordinates, the opponents between the nodes are the same in all steps in one trajectory. However, the position of the nodes represented by the world space coordinates has different value for each step. For example, in Sphere Simple, the world space containing the position of the node where the ball or cloth moves is expressed as 3D coordinates, and the cloth and ball without thickness have mesh space expressed 2D coordinates.
15

---

Published as a conference paper at ICLR 2024
Table 6: Dataset description: Simulators, system timeframe, etc. Self-contact indicates the case where different nodes of the same object collide with each other.
Datasets
At(ms)
Steps k Simulator Mesh Type Self-Contact
System
Dimension
Impact Plate
0.002
52
ANSYS
Triangles
X
Flexible dynamics
2D
Deforming Plate
None
400
COMSOL Tetrahedral
X
Statie
Sphere Simple
10
500
AreSim
Triangles
O
Rigid dynamics
3D
Deformable Plate
None
52
SOFA
Triangles
X
Static
2D
Table 7: Details of features for each dataset.
Datasets
Inputs m.j
Inputs Cig
Inputs
Outputs O2
Impact Plate
Uij:
Uiy Xij.
Xij
Xig:
Xig
ní. (x) x )) Pr, Y,
X1
Deforming Plate
Usjs
Miy Xijs
Xij
Xigs Xiq
n1;
X 0i
Sphere Simple
Uij:
Xijs
Xij
Xigs
Xig
Di, (xf x 1)
X;
Deformable Plate
Uij.
Uij
; Xij=
Xijl
Xig.
Xig
n;
X;
C.I IMPACT PLATE
The evaluation of Impact Plate follows industry-standard practices, primarily assessing the stiffiees of panels - these are commonly used in the display industry (Xue et al., 2013). Impact Plate involves strong impact in short timeframe, representing significant geometric and contact non- linearity in its dynamics. When transitioning from 2D to 3D modeling, the computational cost increases roughly 10-20 times. Therefore, if a ball falls at the center of the plate, there is no need to use a 3D model. Fig. [T depicts two visualizations before and after the ball impact. The most significant part is the transfer of the strong contact energy when the two objects collide. Once the falling height and the mass of ball are determined, the potential energy is calculated. At the moment of collision with the plate, the energy is converted to the internal energy of the panel, generating internal stress in the plate. If the stress value generated in the plate exceeds the ultimate strength of the material, the plate is damaged. Therefore, the final product design is determined through plate thickness and material configurations according to the required energy.
Impact Plate uses an explicit method that typically used to calculate short-term solutions. For example, Deforming Plate represents static system and uses implicit methods. Therefore, there is no need to predict the location of the obstacle (pusher) node because it is determined by the boundary conditions. For flexible dynamic systems, such Impact Plate where ball falls and collides with a place due to gravity, calculations involving the positions of the ball nodes and mesh deformations required. Therefore, in flexible dynamic systems, the location of the obstacle (ball) must be predicted.
This benchmark dataset incorporates various design parameters, including plate properties, thick- ness, density, modulus, drop height, mesh size, and etc. These parameters have been randomly varied generate dataset consisting of 2,000 trajectories, along with 200 validation and test tra- jectories. We use ANSYS (Stolarski et al. 2018) to create the dataset.
C.2 DEFORMING PLATE
Deforming Plate verifies the stress plate when pusher presses on the plate. We excluded two trajectories out of 1,200 training data trajectories. Those two trajectories are excluded because they contain stress singularity (Williams 1952). where excessive strain in single cell causes the stress to become nearly infinite. This exclusion is necessary as it interferes with the learning process. Fig. 12 represents the trajectory number 233 in Deforming Plate. The indicated arrows represent the point with excessive deformation. At the trajectory number 173, stress singularity occurred when the pusher pressed the thin plate, so we excluded it as well.
16

---

Published as a conference paper at ICLR 2024
(a) Before the ball drops
(b) After collision
Figure 11: After the collision, the energy from the ball is transferred to the place, resulting in the deformation of the plate and the corresponding stress distribution. The red area indicates areas of high stress.
(a) Before stress singularity occurs
(b) After stress singularity occurs
Figure Before and after stress singularity occurs due an excessive deformation of the elements. The red arrows indicate the nodes where stress singularity has occurred.
D BASELINE DETAILS
We compare our methodology with two competitive approaches. The first one is MGN (Pfaff et al. 2020 which performs multiple message passing. and the second one is GT (Dwivedi & Bresson
2020 which employs self-attention. We use the official implementation released by the authors on
GutHub for all baselines:
MGN: http:////gttttto.o/o////ggleeeee rree aateemmmster/nnehhaappeee
GT: http:://thh...ooooooo.eeeeeeeeer
MGN To align with the MGN methodology, apply 15 iterations of message passing all datasets. To learn collision phenomena, we added a contact edge encoder and processor, and directly incorporated stress values into the loss. All MLPs have hidden vector size of 128.
For an accurate comparison with HCMT, we use the FFN of the same HCMT without using positional encoding. The encoder/decoder were set to those in MGN. There are 15 transformer blocks with 4 heads. The hidden dimension size inside its Transformer is set to 128. FFN used three linear layers and two ReLU activations. To ensure numerical stability, the results obtained with the exponential term within the softmax function constrained to fall in the range of [-2 2].
GT
17

---

Published as a conference paper at ICLR 2024
E HYPERPARAMETER DETAILS
Table showws the hyperparameters in noise, radius, and the number of training steps applied to each dataset. The radius 7 an important hyperparameterr for collision detection.
Random-walk noises are added to positions in the same way as GNS (Sanceez-Gonzale) et al,, 2020) and MGN (Pfaff al., 2020) for improvements in cumulative errors. The numbers of contact propagation modules and mesh propagation modules are hyperparameters, and the number of blocks = Le + LH
15. Following MGN, the hidden vector size of the encoder/decoder 128 and the Adam optimizer is used. The hatch size is set to and exponential learning rate decay from 10-4 10-6 applied. The hidden vector dimensions dz, d2 for CMT and HMT are set to 128, and the number of heads H is 4. For reproducibility, we introduce the best hyperparameter configurations for each dataset in Table
to
Table 8: best hyperparameters of configures
Dataset
Lc
Lu
X
Noise Std. Dev.
# #Training Steps
Impact Plate
0.003
0.4
2,000,000
Deforming Plate
10
2
0,003
0.03
10,000,000
Sphere Simple
0.001
0.05
10,000,000
Deformable Plate
12
0.001
0.08
675,000
F COMPUTATIONAL EFFICIENCY
Table 9 shows Computational Efficiency. For Deforming Plate and Impact Plate, method show similar learning times to those of MGN. Our model has shorter training time on Sphere Simple compared MGN. In the case of Sphere Simple, self-contacts occur and there are many contact edges. In the case Deformable Plate, edge features are encoded at each level with a fixed number of small nodes and edges, which has negative impact on learning time for our method.
For Impact Plate, the inference time per step for HCMT is 81 ms/ssep. For ANSYS (i.e., numerical solver), it 14,280 ms/step. Compared to ANSYS, HCMT is about 176 times faster.
Table 9: Shows training time/step (mss) for each model.
Model
Impact Plate
Deforming Plate
Sphere Simple
Deformable Plate
GT
79.31
76.93
130.36
56.65
MGN
51.56
51.11
89.28
38.63
HCMT
51.12
53.53
59.02
53.16
18

---

Published as a conference paper at ICLR 2024
G GENERALIZATION ABILITY OF HCMT
HCMT can generalize outside of the training distribution with respect to the design parameters of the system (e.g., plate thickness and drop height of ball) since HCMT uses the relative displacements between nodes as edge features (Sanchez-Gonzale more general-purpose understanding of flexible dynamics used during training. We compare HCMT with MGN to evaluate their generalization abilities toward unseen design parameter distributions during training.
et al., 20200 Pfaff et al. (222)). This provides
We define 3 test datasets using our original Impact Plate dataset to study the generalization ability of HCMT: i) We call the test dataset where we increase the plate thickness from 1.0 1.2 in the range of 0.5 to 1.0 as "Thicker Impact Plate," ii) we denote the test dataset where we increase the drop height ball from 500 to 1000 in the range of 1000 to 1200 as "Higher Impact Plate," iii) we denote the test dataset with increased both plate thickness and ball drop height as "Thicker & Higher Impact Plate." " The 3 test datasets have 50 trajectories generated from untrained design parameters.
Table 10 shows the generalization abilities of MGN and HCMT in RMSE (rollout-all, x10)). The results of HCMT show lower RMSEs than MGN in all test datasets and show better generalization ability.
Table 10: The generalization ability of MGN and HCMT models according to various test datasets not included the training distribution is shown in terms of RMSE (rollout-all x x10)).
MGN
HCMT
Test Datasets
Position
Stress
Position
Stress
Impact Plate
44.18
28,928
20.34
14,447
Thicker Impact Plate
$4.50
25,037
17.38
12,163
Higher Impact Plate
31.23
35,552
29.59
22,509
Thicker & Higher Impact Plate
25.23
24,719
21.52
13,745
H EFFECT OF STRESS SINGULARITY ON LEARNING
In the process simulations using numerical solvers, singularity is a phenomenon in which the behavior of an object rapidly changes specific point large stress. It mainly occurs when the load is concentrated in a very small area. We evaluate how stress singularities affect the learning outcomes of MGN and HCMT. The experiments conducted with trajectories where stress singularities occur in Deforming Plate (see Appendix C.2)
within an object, resulting in very
Table 11 shows the results with and without stress singularities. In the of MGN, when stress aagutieiiiesssao are not included, its position and stress RMSEs decrease by 4.3% and 11.0%, respec- tively. HCMT's RMSEs decrease by 5.1% and 10.4% the same setting. Moreover, HCMT shows better performance than MGN in all cases.
Table 11: Results of whether stress singularity is included or not in Deforming Plate (RMSE, rollout- all, x10*).
With Singularity
Without Singularity
Model
Position
Stress
Position
Stress
MGN
7.98
5.085,048
7.64
4,526.531 4.475,616
HCMT
7.77
4.995.446
7.37
19

---

Published as a conference paper at ICLR 2024
1 ADDITIONAL ABLATION STUDY
I.1 CMT DUAL OR -RANCH
We modify the CMT's dual-branch Layer to single-branch configuration and conduct additional experiments to assess the performance impact of integrating edge mesh and contact edge features. The biggest difference between edge mesh and contact mesh is that contact mesh has no relative displacement in its mesh space.
Table 12 shows the difference between the dual-branch and single-branch configurations within HCMT. It shows that better performance is achieved when using the dual-branch configuration.
Table 12:
Ablation study results on whether to combine mesh and contact edges. The results of ablation studies (RMSE, rollout-all, x10%).
Impact Plate
Deformable Plate
Model
Position
Stress
Position
HMCT with CMT Dual-Brande
20.34
14,447 16,046
7.67
HMCT with CMT Single-Branch
20.30
8.05
1.2 PERFORMANCE IMPROVEMENT WITH REMESHING
Our proposed pooling method has a remeshing step. We perform additional ablation studies to evaluate the performance with and without remeshing. We proceed by setting the level of HCMT to 6. As shown in Table 13 we observe that, particularly for stress, the performance is better when the remeshing step is included.
Table 13: Ablation study on remeshing effect (RMSE, rollout-all, x 10%). A scaled Jacobian closer indicates a mesh with shape closer to an equilateral triangle, signifying higher mesh quality.
Impact Plate
Model
Jacobian
Position
Stress
HCMT without remeshing
0.32
19.21
17,482
HCMT with remeshing
0.51
20.34
14,447
I.3 CLAMPING FOR NUMERICAL STABILITY
The process of clamping, i.c., the clip function in the self-attention of CMT and HMT, is used after taking the exponent of the internal term of the softmax function for numerical stability (Dwivedi & Bresson 2020). We experiment to analyze the effect of clamping (see Equation [[ Equation 2 and Equation 5) Clamping ranges used for CMT and HMT are analyzed sequentially increasing from to 4. We proceed by setting the level of HCMT to 6. Table [4] shows the position and stress RMSEs according to each clamping range. The optimal setting is from -2 to 2.
Table 14: Ablation study numerical stability (RMSE, rollout-all, x x103).
Impact Plate
Clamping range
Position
Stress
-1 to 1
36.36
66.169
-2 to 2
20.34
14,447
to
37.11
58.792
4 4o
42.58
62.913
20

---

Published as a conference paper at ICLR 2024
J
SELF-ATTENTION MAPS
Figs
13
15 16 show the self-attention maps of CMT's dual-branch and HMT.
Contact CMT
(b) Mesh of CMT
(c) Level HMT
(d) Level of HMT
(e) Level 2 of HMT
(I) Level of HMT
g Level 4 HMT
(h) Level HMT
(i) Level of HMT
Figure 13: Self-attention maps of CMT and HMT on Impact Plate. HMT, the top-left represents edges connected to plate nodes, while the bottom-right represents edges connected to ball nodes.
21

---

Published as a conference paper at ICLR 2024
Contact of CMT
(b) Mesh of CMT
(c) Level 0 of HMT
(d) Level 1 of HMT
(c) Level 2 of HMT
(f) Level of HMT
(g) Level of HMT
Figure 14: Self-attention maps of CMT and HMT on Sphere Simple. The reason it exhibits different pattern compared to other domains is due to self-contacts. For example, when a cloth is folded, contacts occur among the cloth's nodes.
22

---

Published as a conference paper at ICLR 2024
(a) Contact of CMT
(b) Mesh of CMT
(c) Level 0 of HMT
(d) Level HMT
(c) Level of HMT
Figure 15: Self-attention maps of CMT and HMT on Deforming Plate. the case of CMT, there is slightly broader area of significance compared to Impact Plate because more of the obstacle's 's surface involved contact. Map of HMT shows an increasing number of meaningful connections as the level increases. The lop-left represents connected mesh edges of the obstacle, while the bottom-right represents connected mesh edges of the plate
23

---

Published as a conference paper at ICLR 2024
(a) Contact of CMT
(b) Mesh of CMT
(c) Level 0 of HMT
2012
(d) Level of HMT
Figure 16: Self-attention maps of CMT and HMT Deformable Plate. Figure shows extensive feature maps in Deformable Plate, with more pronounced deformations the plate than in the ball. The top-left represents edges connected ball nodes, while the bottom-right represents edges connected to plate nodes.
24

---

Published as a conference paper at ICLR 2024
K
RESULT DETAILS
The results each table are the rollout-all RMSE results for MGN, GT, and HCMT from five different seeds.
Table 15: MGN RMSE-all x x10
Deformable Plate
Sphere Simple
Deforming Plate
Impact Plate
Seed
Position
Position
Position
Stress
Position
Stress
10
11.32
32.88
7.64
4,526,531
44.18
28,928
20
10.48
27.46
7.93
4,766,444
43.50
27,323
30
10.63
44.39
7.86
4,618,953
38.63
58,179
40
11.44
26.89
7.66
4,574,221
36.36
38,165
50
10.01
34.70
8.05
4,736,266
40.97
26,760
mean
10.78
33,26
7.83
4,644,483
40.73
35,871
std
0.54
6.33
0.16
92,520
2.94
11,893
Table 16: GT RMSE-all x10
Deformable Plate
Sphere Simple
Deforming Plate
Impact Plate
Seed
Position
Position
Position
Stress
Position
Stress
10
14.13
371.79
11.21
9,014.384
61.05
26,804
20
13.05
189.89
11.44
9,332,153
60.23
25.328
30
14.31
134.32
11.83
9,403,657
64.97
28,948
40
13.82
77.75
11.18
9,043,427
51.44
33,372
50
13.38
445.48
11.04
9,047,868
58.22
82,005
mean std
13.74
243.85
11.34
9,168,298
59.18
39,291
0.47
141.08
0.28
164,941
4.45
21,529
Table 17: HCMT RMSE-all x10
Deformable Plate
Sphere Simple
Deforming Plate
Impact Plate
Seed
Position
Position
Position
Stress
Position
Stress
10
7.67
28.30
7.37
4,475,616
20.34
14,447
20
8.05
30.95
7.56
4,609.359
21.70
15,685
30
7.35
30.06
7.53
4,489,145
20.92
14,257
40
8.18
29.40
7.56
4,534,407
20.11
14,524
50
7.07
33.35
7.44
4,571,253
20.45
14,798
mean
7.67
30.41
7.49
4,535.956
20.71
14,742
std
0.42
1.71
0.07
49,937
0.57
502
25

---

Published as a conference paper at ICLR 2024
Table 18: Detailed measurements
Metries
Dalasels
GT
MGN
HCMT
Deformable Plate
Position
0.886_0.006
0.77910011
0.724:10.066
Sphere Simple
Position
0.071=0:006
0.076100004
0.121
RMSE-1 x10
Position
0.178-0.009
0.129100007
20.08
Deforming Plate
Stress
2.919,024
1.661,029.330..15
1,161,493
Position
0.031
0.029:10.02
0.023
Impact Plate
Stress
676.18
6861:12
682.133
Deformable Plate
Position
13.74:0047
10.7840.54
7.67
Sphere Simple
Position
143.85
33.26:1633
30.41::1.1
RMSF-all x x
Position Stress
11.34:10.28
7.83=10.16
7.49
Deforming Plate
9.168,298=164pp0
464.483a92,520
4,535,956:4:99557
Position Stress
59.18:14.45
40.73:a:294
20.71
Impact Plate
39.291 4:21.599
35,871=11.899
14.7424500
Deformablc Plate
$5.65
38.63
53.16
Training time/step (ms)
Sphere Simple
130.36
89.28
$9.02
Deforming Plate
76.93
51.11
53.53
Impact Plate
79.31
51.56
51.12
26

---

Published as a conference paper at ICLR 2024
L
OTHER VARIABLE CONTOUR AND ROLLOUT IMAGES
Figs. 18
20 are rollout images of Impact Plate, Deforming Plate, Sphere Simple, and De-
formable Plate, over time or step flow.
Ground Truth
HCMT
MGN
GT
model- predicted rollouts compared to the ground truth at Figure 17: The stress contours of various model Impact Plate.
Ground Truth
HCMT
MGN
GT
Figure 18: The stress contours of various model-predicted rollouts compared to the ground truth at Deforming Plate.
27

---

Published as a conference paper at ICLR 2024
Ground Truth
HCMT
MGN
GT
Figure 19: The image of various model-predicted rollouts compared to the ground truth at Sphere Simple.
Ground Truth
HCMT
MGN
GT
Figure 20: The image of various model-predicted rollouts compared to the ground truth at De- formable Plate.
28


Reviews:
[{"review_id": "7Dr6Wb40mW", "reviewer": "Reviewer_CgKu", "summary": "This manuscript presents a novel mesh-based network architecture, Hierarchical Contact Mesh Transformer, that focuses on flexible body dynamics, which can have large amount of non-linear effects (e.g., collision) in a short time. Using a specially designed mesh pooling layer and a hierarchical architecture, HCMT can do long-range interaction directly and propagate information efficiently. The authors compared HCMT with other baselines using previous datasets and showed that HCMT outperforms them.", "questions": "I wonder if this method can enable any downstream tasks that utilize the differentiability of HCMT.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well written. The presentation and visualization make it an easy one to understand.\n2. I personally appreciate the section 3.4 which clarified some confusion when I read there.\n3. I appreciate the open-sourced code attached in the manuscript. I believe it will ease the effort when practitioners reproducing this work.\n4. The results and comparison to previous methods look impressive and detailed.\n5. The ablation study covered a lot of interesting experiments.", "weaknesses": "See questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nSAuEcVnZf", "reviewer": "Reviewer_J9Rp", "summary": "In this paper, the authors introduce the Hierarchical Contact Mesh Transformer (HCMT), a model leveraging hierarchical mesh frameworks to capture long-range dependencies resulting from collisions across spatially distant positions on a body. By using a contact mesh Transformer and a hierarchical mesh Transformer (CMT and HMT) , the HCMT has the ability to quickly propagates collision effects to long range positions. The experiments show that the proposed model can outperform the existing methods on benchmark datasets.", "questions": "- As for the baselines, are there any reasons that only MeshGraphNet (MGN) and Graph Transformer (GT) are selected for comparison?\n- How is generalization ability of the proposed models when applied to similar problems? Does it require training from scratch?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The contact and collision are important and usually challenging problems in physical simulations. The paper aims to address this challenge using transformer for the first time (to my best knowledge). The two transformers architecture i.e., contact mesh Transformer (CMT) for propagating contact messages and hierarchical mesh Transformer (HMT) for handling long-range interactions is intuitive. \n- The experiments show that the proposed method can better capture the long-range interaction collision on the test cases provided in both RMSE and visualized results comparing to baselines.", "weaknesses": "- The proposed method is only compared with two baseline methods from 2020. There are more recent methods mentioned in the related work of the paper, especially for existing work which some components (e.g. pooling method) of the proposed method are inspired by, are not taken for comparison. \n- There are no generalization ability studies for the proposed models. It is unclear how the trained models can be applied to unseen/similar problems without or with minor finetuning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7x10VpWLs1", "reviewer": "Reviewer_zh4D", "summary": "This paper presents a transformer-based neural network model for learning collision/contact responses in flexible dynamics. The network consists of two transformers capturing the mesh and contact dynamics, respectively. The results are evaluated against standard neural network baselines including MeshGraphNet.", "questions": "**Questions about flexible dynamics**\n- Table 1: Are M, D, T constant matrices or do they have a dependency on x (esp. the stiffness matrix) and/or x_dot (esp. the damping matrix)?\n- Table 6: How long did it take ANSYS/COMSOL/etc to simulate these scenes?\n\n**Questions about the mesh edges**\n- Is the rest shape information stored as edge features?\n\n**Questions about the contact edge set**\n- As the contact is detected on edges/nodes, it looks like the method cannot handle vertex-face collisions. I am worried that this may lead to artifacts in 3D examples.\n- I am trying to understand when this set C is computed. Is it dynamically computed for the mesh M^t at every time step, and different time steps may have different C sets? On a related note, could you clarify in what sense C is “fixed” in the description “...C to denote the fixed set of additional contact edges”?\n\n**Questions about node sampling**\n- Resampling nodes and re-meshing via Delaunay triangulation seem to be discrete operators. How did backpropagation work via these non-differentiable operators when training the network using Eqn. (8)?\n\n**Questions for node features**\n- Isn’t Poisson’s ratio part of the node feature?\n- Physical parameters like density and Young’s modulus are stored as node features in this method. Did any examples in your paper use this flexible representation to model spatially varying materials?\n\n**Questions about the experiments**\n- C.1 Impact Plate: Is it a 2D or 3D experiment? The setup seems to be a (symmetric) 3D scene, but the text “Therefore, if a ball falls at the center of the plate, there is no need to use a 3D model” seems to imply that it is solved as a 2D example in the approach.\n- C1 Impact Plate: “along with 200 validation and test datasets.” Is “datasets” a typo for “trajectories”?\n- Table 6: 0.002ms is extremely small. The table indicates that the proposed method is tested on scenes with time steps varying among three orders of magnitude (0.002 vs 10)...Just want to make sure I read it right.\n\n**Minor questions**\n- Table 1 caption: should the stress notation be replaced with \\sigma so that it is consistent with that in the table?\n- The paper repetitively draws comparisons with fluid problems, e.g., EAGLE in Sec. 3.4 and “Notably, the contact edge feature is a novel addition not found in fluid models.” But I don’t get the motivation for such comparisons. The two physical systems are quite different in many ways.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "I’ll admit first that I am not an expert in “flexible dynamics”. Although I recognize the governing equations in Table 1, the concept of “flexible dynamics,” which seems to be rooted in the engineering community, is quite new to me. Overall, I am on the fence but with very low confidence due to this reason.\n\nAt a high level, I think capturing the long-distance collision response with a hierarchical network architecture is an interesting and reasonable idea. While hierarchical networks have been applied in other physical/numerical systems as reviewed in the related work, using them to capture contact/collision seems novel to me.", "weaknesses": "I will briefly summarize my concerns here and elaborate on them in the “Questions” section below.\n- A few technical decisions in the methodology and their implications are unclear to me. See my questions below.\n- I feel that implementing the proposed model seems quite involved since it requires node resampling and remeshing with Delaunay triangulation.\n- I also think the experiments can be improved. I didn’t find too much discussion about its comparisons with the numerical simulators (ANSYS, COMSOL, etc.) or about the method’s generalizability, e.g., to unseen meshes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xCw3VXA8fn", "reviewer": "Reviewer_gSaT", "summary": "In this paper, a novel dual graph transformer architecture is proposed for flexible body collisions. The underlying architecture is built by combining a traditional Collision Mesh Transformer (CMT) with a novel hierarchical dual-branch attention Hierarchical Mesh Transformer (HMT) to enable long-distance interactions that are important for collision dynamics. The paper evaluates their contributions on four datasets (representing different classes of interaction dynamics), showing how the proposed method outperforms existing Graph Transformer (GT) and MeshGraphNet (MGN) architectures in all cases. The authors furthermore perform some ablation studies for some aspects of their methodology on the same datasets. Finally, the authors conclude by pointing out some limitations addressable in future work, e.g., including learnable pooling methods.", "questions": "- Most importantly, how different is the proposed HCMT architecture from existing hierarchical models? Is the novelty just in using contact edges, using a transformer, or something else?\n- From the descriptions, it is unclear how contact edges are computed, i.e., are they based on a radius search? And if so, what search radii are used? What influence does this have on the achieved results, and would using a non-hierarchical transformer with a similar search radius perform well? Also, how large is the used radius in relation to the mesh elements? The appendix makes some mention of a parameter gamma (Table 8), but this parameter seems to vary significantly between datasets, but there is no clear argument as to why this is the case either.\n- A significant concern with the dataset is the apparent exclusion of two trajectories where stress singularities occur. These seem to exist in the solver used for the data generation, so shouldn't a neural network that learns a simulator's behavior be expected to also lead to the same results? Picking and Choosing from datasets is not ideal, and the authors should include a comparison of the different methods with and without these singularities. If the proposed method works best with and without these singularities, then they should be included; if the proposed method works worse than existing methods if they are included, then this should be discussed more straightforwardly.\n\n- Figure 2 and Figure 4 seem to conflict with each other. Figure 2 shows how the interactions appear to be computed simultaneously for different distances, whereas Figure 4 shows how the different levels of the hierarchy are computed one after another. Which one is used and why?\n- The authors mention that they 'clamp' values for numerical stability, which seems important but is not further discussed. Similarly, the authors mention that they use a pre-layer norm, denoted by a hat, but (4) uses another layer norm after applying the transformer?\n- Are the pooling operations performed globally or only per body? The figures seem to indicate the latter, but the text is unclear.\n- Table 3 raises some concerns regarding the reproducibility of the results. First of all, it is unclear how many seeds are used for all evaluations and how the standard deviations are computed, e.g., are they computed within an evaluation dataset or across different seeds? Furthermore, for example, Table 2 shows a position and stress RMSE for the deforming plate of 7.49 pm 0.07 and 4,535,956 pm 49,937, respectively. Whereas Table 3 shows a position RMSE of 7.37, which is 0.12 away from Table 2, and a stress RMSE of 4,475,616, which is 60,340 away from Table 3. Both results, thus, are outside of the given standard deviation. Why is this the case?\n- Why does the error increase significantly with more levels for the deformable plate scenario? The authors note that a decrease in levels decreases the RMSE, but this does not really seem sufficient as not having any levels would mean that there is no need for long-range interactions, which should also be possible to do using MGN, but MGN performs worse than even the longest-range HCMT network.\n\n============\n\nI want to thank the authors for the detailed information and the updates. Overall, I'm still leaning towards the positive side. I would like to slightly update my score, but given the discrete steps I will stick to the current one and let the AC know about my \"slight raise\".", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The originality of the paper in its application domain, i.e., regarding flexible collision dynamics, seems strong as this area of physical simulations is not really explored so far using Neural Networks. However, the originality of the proposed method itself, i.e., the use of two graph transformers, is not as strong.\n\nI believe the main strengths of the contribution of the paper are demonstrated by Tables 2 and 9 as well as Figure 7. The results show a clear benefit to using the proposed transformer-based architecture on the chosen datasets. Furthermore, the authors provide a new dataset useful for future research.\n\nThe writing of the overall paper is good, and the explanations are mostly straightforward to understand from a linguistic standpoint, but not several questions arise when reading the paper.\n\nRegarding Significance: On one hand, the authors state that the proposed network could be useful in product design (as stated in the abstract and conclusions), which is seemingly a positive aspect, but this is difficult to verify from the submission. On the other hand, the usage of two separate graph transformers with different architectures, i.e., non-hierarchical and hierarchical, seems an interesting point that could be explored in other applications areas as well.", "weaknesses": "The overall approach of the paper seems technically sound, but there are several points in the paper that raise questions that should be addressed. Furthermore, the core contribution of the paper is not really clear enough, as it is not clear how novel the use of the double transformer architecture is. The authors point out that several hierarchical approaches already exist in the literature (in the related work section), but it is not clear how different they are from the proposed architecture. Furthermore, the datasets are not easy to understand just by looking at the provided information, i.e., what is actually happening in the impact plate scenario? The following is a list of concerns, weak points and linguistic problems in the paper that the authors could take to heart and clarify if possible (not listed in order of severity):\n- In the abstract, the authors mention that their proposed 'unique flexible body dynamics dataset'  is commonly used for product design. It is not clear how such a dataset is useful for product design, and the use of the word 'commonly' combined with a 'unique dataset' seems contradictory. The authors should clarify how this dataset is useful.\n- The authors claim that GNNs are capable of 'intelligently distributing computation to areas of interest', but this seems somewhat dubious as a claim without any reference to support the argument.\n- The authors claim that classic GNNs 'cannot quickly propagate the influence of collisions over long distances' and justify this by a requirement of needing multiple GNN steps to propagate information. However, hierarchical mesh transformers, as pointed out by the authors in the related work section, are clearly capable of addressing such issues already. Furthermore, the proposed architecture also requires multiple propagation steps.\n- It is unclear how Figure 2 is different from existing hierarchical transformers\n- What is 'N' when describing the computational complexity of Transformers?\n- In the related work section, the authors describe that GNNs are 'expected to be applicable to all systems', but this is clearly not the case, as the authors already demonstrate in their results. This should be clarified.\n- The authors state that slow propagation leads to unnatural dynamics, but this does not seem obvious. If propagation is slow, couldn't one add more layers until the same receptive fields and propagation distances are achieved?\n- What is a \"contact edge\", and how are they computed? The explanations in Section 3.2 are very sparse, and this seems to be a central point of the paper. Also, do they include self-interactions or not?\n- The authors mention 'the concatenated features mentioned earlier', but the discussion on features is moved to the appendix. This should be streamlined.\n- What is the meaning of the double hat notation in (3)? This is made more confusing by the authors using the hat for a pre-layer norm and later using a dot + hat combination in the decoder and updater paragraph.\n- In (1) and (2), the authors should make use of \\left( and \\right) in latex.\n- The authors mention that in 'fluid dynamics, node, and edge lengths are always fixed', which is not true in Lagrangian simulations, e.g., in the CConv approach by Ummenhofer et al.\n- The authors note that Figure 6 'shows that as the level increases, shortcut message passing is possible in our method', but this is not really clear from the figure, e.g., is this related to intra or inter-body messages or both?\n- The self-attention maps shown in the main paper and appendix do not provide many insights into the learned behavior and are not very intuitive to understand.\n- Figure 7 should include colormaps.\n- The use of a scaling factor for the results in Table 2 is not ideal and very confusing to read right next to unscaled results in Figure 9.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)