EGODE: An Event-attended Graph ODE Framework for Modeling Rigid Dynamics
Authors: Jingyang Yuan
Gongbo Sun
Zhiping Xiao
Hang Zhou
Xiao Luo
Junyu Luo
Yusheng Zhao
Wei Ju
Ming Zhang
Venue: NeurIPS 2024 poster (2024)
Keywords: Keywords: Interacting Dynamical System
Rigid Simulation
Neural ODE
Graph Neural Network
Primary area: machine_learning_for_physical_sciences

Abstract: This paper studies the problem of rigid dynamics modeling, which has a wide range of applications in robotics, graphics, and mechanical design. The problem is partly solved by graph neural network (GNN) simulators. However, these approaches cannot effectively handle the relationship between intrinsic continuity and instantaneous changes in rigid dynamics. Moreover, they usually neglect hierarchical structures across mesh nodes and objects in systems. In this paper, we propose a novel approach named Event-attend Graph ODE (EGODE) for effective rigid dynamics modeling. In particular, we describe the rigid system using both mesh node representations and object representations. To model continuous dynamics across hierarchical structures, we use a coupled graph ODE framework for the evolution of both types of representations over a long period. In addition, to capture instantaneous changes during the collision, we introduce an event module, which can effectively estimate the occurrence of the collision and update the states of both mesh node and object representations during evolution. Extensive experiments on a range of benchmark datasets validate the superiority of the proposed EGODE compared to various state-of-the-art baselines. The source code can be found at https://github.com/yuanjypku/EGODE.

EGODE: An Event-attended Graph ODE Framework for Modeling Rigid Dynamics
Jingyang Yuan'
Gongbo Sun? Zhiping Xi Hang Zhou'
Xiao Luo
Junyu Luo'
Yusheng Zhao' Wei Ju Ming Zhang!
SScooo of Computer Science State Key Laboratory for Multimedia
Information Processing, PKU-Anker LLM Lab, Peking University University of Wisconsin-Maddion University of Washington "University of Califormia, Davis UUniversity of California, Los Angeles
yuanjyēpku. edu. en, gsun43ewisc. edu, patxiaoluw edu, hgzhou@ucdaviss edu, xiaoluoθcs ucla. edu, luoj unnyuusus pku. edw. cn, yusheng zhao@stu, pku. edu. cn, juwei@pku cn, mzhang_cs@pkr edu.
Abstract
This paper studies the problem of rigid dynamics modeling, which has wide range of applications in robotics, graphics, and mechanical design. The problem is partly solved by graph neural network (GNN) simulators. However, these approaches cannot effectively handle the relationship between intrinsic continuity and instantaneous changes in rigid dynamics. Moreover, they usually neglect hierarchical structures across mesh nodes and objects in systems. In this paper, we propose novel approach named Event-attent Graph ODE (EGODE) for effective rigid dynamics modeling. In particular, we describe the rigid system using both mesh node representations and object representations. To model continuous dynamics across hierarchical structures, we use coupled graph ODE framework for the evolution of both types of representations over long period. In addition, to capture instantaneous changes during the collision, we introduce event module, which can effectively estimate the occurrence the collision and update the states of both mesh node and object representations during evolution. Extensive experiments on range benchmark datasets validate the superiority of the proposed EGODE compared to various state state-of- th- baselines. The source code can be found at hhttps: //tgh/ubttt.tt///// com/yuanjypkuuEEO00
Introduction
Physics simulations
50
can benefit researchers from many fields by guiding experiments and 558 Among them, simulating rigid collisions has received extensive attention
59
testing their theories
with applications in robotics [19 and graphics [3]. However, high-quality physical simulations usually require complicated computing, which requires extensive computational resources. To solve this issue, data-driven approaches [1 [7 that aim to leverage machine learning for efficient simulators are becoming increasingly popular within the recent years.
In literature, variety of existing approaches have been proposed to model physical systems 20 52 47 2 29) Early attempts usually focus on simulations on regular grids (57 43 40 and
use convolutional neural networks (CNNs). However, in real-world scenarios, objects are rarely located regular ways
55 39 To increase the applicability, many current works use irregular mesh points [47 3 to describe objects in physical systems and utilize graph neural networks capture the interactions between mesh points. In particular, they adopt
(GNNs)
[6
28 38
*Corresponing authors.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

time
time
Ground Truth
EGODE
SGNN
EGNN
Figure 1:
Visualizations of predictions on Physion dataset. EGODE demonstrate the best capability to generate accurate trajectories across diverse scenarios.
architecture, which first maps observations of each mesh point at the current time step into the latent space, and then follows the message-passing mechanism [2X] to update node representations iteratively. Finally, decoder is utilized to generate the predicted trajectories.
an
suffer from three basic obstacles, which could seriously degrade the forecasting performance. Firstly, the majority of existing ap- proaches 12 47 utilize rollout process to model dynamical systems, which take the predictions at the next time as the input an autoregressive process. The discrete rollout process makes difficult to capture the long-tern tendency and continuous evolution of complex physical systems. Secondly, aside from the continuous evolution, rigid dynamics would face instantancous changes 20 54] caused by contact dynamics at certain time whereas the existing approaches often fail to take into consideration. Therefore, we are required build model that can precisely capture the relationships between the dominant continuous evolution and instantaneous changes in complicated rigid-body systems. Thirdly, since each object described by multiple mesh nodes, rigid-body systems intrinsically consist of hierarchical structures across mesh nodes and objects, which increases the difficulty of modeling rigid dynamics.
Despite their notable process, these approaches 20
52 47
To address the aforementioned obstacles, in this paper, we propose a novel framework named Event- attend Graph ODE (EGODE) for modeling rigid dynamics. The core idea of our EGODE to understand the continuous evolution and instantaneous changes in rigid-body systems. To model the hierarchical structures, we introduce both mesh node representations and object representations. To model the continuous dynamics, we adopt its neighbor mesh nodes and the related object to drive the evolution of mesh node representations. Meanwhile, global object representations and the summarized local information jointly determine the evolution of objects. To model the instantaneous changes, we introduce an event module, which estimates the next time when the collision occurs, and then updates both the mesh node representations and the object representations in iterative manner. Finally, we minimize the standard mean square error (MSE) at both node and object levels. We conduct extensive experiments on range of benchmark datasets. A comparison of our EGODE with other baselines on the Physion dataset is depicted in Figure [ The experimental results can validate the superiority of the proposed EGODE over wide range of competing basclines.
In summary, the contributions of the paper are three-fold: (1) We provide new perspective of modeling both continuous evolution and instantaneous changes to study rigid dynamics. To the best of our knowledge, we make the first attempt using graph ODE to simulate rigid-body systems. (2) Our EGODE not only utilizes coupled graph ODE to jointly model the continuous evolution of both mesh nodes' representations and objects' representations, but also introduces an event module to estimate the collision times for instantaneous updating. (3) Comprehensive experiments including

---

quantitative comparison and visualization on different benchmark datasets demonstrate the superiority of our proposed EGODE over wide range competing approaches.
2 Related Work
Data-driven Physical Simulation. To facilitate physical simulations in different areas, wide range of researchers leverage machine learning to build effective data-driven simulators 44 54 52
Early attempts usually adopt convolutional neural networks (CNNs) to model physical systems with grid structures [46). To increase the flexibility of simulators, recent efforts have focused on building simulators on irregular grids 44 20 52 which usually leverage graph neural networks (GNNs) [8 to model the interaction between objects. For example, MeshGraphNel [47] adopts an encoder-processorr processor-decoder architecture to predict the next states for effective mesh- buish-baedddee simulations. EGNN 20 considers the subequivariance of physical systems during the message passing process. GNS
52 have validated the potential of graph neural networks for modeling rigid dynamics. However, these approaches cannot handle intrinsic continuity and discontinuity in rigid models while our EGODE is the first work to introduce graph ODE for effective rigid dynamical modeling.
Graph Neural Networks. Graph neural networks (GNNs) [2
26 25
have been shown efficient
in wide range of vision tasks including cross-modal learning 66 600, object detection
[33 56)
and transfer learning [15] 17 66, [3| (35]. These approaches usually follow the paradigm of mes- sage passing [65) which updates the central nodes by aggregating their neighborhood information
iteratively. Through this process, GNNs can learn from geometric structures for downstream tasks. By combining GNNs with neural ODEs (9) range of continuous GNNs 48
64
49
61 have
been developed, which model the neighborhood aggregation in a continuous way. For example, GDERee [4] combines neural ODE with an attention-ban
GNN to model the interaction signals in recommender systems. However, these approaches usually neglect the instantaneous change in interacting dynamical systems [9. To handle this, we propose new continuous GNN framework named EGODE, which can model the instantaneous updating in rigid-body systems.
Neural Ordinary Differential Equations (ODE). Compared with classic deep neural networks, neural ODEs 9 aim to include continuous layers rather than discrete ones with extensive applica- 8 The updating rule of neural ODE is accelerated by incorporating adjoint functions with neural ODE solvers [12]. Recently, range of approaches [1O] [4, 41 have been proposed to improve the effectiveness of neural ODE, including augmenting the dimension [TO]
tions 4
and regularization terms 14
Neural ODEs have been adopted to model multi-agent dynamical which can deal with irregularly sampled data and partial observations. In
4].
systems 23 24 37
36
this paper, we propose novel neural ODE framework EGODE, which can model both instantaneous updating and continuous evolution in rigid-body systems.
3 Methodology
3.1 Problem Definition
We assume that a rigid-body physical system consists of M objects with N mesh points. The state information of each mesh node i, S S N, includes the observation vectors (i.e., the position vector æ and the velocity vector of at time 1) and static vectors 8; (e.g., friction of the floor) unrelated to the geometric context. The graph structure is constructed based on positions of mesh nodes, i.e.. g = (V. E'} where V collects all the mesh points and E' consists of all the edges at the time step Following previous works [20], we build an edge when the distance between two mesh points is below given threshold. making up the edge set E", Given the initial states GO, we aim to predict the future trajectories X1f where X denotes the position matrix at the time step l.
3.2 Framework Overview
In this work, we study the problem of modeling rigid-body physical systems, which is challenging due to the hierarchical evolution of systems and instantaneous changes from collisions. Towards this end, we introduce new framework named EGODE, which models the evolution of physical systems in continuous manner with the consideration of instantaneous events. In particular, EGODE first drives the dynamics of mesh nodes using both its surrounding mesh nodes and the associated object.
3

---

ODE Solver
Event
Mesh Nodes r
Coupled ODE
dr
Mesh Feature Differential
Objects P
dF
Object Feature Differential
Event Detector
Pi
ri
ii
Ti
T
Event Detected 1
F
F
72
F
Figure 2: Overview of EGODE. We employ coupled graph ODE framework with an event module for collision modeling. The coupled graph ODE structure naturally captures the continuous dynamics inherent in interacting systems. rf and Ft, represents feature of mesh node and object node at time I, respectively. Complementarily, the event module designed to effectively handle potential instantaneous changes, such as those arising from collisions.
Moreover, the object-level dynamics driven by local node information and global system states. To effectively capture the instantaneous collision, we introduce a learnable event module, which detects the potential collision time and updates the instantaneous change iteratively during ODE evolution. An overview of the proposed EGODE can be found in Figure 2 and the details are introduced below.
3.3 Coupled Graph ODE
usually adopt graph neural networks (GNNs) [6 to predict the states of objects at the next step, followed by autoregressive iterations for long-term predictions. However, these approaches cannot capture the long-term tendency and continuous evolution in physical sys- tems [23]. Towards this time, we introduce a graph ODE framework, which can capture continuous dynamics in interacting systems naturally. Moreover, since have both objects and mesh nodes in rigid-body systems, our ODE framework consists of a coupled architecture, which models hierarchical structures in unified way 24 68 20]
Previous works 20 52
In particular, we first initialize each latent state using the static vector, and then concatenate all the dynamical vectors with it for each node into new vector r as follows:
rf = vf h
where h denotes the corresponding hidden state with hς s;- To model the continuous evolution, we introduce a neural graph ODE framework by combining neural ODE [9] with GNNs. Our graph ODE drives the dynamics of the system using the interaction between each node and its neighbors. Formally, we have:
dr! = =
Σ ('(rt,r)))
JEN'(0)
where (1) collects the neighbors of mesh node at the time step t. (..-) aims capture the interaction between each object and its neighbors and (-) produces the summarized influence from neighborhood to drive the evolution the system. Moreover, in rigid-body systems, there are naturally hierarchical structures ranging between mesh nodes and objects. To model the hierarchy effectively, we introduce the states at the object level, by calculating the average of their corresponding observation vectors. In formulation, the object-level vectors be initialized as follows:
Σ
(3)
8: 0:0(1)
boo()==

---

80 i: (i) 5l
Σ
where oit) returns the object j corresponding to the mesh point i. We also utilize h to denote the latent object representation of each object and have Fr oz [ By h which is then incorporated into the evolution of all its corresponding nodes. In other words, we we-te Equation into:
dr! dt
d'(l
Σ (rrr, ('EN(())
where F2, can provide high-level semantics for dynamics modeling. To obtain F. we include another graph ODE drive the evolution the object level. Here, not only connect each object with all the other objects in the system for global understanding, but also learn from its corresponding mesh node for local information. In formulation, we have:
drf
M
Σ 1)) :=o())=
c )
(6)
dt
where (-) and (.,(),,) :) are two learnable functions to object-level updating with different parameters, M the number of objects. the right hand, the first term calculates the interaction between different objects and the second term summarizes the states of its associated node presentations. In the end, we combine both Equation 5 and Equation to jointly solve the coupled ODE, which can not only model the continuous evolution in physical systems, but also output the trajectory at any time step. The whole coupled ODE can be solved by traditional neural ODE solver [9]
3.4 Event Module for Collision Modeling
We have introduced a graph ODE framework to model the continuous evolution in physical systems. However, rigid-body dynamical systems 20. 2 could include instantaneous change during the collision between objects. In this case, our coupled graph ODE could be incapable of sufficiently modeling these discontinuous systems. To tackle this, we include learnable event module to estimate the time of potential collision, which can guide the adjustment to the states of different mesh points and objects 8 47).
One basic solution to model the event (i.e., collision) occurrence is to utilize prior knowledge (e.g., shapes of objects) well position information, which could be unavailable in real-world applications. As consequence, to make our data-driven model more generalized, we utilize a learnable event function condition on the pairwise states of mesh points, which can be formalized as gtt, [ot] vf, o of.). )) This event function is capable of continuously detecting the time when the collision between mesh nodes and i' occurs using the following equation:
(t,, [a] of. axe 1 0.
Note that event fucntion is only calculated between point-pairs whose distance is within a threshold to avoid square complexity. After solving Equation 7 Jusing the neural ODE solver, we can obtain the collision time t". Note that in rigid-body systems, the collisions of mesh nodes from different objects would bring in instantaneous change on all the mesh nodes in their related objects. Here, we update the states of observations using the current states and the object that it collides with. In formulation, the vector after the collision can be written as:
r tot = 6' Σ
(r (""")""))" Fg..
(8)
SEC
Here, Ct* (i) collects all the mesh nodes belonging to the object that it collides with, and d'*(+) and (.) are two new learnable functions for instantaneous updating. Through this, we involve an immediate updating at the time step t" from r to 1++ which can simulate the collision between different objects. Similarly, can update the state of each object as:
gto = 35 [11300(1'
Σ 1),,
(9)
8:000 = (0o()==
5

---

Algorithm Updating Algorithm of EGODE
Require: Observation data, Future ground truth
Ensure: Graph ODE framework;
1: Initialize the vectors rf and r
2: repeat
3: Forwarding our coupled graph ODE Equation SanttiionnSaan 5| Equation
4: Solve Equation to estimate the collision time during the iterative computation of the ODE;
5: Update both node representations and object representations using Equation andd Equation 9
6: until f > T
7: Calculate the loss objective in Equation
8: Update the parameters of graph ODE framework;
where j' denotes the object to have the collision with j. and 5i9* and (h9* are for object-level instantaneous updating. The first term calculates the collision between two objects and the second term models the average of updated mesh node representations. Finally, the whole event-attended graph ODE can solved by iteratively calculating the next collision using graph ODE and updating the corresponding state with Equation X and Equation 9 In this way, integrate the instantaneous updating into the ODE-based continuous evolution to model the rigid dynamics.
3.5 Training Objective
To optimize our graph ODE framework, we first output the observation at different time steps and then minimize the standard mean square error (MSE) loss between the predicted trajectories X and the ground truth X':
C'
Σ
-
X
(10)
Ta-1
Moreover, minimize the MSE loss at the object level as:
T
L
Σ
x
(11)
t= T=
where X is the predicted object-level matrix and X! denotes the ground truth. Finally, we combine both Equation 10 and Equation
as:
C= XC9,
(12)
where À is parameter to balance the losses. The whole updating algorithm can be summarized in Algorithm
4 Experiments
4.1 Experimental Settings
Datasets. Our proposed model EGODE is evaluated two physical dynamics datasets, i.e., Rigid- Fall (30) and Physion [6]. RigidFall simulates collisions and interactions between three rigid cubes during falling under varying gravitational acceleration. Physion, a large-scale dataset and bench- mark for physical system interaction evaluation, models both rigid and soft-body collisions for distinct scenarios, which are Dominoes, Contain, Collide, Drop. Roll, Link, Support, and Drape. Each scenario consists of 2000 training trajectories and 150 testing simulations. The two datasets both treat objects as assemblies of particles.
Baselines and metrics. We compare the performance EGODE with a range classical methods and state-of-art methods including MLP, RNN, SocialODE 62). GNS [52]. DPI-Net (30], EGNN (53), GMN (22). SGNN (20). and SEGNO [32]. We utilize two evaluation metrics: iee. Contact prediction accuracy and Mean Square Error (MSE).
Implementation Details. We implement our baseline models using Pytorch 45 and torchdiffeq 27 We adopt the same hyper-parameters and training strategy for both Physion and RigidFall datasets as

---

Table 1: Results of compared methods on Physion (Accuracy%) Bold numbers highlight the best performance and t represents the standard deviation.
Methods
Dominoes
Contain
Link
Drape
Support
Drop
Collide
Roll
MLP
42.5.04
58.2s1.0
52.3,07
51,9202
54.2.08
42.5.0.
65.4
72.8.05
RNN
52.2.05
57.4
53.1
52.3a0.3
53.810.
53.4
68. 68.4210
74.5.0.
SocialODE
53.2.07
59.1218
54.2
53.5
55. 820.
53. 1
69.2209
75.3
GNS
74.8a1.
72.6
61.02t.7
57.4
64.821.5
63.7
83.8
78.2
DPI
70.6
70.7
66.322.
52.1
65. 620.4
72.8
82.2a1a
79.9
EGNN
70.8=16
67.5at.7
59.2+2.
54. 1a1.4
55.321.6
69.3+1.8
79.7=04
80.9
GMN
54.6=10
66.6+1.2
50.7+4.5
59. 19.0x29
61.8+24
S6.2+16
81.0so7
80.2+09
SGNN
88.8=20
78.3+1.3
72.6+t.
60.6.0.5
71.4
73.9+1.1
83.0=1.5
84.2-07
SEGNO
88.2a1.
76.2
73.5
59.4
68.321.2
72.840.9
85.5
82.4
EGODE
94.7a14
79.0st.3
75.0at1
61.7200.
71.720.8
75.321.
90.0s1e
85.7.08
time
time
Ground
Truth
|II
EGODE
SGNN
DPI
I1
Visualizations of predictions on the RigidFall Dataset. EGODE demonstrates the best capability to generate accurate trajectories.
Figure 3:
mentioned by SGNN [20]. In our method, we initialize all MLP layers with a hidden size of 200. An Adam optimizer with the initial learning rate of 0.0001 adopted during training. We also employ an early stopping strategy of 10 epochs according to validation loss. The batch size is set to I for Physion and 8 for RigidFall dataset. To ensure fair comparison, we initialize baseline models' parameters based on corresponding papers and then fine-tune them to achieve the best results.
4.2 Performance Comparison
The comparison results for Physion and RigidFall datasets are shown in Table and Table 2 ac- cordingly. For the RigidFall dataset, follow the comparison strategy of SGNN to assess the model's performance with different sizes of the training dataset. From the results, we have three observations. Firstly, although the best baseline varies between the datasets, is generally observed that hierarchical models outperform particle-level models, which confirms that hierarchical methods inherently capture the intrinsic attributes in rigid-body systems and reduce the difficulty of modeling dynamics. Secondly, our EGODE demonstrates highest contact prediction accuracy over all bascline models in Physion dataset. In particular, compared to the best baseline SGNN Dominoes and Collide scenario. our proposed EGODE achieved an increase in prediction accuracy of 5.9% and 4.5%, respectively. Thirdly, we observe our EGODE has more robust prediction results than baseline models in long-period prediction on RigidFall dataset, with just small-scale training set. We attribute remarkable performance of proposed EGODE to three key reasons: (I) Introduction neural ODE in EGODE, provides superior generalization for dynamic systems, especially in long-term prediction scenarios. The neural ODE allows for more accurate modeling for continuous systems, thereby enhancing the overall performance of our model. (2) Introduction of coupled architecture.
7

---

Table 2: Prediction MSE (x10-3) of compared methods RigidFall, Bold numbers highlight the
best performance.
ITrainl
500
iTrainl
1000
5000 [Trainl t == 20
Methods
20
t == 40
20
t == 40
t 40
MLP
3.25
8.59
2.05
5.82s344
1,45
4.15s2.68
RNN
2.87a1.44
7.76
2.00
5.35g3.21
1,47
3. 3.6822.37
SocialODE
2.32
6.01
1.45g084
4.13
1.032061
2,9021.87
GNS
2.21+1.03
3.98-2m
1. 1.28+044
2.88+2.32
0.95+0.59
2.67-1.06
DPI
1.62:0.50
4.46:241
0.71 50.58
4.03+2.76
2.68+2.36
EGNN
0.94
2.98a2.00
1.18
2.79
0.90g0.47
2.84
GMN
2.2521.00
5,421281
1.65g165
5,4521.98
1.2220.76
2.65
SGNN
0,3220.35
1.07
0.32a021
0.7320.85
0.1920.9
0,74
SEGNO
0,6420.33
2.19
0.6020.4
2.21
0.38g0.2
1,442092
EGODE
0.17
0.71
0.17
0.49An.42
0.12at.11
0.46:0.42
Table 3:
Comparisons between our EGODE and its variants on Physion.
Methods
Dominoes
Contain
Link
Drape
Support
Drop
Collide
Roll
EGODE w/o O
89 89.82177
78.52u.2
73.5
60.820.
69.920.7
74.1
86.3a12
84,2a0s
EGODE w/o C
90.7a10
78,52L8
74.2a0.9
61. 1
70.4
74.5
88.0s1.1
84,7209
EGODE w/o E
90.3+1.
78.5+13
74.2
60.9
70.0
74.3
86.9-07
84,5+0g
EGODE
94.7a14
79.0.t.
75.0
61.7am6
71.7a08
75.3
90.0s1.
85.7.0s
Our EGODE incorporates both ohjects and mesh nodes in the rigid body, enabling effective modeling of the dynamic system. (3) Introduction of an event module for collision modeling helps EGODE effectively tackle complex and diverse instantaneous events in rigid body motion, thereby enhancing the performance across different scenarios.
4.3 Ablation Study
We analyze our EGODE and evaluate the model's effectiveness in various aspects. In particular, we introduce three model variants as follows: (1) EGODE w/o O which removes neural ODE; (2) EGODE w/o C, which removes coupled architecture; (3) EGODE w/o E, which removes event module for collision modeling; The results are presented in Table We observe that removing any of the three components leads to obvious drop in performance
most datasets and tasks. Notably, EGODE w/o causes the most performance degradation. This indicates the continuous method is insufficient to capture the intricate information inherent in rigid body dynamics. We can also conclude from the experiment results that the coupled architecture and event module are crucial for accurately predicting rigid body systems. by effectively aggregating local information, broadcasting global information, and modeling collision events.
4.4 Sensitivity Analysis
In this section, we investigate how the hyperparameters, i.e. X in Equation 12 and the distance threshold d for mesh graph construction. The results shown in Figure 4 indicate that the model achieves optimal performance when = when other parameters are fixed. The experiments also suggest that our EGODE exhibits overall stability and robustness across different A. The impact of distance threshold d' is also analyzed. When dl relatively small, reduces the connectivity between mesh nodes within an object, hindering information propagation in the network. Conversely, when is relatively large, redundant interactions might be introduced between objects, slightly affecting model performance. The optimal value of d is depending the density of mesh nodes and the scale of the objects. Ultimately, we set = and 0.08 respectively our experiments.
4.5 Generalization Performance
Since the latent embedding rf contains a v/,, h is evident that the left-hand sides of Equation and Equation 0 naturally encompass the acceleration dvf/ dt, thereby adhering to the fundamental

---

100
95
100
95
Accuracy
96
92
96
92
92
84
0.1
0.5
1.0
2.0
10
0.1
0.5
1.0
2.0
10
0.01 0.06 0.08
0.10 0.12
0.04
0.06
0.08 0.10
0.12
d
(n) Impact Dominoes
(b) A Impact Collide
(c) Impact on Dominoes
(d) Impact Collide
Figure Sensitivity analysis our EGODE Dominoes and Collide. The bar charts and error bars describe the accuracy and the 80% confidence intervals, respectively.
time
Ground
Truth
Friction
Ground Truth
Friction
Figure 5: External force field simulation compared with ground truth.
form of Newton's second law. Consequently, by introducing a term representing external forces in Equation 5 we can effectively simulate the presence of additional external forces during the ODE integration process. The external force term in the above setting be an arbitrary function of position, velocity, and object attribute. Therefore, by defining this term as various force field functions, EGODE can readily simulate the motion of rigid bodies under the influence of different external forces. Detailed formulations about the external force can be found in the Appendix A
In our experiment, we employ the most common form of resistive force, which is proportional to velocity. We conducte experiments on the Collision scenario from the Physion dataset. As illustrated in Figure when the resistive force is incorporated into the ODE simulation, notable change in the motion dynamies is observed. In the ground truth, the blue cuboid possesses sufficient energy to collide with and topple the static objects. However, with the introduction of resistive force, the cuboid's velocity and kinetic energy are significantly consumed. These experimental results substantiate the efficacy of our ODE formulation in effectively modeling and transferring motion under influence of force fields. This remarkable generalization capability stems from the inherent continuity and diffcrentiability properties of our proposed EGODE.
5 Conclusion
In this paper, we investigate the problem of rigid dynamics modeling and propose new approach named EGODE to solve the problem. Our EGODE uses both mesh node representations and object representations to describe the rigid system. More importantly, adopts a coupled graph ODE architecture to capture the evolution of dynamical systems. To model the occurrence of collisions, EGODE adopts an event module that provides instantaneous updating for the states of mesh node and object representations. Extensive experiments of various benchmark datasets validate the superiority

---

of EGODE in comparison to different state-of oate-of-thee aff-the-n art methods. In future works, we will extend our EGODE more real-world scenarios including fluid simulation and human trajectory forecasting.
Broader Impacts and Limitations. This study introduces an effective data-driven approach EGODE for modeling rigid dynamics, offering new perspective collision event modeling rigid dynamics. One limitation of our work is that our EGODE is unable to accommodate rigid body hinges and deformable objects. Future works will extend EGODE to these more generalization scenarios.
ACKNOWLEDGEMENT
This paper is partially supported by the National Key Research and Development Program of China with Grant No. 2023 123 103 as well as the National Natural Science Foundation of China with Grant Numbers 62276002 and 62306014.
References
[1] Kelsey R Allen, Tatiana Lopez Guevara, Yulia Rubanova, Kim Stachenfeld. Alvaro Sanchez- Gonzalez, Peter Battaglia, and Tobias Pfaff. Graph network simulators can learn discontinuous, rigid contact dynamics. In Conference on Robot Learning, pages 1 1157-1167. PMLR, 2023.
[2] Kelsey R Allen, Yulia Rubanova, Tatiana Lopez-Guevara, William Whitney, Alvaro Sanchez- Gonzalez, Peter Battaglia, and Tobias Pfaff. Learning rigid dynamics with face interaction graph networks. arXiv preprint arXiv:2212.03574, 2022.
[3] Sheldon Andrews, Kenny Erleben, and Zachary Ferguson. Contact and friction simulation for computer graphics. In ACM SIGGRAPH 2022 Courses, pages 1-172. 2022.
[4] Srinivas Anumasa and PK Srijith. Improving robustness and uncertainty modelling in neu- ral ordinary differential equations. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 4053-4061, 2021.
[5] Mohammadhossein Bahari, Saeed Suadatnejad, Ahmad Rahimi, Mohammad Shaverdikondori, Amir Hossein Shahidzadeh, Seyed-Mohser trajectory prediction works, but not everywhere. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17123-17133, 2022.
and Alexandre Alahi. Vehicle
[6] Daniel M Bear, Elias Wang. Damian Mrowca, Felix Binder, Hsiao- Yu Fish Tung, RT Pramod, Cameron Holdaway, Sirui Tao, Kevin Smith, Fan- Yun Sun, et al. Physion: Evaluating physical prediction from vision in humans and machines. arXiv preprint arXiv:2 106 (06.06666, 2021.
[7] Eric Z Chen, Terrence Chen, and Shanhui Sun. Mri image reconstruction via learning opti- mization using neural odes. In Medical Image Computing and Computer Assisted Intervention- MICCAI 2020: 23rd International Conference, Lima, Peru, October 4-8, 2020, Proceedings, Part II 23, pages 83-93. Springer, 2020.
[8] Ricky TQ Chen, Brandon Amos, and Maximilian Nickel. Learning neural event functions for ordinary differential equations. arXiv preprint vrXiv:2011.0390 2020.
[9] Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In NeurIPS, 2018
[10] Emilien Dupont, Amaud Doucet, and Yee Whye Teh. Augmented neural odes. In NeurIPS, 2019.
[11] Matthias Fey and Jan E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.
[12] Chris Finlay, Jömn-Herik Jacobsen, Levon Nurbekyan, and Adam Oberman. How to train your neural ode: the world of jacobian and kinetic regularization. International conference on machine learning, pages 3154-3164, PMLR, 2020.
10

---

[13] Chuang Gan, Jeremy Schwartz, Seth Alter, Damian Mrowca, Martin Schrimpf, James Traer, Julian De Freitas, Jonas Kubilius, Abhishek Bhandwaldar. Nick Haber, al. Threedworid: platform for interactive multi-modal physical simulation. arXiv preprint arXiv2200700495. 2020.
[14] Arnab Ghosh, Harkirat Behl, Emilien Dupont, Philip Torr, and Vinay Namboodiri. Steer: Simple temporal regularization for neural ode. Advances in Neural Information Processing Systems, 33:14831-14844 2020.
[15] Pallabi Ghosh, Nirat Saini, Larry $ Davis, and Abhinav Shrivastava. Learning graphs for knowledge transfer with limited labels. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 11151-11161, 2021.
[16] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. International conference on machine learning. pages 1263-1272. PMLR, 2017.
[17] Ke Gong, Yiming Gao, Xiaodan Liang, Xiaohui Shen, Meng Wang, and Liang Lin. Graphon- omy: Universal human parsing via graph transfer learning. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7450-7459, 2019.
[18] Junru Gu, Chen Sun, and Hang Zhao. Densetnt: End-to-end trajectory prediction from dense goal sets. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15303-15312, 2021.
[19] Sami Haddadin, Alessandro De Luca, and Alin Achf-fchäffer. Robot collisions: A survey on detection, isolation, and identification. IEEE Transactions Robotics, 33(6):1292-13122 1312 2017.
[20] Jiaqi Han, Wenbing Huang, Hengbo Ma, Jiachen Li, Josh Tenenbaum, and Chuang Gan. Learning physical dynamics with subequivariant graph neural networks, Advances in Neural Information Processing Systems, 35:26256-26268, 2022.
[21] Xiangyu He, Zitao Mo, Peisong Wang, Yang Liu, Mingyuan Yang, and Jian Cheng. Ode- inspired network design for single image super-resolution. In Proceedings of the IEEE/CVF Conference Computer Vision and Patlern Recognition, pages 1732-1741, 2019.
[22] Wenbing Huang, Jiaqi Han, Yu Rong, Tingyang Xu, Fuchun Sun, and Junzhou Huang. Equiv- ariant graph mechanics networks with constraints. arXiv preprint arXiv:2203.064: 2022.
[23] Zijie Huang, Yizhou Sun, and Wei Wang. Learning continuous system dynamics from irregularl--sampled partial observations. Advances Neural Information Processing Sys- tems, 13:16177-16117 2020.
[24] Zijie Huang, Yizhou Sun, and Wei Wang. Coupled graph ode for learning interacting system dynamics. 2021.
[25] Wei Ju, Siyu Yi, Yifan Wang, Qingqing Long, Junyu Luo, Zhiping Xiao, and Ming Zhang. survey of
graph learning. arXiv preprint 00X1v 2024.
[26] Wei Ju, Siyu Yi, Yifan Wang, Zhiping Xiao, Zhengyang Mao, Hourun Li, Yiyang Gu, Yifang Qin, Nan Yin, Senzhang Wang, et al. A survey of graph neural networks in real world: Imbalance, noise, privacy and ood challenges. arXiv preprint arXiv:2403. Xiv:22403.04468, 2024.
[27] Patrick Kidger, Ricky T. Q. Chen, and Terry J. Lyons. "hey, that's not an ode": Faster ode adjoints via seminorms. ICMI. 2021.
[28] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. ICIR, 2017.
[29] Miltiadis Miltos Kofinas, Erik Bekkers, Naveen Nagaraja, and Efstratios Gavves. Latent field discovery in interacting dynamical systems with neural fields. Advances Neural Information Processing Systems, 36, 2024.

---

[30] Yunzhu Li, Toru Lin, Kexin Yi, Daniel Bear, Daniel Yamins, Jiajun Wu, Joshua Tenenbaum, and Antonio Torralba. Visual grounding of learned physical models. In International conference on machine learning, pages 5927-5936. PMLR, 2020.
[31] Xingyu Lin, Yufei Wang, Zixuan Huang, and David Held. Leaning visible connectivity dynamics for cloth smoothing. In Conference on Robot Learning, pages 256-266. PMLR, 2022.
[32] Yang Liu, Jiashun Cheng, Haihong Zhao, Tingyang Xu, Peilin Zhao, Fugee Tsung, Jia Li, and Yu Rong. Improving generalization in equivariant graph neural networks with physical inductive biases. In The Twelfth International Conference on Learning Representations, 2023.
[33] Ao Luo, Xin Li, Fan Yang, Zhicheng Jiao, Hong Cheng, and Siwei Lyu. Cascade graph neural networks for rgb-d salient object detection. In Computer Esion-FECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XII 16, pages 346-364. Springer, 2020.
[34] Junyu Luo, Yiyang Gu, Xiao Luo, Wei Ju, Zhiping Xiao, Yusheng Zhao, Jingyang Yuan, and Ming Zhang. Gala: Graph diffusion- based alignment with jigsaw for source- domain adaptation. IEEE Transactions Pattern Analysis & Machine Intelligence, (01):1-14, 2024.
[35] Junyu Luo, Zhiping Xiao, Yifan Wang, Xiao Luo, Jingyang Yuan, Wei Ju, Langechuan Liu, and Ming Zhang. Rank and align: Towards effective source free graph domain adaptation. arXiv preprint arXiv:2408. 12185, 2024.
[36] Xiao Luo, Yiyang Gu, Huiyu Jiang. Hang Zhou, Jinsheng Huang, Wei Ju, Zhiping Xiao, Ming Zhang, and Yizhou Sun. Pgode: Towards high-quality system dynamics modeling. In Forty-first International Conference on Machine Learning.
[37] Xiao Luo, Jingyang Yuan, Zijie Huang, Huiyu Jiang, Yifang Qin, Wei Ju, Ming Zhang, and Yizhou Sun. Hope: High-order graph ode for modeling interacting dynamics. In International Conference Machine Learning, pages 23124-23139, PMLR, 2023.
(38] Xiao Luo, Yusheng Zhao, Yifang Qin, Wei Ju, and Ming Zhang. Towards semi-supervised uni- versal graph classification. IEEE Transactions on Knowledge and Data Engineering, 36(1) (6:11) 428, 2023.
[39] Yuexin Ma, Xinge Zhu, Sibo Zhang, Ruigang Yang, Wenping Wang, and Dinesh Manocha. Trafficpredict: Trajectory prediction for heterogeneous traffic-agents. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 6120-6127, 2019.
[40] Karttikeya Mangalam, Harshayu Girase, Shreyas Agarwal, Kuan-Hui Lee, Ehsan Adeli, Jitendra Malik, and Adrien Gaidon. It is not the journey but the destination: Endpoint conditioned trajectory prediction. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28 2020, Proceedings, Part II 16, pages 759-776. Springer, 2020.
[41] Pierre Marion, Yu-Han Wu, Michael E Sander, and Gérard Biau. Implicit regularization of deep residual networks towards neural odes. arXiv preprint XiX:22309.01233, 2023.
[42] Nishant Nikhil and Brendan Tran Morris. Convolutional neural network for trajectory prediction. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops, pages 0-0, 2018.
[43] Octavi Obiols-Sales, Abhinav Vishnu, Nicholas Malaya, and Aparna Chandramowliswharan Cfdnet: A deep learning- rearning-based accelerator for fluid simulations. In Proceedings of the 34th ACM international conference on supercomputing, pages 12 2020.
[44] Sunghyun Park, Kangyeol Kim, Junsoo Lee, Jacgul Choo, Joonseok Lee, Sookyung Kim, and Edward Choi. Vid-ode: Continuous-time video generation with neural ordinary differential equation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 2412-2422 2021.
[45] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary Vito,o, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
12

---

[46] Jiang- Zhou Peng. Siheng Chen, Nadine Aubry, Zhihua Chen, and Wei- Tao Wu. Unsteady reduced-order model of flow over cylinders based on convolutional and deconvolutional neural network structure. Physics of Fluids, 32(12):123609, 2020.
[47] Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez,, and Peter W Battaglia. Learning mesh-based simulation with graph networks. arXiv preprint urXiv:2010.03409, 2020.
[48] Michael Poli, Stefano Massaroli, Junyoung Park, Atsushi Yamashita, Hajime Asama, and Jinkyoo Park. Graph neural ordinary differential equations. arXiv preprint arXiv:1911.0755: 2019.
[49] Yifang Qin, Wei Ju, Hongjun Wu, Xiao Luo, and Ming Zhang. Learning graph ode for continuous-time sequential recommendation. IEEE Transactions on Knowledge and Data Engineering. 2024.
[50] Zhengyong Ren and Jingtian Tang. 3d direct current resistivity modeling with unstructured mesh by adaptive finite-element method. Geophysics, 75(1):H7-H17, 2010.
[51] Kai Richter and Rolf Ernst. Event model interfaces for heterogeneous system analysis. In Proceedings 2002 Design, Automation and Test in Europe Conference and Exhibition, pages 506-513, IEBE, 2002.
[52] Alvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter Battaglia. Learning simulate complex physics with graph networks, In International conference on machine learning, pages 8459-8466 PMLR, 2020.
[53] Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural networks. International conference on machine learning, pages
PMLR, 2021.
[54] Yidi Shao, Chen Change Loy, and Bo Dai. Transformer with implicit edges for particle-based physics simulation. In ECCV, pages 349-564, 2022.
[55] Liushuai Shi, Le Wang, Sanping Zhou, and Gang Hua. Trajectory unified transformer for pedestrian trajectory prediction. In Proceedings the Computer Vision, pages 9675-9684, 2023.
the IEEE/CVF International Conference
[56] Weijing Shi and Raj Rajkumar. Point-gnn: Graph neural network for 3d object detection in a point cloud. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1711-1719, 2020.
[57] Qingyang Tan. Zherong Pan, Lin Gao, and Dinesh Manocha. Realtime simulation of thin thin-shell deformable materials using enn-based mesh embedding. IEEE Robotics and Automation Letters, 5(2):2325 2020.
(58) Qixuan Wang and Hao Wu. Mathematical modeling of chemotaxis guided amoeboid cell swimming. Physical Biology, 18(4):045001, 2021.
[59] Shan Wang, J González-Cao, H Islam, M GéstezGe and Guedes Soares. Uncertainty estimation of mesh-free and mesh-based simulations of the dynamics of floaters. Ocean Engineering. 256:111386, 2022.
[60] Yanan Wang, Michihiro Yasunaga, Hongyu Ren, Shinya Wada, and Jure Leskovec. Vqa- gnn: Reasoning with multimodal knowledge via graph neural networks for visual question answering. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 21582-21592 2023.
[61] Song Wen, Hao Wang. Di Liu, Qilong Zhangli, and Dimitris Metaxas. Second-order graph odes for multi-agent trajectory forecasting. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5101-5110, 2024.
[62] Song Wen, Hao Wang, and Dimitris Metaxas. Social ode: Multi-agent trajectory forecasting with neural ordinary differential equations. In European Conference on Computer Vision, pages 217-233. Springer, 2022.
13

---

[63] Yifan Wu, Tom Z Jiahao, Jiancong Wang, Paul A Yushkevich, M Ani Hsich, and James C Gee. Nodeo: A neural ordinary differential equation based optimization framework for deformable image registration. In Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition, pages 11804-208133 2022.
[64] Louis-Pascal Xhonneux, Meng Qu, and Jian Tang. Continuous graph neural networks. In ICML, pages 10432-10441, 2020.
[65] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In ICLR, 2019.
[66] Jingyang Yuan, Xiao Luo, Yifang Qin, Zhengyang Mao, Wei Ju, and Ming Zhang. Alex: Towards effective graph transfer learning with noisy labels. In Proceedings of the 31st ACM international conference multimedia, pages 3647-3656, 2023.
[67] Yawen Zeng, Da Cao, Xiaochi Wei, Meng Liu, Zhou Zhao, and Zheng Qin. Multi-modal relational graph for cross-modal video moment retrieval. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 2215-2224, 2021.
[68] Xiao Zhang, Chongxing Song, Tao You, Qicheng Bai, Wei Wei, and Lei Zhang. Dual ode: Spatial-spectral neural ordinary differential equations for hyperspectral image super-resolution. IEEE Transactions Geoscience and Remote Sensing, 2023.
14

---

A Additional Explanation to Generalization Performance
Since r contains æf, vi ftj. we can effectively simulate the presence of additional external forces during the ODE integration process by introducing slight modification to Equation S
dr! dt (ENA(()
d'(1 Σ (( (0, F(z/, h.)
(13)
where F is arbitrary function of position, velocity, and object properties, can deduce from Newton's ' laws that F corresponds to an additional acceleration term compared to the dynamics described by Equation Consequently, by defining F various force field functions, EGODE can easily simulate the motion of rigid bodies under the influence of different external forces.
In experiments, we use the most common form of resistive force, i.e., F( v/) rof where y denotes a constant resistive coefficient. Then the dynamics of rf can be formulated as follows:
dr di
o'(l Σ ('(ri, (EN'()
F.D F + 00, -oi 0]
(14)
We can obtain the speed components from both ends of the equation at the same time, and then simplify the formulation:
duf = 10'(1 Σ
yo!
dt
KEN((()
duf
+ yo, = l0'(1 Σ
(15)
dt
KEN*(1
d(E d(E
dt = (
Σ
('(ri, M
PEN'()
This function about oa is similar to a classic decay differential equation. In particular, when there are tiny interactions between nodes and no collisions between objects, the right hand of Equation [5] approaches to O, and the solution can be approximated as:
of ~ C₁e
(16)
where C, is constant decided by initial conditions. The expression demonstrates particles whose velocity decays exponentially in space. Although the derivation above is based on various assumptions, experiments in Section 4.5 showed similar results.
B Details of Baselines
Our EGODE compared with a range of competitive methods including MLP, RNN, GNS (52). DPI-Net [30], EGNN [53], GMN (22). SGNN 20), SocialODE (62, and SEGNO 32] The details of each method are depicted as follows:
MLP: A classical machine learning method applied to the Rigid-body collision task.
RNN: A classical method for time-series prediction, used to model rigid-body movement and predict subsequent motion steps.
GNS 52 A discrete method that models physical interactions via particle representa- tion using Graph Neural Networks (GNNs). It consists of encoder- encoder-processor drocessor-decoder architecture
DPI-Net 30 A GNN-based method that learns dynamics via particle representation with different materials, including fluids, gases, soft and rigid objects. showcases the generalization capability with learned particle dynamics model in real-world control tasks.
15

---

EGNN [53] A GNN-based method designed for dynamics modelling. The model subjects to several equivariance constraints including rotation, translation, and permutation, complying with physics rules.
A GNN- method designed for learning dynamics. In addition to rotation, translation, and permutation constraints, GMN also adds geometric constraints, making geometrically equivariant for interacting objects in the real world.
GMN 2
SGNN [00] A discrete method which relaxes the equivariant constraints (rota- to subequivariance due to external fields like gravity. It consists both object-level and particle-level message-passing
tion/translation/permmaaaa
SocialODE
62 An encoder-decoder based architecture that adopts Neural ODE model continuous transition states. The encoder is a spatio-temporal transformer that encodes historical information into latent vector, and sequence of latent trajectories is generated through an Ordinary Differential Equation (ODE) solver and recovered by the decoder.
SEGNO :2]: GNN-based continuous equivariant method using Neural ODE to approxi- mate dynamic trajectories. It also incorporates second-order motion information to enhance modeling capacity.
C Details of Datasets
We mainly conduct experiments on two datasets: Physion (6 and RigidFall [30]. Physion is a large- scale dataset and benchmark for physical system interaction evaluation designed ThrleDWorld [33. It contains realistically simulated scenarios:
Dominoes: Simulation of dominoes being knocked down one after another.
Contain: Simulation involving collisions with concave rigid bodies.
Collide: Simulation of rigid body crashing into other rigid bodies at relatively high speed.
Drop: Simulation of rigid body falling onto other rigid bodies.
Roll: Simulation of rigid body sliding and rolling.
Link:
Simulation of ring-mounted rigid bodics.
Support: Simulation of stack of rigid bodies being hit.
. Drape: Simulation of lightweight flexible object falling rigid bodies.
RigidFall simulates collisions and interactions between three rigid cubes where each cube consists of 64 particles. The three cubes are initially placed in stack in the air and fall under varying gravitational acceleration.
D Details of Evaluation Metric
To compare these baseline models, we utilize two evaluation metrics: i.e., Contact prediction accuracy and Mean Square Error (MSE). The Contact prediction metric is provided by Physion ( and used to evaluate whether two target objects collide not in the whole trajectory. Notably, for MSE, we compare the Euclidean coordinates difference directly rather than calculating the difference between the normalized actual position and predictions. This approach enables more precise quantification of positional discrepancies in three-dimensionnal space.
Contact prediction accuracy:
Acc Σ 1 -.))
Mean Absolute Error (MAE):
m MAE- n
23/.
16

---

Table 4: Prediction MSE of compared methods on Physion, Bold number highlight the best perfor-
mance
Methods
Dominoes
Collide
Roll
Drape
SGNN SEGNO
0.762+am15
3.39+a.2
2.32+0.15
31,3=15 34,3-27
0.774+0m44
3.57-0.1
2.53+0.19
EGODE
0.725
3.04
2.1540144
29,4212
SGNN
EGODE
95
Drop
Contain
80
75
Support
70
Link
Figure 6: Generalization analysis across different tasks. Row/column records the training or testing phase, respectively. EGODE outperforms the best baseline.
In the above expressions, Vj and x, represent the ground truth value, while ūJ and &s represent the predicted value.
E
Additional Implementation Details
We torch-geometric [1 and torchdiffeq [2 to complete code. To enable fair comparison with previous baselines, our model is only given the initial states of the trajectory X to predict future trajectories X for both Physion and RigidFall datasets. An Adam optimizer with the initial learning rate of 0.0001, beta(0.9, 0.999) is adopted during training. A factor 0.8 and patience of is adopted for the Plateau scheduler. We train model for 1000 epochs and early stopping strategy of 10 epochs according to validation loss. To solve the ODE function, we adopt the common Euler ODE solver in experiment and performs well in physics modeling task. By adding adaptive collision event module, our model can easily detect collision in the evaluation stage and compute the contact prediction accuracy. For baseline methods without event detector, follow previous setting of predefined contact threshold to judge collision and compute corresponding contact accuracy during evaluation. We conduct our experiments on server with eight NVIDIA A40 GPUs. Since an OpenGL interface and a monitor are required for the visualization process, we visualize our results using a local PC with single NVIDIA 4090 GPU.
Additional Experiments
F1 Prediction MSE on Physion.
We also investigate the prediction MSE some scenes the Physion dataset our method and compared methods. Note that in some scenes (Contain, Drop, Link, Support), objects are often initially positioned in a centrally symmetric manner, therefore all motion patterns centrally symmetric to the ground truth center are reasonable. In this case, MSE is unable to provide an accurate representation of the predictive performance of the model. The results are demonstrated Table 4 We can indicate that our EGODE outperforms the two strong baselines in all scenes.
17

---

Ground Truth
EGODE
SGNN
time
Figure 7: Additional visualizations of predictions on the Physion Dataset.
F.2 More Generalization Performance
To rigorously evaluate the generalization capability of the proposed EGODE in diverse scenarios, we employ models trained on one specific scenario and test their performance on different scenarios from the Physion dataset. The results illustrated in Fig 6 demonstrate that our proposed method exhibits significantly stronger generalization performance compared to the best baseline, SGNN. This observation highlights the model's proficiency in learning and effectively transferring the underlying principles governing object dynamics and interactions, transcending the specifics of the training scenario. Such superior generalization capability is a testament to the model's ability to capture the intrinsic pattern rigid dynamics, enabling accurate predictions across diverse scenarios without the need for explicit retraining.
18

---

F.3 More Visualization
As shown in the figure, the additional visualizations of predictions on the Physion Dataset showcase that EGODE outperforms the best baseline SGNN to generate accurate trajectories. EGODE yields predictions closer the ground truth compared to SGNN. addition, note that the tip of the green object overlaps with the yellow object in the prediction of SGNN, while our model is more consistent with rigid physical laws. The visualization indicates that EGODE performs better in incorporating physics-based constraints and producing more physically plausible and accurate predictions.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract states that this paper proposes novel approach called EGODE for effective rigid dynamies modeling. Our claims are supported by both theoretical analysis and experimental results.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The conclusion states that our EGODE is unable accommodate rigid body hinges and deformable objects, due to the deficiency of the dataset. These problems will be further discussed in future works.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
20

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The main emphasis of this paper lies in empirically validating the novelty and effectiveness of the proposed framework while leaving theoretical insights for future research.
Guidelines:
The answer NA means that the paper does not include theoretical results,
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: In Section 4.1 "Experimental Settings" and Appendix, we provide detailed explanations of the datasets used and implementation details for our experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibilityy can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the partieular way they provide for reproducibiliity In the case of closed-source models, it may be that access to the model is limited in
21

---

some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
In Abstract, we provide an anonymous link to the code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cr cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ httppecccppbccccccee
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Section 4.1 "Experimental Settings" and the appendix, we provide detailed explanations of the datasets used and implementation details for experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[Yes]
Justification:
We precisely defined and reported the error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
22

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: In the appendix, provide detailed explanations of computer resources used for our experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The research conducted in the paper complies with the NeurlPS Code of Ethics in every respect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The Conclusion states that this study introduces an effective data-driven approach EGODE for modeling rigid dynamics, offering a new perspective on collision event modeling in rigid dynamics. One limitation of our work is that our EGODE is unable to accommodate rigid body hinges and deformable objects. Future works will extend EGODE to these more generalization scenarios.
23

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We explicitly cited the sources of the relevant data and other materials used in the paper.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thood cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
The anonymous link provided in the appendix contains well-documented related materials.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
25

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
26


Reviews:
[{"review_id": "LsogwIv5Ii", "reviewer": "Reviewer_Py8V", "summary": "The paper introduces EGODE, a novel framework for modeling rigid dynamics that has applications in robotics, graphics, and mechanical design. The framework addresses the limitations of existing graph neural network (GNN) simulators by incorporating both mesh node representations and object representations within a coupled graph ODE (Ordinary Differential Equation) structure. EGODE introduces an event module to capture instantaneous changes during collisions, providing a more accurate and continuous model of rigid dynamics. The framework's performance is validated through extensive experiments on benchmark datasets, demonstrating superiority over various state-of-the-art baselines.", "questions": "1. How does EGODE compare with traditional physics engines in terms of computational efficiency and accuracy?\n\n2. Can the authors elaborate on how EGODE might be extended to handle scenarios with rigid body hinges and deformable objects?\n\n3. What are the theoretical underpinnings of the coupled graph ODE framework, and are there any proofs of concept?", "limitations": "1. **Rigid Body Hinges and Deformable Objects**: EGODE currently cannot accommodate complex scenarios involving rigid body hinges and deformable objects.\n2. **Dataset Dependency**: The framework's capabilities are limited by the scope and diversity of the datasets used for training and validation.", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. **Innovative Approach**: EGODE proposes a new perspective on modeling rigid dynamics by combining continuous evolution and instantaneous changes using graph ODEs.\n2. **Coupled Graph ODE Framework**: The use of a coupled architecture effectively models hierarchical structures in rigid-body systems.\n3. **Event Module for Collisions**: The introduction of an event module enhances the framework's ability to handle instantaneous changes during collisions.", "weaknesses": "1. The paper focuses on empirical results, with less emphasis on theoretical insights or proofs of concepts. What are the theoretical underpinnings of the coupled graph ODE framework, and are there any proofs of concept? \n\n2. While EGODE shows good generalization within the tested scenarios, its performance in other types of rigid dynamics needs further evaluation. How does EGODE compare with traditional physics engines in terms of generalization ability, computational efficiency and accuracy?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following.\n\n> Q1. How does EGODE compare with traditional physics engines in terms of computational efficiency and accuracy?\n\nA1. Thanks for your comment. The simulation speed of EGODE is not inferior to numerical simulations. Our EGODE can achieve efficient inference by leveraging GPUs and deep learning toolkits such as torch_geometric. In our paper, each scenario visualization contains thousands of nodes and spans 0.5~1 minutes, yet it only requires less than 1 second to simulate and render on a single laptop. \nDue to the heavy computation burden of data adaptation to traditional physics engines, we will provide the experimental speed comparation during the interaction periods in several days.\n\n> Q2. Can the authors elaborate on how EGODE might be extended to handle scenarios with rigid body hinges and deformable objects?\n\nA2. Hinges and deformable objects are both meaningful and more challenging than rigid body dynamics. For hinges, each block can be treated as an independent rigid body, and special edges can be used in our EGODE to model the interactions between different parts at the hinge points, maintaining geometric and mechanical consistency. The fixed parts of the hinges can also be considered a special case of rigid body dynamics under constraints, where we evaluate our EGODE on a similar setting called Contain in Table 3 where container objects restrict other objects movements. The model can be trained by collecting sufficient data on hinge dynamics scenarios to improve its performance. While we did not specifically evaluate our model on single deformable objects, our framework is designed to handle more complex scenarios involving interactions between deformable objects and rigid bodies. As an example, we have evaluated our method on a scenario called \"Drape\" and the results are reported in Table.3, where deformable objects (cloth) interact with rigid bodies. This evaluation demonstrates the potential of our approach to handle deformable object dynamics and its generalization ability.\n\n> Q3. What are the theoretical underpinnings of the coupled graph ODE framework, and are there any proofs of concept?\n\nA3. In Appendix B, we have provided the theoretical underpinnings to show that it is possible to propagate gradients across the event times to the input arguments of the system and therefore our method can be optimized by SGD. Besides, our work focuses on empirical results since both neural ODEs and physical scenarios involve too many parameters. In future work, we consider simplifying the model and developing more interpretable models with theoretical underpinnings. We will include this in the revised version.\nThanks again for appreciating our work and for your constructive suggestions. Please let us know if you have further questions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LcFinEz9kx", "reviewer": "Reviewer_rwMo", "summary": "This paper introduces EGODE, a method for modeling rigid dynamics. To do this, they introduce a framework which integrates neural ODEs and GNNs, and they integrate this with an event module approach for collision modeling. They demonstrate superior performance to baselines on two standard benchmarks and demonstrate via ablations the contribution of components of their model.", "questions": "Just reiterating the above — could you provide confidence intervals on experiments?", "limitations": "See weaknesses re: deformable bodies.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper is overall clearly written, with a clear and explicit methods section, good motivation, and well-structured results. Methods are (to my knowledge) novel, with particular interest to the graph ODE framework. Experiments appear to use sound approaches and appear significant, with fairly good improvements over a wide range of popular baselines. The ablation study is convincing.", "weaknesses": "I’ve got some minor clarity comments.\n\n-It would be helpful to be a bit more concrete with existing model shortcomings in the introduction. It is not completely clear what it means for existing approaches to “fail to take…into consideration” instantaneous changes. I think I know what you’re saying (e.g. GNNs model large instantaneous changes via iterative message passing, which can lead to issues — is that right?) but it would be good to be a bit more explicit in outlining these issues.\n\n-Benchmarks could be described better. E.g. I assume the success metric shown for Physion is the accuracy on whether the objects of interest collided, right? Yet as far as I can tell, this is never mentioned.\n\nIn judging significance, if I’m understanding things correctly, you’re providing standard deviation estimates in the tables. It would be helpful to report SEM or give confidence intervals. The reported +- are often not particularly small relative to the improvements, to the extent that a number of them appear like they may not be significant. It would be helpful to clarify this.\n\nMore broadly, it makes sense to focus on rigid bodies, but much related work is for deformable bodies as well, and arguably, a major reason for using these complex neural models is to accommodate different sorts of media. It would be helpful to at least give a sense for how this method could be extended, or what challenges there are in extending it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are truly grateful for the time you have taken to review our paper, your insightful comments and support. Your positive feedback is incredibly encouraging for us! In the following response, we would like to address your major concern and provide additional clarification.\n\n> Q1. It would be helpful to be a bit more concrete with existing model shortcomings in the introduction. It is not completely clear what it means for existing approaches to “fail to take…into consideration” instantaneous changes. I think I know what you’re saying (e.g. GNNs model large instantaneous changes via iterative message passing, which can lead to issues — is that right?) but it would be good to be a bit more explicit in outlining these issues.\n\nA1. Thanks for your comment. Your understanding is correct. GNN methods utilize iterative message passing to model the spatial relationships based on current distances, which are hard to capture large instantaneous changes. We will include it in the revised version. \n\n> Q2. Benchmarks could be described better. E.g. I assume the success metric shown for Physion is the accuracy on whether the objects of interest collided, right? Yet as far as I can tell, this is never mentioned.\n\nA2. Thanks for your comment. The success metric denotes the accuracy of predicting whether the objects of interest collided. We will include it in the revised version. \n\n> Q3. In judging significance, if I’m understanding things correctly, you’re providing standard deviation estimates in the tables. It would be helpful to report SEM or give confidence intervals. The reported +- are often not particularly small relative to the improvements, to the extent that a number of them appear like they may not be significant. It would be helpful to clarify this.\n\nA3. Thanks for your comment. We have conducted one-sample paired t tests to justify that all of the improvements with the best baseline are statistically significant with p-value < 0.01. Our problem focuses on the classification tasks and trajectories regression tasks, which is not a parameter inference task and thus cannot have SEM or CI for performance comparison. \n\n> Q4. More broadly, it makes sense to focus on rigid bodies, but much related work is for deformable bodies as well, and arguably, a major reason for using these complex neural models is to accommodate different sorts of media. It would be helpful to at least give a sense for how this method could be extended, or what challenges there are in extending it.\n\nA4. Thanks for your comment. Deformable body dynamics involve complex interactions and behaviors not present in rigid body dynamics, such as stretching, bending, and compressing, which require different modeling approaches. Although we do not have specific low-level modules designated for deformable objects, our model can still handle complex scenarios involving interactions between deformable objects and rigid bodies due to its generalization ability. For instance, Table 3 shows our model's ability to model deformable objects' interaction with rigid bodies in the Drape scenario, where deformable objects are represented as cloths. This result showcases the potential application of our approach in modeling deformable object dynamics. In future work, we will extend our methods to deformable bodies on more complex and challenging datasets and may design modules specifically for processing deformable objects.\n\n> Q5. Could you provide confidence intervals on experiments?\n\nA5. Thanks for your comment. Please refer to A3.\n\nThanks again for appreciating our work and for your constructive suggestions. Please let us know if you have further questions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0G3vpZE2T6", "reviewer": "Reviewer_ZzdH", "summary": "The paper presents a Graph ODE framework to model rigid body dynamics. In a departure from previous works on Graph ODEs, the proposed framework incorporates a hierarchical structure by explicitly modeling both mesh-based representations and object-level representations of the rigid bodies. Furthermore, the framework also introduces a learnable event-detector and an event-processor. The event-detector module is used to estimate the time at which a potential collision occurs and the event-processor deals with the instantaneous state update after a collision event. The method is evaluated in the RigidFall and Physion datasets and consistently outperforms other baselines.", "questions": "- How fine-grained can the mesh-based representation be without deteriorating the performance of the approach? \n\n- Are there any scenarios where using standard mean square error (MSE) loss leads to poor performance? Would incorporating physics priors in the loss function further improve the performance or would it potentially reduce the amount of data required for learning?", "limitations": "Yes. Limitations of the method are mentioned.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper is well written and the overall ideas are clear.\n\n- The proposed method is evaluated thoroughly in challenging benchmarks and it consistently outperformed state-of-the-art baselines in terms of contact prediction accuracy and in terms of the mean-squared error of the prediction of Euclidian coordinates.\n\n- The framework has low sensitivity wrt to hyperparameters.", "weaknesses": "- The learnable event function, which operates on pairwise mesh points, requires exhaustive evaluation which might hinder the scalability of the approach in terms of the number of rigid bodies that can be simulated and in terms of the mesh resolution of the simulated rigid bodies.\n\n[Minor]\nL77. The wording might be lacking a negation. Are other approaches able to \"handle intrinsic continuity and discontinuity in rigid models\"?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are truly grateful for the time you have taken to review our paper, and your insightful comments and support. Your positive feedback is incredibly encouraging for us! In the following response, we would like to address your major concern and provide additional clarification.\n\n> Q1. The learnable event function, which operates on pairwise mesh points, requires exhaustive evaluation which might hinder the scalability of the approach in terms of the number of rigid bodies that can be simulated and in terms of the mesh resolution of the simulated rigid bodies.\n\nA1. We agree that the pairwise evaluation of mesh points for event detection could potentially limit the scalability as the number of rigid bodies and mesh resolution increase. However, our event function does not directly process all pairwise mesh nodes, but instead performs rule-based filtering first (which was not elaborated in the main text due to space limitations). Specifically, Equation (7) first uses the distance condition $d(x_i, x_j)<d_{threshold}$ to filter out the node pairs that require event function computation, and then uses GNN to propagate information. This significantly reduces the time complexity of the function. Moreover, as the nodes become denser, we can also reduce $d_{threshold}$ to alleviate the computational burden of the event function. Since denser nodes make the collision analysis more detailed, reducing $d_{threshold}$ does not significantly affect the performance. We are conducting experiments to examine the performance and efficiency pratically, in terms of the number of rigid bodies and mesh node number, the results will be provided during the interaction periods in several days.\n\n> Q2. L77. The wording might be lacking a negation. Are other approaches able to \"handle intrinsic continuity and discontinuity in rigid models\"?\n\nA2. Thanks for pointing this out, We will fix it in the official release. In our work, we utilize graph ODE and event function to handle intrinsic continuity and discontinuity in rigid models. We believe that other potential methods are also worth discussing, such as using 3D space or physical inductive biases of rigid bodies. We will further explore these approaches in future work to find more solutions to this problem beyond our proposed EGODE. \n\n> Q3. How fine-grained can the mesh-based representation be without deteriorating the performance of the approach?\n\nA3. We have already discussed in A1 how to reduce the complexity of the event function. By reasonably selecting $d_{threshold}$, the event function can be controlled within $O(N)$, where $N$ is the number of nodes. We are conducting an experiment about the most fine-grained the graph can achieve without deteriorating the performance. Due to the heavy computation burden, we will provide the results during the interaction periods in several days.\n\n> Q4. Are there any scenarios where using standard mean square error (MSE) loss leads to poor performance? Would incorporating physics priors in the loss function further improve the performance or would it potentially reduce the amount of data required for learning?\n\nA4. Although we currently achieve good results using the MSE loss, incorporating physics priors into the MSE is a novel idea. We have tried supervising the center of mass position $X_c^t$ using MSE as well. Due to the heavy computation burden, we will provide the ablation study during the interaction periods in several days.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eOi6S2mWvL", "reviewer": "Reviewer_uFcU", "summary": "The paper introduces EGODE, a method simulate rigid-body dynamics with contacts using a hierarchical representation. The main system consists of two parts: a mesh node representation and object representation that both are coupled inside a neural ODE that simulates the rigid body system dynamics. In addition, an extra event module processor is learned to make instant changes to the system state in case of a collision. This combination enables the model to model the  dynamic effects more accurately in several simulation settings and even generalize to novel scenarios involving external forces.", "questions": "- Is there a reason to leave out FiGNet [1] as a baseline when its mentioned in the introduction and its limitations are discussed in Related Work? Can you add the baseline to the experiments?\n- Could you elaborate on the factors contributing to EGODE's higher relative performance on RigidFall compared to Physion? Are there specific characteristics of these benchmarks that favor EGODE's architecture?\n- For the generalization experiment shown in Figure 5 could you add some baselines like SEGNO for comparison? Its hard to see how good the proposed method generalized without any baseline references", "limitations": "Yes.", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- the idea to combine GraphODE with event modules  looks novel and is well motivated given the shortcomings of current methods that typically apply GNNs, which cannot handle instantaneous changes of collisions well\n- Overall the paper is structured well and is easy to understand even for people from other domains. The introduction does a good job to motivate the proposed method. Related work covers various recent approaches and the differences to EGODE. \n- Detailed ablation studies are conducted to justify various design decisions of the framework and the method is tested against many recent baselines on two benchmarks and surpasses them in all tested environments", "weaknesses": "- Fignet is mentioned in the introduction multiple times but it is not used as a baseline\n- Generalization experiment has no reference baselines, which makes it hard to asses the performance of the proposed method", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following.\n> Q1. Is there a reason to leave out FiGNet [1] as a baseline when its mentioned in the introduction and its limitations are discussed in Related Work? Can you add the baseline to the experiments?\n\nA1. We appreciate your suggestion to include FiGNet [1] as a baseline in our experimental evaluation. Unfortunately, due to the unavailability of the original code publicly, we didn't implement FiGNet in our scenario as a baseline for comparison in the first time. However, we understand the importance of this baseline and have decided to replicate FiGNet using the information provided in the original paper. Given the computational resources required to run FiGNet and the need to ensure a fair comparison, we plan to provide the results in an updated version of our submission or as supplementary material. We expect to complete this within the next few days.\n\n>Q2. Could you elaborate on the factors contributing to EGODE's higher relative performance on RigidFall compared to Physion? Are there specific characteristics of these benchmarks that favor EGODE's architecture?\n\nA2. The superior performance of EGODE on the RigidFall benchmark can indeed be attributed to the simpler nature of the physical interactions present in this dataset. RigidFall consists of simulations involving just three cubes under varying gravitational conditions, all with uniform internal properties such as particle numbers, shapes, and friction coefficients, as shown in Figure.3. In contrast, Physion is a more complex dataset, encompassing eight distinct scenarios with multiple objects that vary in size, shape, and friction. For instance, Figure.1 showcases complex objects such as tower-shaped and torus-shaped objects. Additionally, Physion includes not only rigid-rigid interactions but also rigid-deformable interactions. The simplicity of the RigidFall dataset aligns well with the strengths of our method, which excels at modeling straightforward physical interactions. In addition, the performance of EGODE across both datasets demonstrates its robustness and adaptability in handling a range of physical phenomena.\n\n>Q3. For the generalization experiment shown in Figure 5 could you add some baselines like SEGNO for comparison? Its hard to see how good the proposed method generalized without any baseline references\n\nA3. We agree that including additional baselines would enhance the interpretability of our generalization experiment presented in Figure 5. To address this, we will incorporate SEGNO and other relevant baselines for comparison. This will provide a clearer context for evaluating the generalization capabilities of our proposed method. We aim to finalize these additions and submit the updated figures within the next few days.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "a1yKzpeLXQ", "reviewer": "Reviewer_KaXG", "summary": "This paper presents a graph-ODE simulator for capturing rigid body dynamics with collision events. It introduces a design based on object-level and mesh-level representations. It also introduces an event module to capture collision and incorporate it into the network. The paper conducts an evaluation on two rigid-body datasets Rigid-Fall and Physion and compares its performance with multiple learning-based simulators.", "questions": "See Weakness.", "limitations": "See Weakness.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper picks an interesting and potentially impactful research topic. Rigid-body simulation with contact and collisions has wide applications in ML/Robotics, perhaps much more than simulations of any other physics systems (fluids, deformable volumes, cloths, elastic rods, etc.). One of the most critical technical components in such simulations is contact and collision handling, which is also the source of many interesting behaviors of rigid bodies. Rethinking this problem with learning-based techniques may lead to new opportunities, e.g., ML-friendly rigid-body simulators that can be seamlessly integrated into a modern deep-learning pipeline.", "weaknesses": "I will combine my comments on “Weaknesses”, “Questions”, and “Limitations” here.\n\nI am confused by this paper’s view on rigid-body simulation. State-of-the-art numerical simulation of rigid bodies with contact, e.g., Isaac Sim and Mujoco 3, is quite powerful. With efficient contact handling algorithms and GPU acceleration, they can simulate fairly complicated (articulated-)rigid bodies like humanoids and quadrupeds at a speed of several millions of time steps per second (https://github.com/google-deepmind/mujoco/discussions/1101). To me, the Rigid-Fall and Physion examples are trivial and solved problems for modern rigid-body simulators. However, the paper did not include these simulators as baselines and limited its comparison with learning-based simulators only. I struggle to see a strong motivation that can support this (lack of) comparison.\n\nThe paper also contains several confusing statements regarding physics simulation in its introduction, which I feel are bending the storytelling in the paper’s favor:\n1. Paragraph 1: Each sentence alone is technically correct, but combining them gives readers the impression that simulating rigid collision is computationally expensive and data-driven approaches are promising. I am not sure this is the right impression to make. Simulating fluids and deformable bodies is potentially expensive, but simulating rigid bodies with collisions is quite cheap using modern algorithms and hardware.\n2. Paragraphs 2-3: Following paragraph 1, these paragraphs lead readers to focus on GNNs (plus standard message passing) and their difficulties in rigid-body simulation. These difficulties do not have to exist in the first place. To me, using graphs to capture rigid-body dynamics is a somewhat contrived idea, because each rigid body contains a small and constant number (6) of DoFs regardless of its mesh resolution. Storing DoFs like x and v on mesh nodes is highly inefficient and unnecessary because they are governed by the 6 DoFs only. GNNs are more suitable for capturing fluids and deformable bodies because their governing equations involve PDEs with spatial derivatives (so information exchange between neighbors is needed) and high DoFs after discretization. In this sense, choosing GNS and DPI as baselines is also contrived: They are not designed specifically for rigid bodies, and they are capable of capturing much more complicated dynamics. Instead, I feel that classic numerical simulators should have been a baseline.\n\nI also have a few more comments regarding the technical method and experiments:\n1. It looks like the positions and velocities of all mesh nodes are included in the state variable and evolved in the ODE (Eqns. 1-2). The number of these variables grows as mesh resolution increases, but they are essentially governed by their underlying rigid-body DoFs (6 per rigid body) only. Is it necessary to assign and evolve DoFs at each mesh node?\n2. Eqns. 3-4 roughly captures the linear motion of the “center of mass” at each rigid body. Using them as the object-level state does not seem to capture the angular motions and angular velocities of each object. Missing angular information on the object level is a bit counter-intuitive from a physics perspective.\n3. The collision events visualized in the figures seem between parametric surfaces (cubes and spheres) only. Such collision events could be easily resolved with closed-form solutions. These examples do not seem to show the benefits of having a mesh representation in collision detection. A more complicated scene with multiple organic surfaces would be a better example to necessitate these meshes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following.\n> Q1. The Rigid-Fall and Physion examples are trivial and solved problems for modern rigid-body simulators. However, the paper did not include these simulators as baselines and limited its comparison with learning-based simulators only. I struggle to see a strong motivation that can support this (lack of) comparison. \nThe paper also contains several confusing statements regarding physics simulation in its introduction, which I feel are bending the storytelling in the paper’s favor.\n\nA1. Although modern numerical simulations can effectively solve rigid body dynamics problems, learn-based methods also possess unique research value. It is based on long-standing considerations followed:\n\n1. Numerical simulations are not easily integrated with learn-based neural network methods. Specifically, when simulating complex dynamic systems involving both rigid bodies and fluids, it is challenging to delegate fluid simulation and rigid body simulation separately to neural networks and numerical simulators. This makes it difficult to leverage the advantages of neural networks in fluid simulation. On the other hand, end-to-end neural networks can adapt to such complex scenarios by incorporating the simulation of rigid bodies, fluids, and their interactions together (which we are currently exploring). Although this paper does not address fluid simulation, the exploration of learn-based simulations of rigid bodies constitutes a significant precursor to the complex system with rigid and non-rigid. I believe this is one of the most impressive directions for practice and development of our EGODE.\n\n2. Besides practical implications, our EGODE also has methodological development. We adopt an event module with coupled ODE architecture to model the instantaneous updating of object states. This novel approach can be used not only in rigid body collision simulations but in any dynamic processes involving impulse or angular impulse, and even more broadly in systems including instantaneous state changes (eg. crushing and deformation).\n\n3. Based on the aforementioned reasons, we have sufficient motivation to study and improve learn-based methods for simulating rigid body dynamics and leverage their strengths in appropriate scenarios, so we only utilized learn-based methods as our baselines for performance comparison. \n\n4. Moreover, the simulation speed of EGODE is not inferior to numerical simulations. You mentioned that numeral simulators can compute fairly complicated rigid bodies at a speed of several millions of time steps per second. Similarly, our EGODE can also achieve efficient inference by leveraging GPUs and deep learning toolkits such as torch_geometric. In our paper, each scenario visualization contains thousands of nodes and spans 0.5~1 minutes, yet it only requires less than 1 second to simulate and render on a single laptop.\n\n> Q2. It looks like the positions and velocities of all mesh nodes are included in the state variable and evolved in the ODE (Eqns. 1-2). The number of these variables grows as mesh resolution increases, but they are essentially governed by their underlying rigid-body DoFs (6 per rigid body) only. Is it necessary to assign and evolve DoFs at each mesh node?\n\nA2. The positions and velocities of mesh nodes are essential for modeling the interaction and inner-action of objects (eg. contacting, sliding, and pressing), initiating collision events, and performing dynamics calculations. Although the number of these variables increases with higher grid point density, it represents a trade-off between computational cost and performance. Furthermore, the degrees of freedom of surface and interior mesh nodes enable more refined modeling of dynamic phenomena such as compression, friction, and adhesion. Therefore, unless there is a complete absence of interaction between objects, recording the states of these nodes is beneficial.\n\n> Q3. Eqns. 3-4 roughly captures the linear motion of the “center of mass” at each rigid body. Using them as the object-level state does not seem to capture the angular motions and angular velocities of each object. Missing angular information on the object level is a bit counter-intuitive from a physics perspective.\n\nA3. In Eqns. 3-4, linear motion is explicitly recorded at the object nodes of rigid bodies because $x$ and $v$ directly influences the state updating of simulation ($x_i^{t+1}$ is largely determined by $x_i^{t}$and $v_i^{t}$). Although angular momentum and angular velocity are also important, they can be implicitly embedded with other valuable information in hidden state $h^t$, rather than explicitly represented. The model learns to express and utilize them in an optimal way.\n\n> Q4. The collision events visualized in the figures seem between parametric surfaces (cubes and spheres) only. Such collision events could be easily resolved with closed-form solutions. These examples do not seem to show the benefits of having a mesh representation in collision detection. A more complicated scene with multiple organic surfaces would be a better example to necessitate these meshes.\n\nA4. More visualization with more complex objects can be found in Figure.1 and Figure. 7. Specifically, Figure 1 showcases complex objects such as tower-shaped and torus-shaped objects. Our EGODE is designed compatible with rigid bodies of arbitrary shapes, not limited to parametric surface objects. Moreover, our baseline Physion dataset includes objects with various geometries and trajectories of various angles of collision and contact, to validate the generalization capability of our proposed EGODE. Additionally, the results of a complex scenario like Drape, which models interactions between deformable objects and rigid bodies, are reported in Table 3. The visualization of complex scenarios like Drape will be added in the revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The submission initially received mixed reviews; the authors did a great job during the rebuttal, after which all reviewers became positive about the submission.  The AC agrees with the recommendations.  The authors should incorporate the rebuttal into the camera ready.
Decision: Accept (poster)