Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors introduce a Lagrangian graph neural network to predict the trajectory of a rigid body. The rigid body is represented as a graph where edges represent rigid bodies – e.g., links in a chain – and nodes represent connections between those rigid bodies. The position and velocity of each node is assumed to be given. From the input graph, the authors construct two graph neural networks which are trained simultaneously. One learns to predict the kinetic energy of the rigid body and the other learns to predict the potential energy of the rigid body and the Lagrangian is formed by computing the difference of these quantities. The networks are trained by minimizing the difference between the predicted acceleration (determined by the EL equations) and the ground truth acceleration of the system. Against previous work, the proposed approach shows strong predictive performance and an ability to generalizability to unseen structures. The mass and moment of inertia of all members in the graph are the same irrespective of their length.
strengths_and_weaknesses: - Originality: The method appears to be new. The related work is well organized and the proposed approach is adequately situated in the context of existing methods.
- Quality: The submission appears to be technically sound, with the claims well supported by the empirical analysis.
- Clarity: The paper is well written and organized
- Significance: The paper addresses the issue of predicting physically plausible trajectories of a rigid body system, a challenging problem which has the potential to impact scientific discovery. However, there are several limitations of the approach, i.e., The mass and moment of inertia of all members in the graph are the same, and the constraints of the system are assumed to be known.
questions: - Questions:
  - [55] should it be \psi = sin^(-1)((y_i - y_{i-1})/x_i - x_{i-1}))?
  - Eqn 5: it looks like the kinetic energy is mass times position, shouldn’t it be mass times velocity
  - [208] what is || in eqn 9?
  - [309] How does the learned mass matrix compare to the gt mass matrix?
- Comments:
  - [47] Possibly relevant citations [1].
  - The notation in eq 5 is a bit challenging (e.g., x_i^cm2)
  - Consider using \mathcal{l}(\ddot{q}, \hat{\ddot{q}}) for the loss since \mathcal{L} is used for Lagrangian
- Possible typos:
  - [37] abbreviation EL is used before it is defined
  - [143] “This is because…”
  - [152] “significantly smaller that” → “significantly smaller than”
  - [184, 185] node → nodes
  - [211] N_i = {u_j | (u_i, u_j) \in E}
  - [215] “denoted as” → “denoted by”

[1] Duong, Thai, and Nikolay Atanasov. "Hamiltonian-based Neural ODE Networks on the SE (3) Manifold For Dynamics Learning and Control." Robotics: Science and Systems (RSS). 2021.

limitations: The authors note that the proposed approach assumes knowledge of system constraints.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: A novel framework is introduced for Lagrangian Neural Networks using graph networks to model rigid body dynamics. Comparisons are made with classical Lagrangian and Hamiltonian Neural Networks, showing the generalizability to arbitrary topologies and system sizes.
strengths_and_weaknesses: The contributions are very clear, and the authors did a great job explaining their framework for rigid bodies. The benchmarks are clear, and the non-diagonal mass matrices show the capabilities nicely of the model. Some steps could, however, involve a few more explanations for readers that are not familiar with the topic, such as in line 100 the $\nabla_{\dot{q}_i \dot{q}_i}$ operator, would that be a second-order derivative? It is not a notation I have seen before. Also, in line 121 you mention solving and substituting the lambda, it would be great if this derivation were present in the Appendix.

The information presented in Figure 2 and 3 were a bit overlapping, and it could perhaps be combined to make the paper visuals more concise. Figure 4 also has text "ABC" which are not really explained (can be inferred from caption), and ideally the purple lines can be explained in the caption as well.

The extension to the Hamiltonian feels oversimplified, though I'm not sure about the topic. It would absolutely be worth investigating. If no experiments were performed, section 4.2 could be left out.

In line 276 the authors refer to an appendix, but no appendix was present in the file. 

All in all, more results with exciting articulated bodies could strengthen the paper a lot, and especially scenarios such as Pfaff et al. 2020 (Learning mesh-based simulation with Graph Networks) with deforming plate and waving flag can be interesting to try.


questions: 1) Same from before: line 100 the $\nabla_{\dot{q}_i \dot{q}_i}$ operator, would that be a second-order derivative?
2) How did the choice of Section 4.1 come about, where the edges represent rigid bodies and the nodes represent connections between the rigid bodies? What benefit/drawback does this have?
3) Did you perform any experiments on the Hamiltonian extensions? 
4) Figure 5 shows a quasi constant error between the two methods, is there a reason for this? Is this not the initial error propagating throughout the rollout trajectory? 
5) Why is the GNS result in Figure 6 so smooth, while the LGNN is more chaotic?
limitations: The limitations are clearly addressed, though focusing more on that contact modeling is a limitation would be preferred, since the main focus is rigid body mechanics, where contact and collision is essential. Adding some initial ideas towards addressing contact would be appreciated as well.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: In this paper, the authors propose a Lagrangian graph neural network (LGNN) that can learn the dynamics of rigid bodies which can be modelled as a graph. The LGNN consists of two GNNs, which learn the potential energy and kinetic energy of the system Lagrangian respectively. The performance of the model is verified on 2D articulate body simulations based on in-extensible chains and rods. The experiments show that the proposed method can outperform baseline method Graph Neural Simulator (GNS). The generalizability of the model is evaluated on system with larger sizes and unseen topology. 
strengths_and_weaknesses: Strengths 
- Learning the Lagrangian mechanics using GNN is interesting and the experiments results and the videos shown in the supplemental materials are neat.
- The paper is written clearly and well organized.
- The introduction to the background knowledge helps understand the proposed idea.

Weakness
- The method is validated on a simple (less than 20 links) rigid body simulation scenario, where the traditional numerical methods can fairly accurately and effectively solve the problem. Therefore, the key point I am concerning is what the advantage of the learning based method proposed comparing to traditional method. The advantages can be either more efficient than traditional simulations or exploring dynamics the traditional simulations do not easy to capture. However, the comparisons on such cases are missing which makes the contribution somehow hard to evaluate. I think experiments on more large scale or complex scenarios will make the proposed method more convincing.
questions: - The simulations shown can be solved by traditional numerical methods fairly accurately and effectively. I think experiments on larger scale or more complex scenarios where the traditional numerical methods are slow or struggling, will greatly enhance the statements made in the paper.
- The term `rigid body dynamics` is somehow too general I think. Actually the simulations learned here are limited to articulate-body dynamics which can be modelled as a graph. 
- The term `zero-shot generalizability` is mentioned in the paper multiple times but without explanations or references. It will be helpful to explain what the `zero-shot` here indicates.  
- Videos shown in the supplemental materials demonstrate a simulation of 1s physical time. However, in the paper, the Figure 5 and Figure 6 only show the errors  for 0.1s and 0.3s physical time. It will be more convincing to show how the errors evolving during the whole 1s simulation.
- There are no visual results of the simulation shown in the paper. Putting some key snapshots of the videos will help the readers to easily understand the problems to solve here.
- In the appendix A.6 training details, the author mentioned the baseline method Lagrangian Graph Network (LGN), I am wondering what is the difference between the LGN and proposed LGNN?

Minors:
- Mixed usages of `colon` and `period`, e.g., line 221 `loss function` uses colon while other paragraph titles in the same section uses period; the paragraph titles 5.1 use colon but that of 5.2 use period.


limitations: The authors have discussed the limitations in the conclusion to some extent.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper presents a GNN-based Lagrangian formulation for learning the dynamics of chains and ropes, and achieve better accuracy and energy behavior than an unconstrained model (GNS).
strengths_and_weaknesses: Most papers on HNN, LNN approaches only demonstrate results on mass/spring and n-body system, and the premise of using these methods on more complex, realistic systems is alluring. However, while the paper is motivated by the challenges of rigid-body simulation, I'd argue the method doesn't actually tackle any issue that make rigid-body simulation a hard problem, such as collision between complex object shapes, frictional contact, or even 3D rotation for objects with nontrivial inertial tensor. 

Contrary to what the introduction states, 2D ropes can be solved pretty well with simple particle-based methods (see e.g. the 2016 Interaction Nets paper). The non-diagonal mass matrix and inertial tensor is (a) only a problem for method which have an explicit formulation for it (e.g. LNN/HNN), end-to-end learned models (such as plain GNNs) can learn the effect from data and neighborhood relations and (b) the effect disappears in the limit of small edge lengths.

The main contribution of this paper hence is demonstrating that graph-based LNN approaches work for bars/chains/ropes, and make use of the energy-stability of those methods. However, it's worth noting that properties such as length preservation, generalization that the paper shows stem from the fact that very little is learned here; all the constraints (segment length constraint, friction, drag etc.) are manually encoded, only inertial dynamics are learned, and those equations are both universal, and quite straightforward to infer from geometry. Normally you'd want a learned method to do the opposite, i.e. learn complicated interactions/friction models that may be hard to measure for real systems, and manually encode the known priors. The only case I could think of where the proposed approach would be useful is for inferring non-trivial mass distributions which may not be visible (say, on a drawbridge made from composite anisotropic material); but those use cases would need to be demonstrated.

So while I do think there's value in expanding the type of systems that can be tackled with LNN/HNN methods, the paper in its current form is overclaiming its contributions (rigid dynamics) and quite limited in what it adds. It could be extended into a much stronger paper by showing how any of the harder aspects of rigid simulation can be tackled, or how the tricky bits (e.g. constraints, friction, ...) can be learned.
questions: - the terms gamma, a(q) etc. need to be properly defined in the methods section
- why do you even need a full GNN with L layers of message passing? kinetic/potential energy are local, so it at most would need 1 round of collecting neighborhood information
- it sounds like the paper makes a distinction between bars, chains and ropes. In a rigid-body approximation, aren't those exactly the same (the only difference being that ropes having shorter segment lengths than bars) ?
- in the appendix, why is LGN (which I think refers to a _Hamiltonian_ GNN) much worse? The core method is very similar, and I'd expect it to preserve energy very well.
limitations: The method is quite limited in what it can be applied to, which is pointed out briefly in the conclusion section.
review_title: null

Meta-review: metareview: The authors propose the use of two separate networks to learn the kinetic and potential energy of objects made of chains of rigid bodies. Their neural net architecture uses knowledge of constraints in the system, and improves on previous GNS work.
Reviewers pointed out that experiments were too simple with a small number of DoFs where traditional methods already perform very well. The authors replied that their method can learn from trajectories only, and include complex settings, for example with dissipative drag forces. They also added a number of much more complex experiments in a revised version.
Overall, this is a borderline paper given the high standard required for NeuIPS publication. I lean towards recommending this paper for acceptance, with one condition: as pointed out by reviewers VUdP and EADF, the title and abstract should be changed to reflect the fact that a) this paper is not solving "rigid dynamics", but only works on "articulated rigid body"; b) the graph net doesn't actually perform message passing.
confidence: Less certain
Decision: Accept