Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposes a new graph neural network model for predicting the dynamics of n-body systems. The two main innovations of this model are:
1. Ability to handle systems with rigid-body constraints. 
2. A new 3D transformation equivariant layer with universality properties.  
main_review: # Strenghts 

- The introduction does a good job of motivating the need for the proposed work. 
- The description of EGNN and the preliminary material more generally provide a good understanding of the relevant prior work. 
- The model essentially relies on an object-centric view and the dynamics of each object are treated unitarily. This could also be thought of as a novel instance of object-centric-based learning. 
- The theoretical analysis around the newly proposed equivariant layer is nice and useful. 
- Since the EGNN layer is a particular case of the proposed one, the derived result also has theoretical implications for this prior work and improves our understanding of its limitations. 
- The evaluation equips all the baselines with minimal information about the presence of the rigid objects (i.e. the edge-type indicator functions). This ensures a certain level of fairness of the evaluation methodology. 
- I like that the evaluation is nicely factorized into different settings (e.g. small vs large training sets and results across different types of systems). This provides further insights into how the model performs in different regimes. 
- Table 2 provides a nice empirical confirmation that the model works as expected and that the constraints are satisfied. 
- The generalization of the proposed models to novel system configurations is tested in Table 3 and the results look good. 
- The ablation study confirms the intuition / theoretical motivations provided for various design decisions described in earlier parts of the paper. 

# Weaknesses

-  The evaluation only looks at a single simulated dataset. While these simulations are nice and insightful, I would have expected some real-world evaluation as well. Even the original work of Kipf considered a few real-world datasets. For instance, the authors could have looked at some motion capture data where different parts of the human body could be treated as rigid-body constraints. 
- Related to the point above, the forward kinematics part of the model relies on having a perfect understanding of the nature of the constraint and relies completely on domain knowledge. This raises two issues: 1) What happens when the constraint is so complicated we might not be able to derive some forward kinematics equations like for the simple hinges and sticks? 2) This additional domain knowledge provides very important information that the baselines do not necessarily have. For these two reasons, it would have been useful to also consider a model where Equation (9) is learned by an abstract learnable function. This would have provided a model less reliant on very specific domain knowledge and more centered on the inductive bias of object-centric modeling.
- The considered constraints are relatively simple (i.e. small rigid bodies) and do not really test how the model can cope with more complex constraints. For instance, the sum in Equation (7) might increase too much in magnitude as the object is formed of many more parts. This naturally brings the question of whether the model can cope in practice with very large rigid objects? 
- None of the reported results have standard errors around them. The experiments should be run at least across multiple initializations in order to assess the statistical significance of the results.  


summary_of_the_review: Balancing the weaknesses and strengths from the list above, the paper is, in my view, at the threshold between acceptance and rejection. Since there is no neutral score I can select, I am voting for weak reject for now, but I am open to changing my score during the discussion period. 

EDIT: Updated my score to 8.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper develops a model that augmented interaction networks with mechanical constraints and develops equivariant message passing schemes to ensure physical realism in the model outputs. The work focuses on a set of toy systems that contains particles connected by sticks and hinges and the developments are made for these systems in particular. There are some results on generalised equivariant message passing and the universality of the proposed method. 
main_review: **Strengths**

The presentation is clear and professional.

The task is well defined and has some physical relevance.

The supplementary materials are extensive and include a quality code repository that is well documented.

**Weaknesses**

There is a lack of novelty and scope in the work. The following problems are fundamental to the contribution and it is not clear to me how the authors could remedy these faults within the rebuttal period unless they would like to flatly refute the points:

1. Most of the core ideas presented here are captured and applied more widely, including to real-world scenarios such as the case of robotics mentioned in the introduction of this work, in ​​Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning Lutter et al. ICLR 2019. (This reference is missing.)

2. The scope of the theoretical and empirical contributions of the paper are limited to very small systems of sticks and hinges. There are existing works in this area that scale to more complicated rigid systems without the need for hard coded physics (Sanchez-Gonzalez et al. ICML 2018), to large-scale systems of thousands of particles (Sanchez-Gonzalez et al. ICML 2020), and to systems with constraints (Pfaff ICLR 2021). The contributions are thus quite limited in the context of the achievements of the field (and in particular when the work of Lutter et al. is considered.)

3. The general formulation of an equivariant message passing layer given in Equation 11 does appear novel, although the actual use case with $Z = (f,x,v) = (ma, x, v)$ is very close to the extension given by ​​Satorras et al. (EGNN) in their equation 7 that shows how to extend their method to using velocities. It is trivial to extend their method to include an equivariant treatment of accelerations ($v^{l+1}_i = v^{l}_i + a^{l+1}_i$). It is also in the spirit of the work presented here to include such a physical inductive bias (i.e. that the rate of change of the velocity $v$ is set to be exactly the acceleration $a$ and so on).

All the tables should include errors. I appreciate that the results suggest that the GMN is the dominant model in this system but it is still necessary to indicate the variation in the recorded performances.

It is also not correct to state that ‘GMN meets constraints spontaneously.’ The constraint satisfaction has been hand crafted into the model whereas the compared models are required to learn the constraints. This is nearly a minor point but it does make an incorrect claim.

The self-containment of the paper, as described in Appendix B with reference to the derivation of the dynamics of sticks and hinges, whilst admirable, is not relevant to the novelty or quality of the scientific contribution of the work.

**Minor comments**

The following notes are very minor and may be subjective and as such I have not factored them in to my score for the paper. I would not update the score in either direction if the authors ignore or act on these notes, they're just things that I spotted as I went through the paper.
- It's unclear to me what the first sentence of the abstract is trying to say. 'Prevailing yet challenging' is a little confusing to me, is it not the case that challenging topics are also prevalent?
- 'mainly stem from that...' in the second sentence of the abstract needs to be rephrased
- 'just like what physicists used to do' in the introduction is a little childish, though perhaps we should wonder what it is that physicists are now getting up to
- 'Coulombian force' is an unusual variation on 'Coulomb force'
- Page 4 'to inference the acceleration' should be 'to infer'


### Rebuttal Update
I have raised my score (3->5) in light of the additional experiments and lowered my confidence (3->2) as I have not been able to engage as thoroughly as the other reviewers who are apparently satisfied with the paper.
summary_of_the_review: The ideas presented are physically well motivated but only incrementally advance on the state of the field by combining many existing ideas, and the application scope is very narrow so it is not the case that the paper derives a significant contribution by showing that these techniques taken together are greater than the sum of their parts etc. The presentation is very professional and their is some novelty in extending the equivariant message passing scheme of Satorras et al., but these are not strong enough contributions to justify acceptance alone.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper focuses on the formulation of a graph neural network approach towards the learning of update rules of constrained systems, extending prior work (Interaction entwork, EGNN) by learning the updates of constrained components in the generalized coordinates expressing those constraints and incorporating forward and inverse kinematics in the algorithm.
Comparisons are made with Linear, Basic GNN, TensorForce Networks, SE3-Transformers, Radial Field and EGNNs by evaluating them on a constrained version of the N-body problem introduced by Kipf et al. 2018 and it is shown that the proposed method better adheres to constraints and generalizes better on the studied problem.
main_review: Strengths:

The paper is decently clearly written (modulo some typos/phrasings noted below, but nothing that impedes the understanding of the reader), provides all necessary proofs and makes 2 main contributions

- formulating an algorithm which incorporates Generalized Coordinates and inverse/forward kinematics
- proposes an extension of the EGNN framework from working on single vectors to matrices, including some proofs about the equivariance and expressivity.


Incorporating the constraints clearly helps generalization on  the experiments

Weaknesses:

- My main criticism is with regards to lack of novelty/generality: one has to manually derive the forward/inverse kinematics of the constrained system, which requires knowledge of the underlying physics already...at which point one might want to go one step further and derive some approximation as well? Alternatively, I would have expected the incorporation of the generalized coordinates to be done in a more black box fashion, i.e. encoding the knowledge that there are *some* generalized coordinates in the architecture and letting the models derive *which* we are talking about (this could possibly be attempted with invertible neural networks for the kinematics and some form of constraint-seeking algorithm that tries to minimize the total number of dimensions used in the generalized coordinate latent for example?)
- only studied on a single setting, but this is a minor nitpick as it's similar to other previous works
- reporting the constraint violation in the main body is only marginally meaningful as of course a system which enforces the constraints *by design* will achieve 0 constraint violation and that part in particular is not learned. 

Minor nitpick:
- the paper mentions rotation and translation in the intro but then focuses mainly on rotation? doesn't change much for the final results but it confused me a bit 


Typos/phrasings  (not accounted for in my review, just as feedback for the authors):

- page 2: "have achieved desired performance on the N-body system...lacks of constraint" is meant to  say "have achieved desirable performance ....lack constraints"?
- page 3 "is proportion to the its" "is proportional to the" ?
- page 4: "despite the desired performance" => "desireable" again?
- page 5: "can be generalized to the function with multiple input vectors" => to functions with multiple input vectors? 
summary_of_the_review: The paper presents and incremental improvement over previous works and was done cleanly, so a marginal accept. If it had been exploring a fully learned generalized coordinate transformation that uses an inductive bias but does not need to hand design the coordinates and kinematics I'd give a higher score.

://EDIT: updated my score
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors propose Graph Mechanics Networks (GMN), a E(n) equivariant model that explicitly models systems with rigid constraints, with exact constraint preservation by means of using generalized coordinates. In my opinion the main contributions are as follows:

C1. Establish math for writing models that operate on cartesian coordinates, make predictions in very generalized coordinates and then converted back to cartesian. This is an approach that has been used in previous work, but is usually hidden as an implementation detail.

C2. Extend the E(n) (Satorras et al.) equivariant message function to take and output a variable number of vectors with a proof demonstrating that the approach is flexible enough to actually approximate any E(n) equivariant function. I think this is a novel low level modeling contribution.

C3. Extent the E(n) (Satorras et al.) model to build an E(n) equivariant model for exact constraint preservation. Although a bit niche, this is the first model of that nature.
main_review: Strengths:

S1. The GMN model delivers exactly what it promises: exact constraint preservation, with E(n) equivariance. 

S2. Claim C2, seems like a good idea in general. In fact, leaving hard constraints aside, it seems like a nice alternative to model in Eq. (7) of the E(n) paper, to be able to directly feed the velocity vector to the message function, and perhaps give more expressive power.

S2. The model shows expected advantages in terms of data efficiency over the baselines.

S3. The model shows expected out of distribution generalization properties.

Weaknesses:

W1. The model requires specifying the forward kinematics function for the constraint (to go from generalized positions and velocities to cartesian positions and velocities) as a differentiable function, which in many cases is of a big ask. Furthermore, the approach for writing this constraints does not seem very compositional, for example two separate analyses (forward functions, sets of generalized coordinates, etc) are used to distinguish the hinge and the stick constraints, when in practice a hinge is pretty much two sticks where one of the particles is shared. Would it not be possible to somehow share some of these mechanisms between the hinge and the stick (e.g. just writing the hinge as two sticks in sequence?).

W2. The work focuses a lot on the specific combination of adding exact constraint preservation to the E(N) equivariant model. I believe in most cases the E(n) equivariant model is often used for domains where hard mechanical constraints are not that common (e.g. molecule property prediction, or force prediction). I think it is great to extend the E(N) model this way, but I am not 100% sure how many practitioners interested on the E(n) model will actually end up using the constrain part.

W3. On the other hand there are other baselines from the literature that have looked more explicitly into mechanical systems with hard constraints, which are not currently referenced in the paper. For example, https://arxiv.org/pdf/1810.01566.pdf models a fluid with a floating rigid box made of 64 particles by making a whole prediction for the canonical coordinates of the whole box, and then using the forward kinematic model of the box to obtain the cartesian positions, with an approach that is very similar to equations (6,7,8,9) in this work, so this should probably be referenced. Another common strategy for learning constraints without having to specify the forward kinematics function is to add noise to the training dataset, and ask the model to correct the noise (mujoco rigid systems: https://arxiv.org/pdf/1806.01242.pdf, particle systems: https://arxiv.org/abs/2002.09405, meshes https://arxiv.org/abs/2010.03409), so the model learns to correct constraint violations, and keep constraint violations low during a long rollout, so this should probably be used as a baseline (giving the length of the sticks as an additional static feature). I believe the approach proposed in this paper would probably still do better than training with noise, but it is also much more complex in terms of assumptions that it makes, so it would be an interesting trade off to show. Note that in these type of domains full rotational equivariance is usually not as important since gravity often breaks the symmetry for at least one of the axes.

Other minor comments/observations:

O1. "first recording data and then inducting formulas". If feel the this sentence is a bit extraneous to the paper contributions, but if you want to make that point, probably worth referencing this: https://arxiv.org/abs/2006.11287

O2. "However, all above approaches have ignored the symmetry in physics". There is a large subset of models that partially implement symmetries by enforcing translation equivariance but not rotation equivariance, which should probably be mentioned and referenced in the related work (e.g. https://openreview.net/pdf?id=B1lDoJSYDH, https://arxiv.org/abs/2002.09405, https://arxiv.org/abs/2010.03409).

O3. "Euclidean transformation (translation or rotation) of the input states". I think "reflection" should be part of that list too, since the paper uses E(n) assumptions for the proofs (otherwise this would involve the SE(n) group, and I think the appendix proof would not hold).

O4. I think equation (2) is technically incorrect, since in the E(n) paper the function used to update the hidden node embedding (phi_h in e.q. 6 from E(n) paper), is applied after the sum aggregation, which is not possible in Eq(2). Similarly, the two outputs in eq (5) are confusing, since the actual functional form of phi for vector and non vector outputs will end up having different functional form (as per the last sentence of section 3.3, which again does not clarify whether rho_w is shared or not between the vector and scalars outputs).

O5. Corollary 1. "assume the entries of Z are drawn independently", actually this is quite a strong assumption, there may be many cases in systems where forces, velocities and relative positions are actually parallel to each other (e.g. an object falling in a uniform field with initial zero velocity), so the rank of the matrix would be 1. So probably worth skipping Corollary 1 and just going to Corollary 2.

O6. "the generalized coordinates include the states of particle 0 denoted as ... R^3  the rotation Euler angles of stick 01 ... R^3 ... and 02 .. R3". Actually, if the particle 0 has only 3 degrees of freedom this means they are point particles (e.g. no solid rotation), so would not this mean then sticks 01 and 02, would only have 2 generalized coordinates each (e.g. latitude and longitude), instead of 3?

O7. "The edge feature is provided by a concatenation of the product of charges" Did the model had any trouble learning to perform this product if instead just giving the charge as a node feature of the inputs?

O8. "EGNN deliver much worse performance when the training size is small". Do you have an intuition of why GMN is more data efficient than EGNN, considering both models are similarly equivariant?

O9. Table 3. "are comparable with the original environment" would be possible to add an extra row with the performance GMN trained on those datasets (e.g. what the test performance would be if the model actually saw data from that distribution during training?).

O10. Table 4. Is there an interpretation of why the shared model works worse for the phi_2 that receives the three inputs than the unshared one, but in the case of phi_2 receiving just one input is the other way around?


Small typos/grammar:
"Considering constraints, by no means, is not easy" weird grammar.
"How the size of the training set influences." --> Data efficiency
"We have introduced an normalization" (typo)


summary_of_the_review: The paper has a reasonable balance of novel contributions along different axes. I am not sure any one of them is sufficiently novel/interesting to meet the ICLR bar on its own, but put together and following corresponding revisions, I think the could make make a bigger ICLR-worthy contribution. My only concern is that it may be hard to reach the right audience, since I suspect many readers would be interested on each of the contributions individually, but not sure many will be interested on all 3 of them simultaneously. I would perhaps recommend including "E(n)" or "equivariant" in the title of the paper to reach a larger audience.
review_title: null

Meta-review: The manuscript develops a new kind of graph neural network (a Graph Mechanics Network; GMN) that is particularly well suited to representing and making predictions about physical mechanics systems (and data with similar structure). It does so by developing a way to build geometric constraints implicitly and naturally into the forward kinematics of the network, while still allowing for effective learning from data. The manuscript proves some essential properties of the new architecture and runs experiments both with simulated particles, hinges, sticks (and their combination), as well as with motion capture data. 
Reviewers were generally impressed by the writing and clarity of the work, as well as the main results. In addition, in those cases where reviewers thought that the experiments were lacking, the authors delivered effective new experiments to address those concerns (e.g. looking  at mocap and molecular datasets). One reviewer initially scores the manuscript as a Reject/3 on the basis of concerns about novelty and the scope of the theoretical and experimental contributions of the paper. However, they adjust their score 3->5 based on the rebuttal presented by the authors (including new experiments). The reviewer also downgrades their certainty (from 3->2) on the basis of the engagement from reviewers offering higher scores.
Overall, the manuscript presents a promising contribution to the graph networks literature and I agree with the general consensus in favour of publication.
Decision: Accept (Poster)