Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The manuscript improves the equivalent graph neural network (GNN) to tackle the possible inefficiency when the model predicts the dynamics of a physical system when the system symmetry is partially broken by an external force such as gravity.  The subequivariant GNN (SGNN) proposed in this work introduces the hierarchical architecture in terms of the particles and objects where the subequivariant message passing is incorporated into the model to deal with complicated object interactions. The approach with the additional freedom can be more accurate in the task to evaluate the physical dynamic of objects from vision and achieve an impressive generalization compared with GNS model etc. 
strengths_and_weaknesses: Originality:  The work introduces the concept of subequivariance based on the hierarchical network to extend the capability of GNN in the task of learning the physical dynamics from vision information. The work has very good originality. 

Quality: The model is developed rigorously based on the understanding of the rotation equivariant group. The design and test of SGNN with Physion dataset are thoughtful and convincing. 

Clarity: The presentation of the manuscript is clear. However, the background and the motivation are confusing. It is a bit hard to understand the challenge of the problem until I read the online introduction of Physion dataset. 

Significance: Though I may not consider this work as a breakthrough of GNN model development in the focused task, the fresh idea, such as the subequivariance described in the manuscript, indeed helps the neural network achieve better accuracy and generalization. The work can bring people to pay attention to the importance of developing a model with appropriate physical constrain for a similar task.  

questions: Q1: As mentioned previously, the author should make a better introduction to make people easy to understand the background and motivation as well as the challenge of the task. 

Q2: It seems the work is similar to the development of the machine learning force field (MLFF) in molecular dynamic simulation. Probably, the task predicting the physical dynamic of the objects has less information from the vision system than that can be obtained in an atomic system. It would be better if the author could clarify the difference between the two tasks. 

Q3: If the SGNN mode is capable of predicting the force field of the atomic system, it would be interesting to compare with the MLFF models. There are many recently developed models such as Nequip and TorchMD incorporating the rotation equivariance and achieving impressive high accuracy while predicting the atomic force field.

Q4:  Besides Physion, is there any other dataset for further verifying the performance of the model? Unlike SGNN, it seems the compared models, such as GMN and EGNN are general models and not specifically developed for this task. The comparison may not be fair. It would be better to compare with the model which is dedicated to this task. Otherwise, the SGNN should be compared with other models in different tasks.

limitations: Yes. This is a work dedicated to developing the machine learning model for science. There would be no potential negative societal impact.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper targets an interesting question of embedding equivariance into graph neural networks for physical dynamics recovery. The main focus is to 1) relax the strict constraint for cases with gravity and 2) consider the differences in self- and mutual interactions during learning.
strengths_and_weaknesses: This work targets the learning of physical system dynamics, which is an important topic and needs more attention in the community. The authors provide a clear illustration of the relevance to existing works on 1) graph neural networks to model interactions among particles/objects and 2) using physical constraints as an inductive bias to improve generalizability. The authors mainly propose subequivariance against the case with gravity and design multi-stage modeling to account for differences in object properties such as shape. To the best of the review’s knowledge, this work is new. 


questions: The review has some questions about the work.
1.	In Sec. 3.1. the authors define the problem target as to predict the position of the next time step (x^{t+1), where the position of the current time step is input (x^t). The reviewer would like to know if the proposed method is generalizable to a broader problem setup. There are many physical systems whose modeling can be seen as a mapping from one input quantity to one output quantity (xy). The graph structure is also applicable. How to generalize the proposed method then?
2.	The structure of the proposed Subequivariant Graph Neural Networks needs to be clarified. The reviewer suggests adding a flow chart around Sec. 3.2 -3.3. 


limitations: Please see the questions above.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: In this work, authors present a new formulation of the equivariant graph neural network, namely, subequivariant GNN (SGNN), which allows the modeling of systems with symmetry breakage such as gravity. In addition, to model systems with different shapes and geometry, an additional feature that represent the object type is added so as to distinguish the intra-object (elasticity/rigidity/plasticity) and inter-object interactions (collision/repulsion/weak attraction). A hierarchical modeling approach is implemented to address particle- and object-level interactions separately. By considering different physical scenarios, for instance, collision and contact prediction, the superior performance of SGNN is demonstrated.
strengths_and_weaknesses: Overall, the work is well-written and clearly presented. It builds on the equivariant GNNs and modifies it to present the idea of SGNN, which can include directional symmetry breaking such as gravity. TSome of the main comments regarding the work are as follows. 
1) While the idea is useful, the implementation and proofs are fairly straightforward extension from the equivariant case. Also, no additional inductive biases to preserve the physics (as in the case of Hamiltonian or other physics-informed GNNs) are implemented. This raises a question on the validity of the trajectory predicted. Specifically, no comments on whether the trajectories represent a physically feasible realization is not discussed. This is important because one of the major advantages the authors claim for SGNN is the ability to "learn" the dynamics. 
2) Authors refer to previous works such as Hamiltonian GNNs, and mention that they do not consider rotational equivariance. This is incorrect. In Hamiltonian GNNs, the edge embeddings can be modified to have the distance (L2 norm of the difference in positions), instead of giving simply the vectorial difference. Since the functional form to be learned in the case of a Hamiltonian is scalar, this approach also works very well and is both translationally and rotationally invariant. 
3) The examples demonstrated in the work are those, where contact seems to be of main interest. It is not clear how much better the model would perform in other cases where contact is not necessarily the primary interest but dynamics is. Now, if the main focus of the work is simulate contact, then there has been several works which attempted to do this (for instance, Zhong, Y.D., Dey, B. and Chakraborty, A., 2021. Extending lagrangian and hamiltonian neural networks with differentiable contact models. Advances in Neural Information Processing Systems, 34, pp.21910-21922.), which employ a similar to idea of cutoff-based contact detection. Indeed, they do not employ a graph-based approach. However, in the present case although the particles are considered, there do not seem to be any deformation simulated and hence, these approaches referred should also stand equally valid as SGNN.
4) Also, the experiments chosen seem to be favorable for SGNN in comparison to the other baselines. For instance, implementing EGNN and GMN, exactly as they are, with gravity is expected to yield poor performance as the architecture expects the data to be rotationally invariant. Similarly, GNS and DPI learns purely from data and hence are unaware of symmetry unless trained specifically for it. More interesting examples where the situations where GNS and EGNN have been shown to yield SOTA performance could give a more realistic representation of how better SGNN is in comparison to these models.
questions: Some of the questions that naturally follow from the previous sections and some additional questions the authors should address are mentioned below.
1) Since one of the main aim of the present work is to simulate realistic contact models, additional baselines which employ contact models in the Lagrangian/Hamiltonian neural network framework may be considered. Indeed, they may not scale as the GNNs, but they can potentially give improved conservation of physical laws than purely data-driven approaches. Also, this will provide insights into the deficiencies of SGNN.
2) Authors have not evaluated how realistic the trajectory is with respect to the physical laws. For instance, is the energy and momentum conserved in the collision, is the coefficient of restitution 1, etc. These are important to analyze, since the claim is that the SGNN provides improved dynamics (Q2 in results). 
3) Again, since the contact is of main concern in the present work, authors should evaluate how the model performs in situations where there is self contact. At present, it is not clear whether the formulation is capable of dealing with such scenarios.
limitations: There are several limitations for the present work, which the authors should consider.
1) Although a particle-based approach is employed, unlike previous works such GNS, there are no discussions on deformable systems. It is not clear how much improvement SGNN can give for deformable systems.
2) As mentioned in the questions, discussions on self-contact and how this is incorporated is missing.
3) Performance of SGNN on systems with drag and other dissipative forces are lacking.
4) In the case of a deformable system, consider a scenario where a particle from a given object breaks away, gets reflected by the wall, and then comes back to interact with the same object. In such case, if the particle comes with the \varepsilon cutoff distance, does SGNN model this as a contact or as a particle of the object. In other words, does the particle "heal" with the object or not. If yes, this is unphysical. It is not clear if the model can address this issue.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper introduces a new model for learning physical dynamics. It introduces a concept that is termed subequivariance, which appears similar to restricted representations from group theory. The paper also introduces a hierarchical message passing to better enable long range interactions between particles. The paper also expands the input feature space by computing geometric features of the input particles and their objects. The paper experiments on a range of dynamics datasets, namely Physion and RigidFall. 
strengths_and_weaknesses: Strengths:
* The paper adds a "binary operation" which is the difference in position between adjacent features which adds a translation invariant representation to the input feature space, although the benefits and justification for this are not adequately detailed. 
* The experiments seem thorough and multiple baselines are used, although I am not famailir with these baselines so I cannot comment on whether any relevant baselines have been missed.

Weaknesses:

[major]
* A key concept of the paper is characterising "subequivariance" as a relaxation of equivariance focusing on the group E(3). Although this concept has already been considered previously in [1,2] by using restricted representations of the group rather than the regular representation.
* A criticism of GNNs made in the paper is that they seldom explicitly involve the object information into the message passing, but this feels like a specific decision choice of the input features rather than some fundamental issue of GNNs which a paper can solve. Also, there are papers which consider specific object information such as [3].
* The concept of having different edges which pass different information appears strongly related to a relevant sub-field of GNNs exploring automorphism equivariance which has been ignored in this paper [4,5,6].
* In the experiments only rotations around the gravity axis are used. Surely, all rotations should be possible and all the models which do not correctly account for the break in symmetry caused by gravity should produce unrealistic results, while if this model works correctly it should break the symmetry and produce realistic results. I feel like this would be a far stronger result proving the correctness of the model. 

[minor]
* Multiple typos / grammatical mistakes makes reading the paper difficult. (L44, 59)
* The object features appear to be the pooled features of its particles, which is used to create a feature space by subtracting these from each of the particles features. This seems to be a complexly worded way of saying subtracting the mean from the particle features which is a common approach in ML.


[1] Weiler, M. and Cesa, G., 2019. General e (2)-equivariant steerable cnns. Advances in Neural Information Processing Systems, 32.
[2] Cesa, G., Lang, L. and Weiler, M., 2021, September. A Program to Build E (N)-Equivariant Steerable CNNs. In International Conference on Learning Representations.
[3] Pfaff, T., Fortunato, M., Sanchez-Gonzalez, A. and Battaglia, P.W., 2020. Learning mesh-based simulation with graph networks. arXiv preprint arXiv:2010.03409.
[4] de Haan, P., Cohen, T.S. and Welling, M., 2020. Natural graph networks. Advances in Neural Information Processing Systems, 33, pp.3636-3646.
[5] Thiede, E., Zhou, W. and Kondor, R., 2021. Autobahn: Automorphism-based graph neural nets. Advances in Neural Information Processing Systems, 34, pp.29922-29934.
[6] Mitton, J. and Murray-Smith, R., 2021. Local Permutation Equivariance For Graph Neural Networks. arXiv preprint arXiv:2111.11840.
questions: * How does subequivariance differs from using the restricted representation?
* How does including object information differ from being a specific feature space choice?
* How does the choice of using different edges to pass between objects/particles differ from an automorphism equivariant network?
* Why are only rotations applied around the gravity axis in the experiments?
limitations: N/A
review_title: null

Meta-review: metareview: Overall this is an interesting paper. It proposed a new formulation of the equivariant graph neural network, subequivariant GNN. Reviewers agree that the proposed idea could be useful to the community, albeit with perhaps small application scope. So on the novelty side, this paper is okay. The biggest concern among the reviewers is about the experiments, i.e., mostly the fair comparision. I feel the authors did a reasonable job to explaining why the current baselines were chosen and provided additional experimental evidence. Authors could take in account the comments from the reviewers to improve the overall presentation of the paper.
confidence: Less certain
Decision: Accept