Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes the use of local coordinate frames for dynamics modelling in graph neural networks. 6dof pose, with angles estimated using translational velocity, is used to determine object pose, and graphs constructed in coordinate frames of these objects. Standard GNN dynamics models can then be trained, with the addition of an anisotrophic filter. Results on a range of experimental settings (synthetic, charged particles, mocap) show that dynamics modelling in local coordinate frames is highly effective and results in improved performance when compared with the arbitrary global frames currently in use.
main_review: Originality:

The idea of dynamics modelling in local coordinate frames is not new, but this is the first work I have seen to do so in a graph neural network.

Quality:

The paper is clear and well structured, and experiments are concise but thorough, clearly highlighting the value of the proposed approach. Ablations show that coordinate frame changes bring about the greatest improvement in performance. This was the best paper in my set.

Clarity:

The paper is clearly laid out, but I found it extremely formally written, and in many cases simple ideas are described using jargon and terminology that "...obfuscates and impresses rather than clarifies."  to quote troubling trends in machine learning scholarship, Lipton 18. This is an extremely simple, but clearly effective idea, but I found that the paper hides this under layers of unnecessary complexity.

Significance:

I believe the performance improvements obtained by using local coordinate frames are significant enough to warrant publication, and that there is enough in this paper for interested parties to build on.
limitations_and_societal_impact: The paper briefly discussed limitations, although I would have liked more detail on failure modes, downsides - when will it fail, is it vulnerable to gimbal lock, is the pose estimation affected by symmetry's, are there better orientation representations, etc.

There may be potential societal impacts for tracking applications which could adopt this work, but I believe this work is still in such an early research stage that this is unlikely to be the case.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes a to use local coordinate system per object in a spatio-temporal graph (with a GNN) of multiple object in order to learn and predict the dynamics of the objects interaction. This enables adding invariance, equivariance, and use anisotropic filters between GNN layers to improve learning performance.

main_review: Strengths
+ The addition seems simple and useful for a wide range of problems, at least ones that use graph node features as euclidean states of position and velocity.
+ Experiments demonstration notable improvement over baselines.
+ Paper is mostly well motivated (exploit invariance and symmetry for dynamic tasks, prior work is focused on static datasets) and clearly explained.

Weaknesses
- The overall contribution falls a bit on the weaker side. Besides the insight to swap to a local coordinate system, Section 3.1 is mostly simple linear algebra and rigid body mechanics, commonly employed in graphics, vision, and robotics. Related to that, since rotation matrices are already used why not consider working with SE2 / SE3 Lie groups and make the math less clunky?

- Section 3.2, 'In practice, we do not always have perfect information about the object states,' what application is this with respect to? Is the idea to find the next state from the current velocity i.e. the temporal model assumes a linear dynamical system?

- An example in the introduction would help ground where symmetry arises and why it leads to subpar leaning. '... exhibit symmetries that if left to their own devices lead models to subpar learning.'

- More insight with examples would be helpful in explaining how the paper converged to idea of speed normalization?

limitations_and_societal_impact: Limitations were adequately discussed.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper investigates the problem of simultaneous learning of interactions and dynamics of dynamical systems of multiple interacting objects. The proposed method is built upon dynamics NRI and incorporate a design Roto-translated invariance. The key of the Roto-translated invariance design is a local coordinate frame design - one for each object. The feature vectors that are fed into GNN are based on local coordinate frame representations instead of global coordinate frame representations. The results shows a clear advantage over previous methods. 
main_review: The paper leverages symmetries and built it into the computation graph to improve the learning of interactions and dynamics. The idea is straight forward, but the writing of the paper need to be improved to make it easier for readers to appreciate it. For example, Figure 1 is very helpful for people to understand the high-level idea, but it is too far away from the presentation of the idea (Section 3.1) and it is not referred to in the text. The authors refer to Figure 6 in Section 5.2 and Figure 7 in Section 5.3, but those figures are not in the main text. It seems they are not referring to Figure 6 and 7 in the supplementary materials either. 

I have one key question: Your augmented state only consists of linear position, angular position and linear velocity. You are not including angular velocity. I guess this is because you assume they are point mass and cannot rotate. In your examples, the particles indeed cannot rotate. If this is the case, I wonder why including the angular position and the rotation transformation is necessary. In other words, what if you only have the translation transformation to form each local frame? In your framework, this reduces to the case that you don't have angular position data and you approximate the angular positions as the global x direction instead of the direction of velocity vector. I think it would be good to add it as an ablation study so that we know the importance of rotation translation. Intuitively I can't think of any reason rotation transformation would help if you only have particles instead of 3D rigid bodies. 

One nitpik: Figure 1 is a little misleading since the visual orientation of the cartoons are fixed across figures. For example, the polygon is visually at the same orientation even though the arrow indicates the orientations should be different. It's easier to change them all to circles but I guess the authors use different shapes to be color-blind friendly. 

limitations_and_societal_impact: Yes, the authors have addressed the limitations. Societal impact doesn't apply in this case.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper presents an approach to  enable roto-translation invariant edge prediction and roto-translation equivariant trajectory forecasting. Rotation and translation invariants are achieved by transforming global coordinate into local coordinate frames of each object in the graph neural network. Experiments are done on forecasting trajectories of charged particles, traffics and human motions.
main_review: Pro:

1. The idea of representing neighbors coordinate in its own local coordinate frame is simple and straightforward. And the improvement seems substantial in the experiments presented.

2. Detailed derivation provided to support the soundness of the method.

Questions

1. It is not clear to me how you get the angular position approximation when the velocity is 0.

2. I am also not sure how you can represent full 3D orientation using euler angle only since it will encounter singularity at some point.

3. Figure 3 doesn't really tell us anything without the ground truth. Even though more data are provided in the appendix, figure 3 seems redundant. 

4. I don't follow why we are doing a speed normalization ablation on dNRI, shouldn't it perform on LoCS? i.e, compare LoCS with or without speed normalization.

5. What does GRU in Figure 4b, Figure 5c refer to?
limitations_and_societal_impact: Yes, the authors address the limitation.
review_title: null

Meta-review: As all reviewers agree, the paper discusses an interesting approach to the problem of learning complex dynamical systems with notable experimental supports. Although some reviewers have concerns on the technical part, the authors' responses have resolved those. The paper still have some weak points, especially in the presentation, but I think I can expect the authors modify the paper in the camera-ready by reflecting the discussion. Based on these, I recommend acceptance (poster) for this paper.

Decision: Accept (Poster)