Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This work presents an Autoregressive Graph Network (AGN) that learns physics models. Given the state of a system for $k$ time-steps, an autoencoder is used to generate node (and edge) embeddings for each particle (and pairs of particles) in the system. Importantly, the autoencoder is constrained to impose an autoregressive structure that captures temporal dependencies. The node and edge embeddings are then input to a message-passing network, whose output is then used to predict the target state.  
Finally, experiments are performed to study the performance of the AGN at multi-step forward prediction on datasets featuring different particle interactions.
strength_and_weaknesses: The experimental results demonstrate that the AGN outperforms the baseline GNs on most datasets presented.

However, some aspects of the approach were not completely clear. 
- In the introduction, the authors mention that the encoder learns the arrow of time. But based on the discussion following Eq 2, it looks like the mask M explicitly induces the desired temporal dependencies? 
- In the last paragraph of section 3.2.1 (VAR encoder), the input to the VAR edge encoder includes the derivatives of the full state: $\dot{\boldsymbol{x}_i}(t) - \dot{\boldsymbol{x}_j}(t)$. According to the definition following Eq 1, these derivative terms already include the target $\ddot{\boldsymbol{r}}$. Is this a typo or intended? Also, why does Eq 5 include both $H_i - H_j$ and $|H_i - H_j|$ terms?
- If the temporal dependence is already captured by the embeddings, why are there $L$ message-passing steps? How much more computationally efficient is the AGN compared to the baseline GNs used in the experiments? 
clarity,_quality,_novelty_and_reproducibility: The description of the approach in Section 3 was a bit difficult to parse. This section could be better developed so that the reader can appreciate the important aspects of the approach.
- In section 3.1, the dimensions of $\boldsymbol{r}$ should be $N \times d$ instead of $Nxd$ ?
- In the text following Eq 2, it should be Eq 2 instead of Eq 1 ?
- I presume $act$ is an activation function. It would be helpful to clearly state so. 
- Terms like $H(x)$ are not clearly defined. It would be useful to clearly define what output it corresponds to. 
- The definition of the quantity $m(s,t)$ is hard to parse, and it is not immediately clear what $m$, $s$, $n$, $p$ stand for
- The input to the MLP edge encoder already contains the acceleration terms, is this correct?

It would also be useful to have a legend or more detailed captions for figures 7 and 8. It looks like the colored traces are the ground truth trajectories, and the gray traces are the estimates? 

There are also several typos and grammatical errors. 
summary_of_the_review: This paper presents a Graph Network that imposes an autoregressive structure on its input embeddings to capture temporal dependencies.  Experimental results indicate that the approach outperforms baseline GNs on multiple datasets. However, a few important aspects of the approach were unclear. Improving the presentation, for section 3 in particular, would help the reader better understand these aspects of the approach.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes an additional autoregressive module to the vanilla graph network-based simulator (GNS), which can outperform GNS while maintaining high efficiency. Specifically, it learns spatiotemporal embedding for simulating spatial-temporal trajectories, which somehow solves the problem of high computational costs caused by long-term memory. It conducts experiments on multiple scenarios such as gravity, spring, charge, etc.,  to demonstrate the effectiveness of the proposed method.   
strength_and_weaknesses: ### Strength
1. The problem of designing graph networks for spatial-temporal simulation is important. 
2. It somehow solves the problem of high computational costs caused by long-term memory. 
3. It outperforms vanilla GNS while maintaining high efficiency.

### Weakness
1. The authors claim that this autoregressive model is better than other sequential models, such as RNN and Transformer, but there is no such comparison. I think the comparisons of both accuracy and efficiency are necessary. I am not fully convinced that this model is better than other commonly-used sequential models. 
2. The idea of the autoregressive model seems not very novel. 
3. The writing can be improved. 
clarity,_quality,_novelty_and_reproducibility: The idea of the autoregressive model is not very novel. This paper lacks some necessary performance comparisons with other sequential models. The codes and data are provided, which is highly appreciated. 
summary_of_the_review: I think the problem is important and the proposed model seems effective, however, my main concern is about the advantage of this model over other sequential models, which is not supported by experiments.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes to add a linear node and edge encoding layer before the message passing steps of a graph network in order to improve inference performance. This encoder is employed for physics problems where the networks have the task to infer accelerations for time integration of the system under consideration, and its main job seems to be a selection of features from current and previous time step information. This approach is demonstrated for four different, but relatively simple types of particle-based systems.

strength_and_weaknesses: The main goal of the paper - more accurate predictions for learned physical systems - is an important and relevant one. Also, the evaluations seem to show improvements for the proposed architectures. However, I see quite a few problems and unclear points with the algorithm and manuscript:

Among others, I was surprised about the focus and attention that is given to the "autoregressive" label. This is a fairly standard approach for time integration methods, and the necessity for multiple previous steps is an inherent drawback of learned algorithms: from classical physics we know that the evolution of a deterministic physics system is uniquely prescribed by a single state. The necessity for working with multiple time steps ("lookback" as its called here) is not necessarily a positive aspect. The use of linear encoder/decoder blocks for graph networks is also not exactly new, e.g., see Stachenfeld et al. 2021 "Learned coarse models".

In addition, I was missing an explanation for why these additional linear layers should have a significant impact on the learning task (beyond adding more weights). This is likewise something that could be handled by the MLPs in the message passing layers. Here the paper also does not make clear how the weight count changes when including the proposed encoders.

clarity,_quality,_novelty_and_reproducibility: As mentioned in the previous section, several aspects regarding motivation and architecture remain unclear. I found the explanations hard to follow, in general. 

The appendix contains many graphs, but doesn't give much new information or analysis. Some figures, like figure 3, also seem to show that there's little advantage gained from the proposed method.

summary_of_the_review: Based on my current understanding of the architecture, I'm not clear whether the improved error measurements stem from more weights or inherent advantages of the encoding layers. So, based on my current understanding, I don't think I can recommend accepting this paper. I would recommend that the authors spend time to more clearly analyze and explain why their method leads to improvements, rather than long lists of graphs. This would also potentially make it possible to assess how well proposed changes would carry over to other settings, e.g., other types of physics simulations.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors propose a new graph network (GN) architecture for learning dynamics models of particle-based systems. Their architecture is structured as a GN applied autoregressively in the temporal direction. They compare their method to various alternative GN architectures (vanilla, physics-inspired) on a few different dynamical systems.
strength_and_weaknesses: Strengths:
1. The proposed method is extensively evaluated, with many experiments on a range of tasks.

Weaknesses:
1. The proposed method does not feel especially novel. One of the main contributions discussed, the concept of "vector autoregression" feels like a trivial extension of existing work, either of [1] by extending it to be autoregressive over time, or of [2] by extending it to particle systems via a GN.
2. Some of the arguments about benefits of the proposed method feel tenuous at best. For example, I have a hard time following the discussion about translation invariance. The authors claim that "By providing absolute positions as input, the autoregressive encoder learns translation invariant features." -> Why exactly does this happen?


[1] Learning to Simulate Complex Physics with Graph Networks. Sanchez-Gonzalez et al, https://arxiv.org/abs/2002.09405.
[2] Prediction and control with temporal segment models. Mishra et al, https://arxiv.org/abs/1703.04070.
clarity,_quality,_novelty_and_reproducibility: The paper is clearly written. 
As discussed in "Weaknesses", I don't think the proposed method is particularly novel.
I am also not sure if it could be reproduced from the details proposed in the supplementary material. I also wonder if more details could be provided about the Lagrange and Hamiltonion GN baselines that the authors compared their method to.
summary_of_the_review: Combining the concerns about novelty (see "Strengths & Weaknesses") and the sparse explanation of the baselines (see "Clarity, Quality, Novelty And Reproducibility"), I have a hard time understanding why the proposed method outperforms the baselines. Based on the provided results, it does seem to perform well empirically, but I'm wondering if there is some other feature of the proposed method (e.g. some architectural choice) or some overlooked flaw of the baselines or that explains this? Without a clear understanding of the contributions nor the empirical success, I would have a hard time recommending this paper for acceptance.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper introduces an  Autoregressive Graph Network (AGN) for particle-based prediction in physical problems. The idea is to augment standard graph neural networks (GNN) with an autoregressive structure in the temporal domain. The model is shown to perform on par with Graph Transformers while being much more efficient.
The paper initially received two reject (3) recommendations and two borderline reject (5) recommendations. The main concerns pointed out by the reviewers related to clarifications in the method and contributions, and requested consolidated experiments including additional baselines for comparison. The answers provided by the authors during the discussion period were not sufficient to convince the reviewers to increase their grades, and the reviewers considered that the paper should be re-written and re-submitted. After rebuttal, there remained a consensus that the paper should be rejected. 

The AC carefully reads the submission and discussions. The AC considers that the description and positioning of the paper should be improved and clarified, especially to consolidate the originality of the approach beyond combining auto-regressive models and GNNs, which seems too limited. The experiments should also include more comparisons to standard and stronger baselines. Therefore, the AC recommends rejection.
justification_for_why_not_higher_score: - The novelty and clarity of the method should be improved
- A more thorough experimental evaluation is needed
Decision: Reject