Dynamic Relational Inference in Multi-Agent Trajectories
Authors: Ruichao Xiao
Manish Kumar Singh
Rose Yu
Venue:  (2021)
Keywords: deep generative model
relational inference
trajectory modeling
multi-agent learning

Abstract: Unsupervised learning of interactions from multi-agent trajectories has broad applications in physics, vision, and robotics. However, existing neural relational inference works are limited to static relations. We consider a more general setting of dynamic relational inference where interactions change over time. We propose  DYnamic multi-Agent Relational Inference (DYARI) model, a deep generative model that can reason about dynamic relations. Using a simulated physics system, we study various dynamic relation scenarios, including periodic and additive dynamics. We perform a comprehensive study on the trade-off between dynamic and inference period, the impact of the training scheme, and model architecture on dynamic relational inference accuracy. We also showcase an application of our model to infer coordination and competition patterns from real-world multi-agent basketball trajectories.

Under review as a conference paper at ICLR 2021
DYNAMIC RELATIONAL INFERENCE IN MULTI-AGENT TRAJECTORIES
Anonymous authors Paper under double-bling review
ABSTRACT
Unsupervised learning of interactions from multi-agent trajectories has broad ap- plications in physics, vision and robotics. However, existing neural relational inference works are limited to static relations. In this paper, consider general setting of dynamic relational inference where interactions change over time. We propose namimc multi-Agent Relational Inference (DYARI) model, deep gen- erative model that can reason about dynamic relations. Using a simulated physics system, we study various dynamic relation scenarios, including periodic and addi- tive dynamics. We perform comprehensive study on the tradc-off between dynamic and inference period, the impact of training scheme, and model architecture on dynamic relational inference accuracy. We also showcase an application of our model to infer coordination and competition patterns from real-world multi-agent basketball trajectories.
INTRODUCTION
Particles, friends, and teams multi-agent relations at different scales. Learning multi agget interactions is essential to our understanding of the structures and dynamics underlying many systems. Practical examples include understanding social dynamics among pedestrians (Alahi al. (0166).
learning communication protocols in traffic (Sukhbaatar et al. 2016 Lowe et (2017) and predicting Gonbalez al. 2020).
physical interactions of particles (Mrowca al.) 2018: Li al. [2018] Most existing work on modeling relations assume the interactions are observed and train the models with supervised learning. For multi-agent trajectories, the interactions are hidden and thus need to be inferred from data in an unsupervised fashion. While one could impose an interaction graph structure (Battaglia et al. 2016). it difficult find the correct structure as the search space is very large (Grosse et al. 2012). 2). The search task is computationally expensive and the resulting model can potentially suffer from the model misspecification issue (Koopmans & Reiersol 1950).
Relational inference aims to discover hidden interactions from data and has been studied for decades. Statistical rela- tional learning are based probabilistic graphical models such as probabilistic relational model (Kemp & Tenen- Getoor et al. [0001] Koller et al.) 2007 (Shum 2019 However, these methods may require sig- nificant feature engineering and high computational costs. Recently, [Battagia et al. (2016); (anttro et al. (2017) pro- pose to reason about relations using graph neural networks but still requires supervision. One exception is Neural Relational Inference (NRI) (Kipf al. 2018), flexible deep generative model that can infer potential relations in an unsupervised fashion. As shown in Figure NRI simul- taneously learns the dynamics from multi-agent trajectories and infers their relations. In particular, NRI builds upon variational auto-encoder (VAE) (Kingma & Welling, 2013) and introduces latent variables to represent the hidden relations. Despite its flexibility, major limiting factor of NRI is that assumes the relations among the agents are static. That is, two agents are either interacting or not interacting regardless of their states at different time steps, which is rather restrictive.
baum 2008
et al.
Observed dynamics
Interaction graph
Neural Relational Inference for learning the interaction graph. Picture taken from (Kipf et al.! (0188)
Figure 1:

---

Under review as a conference paper at ICLR 2021
In this paper, we study more realistic setting: dynamic relational inference. For example, in game plays, players can coordinate and compete dynamically depending on the strategy. We propose a novel deep generative model, which we call DYnamic multi-Agent Relational Inference (DYARI). DYARI encodes trajectory interactions at different time steps. It utilizes deep temporal CNN models with pyramid pooling to extract rich representations from the interactions. DYARI infers the relations for each sub-sequence dynamically and jointly decode a sequence of relations.
As relational inference is unsupervised, we use simulated dynamics physics systems as ground truth for validation. We find that the performance of the static NRI model deteriorates significantly with shorter output trajectories, making it unsuitable for dynamic relational inference. In contrast, DYARI is able to accurate infer the hidden relations with various dynamics scenarios. We also perform extensive ablative study to understand the effect of inference period, training schemes and model architecture. Finally, We showcase our DYART model on real-world basketball trajectories.
In summary, our contributions include:
We tackle the challenging problem of unsupervised learning of hidden dynamic relations given multi-agent trajectories.
We develop novel deep generative model called DYARI to handle time-varying interactions and predict a sequence of hidden relations in an end-to-end fashion.
We demonstrate the effectiveness our method on both the simulated physics dynamics and real-world basketball game play datasets.
RELATED WORK
Deep sequence models Deep sequence models include both deterministic models (Alahi al.
2016 Li et al. (2019] Mittll et al. 2020) and stochastic models (Chung et al.) 2015 Fraccaro et al.
2016 Krishnan et al. 2017] Rangapuram al.
2018: Chen 20118 Huang et al., [2018] Yoon
it al. 2019). For GAN-like models, Yoon et al. 2019) combine adversarial training and supervised
learning objective for time series forecasting, Liu al. (2019) propose non-autoregressive model for sequence generation. Compared with GANs, VAE-type models can provide explicit inference and are preferable for our purpose. For instance, Chung et al. (2015) introduces stochastic layers in recurrent neural networks to model speech and hand-writing. Rangapuram et al. (2018) parameterizes a linear state-space model for probabilistic time series forecasting, Chen et al. (2018); Huang et al. (2018) combine normalizing flows with autoregressive models. However, all existing models only model the temporal latent states for individual sequences rather than their interactions.
Relational inference Graph neural networks (GNNs) seek to learn representations over relational data, several recent surveys on GNNs and the references therein, c.g. (Wu et al. 2019 Goyal & Ferrara (2018). Unfortunately, most existing work assume the graph structure is observed and train with supervised learning. In contrast, relational inference aims to discover the hidden interactions and is unsupervised. Earlier work in relational reasoning (Koller et al.) 2007) use probabilistic graphical models, but requires significant feature engineering. The seminal work of NRI (Kipf et al.) 2018) use neural networks to reason in dynamic physical systems. Alet et al. (2019) reformulates NRI as meta-learning and proposes simulated annealing search for graph structures. Relational inference is also posed as Granger causal inference for sequences (Louizos et al. 2017: Löwe et al. 2020).
Nevertheless, all existing work are limited to static relations while we focus on dynamic relations.
Multi-agent learning Multi-agent trajectories arises frequently in reinforcement learning (RL) and imitation learning (IL) (Albrecht & Stone 2018 Jaderberg el al., 2019). Modeling agent interactions given dynamic observations from the environment remains a central topic. In the RL setting, for example, Sukhbaatar et al. (2016) models the control policy in a fully cooperative multi-agent setting and applies GNN to represent the communications, [Le et al. (017) models the agents coordination as latent variable for imitation learning. [Song et al. (2018) generalizes GAIL (Ho & Ermon] (0166) to multi-agent through a shared generator. However, these coordination models only capture the global interactions implicitly without the explicit graph structure. Tacchetti et al. (2019) combines
GNN with a forward dynamics model to model multi-agent coordination but also requires supervision. Grover et al. (2018) directly models the episodes of interaction data with GNs for learning multi- agent policies. Our method instantiates the multi-agent imitation learning framework, but focuses on relational inference. Our approach is also applicable to dynamic modeling in model-based RI..
2

---

Under review as a conference paper at ICLR 2021
3 DYNAMIC MULTI-AGENT RELATIONAL INFERENCE
Given collection of multi-agent trajectories, we aim to reason about their hidden relations over time. First we describe the underlying probabilistic inference problem.
3.1 PROBABILISTIC INFERENCE FORMULATION
For each agent i E (1,...,, ,N. define its state (coordinates) as 3y E RRD. A trajectory -(0 (#1, 22, IT) ) is sequence states that are sampled from a policy. Given trajectories from N agents (-(0) j==1 dynamic relational inference aims to infer the pairwise interactions of N agents every time step. Mathematically speaking, the joint distribution of the trajectories can be written as:
p(7(1) ... (N)) II p(x ,x))
where p(X++1 |x,,-., ,x) represents the state transition dy- namies. We the bold form X; (s.²) (.. or ( to
indicate the concatenation of all agents observations and Xet = (X), )1).
We introduce latent variables (t1) to denote the interactions between agent and j time t. To make the problem tractable, we restrict 2, (3)) to be categorical, representing discrete interactions such coordination competition. We assume that the dynamics model can be decomposed into the individual dynamics, in conjunction with the pairwise interaction. This substantially reduces the dimensionality of the distribution and simplifies learning. Therefore, we can rewrite the transition dynamics as:
Figure 2: Probabilistic graphical model representation of dynamic multi-agent relational inference. Shaded nodes are observed and unshaded nodes are hidden variables.
N
N
N
p(x1+1/x<1) 20
/
III
p( P1
II
II j=1.
p(2)
(4)) pe
12
(3) ct: dz
(2)
captures the state transition dynamics of single agent. p(z |2 x22) represents the latent interactions between two agents. Figure 2
Here each p(x 12 (6)
visualizes the graphical model representation for three agents over number of time steps. The shaded nodes represent observed variables and the unshaded nodes are latent variables. Dynamic relational inference is to estimate distributions of the hidden variables {2( (19) at different time steps.
3.2 DYNAMIC MULTIAGENT RELATIONAL INFERENCE (DYARI)
We propose a deep generative model: Dynamic multi-Agent Relational Inference (DYARI). Given the trajectories (xI,... XT) of all agents, DYARI first concatenates the trajectories based on fully connected graph. The concatenated trajectories are used as interaction features for the encoder. Then sample the sequence of relations from the encoded hidden states. Finally, we generate the future trajectory predictions conditioned on the sampled relations. Figure visualizes the overall architecture of model which encodes and decodes multi-agent trajectories. The bottom cut-out diagram shows the architecture of our encoder.
Encoder. A key ingredient of DYARI is encoder that is inspired by PSPNet Zhao 2017 to learn rich representations of trajectories at different scales. In particular, we define residual block as two-layer CNN with residual connections (He al.) 2016). Our encoder has four modules: feature extraction, pyramid pooling, an aggregation module, and an interpolation module.
Feature extraction: the feature extraction module consists of multiple residual blocks interleaved with pooling layers to extract rich temporal features.
Pyramid pooling: the pyramid pooling module learns multi-scale temporal representations from the extracted features. First, the output of the feature extraction module is downsampled by 2x
3

---

Under review as a conference paper at ICLR 2021
Relational Inference
Interaction features
Trajectory Input
1rl.1. (1.2)
LLL. Itl.t.
Trajectory Ousput
I.l.1.
Gumbel
((3)
+(N)
Encoder
W... lalat
Decader
t=1-T
2
Pooling layer
Concat layer
Interpolats layer
Reedual blocks
Convta
Figure 3: Visualization of the DYARI model. It infers pairwise relations at different time steps given trajectories. The bottom diagram shows the details of the encoder and the decoder is the same as NRI.
and 5x through average pooling. Then, the downsampled features are passed through two residual blocks and finally upsampled by 2x and to generate features which are of the same size as the input. The representations learned at 2x and 5x resolutions are concatenated with the input to generate composite multi-scale features.
Aggregation module: 1-D convolution that aggregates the multi-scale features.
Interpolation module: it average-pools the aggregated features corresponding to the dynamic period. Then the outputs upsampled through nearest neighbours interpolation to obtain the hidden presentations for the relations.
Sampling. We utilize variational inference (Kingma & Welling 2013) to sample the latent variables from hidden representations. Specifically, assume the interaction posterior 2, (19) to be categorical:
xx1) Cat(pı.
Using the Gumbel-Max trick (Jang et al. 2017). we can reparameterize the categorical distribution as: (id) Sof toftmax(h"" (i5) dt (.)).. Here h is the hidden states of the encoder and gi (t5) is random
(35)
2t
Gumbel vector. Note that defining feature of DYARI is that the latent variable 2((/) time-dependent, requiring fine-grained modeling. Our encoder ensures that the learned representations are expressive enough to capture such complex dynamics.
Decoder. Given the sampled latent variables, the decoder generates the prediction auto-regressively following Gaussian distribution:
p
X <t 24 (15) (r
(3)
14+1 face Ej Σx
(if) 2 ugi 0), us (1 (2) x11)
(4)
Here the output # is reparameterized by Gaussian distribution with mean M441 and a fixed standard deviation 2² The mean vector 144 of agent is computed by aggregating the hidden states of all other agents. We use a separate MLP to encode the previous inputs into different type of edges
(e)
in k-dimensional one-hot vector 2, (if) To generate long-term predictions using the model in Eqn. (4), we can also incorporate the predictions from the previous time step. The decoder architecture is the same in NRI at given time step, which consists of message passing GNN operations, followed by a GRU (Cho et al. 2014) decoder.
Inference. At every time step f, we learn different distribution for the hidden relation 2, (t) assume uniform prior for po(z.) and use ELBO as the optimization objective:
We
LELBO
Ellog (x zer)]] Bdxa. (9s x xxx)x))la (co()lll(zee)))
(XeT
(5)
(m.
(6)
xz (1)
)
Σ
H (g
(1)) (x))
202
4

---

Under review as a conference paper at ICLR 2021
(6) where the mean vector j4, is parameterized by the decoder. H/ is the entropy function and / balances the two terms in ELBO (Higinns et al. 2016).
EXPERIMENTS
We conduct extensive experiments on simulated physics dynamics and real-world basketball trajecto- ries. The majority of our experiments are based on the physies simulation in the Spring environment. This is ideal for model verification and ablative study as we know the ground truth relations.
4.1 PHYSICS SIMULATIONS
Data Generation The Spring environment (Kipf et al. 2018) simulates the movements of group of particles connected by spring in a box. The hidden relation is whether there is spring connecting the two particles. To simulate dynamic relations, we generate the trajectories by removing and adding back the springs following certain patterns. Figure visualizes the trajectories resulting from such dynamic relations. Starting from the bottom, the two-particle trajectories appear as straight lines and bend in the middle due to the spring force, and return to straight lines after the removal of the spring.
0.8
0.6
0.4
0.2
0.1
0.0
0.1
first state changes
particle particle
We define the number of time steps between the change of relations as dynamic period. The primary challenges for dynamic relation inference arise along two dimensions:
second state changes
Figure 4: Example trajectories of two particles. with two relation changes. The trajectories start from the end with lighter color and grad- ually become darker.
1. The shorter the dynamic period, the more frequent the rela- tion changes. Hence, becomes more difficult to infer relations with shorter dynamic period.
2. If the dynamic period itself also changes, then the task be- comes much harder because the model also needs to adapt to the unknown period in the changing relations. Note that the way relations change in the trajectories must follow certain pattern and not be completely random. Otherwise it would be impossible to learn anything meaningful.
We experiment with two types of dynamic relations: periodic dynamics and additive dynamics. For periodic dynamics, we generate the trajectories by periodically removing and adding back the springs. We investigate the model performance by generating data with different frequencies of periodicity. For additive dynamics, we assume the dynamic period is increasing arithmetically. Each trajectory is of length 50 and the decoding length 40, see details of the generated dataset in Appendix.
Baselines and Setup We consider several baselines for comparison: (1) NRI (static): unsupervised NRI with encoder trained using the entire trajectory and infer repeatedly over time. This corre- sponds to NRI (learned) in (Kipf et al., 2018). (2) NRT (adaptive): NRI (static) with an encoder trained over sub-trajectories. The encoding length corresponds exactly the dynamic period of the dataset. We use the NRI (static) decoder to predict the entire trajectory in an auto-regressive fashion. (3) Interaction Networks (IN) (Battaglia al. 2016): supervised GNN model which uses the ground truth relations to predict future trajectories. We include this supervised learning model as the "gold standard" for our inference tasks. is important to note here that (Graber Schwing) (020) also propose a model, dNRI, for this problem but the focus of their work is trajectory prediction whereas we focus on unsupervised relational inference. In our experiments with their model, we observed relational inference accuracy of 0.505 hand, the same model gives an accuracy of 0.66 on the 3-particle synthetic data presented in their paper. Therefore, dNRI unable to infer relations in unsupervised setting.
periodic data with dynamic period 20. On the other
In practice, we do not know the dynamic period beforehand. Therefore, how often we infer the relations is a difficult choice: rare inference would miss the time steps where relations change while predicting too frequently introduces more latent variables and complicates the inference. To investigate this tra we define inference period the number of time steps between two
5

---

Under review as a conference paper at ICLR 2021
predicted relations. Unless otherwise noted, the inference period in our experiments is the same as the dynamic period. All the models are trained to predict the sequence an ruto-segressive fashion: the prediction of the current time step is fed the input to the next time step. We use Adam (Kingmal Ba 2014) optimizer with learning rate 5e-1 and weight decay le and train for 300 epochs.
4.1.1
PATHIOLOICCAL CASES OF NEURAL RELATIONAL INFERENCE
known that latent variable models suffer from the problem of identifiability (Koopmans & Reiersol 1950), which means certain parameters, in principle, cannot be estimated consistently, NRI infers correlation-lik relations between trajectories which highly depend on the length the time lag. To test this hypothesis, follow the exact same setting Kipf al. (2018) to infer the interaction graph. Instead of decoding 50 time steps, we vary the length of input and output sequence.
Table D summarizes the inference accuracy with different sequence length in the encoder and decoder. We can see that the performance of NRI deteriorates drastically with shorter training sequences, simply increasing the capacity of the encoder (NRI++) does not help. One plausible explanation is that NRI is learning correlation-like interactions. Shorter decoding sequences carry less information about correlations, making it harder learn. Meanwhile, also observed that using auto-regressive can achieve better inference accuracy compared to teacher forcing. The pathological cases highlight the issue NRI for dynamic relational inference. the interactions change frequently every few time steps, repeatedly applying NRI to different time steps would suffer from short decoding sequences. Therefore, having a model that can jointly infer a sequence relations critical.
Table I: Inference accuracy (%) of trained with trajectory lengths. Note that the performance deteriorates significantly when the output length decreases. For NRI++, we added two more hidden layers to the MLP encoder of NRI.
Teacher Foreing
Auto-regressive
Output Length
40
20
8
4
40
20
8
4
NRI
0.99
0.65
0.63
0.54
0.99
0.81
0.80
0.69
NRI+
0.99
0.66
0.63
0.53
0.99
0.80
0.80
0.70
4.1.2 DYNAMIC RELATIONAL. INFERENCE COMPARISON
We compare the performance different models for dynamic relational inference tasks.
Periodic dynamics In the periodic scenario, the dynamic period is fixed. We generate four datasets with a dynamic period of 40, 20, 8, to simulate relational dynamics with increasing frequency. Table. collumss "40, 20, 8, 4" show the trajectory prediction mean square error (MSE) and interaction inference accuracy comparison of different methods.
We can see that all methods can achieve almost perfect predictions of the trajectories with very low MSE. However, there is sharp difference in relational inference accuracy. NRI (static) is incapable of learning dynamic interactions. NRI(adaptive) can learn but has lower accuracy due to short decoding sequences. With more expressive encoder and joint decoding, DYARI is able to reach higher accuracy. When the dynamic period very small at 4, even DYARI struggles slightly, suggesting the fundamental difficulty with frequently changing dynamics.
Table 2: Performance comparison for ours and the baselines in both the periodic (40,20,8,4) and additive (Add) dynamic scenarios. MSE for trajectory prediction and Accuracy quantifies the dynamic relational inference performance.
Dynamic Period
MSE J 8
Accuracy 1 20
40
20
4
Add
40
8
Add
NRT (static)
2.2e-4
5.2e-3
2.7e-3
2.4e-3
3.6e-3
0.99
0.52
0.51
0.50
0.53
NRT (adaptive)
2.2c44
2.7e-3
1.3e-3
5.9e-4
3.
0.99
0.81
0.80
0.69
0.81
DYARI
2.6e-5
4.1e-5
4.6e-6
3.6e-6
7.6e-6
0.99
0.92
0.91
0.74
0.87
IN
2.9e-5
2.3e-5
4.3e-5
4.7e-5
3.9e-5
0.99
0.99
0.99
0.98
0.99

---

Under review as a conference paper at ICLR 2021
Validation Accuracy
1.0
Dynamic Period: 40
1.0
Dynamic Period: 20
1.0
Dynamic Period:
0.9
0.9
0.9
0.8
0.8
0.8
0.7
0.6
0.6
0.6
0.5
0.5
100
200
300
100
200
300
0
100
200
300
Training Epoches
teacher forcing
auto-regressive
hyorld
Figure 5: Learning curve of DYARI trained with teacher forcing (blue), auto-regressive (yellow), and hybrid (green) in decoder. Hybrid model is first trained with teacher forcing in the beginning 30 time steps and then auto-regressively in the later 10 time steps. We report the relational inference accuracy on the validation data for different dynamic periods.
Additive Dynamics In the additive scenario, allow the dynamic period itself increase arith- metically. We increase the dynamic period in steps of 4 starting from dynamic period of 4 timesteps. In a sequence of 40 timesteps, this implies that the relations (spring connection) get flipped at timesteps 4, 12 and 24. We use four NRI(static) models, each trained separately with 4, 8, 12 and 16 encoding timesteps. We combine the ensemble model predictions into NRI(adaptive). Table. 2 "Add" columns show the performance comparison. Similar to the periodic scenario, DYARI outperforms the baselines in this challenging task as well. Note that NRI(adaptive) is close competitor w.r.t inference accuracy, but it is four model ensemble and takes long time to train.
4.1.3 ABLATIVE STUDY
We perform ablative study to further validate our experiment design and understand the behavior of DYARI. In partieular, study the trade-off between dynamic and inference period, the effect of training scheme, as well as the ablative study of model architecture design.
Dynamic vs. Inference Period, To understand the relations between dynamic and inference period, re- peat the periodic scenario experiments by varying both dynamic and inference period in 40, 8, time steps.
Table 3: Inference accuracy for different combinations of dynamic and inference pe- riods with DYARI.
In T we observe that dynamic relational inference reaches the highest accuracy when the inference period matches the dynamic period. If the inference period is longer than the dynamic period, the model can miss the changes in the relations and completely fails to perform inference. Meanwhile, if the inference period shorter than the dynamic period, the model still can learn but suffers from low accuracy. This is potentially due to the extra uncertainty introduced by estimating more latent variables.
Dynamic Period
40
8
4
40
0.99
0.50
0.50
Inference Period
8
0.88
0.80
0.50
4
0.80
0.76
0.74
Decoder Training Scheme Another fundamental challenge in sequence prediction is covariate shift (Bickel et al. 2009)- mismatch between distribution in training and testing due to sequential dependency. Common solutions to mitigate covariate shift include teacher forcing (Williams & Zipser 1989 and scheduled sampling (Bengio et al., 2015). However, all these work are focused the prediction of observed sequence while our sequence predictions are on the latent variables. not evident that covariate shift exists in this setting, We demonstrate the empirical evidence for the effect of different training schemes on the accuracy of relational inference.
Quite surprisingly, we found that auto-regressive training is most effective for dynamic relations inference. Figure. Figure..Jsuummaaaa summarizes the difference in learning curve between using teacher forcing and regressive for different dynamic periods. We also include version of scheduled sampling (hybrid)) in the first 30 sime-steps, we train the model with teacher forcing and then switch to auto-regressive in the last 10 time-steps. We observe that while teacher forcing converges faster,
7

---

Under review as a conference paper at ICLR 2021
Figure 6: Visualization the inferred relations (dashed links) in the basketball players trajectories over time by DYARI with an inference period of 8. The blue dashed links in the top the inferred interactions from the same team (coordination) and red dashed links in the bottom are from different teams (competition). Different columns represent different time steps.
MSE
NELBO І
Inference Period
40
20
8
4
40
20
8
4
NRI(static)
2.3e-3
13.71
NRI(adaptive)
2.3c-3
3.0e-2
3.3c-2
9.7c-3
13.71
303.10
337.54
96.76
DYARI
2.2e-3
8.4e-4
4.6e-4
1.8e-4
12.65
6.16
4.38
3.67
Table Performance comparison for DYART and baselines on the real-world baskethall trajectory dataset with different inference periods 40, 20, 8 and 4.
leads to lower accuracy. This observation is consistent across different dynamic periods. Therefore, auto-regressive training is preferred for dynamic relation inference.
4.2 REAL-WORLDD BASKETBALL DATA EXPERIMENTS
To showcase the practical value dynamic relational inference, we apply DYARI to a real-world basketball trajectory dataset. The goal of the experiment is to extract meaningful "hidden" relations in competitive basketball plays. The basketball dataset contains trajectories for 10 players in a game. the trajectory prediction MSE and negative ELBO
As the ground-truth relations unknown, as in-direct measures for the dynamic relational inference performance. We assume there are two types of hidden relations: coordination and competition. We defer the details of the dataset and training setup to the Appendix.
We report performance comparisons for different inference periods. As shown in Table 4 we observe lower MSE loss and negative ELBO with shorter inference period. Intuitively, the interactions in the real world may change constantly, thus shorter inference period can capture the dynamics better. DYARI outperforms the baselines in trajectory prediction MSE and negative ELBO loss. Notice that NR((adaptive) is using encoder and decoder that trained separately and this results in a high negative ELBO loss on the test set. Fig. visualizes sample trajectory of 10 basketball players with inferred relations from DYART over different time steps. We separate coordination and competition interactions in different rows. In Fig. 66 Kobe Bryant is moving along with three-point line and guarded by defender. We can see clear attention drawn to the specific players throughout the play. See Appendix for other inferred relations.
5 CONCLUSION
We investigate unsupervised learning of dynamic relations in multi-agent trajectories. We propose novel deep generative model: Dynamic multi-Agent Relational Inference (DYARI) to infer changing relations over time. We conduct extensive experiments using simulated physics system to study the performance DYARI in handling various dynamic relations. We perform ablative study to understand the effect of dynamic and inference period, training scheme and model design choice. Compared with static NRI and its variant, our DYARI model demonstrates significant improvement in simulated physics systems well in real-world baskethall trajectory dataset.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Alexandre Alahi, Kratarth Goel, Vignesh Ramanathan, Alexandre Robicquet, Li Fei-Fei, and Silvio Savarese. Social Istm: Human trajectory prediction in crowded spaces. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 961-971, 2016.
Stefano V Albrecht and Peter Stone. Autonomous agents modelling other agents: A comprehensive survey and open problems. Artificial Intelligence, 258:66-95, 2018.
Ferran Alet, Erica Weng, Tomás Jozano-Pérez, and Leslie Pack Kaelbling. Neural relational inference with fast modular meta-learning. In Advances in Neural Information Processing Systems, pp. 11804-11815, 2019.
Peter Battaglia, Razvan Pascanu, Matthew Lai, Danilo Jimenez Rezende, et al. Interaction networks for learning about objects, relations and physics. In Advances in neural information processing systems, pp. 4502-4510 2016.
Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. Scheduled sampling for sequence prediction with recurrent neural networks. Advances in Neural Information Processing Systems, pp. 1171-1179, 2015.
Steffen Bickel, Michael Brückner, and Tobias Scheffer. Discriminative learning under covariate shift. Journal of Machine Learning Research, 10(Sep):2137-21555 2009.
Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differen- tial equations. In Advances in neural information processing systems, pp. 6571-6583, 2018.
Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint orXiv:/406.11 1078, 2014.
Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron Courville, and Yoshua Bengio. A recurrent latent variable model for sequential data. In Advances in neural information processing systems, pp. 2980-2988, 2015.
Marco Fraccaro, Søren Kaae Sønderby, Urrich Paquet, and Ole Winther. Sequential neural models with stochastie layers. In Advances in neural information processing systems, pp. 2199-2207, 2016.
Lise Getoor, Nir Friedman, Daphne Koller, and Avi Pfeffer. Learning probabilistic relational models. In Relational data mining, pp. 307-335. Springer, 2001.
Palash Goyal and Emilio Ferrara. Graph embedding techniques, applications, and performance: A survey. Knowledge-Baud Systems, 151:78-94, 2018.
Colin Graber and Alexander Schwing. Dynamic neural relational inference. Proceedings of the IEEE conference on computer vision and pattern recognition, 2020.
Roger Grosse, Ruslan R Salakhutdinov, William T Freeman, and Joshua B Tenenbaum. Exploiting compositionality to explore large space of model structures. arXiv preprint arXiv: 48566, 2012.
Aditya Grover, Maruan Al-Shedivat, Jayesh K Gupta, Yura Burda, and Harrison Edwards. Learning policy representations in multiagent systems. Proceedings of the 35th International Conference on Machine Learning, 2018
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and patlern recognition, pp. 770-778, 2016.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. 2016.

---

Under review as a conference paper at ICLR 2021
Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. In Advances in neural information processing systems, pp. 6565-4573, 2016.
Chin- Wei Huang, David Krueger, Alexandre Lacoste, and Aaron Courville. Neural autoregressive
flows. In International Conference on Machine Learning, pp. 4078-2087, 2018.
Max Jaderberg, Wojciech M Czarnecki, lain Dunning, Luke Marris, Guy Lever, Antonio Garcia Castaneda, Charles Beattie, Neil C Rabinowitz, Ari S Morcos, Avraham Ruderman, al. Human- level performance in 3d multiplayer games with population- based reinforcement learning. Science, 364(6443):859 865, 2019.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparametrization with gumble-softmax. In International Conference Learning Representations (ICLR 2017). OpenReview. net, 2017.
Charles Kemp and Joshua B Tenenbaum. The discovery of structural form. Proceedings of the National Academy of Sciences, 105(31):10687-166)) 2008.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint 1r1iv::412.6980 2014.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. 1312.61112.61111 14, 2013.
arXiv preprint
Thomas Kipf, Ethan Fetaya, Kuan-Chieh Wang, Max Welling, and Richard Zemel. Neural relational inference for interacting systems. In International Conference on Machine Learning, pp. 2688- 2697, 2018.
Daphne Koller, Nir Friedman, Sašo Džeroski, Charles Sutton, Andrew McCallum, Avi Pfeffer, Picter Abbeel, Ming-Fai Wong, David Heckerman, Chris Meek, et al. Introduction to statistical relational learning. MIT press, 2007.
Tjalling C Koopmans and Olav Reiersol. The identification of structural characteristics. The Annals of Mathematical Statistics, 2162):165-181, 1950.
Rahul G Krishnan, Uri Shalit, and David Sontag. Structured inference networks for nonlinear state space models. In Thirty-first aaai conference on artificial intelligence, 2017.
Hoang M Le, Yisong Yue, Peter Carr, and Patrick Lucey, Coordinated multi aulti-agent imitation learning. In Proceedings of the 34th International Conference Machine Learning- Volume 70, pp. 1995- 2003. JMLR. org, 2017.
Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. In Advances in Neural Information Processing Systems, pp. 5244-5254, 2019.
Yunzhu Li, Jiajun Wu, Russ Tedrake, Joshua B Tenenbaum, and Antonio Torralba. Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids. arXiv preprint crXivv1810.011 2018.
Yukai Liu, Rose Yu, Stephan Zheng, Eric Zhan, and Yisong Yue. Naomi: Non-autoregressive multiresolution sequence imputation. In Advances in Neural Information Processing Systems, pp. 11236-11246, 2019.
Christos Louizos, Uni Shalit, Joris M Mooij, David Sontag, Richard Zemel, and Max Welling, Causal effect inference with deep latent-variable models. In Advances Neural Information Processing Systems, pp. 6446-6456, 2017.
Ryan Lowe, Yi I Wu, Aviv Tamar, Jean Harb, OpenAl Pieter Abbeel, and Igor Mordatch. Multi-agent actor-critic for mixed cooperative-compettii environments, In Advances in neural information processing systems, pp. 6379-6399. 2017.
Sindy Löwe, David Madras, Richard Zemel, and Max Welling. Amortized causal discovery: Learning to infer causal graphs from time-series data. arXiv preprint arXiv:2006. 10833, 2020.
10

---

Under review as a conference paper at ICLR 2021
Sarthak Mittal, Alex Lamb, Anirudh Goyal, Vikram Voleti, Murray Shanahan, Guillaume Lajoie, Michael Mozer, and Yoshua Bengio. Learning to combine top-down and bottom-up signals in recurrent neural networks with attention over modules. arXiv preprint arXiv:2006.16981,, 2020.
Damian Mrowca, Chengxu Zhuang, Elias Wang, Nick Haber, Li F Fei-Fei, Josh Tenenbaum, and Daniel L Yamins. Flexible neural representation for physics prediction. In Advances in neural information processing systems, pp. 1799-8810, 2018.
Syama Sundar Rangapuram, Matthias W Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, and Tim Januschowski. Deep state space models for time series forecasting, In Advances in neural information processing systems, pp. 7785-7794, 2018.
Alvaro Sanchez- Gonzalezz Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Pe- ter W Battaglia. Learning to simulate complex physics with graph networks. arXiv preprint arXiv:2002.09405 2020.
Adam Santoro, David Raposo, David G Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, and Timothy Lillicrap. simple neural network module for relational reasoning. In Advances in neural information processing systems, pp. 4967-4976, 2017.
Michael Shum, Max Cleiman-Weiner, Michael L Littman, and Joshua B Tenenbaum. Theory of minds: Understanding behavior in groups through inverse planning. In Proceedings of the AAAI Conference Artificial Intelligence, volume 33, pp. 6163 61706170,20 2019.
Jaaming Song, Hongyu Ren, Dorsa Sadigh, and Stefano Ermon. Multi-agent generative adversarial imitation learning. In Advances in neural information processing systems, pp. 7461-7472, 2018.
Sainbayar Sukhbaatar, Rob Fergus, et al. Learning multiagent communication with backpropagation. In Advances neural information processing systems, pp. 2244-2252, 2016.
Andrea Tacchetti, Francis Song, Pedro AM Mediano, Vinicius Zambaldi, Neil C Rabinowitz, Thore Graepel, Matthew Botvinick, and Peter W Battaglia. Relational forward models for multi-agen learning. International Conference on Learning Representations (ICLR 2019), 2019.
Ronald J Williams and David Zipser. A learning algorithm for continually running fully recurrent neural networks. Neural computation, 1(2):270-280, 1989.
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang. and Philip Yu. comprehensive survey on graph neural networks. arXiv preprint wxXiv:9911.0056, 2019.
Jinsung Yoon, Daniel Jarrett, and Mihacla van der Schaar. Time-series generative adversarial networks. In Advances Neural Information Processing Systems, pp. 5509-5519, 2019.
Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2881-2890, 2017.

---

Under review as a conference paper at ICLR 2021
A MODEL IMPLEMENTATION DETAILS
In this section, we include some details about the model implementation, especially the encoder part. Our encoder analogical to ResNet CNNs (He et al., 2016) used field of image recognition, where the task can be abstracted to be classification problem ID dimension. Meanwhile, inspired by PSPNet used in visual scene semantic parsing (Zhao et al. 2017). we add additional global feature extractors to combine the whole-sequence (global) features and the sub-sequence (local) features.
for DYARI, each residual block shown in Fig. consists of 4 skip connections structure.
def conv. (in
(aen_nnnnnngeeee planes, out plapae,,, karnel size=3, strids=1): #oo1 dimensionsl convolution paddingaes
return nn. in.Convidiin_p anes, out out_planes, srimel_sizenkernl_, strideml, paddi ngskengs-ereel_gg peipkernel_szze/2,, biassTrue)
class expansion
def init (self, implanes, planes, kernel_sizes3, strides1, downsamplenlione): (Bssiclkss,, self).,_init___ self. conv_ldt implanes, planes, simnelsiieeeeeeeessi stride=stride) self-bni mmBBatehMormlo(plmees
self.relu mmL.evyeeellimmmleeeerre self.conv2 conv_id(
planes, planes, kernel simmml_siieeeeeee_siiee stride=stride)
self.bn2 m.BBatchNermmddooooms)) self.downsample downsample
self.stride stride
def forward(salf, x):
residual
(x)
out belluut out self relu(out)
out (out.c self bn2(utt
if self. downsdaee is not None:
residual dol
residual
out self relu(our))
Figure 7: Pytorch code snippet of the Residual Block used in DYARI encoder.
B EXPERIMENTAL DETAILS
Particle dataset In general, we use the same pre-processing in NRI. Each raw simulated trajectory has length of 5000 and sample with frequency of 100 so that each sample has length of 50 in our dataset. Correspondingly, the value of dynamic period/inference period matches the length of sample our dataset. For instance, dynamic period 10 means that the the raw trajectory, the state of node changes every 1000 time steps. In addition, The value of trajectories are all normalized to range of (0, 1] and the evaluation done on the same range as well.
Basketball dataset details The baskethall dataset consists of trajectory from 30 teams. The raw trajectory is captured with frequency of 25 ms. For our experiment, sample the trajectory with frequency of 50 ms for more evident player movements. We use a inference period that matches the length of sample. For instance, inference period 10 means that our model produce prediction every 500 ms. The resulting dataset include 50,000 training samples, 10,000 validations samples and 10,000 test samples.
We normalize the values of the trajectories to range [0,1] I] and train all the models an auto-regressive fashion. We the same training set up in physics simulation experiments with batch size of 64.
12

---

Under review as a conference paper at ICLR 2021
Figure 8: Visualization the basketball players trajectories with inference period The top row visualizes the inferred interactions from the same team (coordination) and the bottom row visualizes the inferred interactions from different teams (competition). Different columns represent different time steps.
C ADDITIONAL EXPERIMENTS
Stochastic dynamics In order to make the problem even harder and to unify all the previous settings, we generate a dataset where the edge types are flipped randomly with probability p after each dynamic period of timesteps. The static data generation corresponds to = and the periodic dynamics corresponds to p 1. Table 5 shows the MSE and inference accuracy of NRI, DYARI and Interaction Networks on the stochastic dataset for flipping probabilities 0.8 and p = 0.9.
Table 5: Qualitative results for stochastic dynamics. Accuracy improves by increasing the model capacity. In the training, The inference period of the two DYARI match with the dynamic period.
MSE
Accuracy
Flipping Probability
0.8
0.9
0.8
0.9
NRI
1.4e-3
2.3e-3
0.59
0.60
DYARI
8.3e-4
1.8e-3
0.57
0.63
IN (Supervised)
4.5e-5
4.2e-5
0.99
0.99
Table 6: Results with and without average pooling in the interpolation module of DYARI.
MSE
Accuracy
DYARI without average pooling DYART with average pooling
1.8e-5
0.59
4.1e-5
0.92
The Effect of Average Pooling We perform an ablation study where we remove the average pooling corresponding the inference period the interpolation module study the effect of this average pooling the results. We find that without average pooling, inference accuracy decreases from 0.92 to 0.59 (as shown in Table for inference period 20. Here we drop average pooling corresponding to the inference periods and directly interpolate to the sequence length. This shows that intermediate average pooling is critical for the relational inference performance.
Additional Inferred Relations in Basketball Trajectories We set the number of relations in Basketball dataset as two. In Sec 4.2, we visualized one of the inferred relations. Table visualizes the second relations. Notice that the first relation captures focus on the rightmost red player while here the relation captures focus on the leftmost red player.
13


Reviews:
[{"review_id": "vVoOwTqGlMn", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose a novel Relational Inference system that learns to predict the graph structure underlying the data as well as the updated state of the system. Relational reasoning has received considerable attention in recent year. Predicting the graph structure underlying a system from data in a dynamic way is an great next step, which could help alleviate some of the scalability issue currently afflicting these methods.\n\nThe way I read this paper, there are two distinct high level contributions:\n1. Dynamically infer the graph structure underlying a trajectory, from data and use it to predict the next step\n2. A specific method to accomplish the first contribution.\n\nIt is encouraging to see that DYARI outperforms NRI in the cases presented. However, it is possible to imagine a dynamic version of NRI (where the inferred graph structure is updated every time-step after a burn-in period).\nMoreover, while NRI is a strong baseline, other similar methods have been proposed to tackle relational inference e.g. Graph Attention Networks, Transformers and even straight Graph Networks. Similarly to NRI, with a simple thresholding mechanism these could be used to predict the graph structure as well, both statically and dynamically.\n\nIt would be really helpful to better place this work in the context of existing methods and to try and tease out the two contributions. Is it a good idea to dynamically infer the graph structure, and if so should we use this specific method?\n\nFinally, predicting the graph structure lends itself to really interesting analysis, and the dynamic aspects of this are completely unexplored. Imagine finding that your method infers that particle clouds form disjoint objects that become connected when they collide, wouldn't that be something?\n\nWith a broader outlook, we should note that Relational Inference and Graph-based models have received a lot of attention in recent years. Methods have scaled to thousands of nodes and very complex physics (see e.g. Learning to simulate complex physics with graph networks). The datasets reported in this work include up to 10 interacting objects. Since the dynamic inference of the connectivity structure is especially apt to larger scale experiments, it would be really great to see results on more complicated datasets that include computational cost considerations when compared to static graph baselines, although this might be outside the scope of this initial paper.\n\nThank you very much for sharing these cool ideas. I genuinely believe this is a research direction worth investigating. I look forward to our discussion.\n\nAll the best!\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "lFIJQGogQj", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper builds on Kipf et al. (2018)’s Neural Relational Inference. In particular, this work introduces a latent variable model which treats the interactions (i.e. relations) between different agents as dynamic and time-varying. As in NRI, the interaction variable between any two agents is conditioned on the history of those agents’ states. An agent’s future state is conditioned on its history of states as well as its interaction variables with other agents.\n\nThe results from DYARI are interesting but I am concerned about the intricacies of setting the “inference period” to be aligned with the “dynamic period.” I would have expected that choosing a smaller inference period (e.g. half the dynamic period) should lead to little to no loss in performance. The authors ascribe the observed loss in performance to “the extra uncertainty introduced by estimating more latent variables.” I’m not totally convinced. \n\nPhysical systems like springs have an inherent temporal invariance which you don’t seem to be exploiting. Did you consider (a) processing timesteps in a moving window of size T (i.e. z^{ij}_t is conditioned only on the last T observations) or (b) using a recurrent encoder to process observations sequentially, rather than process all timesteps at once with the PSPNet encoder? That might help with the inference challenge you're facing.\n\nMore questions and requests for clarification:\n- What if the inference period is out of sync with the dynamic period (e.g. 3 steps versus 5 steps)? Does the model still perform reasonably?\n- Why not model relations as continuous latents (setting aside the fact that NRI used discrete variables)? That way you can model negative interaction (e.g. competition), zero interaction, as well as the magnitude of interaction.\n- I don’t understand Figure 8 at all. Consider the two blue players who stay on the leftmost side of the basketball court. They have a dotted blue line between them in all the “coordination” plots and a dotted red line between them in all the “competition” plots. Could you please explain what's going on? I would expect only one relation to be on at any time step.\n- In Table 6, you show average pooling leads to higher accuracy but also a higher MSE. Shouldn't you have a lower MSE with a higher accuracy?\n- Could you please add error bars for your reported scores over independent runs?\n- What value of Beta did you use for your loss? What was the effect of varying this hyperparameter?\n\nMinor:\n- In equation 2, I assume the density p(x^(i)_{t+1} | x^(i)_{< t}, z^{ij}_t) was meant to be conditioned on N-1 latents not just a single j? In fact, j is undefined in the equation at that point. \n- Figure 2 shows the hidden interaction nodes z^{ij}_t are not conditioned on any other nodes. That is not true from equation 2: each z^{ij}_t seems conditioned on agent i and j’s past trajectories.\n- Could you please use x_{\\leq t} instead of x_{< t} to denote states up to and including time t?", "extra_scores": {}, "extra_text": {}}, {"review_id": "X8YgDOhQcef", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper introduces DYnamic multi-Agent Relational Inference (DYRI), which is an extension of Neural Relational Inference (NRI) for dynamic relations.  This extension shows improved performance on the real-world basketball trajectory dataset.\n\nThe article is fairly well structured, apart from the literature review, which is somehow missed very important and very related works. The paper is clearly motivated and seems to be technically correct. \n\nMy main concern with this paper is a novelty. The main argument is to increase the expressive power of static NRI for dynamic relations, which I think is a very valid motivation for this paper; however, this idea has been already published as Dynamic Neural Relational Inference (dNRI; Graber & Schwing, CVPR 2020). Based on my understanding, the model and inference of DYRI are the same as dNRI.  The authors have to clearly discuss their difference with dNRI if they still think those are different. Also, I would have expected to see it as a baseline.\n\nAs I mentioned above, one of the main drawbacks of the paper is its literature review and baseline methods. Besides dNRI, this problem has been tackled in other papers with different structures. For example, Graph-VRNN (Chen Sun, ICLR 2019) combined variational RNNs and NRI for multi-agent interactions and applied it in the same basketball dataset. Also, Dynamic Neural Relational Inference for Forecasting Trajectories has been proposed in CVPR 2020. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "PYQu7TKUrvd", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a method for dynamic relational inference for multi-agent trajectory prediction. The method extends the neural relational inference (NRI) (Kipf et al., 2018) by changing the static relations between agents to dynamic relations. This equates to inferring time-varying latent variables $z_t^{ij}$ as opposed to learning time-independent latent variables $z^{ij}$. The paper conducts experiments on physics simulations and basketball trajectories to show the superiority of the proposed method against different variants of NRI.\n\n\n**Pros:**  \n++ The paper is well motivated and clearly written.  \n++ The paper performs detailed experiments and ablation studies to analyze the behavior of the proposed model and the pitfalls of NRI.\n\n\n**Concerns:**  \n-- My major concern of this paper is that the almost same idea has been proposed in prior work [1]. The probabilistic formulation in that paper is essentially the same as this paper, which also introduces time-varying latent variables $z_t^{ij}$ and infers them within a VAE model. This drastically reduces the novelty of this paper. [1] also performs more extensive experiments on simulation, human motion, basketball, and traffic trajectory.\n\n-- My second concern is about the design “the inference period in our experiments is the same as the dynamic period”. This is a rather strong assumption, which cannot be realized in real applications since the dynamics period is not known. This assumption also seems to defeat the purpose of dynamic relational inference, which is to infer the change in dynamics so the model should be able to have a much longer inferior period than the dynamics change period. From table 3, it becomes evident that the model only performs well when the inference period is shorter than the dynamics period, which means when the relationship between agents stays unchanged during a single inference period. \n\n-- The decoder distribution in Eq. 2 should have multiple $p(x_{t+1}^{(i)}|x_{<t}^{(i)}, z_{t}^{(ij)})$ for different $j$. In the current form, it seems only one $j$ is used and not clear which one.\n\n[1] Dynamic Neural Relational Inference. Graber et al. CVPR 2020.\n\n**Rating Justification:**  \nA very similar idea to this paper has been proposed in prior work [1], which significantly reduces the novelty of this paper. There are also concerns about the model’s design (e.g., inference period). Thus, I vote for a reject.\n\n**Additional Comments:**  \n-- the notation $<t$ is used to denote observations that also include t, which is quite confusing. $\\leq t$ is probably better.  \n-- Fig 2 does not match Eq 2. Since the $x_2^{(1)}$ is generated by  $z_2^{(12)}$ and $z_2^{(13)}$, so $z_t^{ij}$ in Eq 2 should be $z_{t+1}^{ij}$ to match the time index of $x_{t+1}^{(i)}$\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a method for relational inference in multi-agent/multi-object trajectory prediction tasks. Different from the neural relational inference (NRI) model [1], the presented method is able to model time-varying relations. Experimental results on physics simulations and sports games (basketball) show benefits over variants of the NRI model.

The reviewers agree that the presented method is mostly solid, that the experiments are insightful, and that this is generally a well-written paper. The authors, however, have apparently overlooked recent related work [2] (dNRI) that proposes a very similar model. In the light of dNRI, it is difficult to argue for the novelty of the presented approach, and the paper needs to undergo a revision in order to more clearly differentiate it from the dNRI model, and to resolve the other concerns raised by the reviewers.

[1] Kipf et al., Neural Relational Inference for Interacting Systems (ICML 2018)
[2] Graber et al., Dynamic Neural Relational Inference (CVPR 2020)
Decision: Reject