Online Relational Inference for Evolving Multi-agent Interacting Systems
Authors: Beomseok Kang
Priyabrata Saha
Sudarshan Sharma
Biswadeep Chakraborty
Saibal Mukhopadhyay
Venue: NeurIPS 2024 poster (2024)
Keywords: Neural Relational Inference
Online Learning
Multi-agent System
Primary area: graph_neural_networks

Abstract: We introduce a novel framework, Online Relational Inference (ORI), designed to efficiently identify hidden interaction graphs in evolving multi-agent interacting systems using streaming data. Unlike traditional offline methods that rely on a fixed training set, ORI employs online backpropagation, updating the model with each new data point, thereby allowing it to adapt to changing environments in real-time. A key innovation is the use of an adjacency matrix as a trainable parameter, optimized through a new adaptive learning rate technique called AdaRelation, which adjusts based on the historical sensitivity of the decoder to changes in the interaction graph. Additionally, a data augmentation method named Trajectory Mirror (TM) is introduced to improve generalization by exposing the model to varied trajectory patterns. Experimental results on both synthetic datasets and real-world data (CMU MoCap for human motion) demonstrate that ORI significantly improves the accuracy and adaptability of relational inference in dynamic settings compared to existing methods. This approach is model-agnostic, enabling seamless integration with various neural relational inference (NRI) architectures, and offers a robust solution for real-time applications in complex, evolving systems.

Online Relational Inference for Evolving Multi-agent Interacting Systems
Beomseok Kang, Priyabrata Saha, Sudarshan Sharma, Biswadeep Chakraborty,
Saibal Mukhopadhyay
Georgia Institute of Technology
{beomseok, wtakkooaaaayyyyyyyyaayeeeee edu
Abstract
We introduce novel framework, Online Relational Inference (ORI), designed to efficiently identify hidden interaction graphs in evolving multi-agent interacting systems using streaming data. Unlike traditional offline methods that rely a fixed training set, ORI employs online backpropagation, updating the model with each new data point, thereby allowing it to adapt to changing environments in real- time. A key innovation is the use of an adjacency matrix as a trainable parameter, optimized through new adaptive learning rate technique called AdaRelation, which adjusts based on the historical sensitivity of the decoder to changes in the interaction graph. Additionally, data augmentation method named Trajectory Mirror (TM) is introduced to improve generalization by exposing the model to varied trajectory patterns. Experimental results on both synthetic datasets and real- world data (CMU MoCap for human motion) demonstrate that ORI significantly improves the accuracy and adaptability of relational inference in dynamic settings compared to existing methods. This approach is model modeel-ggoostcc enabling seamless integration with various neural relational inference (NRI) architectures, and offers robust solution for real-time applications in complex, evolving systems. Code is available https: //g gi thub. com/ beonseokg/ORI.
Introduction
Multi-agent interacting systems have been studied in various fields, including particle-basee physical simulations [1-3], traffic systems [4, 5], and social networks [6-8]. Interaction among agents is crucial information to accurately model such systems and also provides interpretability in agent behaviors [9]. However, external observers can only access the trajectory of agents without knowing interaction graphs. Consequently, identifying unknown interaction graphs from observable trajectories of agents has been emerged as a specific problem referred to as relational inference [10].
In recent years, neural relational inference (NRI) and its variants have shown promising performance in synthetic and real-world environments [11-16]. Prior studies mostly aimed to present better network for NRI based on variational autoencoder (VAE) built with graph neural networks (GNN) [14, 15, 12, 17]. These methods involve an encoder to infer an interaction graph an adjacency matrix from observed trajectories and decoder to predict the future trajectories employing the inferred interaction graph. They generally perform training offline assuming the well- well-aligned distribution in training and testing data. Unfortunately, such assumption is frequently violated in practice due the shifts in test condition, including sudden changes in the interaction, evolving system parameters and even dynamics itself. Building relational inference model generalizable to all the different scenarios challenging [18-20]. In this case, online learning is attractive approach to continuously adapt the model to newly observed environments [5]. However, online learning for relational inference has been rarely explored.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Table I: Comparison of key features between prior works and this work.
Model
Consider Evolution in
Criteria
Method
Description
Agnostic Interaction Parameter Dynamics
Acc. MSE
Prior offiine works
offfine backpropagation
[11, 7, 12, 23, 16]
novel encoder and decoder
x
x
Prior online work
online convex optimization
[5]
constant learning rate
online hackpropagation
This work
AdaRelation; Traj. Mirror
indicates the presence of a feature, and indicates the absence of a feature.
Online backpropagation using gradient descent is widely used online learning method as it is compatible with various neural network designs [21]. However, online backpropagation on existing NRI models significantly degrades the accuracy relational inference, since their decoder quickly learns the trajectory prediction even before the encoder generates reasonable interaction graphs. It is important note that while the models are trained in self-supervised manner (i.e., trajectory prediction), true labels of interaction graphs are never provided to the models, indicating that identifying interaction graphs is essentially unsupervised. That is, optimizing the unsupervised encoder is more challenging than the self-supervised decoder in nature. The key problem is how to match the learning speed between the interaction identification and the trajectory prediction so that both the tasks can be collaboratively optimized.
In this paper, we propose novel framework named Online Relational Inference (ORI) to effi- ciently identify hidden interaction graphs in evolving multi-agent interacting systems. Our method strategically allocates adjacency matrix representing the interaction as a trainable parameter in the model and directly optimizes it through online backpropagation on the predicted trajectories. It effectively accelerates the update of the adjacency matrix than the encoder-based approach and ensures the following decoder to be learned with reasonable adjacency matrices from the early stage of training. ORI can seamlessly integrate with prior NRI models, offering architectural flexibility (i.e., model-agnostic). Moreover, we developed an adaptive learning rate technique named AdaRelation particularly designed for relational inference in the evolving systems. employs the historical adja- cency matrix to indirectly estimate the decoder's 's sensitivity over the adjacency matrix and determine whether the learning needs to be accelerated. In addition, we introduce a data augmentation technique named Trajectory Mirror (TM) to expose various trajectories by flipping the axis in the systems. We experimentally demonstrate the effectiveness of ORI on various NRI models in both the synthetic and real-world (CMU MoCap for human motion [22]) datasets. Our key contributions are as follows:
To the best of our knowledge, ORT is the first model-agnostic online relational inference framework for evolving multi-agent interacting systems. ORI employs online backpropaga- tion to optimize an adjacency matrix from the trajectory information without any assumptions the model architecture.
We experimentally demonstrate that ORI identifies unknown interaction graphs in various evolving multi-agent interacting systems, such as sudden changes in interaction (Figure 2), parameters in the dynamics (Figure 3), and even dynamics itself (Figure 3), outperforming existing NRI models (Table 2).
We propose AdaRelation, a novel adaptive learning rate technique particularly designed for relational inference in evolving multi-agent systems. It automatically tunes the learning rate for the adjacency matrix when interaction or/and dynamics in the system suddenly change (Figure 3).
We propose Trajectory Mirror, a data augmentation technique to ensure the reasonable relational inference regardless of the trajectory axis. significantly improves the conver- gence speed and overall interaction prediction accuracy in the several evolving scenarios (Supplementary).
2 Related Works
Neural relational inference. NRT [11] the first work that formulated the problem of unsuper- vised relational inference from observed agent trajectories. They implemented a VAE-based graph

---

neural network that encodes trajectories into an adjacency matrix, representing relation types, and then decodes the trajectory using the predicted adjacency matrix. This approach has been established as standard in several follow-up works. For example, dNRI [12] and ExolveGraph [15] introduced graph recurrent network-based architectures to encode and generate dynamic priors for predicting evolving interactions. A Additionally, a memory-aaggennce architecture has been proposed for en- hanced long-term prediction using external associative memory pools [24]. More recently, finer granularities in relation modeling have been considered, such as edge to edge interaction [23], relation potentials using an energy- -barey function [25], and disentangled edge features [13]. However, all of these methods commonly rely encoder-decoder pair in offline setting, supported by large amount of batched training samples, without considering various evolving scenarios.
Evolving multi-agent systems. Apart from relational inference, trajectory learning of evolving multi-agent interacting systems have actively explored using graph neural networks [18, 26, 6]. In particular, graph neural ordinary different equations (ODE) have been applied various evolving scenarios, including evolving environments (e.g., temperature and pressure) [18, 20] and stochastic motion [19]. However, these approaches assume known graph structure, focusing on accelerating the simulation of physical dynamical systems [1, 7J. Additionally, each model addresses a specific type of evolution, raising questions about its generalizability to other scenarios. Moreover, adapting neural ODEs. potentially through online backpropagation, is a challenging problem since they are optimized from initial values [27].
Online learning for multi-agent systems. Online learning for relational inference in multi-agent systems a rarely explored research problem despite its significance. One primary related work [5] also treats the adjacency matrix trainable parameter without using an encoder, optimizing it with an online expert mixture algorithm, a type of online convex optimization algorithm. Consequently, the loss function needs to be convex over the adjacency matrix to guarantee optimization. Their model architecture is specifically designed to place message passing in the last layer of the model to ensure the model output is linear combination of hidden states and the adjacency matrix, maintaining the convex property. However, this simple architecture significantly differs from recent architectures (e.g.. NRI and its variants), making it challenging to model nonlinear and complex interactions. Additionally, this work only reports trajectory prediction errors without providing any accuracy metrics relational inference. Please note that comparison with ORI is provided in the supplementary material.
3 Proposed Approach
3.1 Background
Neural relational inference. Relational inference aims to identify unknown interaction graph represented by an adjacency matrix I E RNNNNN xm, where N is the number of agents and m is the number of interaction types, from their trajectory in given time period Tis+AA which generally include positions and velocities of agents. Existing approaches are mostly designed with a pair of neural network- based encoder and decoder, where encoder (g&) predicts the adjacency matrix I, from an observed trajectory (i.e., 9 (r,,-AAt)) and then decoder predicts the future trajectory (fra+ by fa(a,- ( It). That is, the encoder is optimized return the adjacency matrix for lower error on the predicted trajectory (decoder output), assuming /" = arg mins, L(ErAtt fo Attt, Tt).
Online learning. Online learning problems have traditionally been formulated within the online convex optimization (OCO) framework, as introduced in [28]. Conventional gradient descent- online optimization projects the updated variables onto a convex set at each step [29]. Such learning settings provide theoretical convergence guarantees when the loss function is convex with respect to the optimization variables. However, applying online gradient descent to deep neural networks (DNNs) is challenging due to the nonconvex nature of the loss function, and standard backpropagation performs poorly in an online setting [21]. Two primary directions of study have emerged to tackle the challenges of online backpropagation. One approach involves employing a flexible network structure, where the DNN architecture evolves over time [21, 30). The other approach focuses on using an adaptive learning rate, where the learning rate is adjusted over time [31, 32]. Since our primary focus
3

---

X
Model-aghoetic
X
Apxede
CRUN
GNN
Trai
Lmaa
Trafectory Miror
(T (0 10p ((O)
dl-mmse
sf(())
AdaRelation
0.73
((()
high (FCc) -0----C will
decresss g [r())
incroase
tip sads
ni cods
ni both
2.00
(t)
Relation changes
Dynamies changes
Trainable
[(t] gAxns
Figure 1: A brief illustration of the proposed Online Relational Inference (ORI) framework.
on learning multi-agent interactions online without relying any specific network structure, concentrate on adapting the learning rate based on the evolution of the adjacency matrix.
3.2 Motivation of ORI
Our objective to discover hidden relations between agents evolving multi-agent interacting systems using their streaming trajectories. The motivation of this work comes from the primary challenges to apply existing methods to implement a fast-adapting, accurate, and stable online relational inference framework. Accordingly, we summarize our key motivations into three categories.
Why we consider adjacency matrix as trainable parameter? may not be effective to simply apply the encoder and decoder-based existing methods evolving multi-agent systems in the online setting. The primary challenge is that the intricate encoder is slowly trained with streaming and evolving data. It eventually degrades the decoder well since the encoder and decoder are jointly trained, influencing each other. Whereas, ORI allocates the adjacency matrix as a trainable parameter, significantly enhancing the training speed in both the matrix and decoder. While such allocation is motivated from [5], this work still faces following crucial challenges.
Why we need model-agnostic learning? The relation inference performed through the trajectory prediction without explicit supervision on graph structures (i.e., true relation is not observable). This means the only supervision is defined by the predicted trajectories from the decoder, and hence largely depending on how effectively the decoder responds to changes in the embedding adjacency matrix. If learning method is constrained to a specific decoder design, like [5], the performance achievable by the method can be constrained by the decoder design. Ideally, the learning method should offer the flexibility to seamlessly integrate to various decoder designs.
Why we need adaptive learning rate? The choice of a learning rate is particularly important in the evolving systems since the loss landscape can significantly vary with the evolution in the systems. For example, low learning rate may be suitable for a slowly evolving dynamics A, but may high one needed for stable operation leads to a slow convergence or/and potentially a sub-optimal performance the dynamics evolves over time. Ideally, the learning rate needs to be automatically tuned over time, avoiding a trade-off between faster convergence and unstable learning.
fast evolving dynamics B. constant (or decaying) learning rate
3.3 Training Procedures of ORI
First, the key difference in our training setup compared to the offline learning setup is that both batch and epoch are (i.e., streaming data). There is no separate validation or test dataset in online learning. Training of ORI is performed by online backpropagation on each individual training sample every iteration. Figure describes the proposed approach. Note that the input and output to the decoder in the figure (denoted as GNN) and its optimization essentially the same existing methods. Our

---

contribution is primarily in the optimization method for the adjacency matrix given by:
(1++1)=11 = (,, dl dm (ōt 154-A41:+A4
(1)
dI
where My (Di(),, (t)) E RR is the relation-aware adaptive learning rate, elaborated in the following paragraph. Initially, the adjacency matrix (I(1) E RNXNX="; (t)(() E [0, ID is filled with 0.5. Each training sample given the model involves single trajectory for a time period of At tt At (i.e., -t-At: The decoder (fe) observes 2t-At:t and then predicts the future trajectory by Ber+ar f= (t)) in an autoregressive manner. It also reconstructs the trajectory 2t-ac- during the observation. ORI estimates a MSE loss on the predicted and reconstructed trajectory at and then updates the decoder and adjacency matrix using gradient
t + At (i.e., Lusse (f2- descent. Note that, the adjacency matrix is updated only once at each iteration. The model infers the relation in the given trajectory at the last time step. While the training objective is to minimize the overall MSE loss throughout the streaming data, our primary evaluation criterion is the relation accuracy representing the proportion of true positive and true negative the adjacency matrix.
Adaptive relation-aware learning rate. We propose AdaRelation, a learning rate adaptation technique specifically designed for relational inference. AdaRelation adjusts the learning rate for the adjacency matrix (not the decoder) based changes in the norm of the adjacency matrix. For example, if the norm exceeds a certain threshold, it gradually decreases the learning rate within a defined range.
The mechanism is intuitive: a good trajectory predictor should show a large output variance depending on the adjacency matrix, meaning the quality of a predicted trajectory should vary clearly with changes in the adjacency matrix. We observe that the norm of gradient (l) dzuss (11) indeed increases as the model adapts to the evolved system, often making the adjacency matrix unstable (see (1)). Conversely, decreases when the system suddenly evolves, slowing down the update of the adjacency matrix. Therefore, the gradient norm indicates when the update of the adjacency matrix needs to be stabilized or accelerated. Given the adjacency matrix is the sum of this gradient over time, comparing the current matrix with past essentially reflects changes in the gradient norm. Accordingly, define Dr()) to estimate the evolution in the L1 norm of the adjacency matrix over w time steps as follows:
1
DIO = 2 T(() - T(L - w)ll
This measures how much the predicted interaction strength (k),,.) changes on average over w time steps. We define a threshold parameter to restrict changes in Lij.A to be remain near this threshold. The update of v1( involves adding or subtracting a, an adaptation step size, determined by the range of Di()) and the threshold:
{ fa if Dr() >
An(()
a
otherwise
Consequently, the next learning rate (17(t + 1)) is represented by (,,( n(t)):
hr h,,(Dr()) (t)) = clip immaa (n(() + An()))
where clip, Mminmmax limits the learning rate within the lower bound (Amin) and upper bound (7max).. This effectively controls the learning rate to automatically stabilize and accelerate the update of the adjacency matrix. More intuition behind AdaRelation discussed in the supplementary material.
Trajectory Mirror. The models trained by the online backpropagation are often prone to be biased certain training samples. It also happens in multi-agent interacting systems. For example, the coordinates and velocities of agents in the currently observed data samples can be biased. Such scenario has not been discussed much in literature because existing works expose the model to huge amount of simulations with short-term trajectories, ensuring several different initial positions and velocities. However, our problem addresses streaming trajectories in a much longer-term, where we do not have an access to initialize their positions and velocities. Ideally, the relation between agents should be correctly inferred, regardless where the model observes them. Accordingly, we consider data augmentation technique named Trajectory Mirror, which flips the axis and generates,
5

---

for example in two-dimensional space, three additional trajectories (see Figure 1). It is simple yet effective avoid the model bias and significantly enhance the convergence speed of the model without storing the past trajectories. Ablation studies are available in the supplementary material.
Technical novelty. ORT is a novel model -ggeos-g online relational inference framework, incor- porating a strategic combination of two different learning methods for the adjacency matrix and decoder. ORI introduces AdaRelation, an adaptive learning rate technique designed for relational inference, and Trajectory Mirror, simply yet effective data augmentation technique proved in several evolving scenarios (results are supplementary). Our approach clearly differs from existing methods which perform end-to-end gradient descent on the encoder [11, 12, 23] or online convex optimization constrained to a specific decoder design [5].
4 Experimental Results
Dataset. We experimented with three common benchmarks, synthetic springs and charged systems, and CMU MoCap (human motion) [22]. The synthetic datasets were generated using the open-source code from NRI [11]. For the springs and charged systems, 10 simulations with 90k time steps each were created, involving 10 agents with different random interactions (p- = 0.5). These simulations are sequentially presented to models. The evolving relation dataset features different interaction graphs in each simulation with fixed spring k = 0.1 and ke = 1.0. Another dataset varies these constants, generated from uniform distributions [0.1, 0.2] for springs and [1, 2] for charged constants. The evolving relation and dynamics dataset randomly selects each simulation from either the springs or charged systems. The CMU MoCap dataset, processed using dNRI's open-source code [12], includes human motion data with 31 joints and various actions, captured at 120Hz.
Baseline and implementation detail. ORI a model-agnostic online learning framework involving the trainable adjacency matrix. The adjacency matrix is initialized only at the initialization stage. The same adjacency matrix is used throughout the entire samples and simulations without assuming that we know when the interaction evolves. ORI can integrate with various models as long as they use the adjacency matrix as input. Most prior works use decoders based on graph layer perceptron (MLP) and graph recurrent neural networks (RNN), as seen in NRI [11], incorporating node-to- edge and edge-to-node message passing. Additionally, MPM [23] introduces graph RNN and an attention-based graph RNN for more efficient edge-to-edge message passing. To demonstrate ORI's effectiveness, we use four different trajectory predictors: NRIm (ML.P-based), NRIr based), MPMr (RNN-based), and MPMa bbased) Since ORI employs the decoders in NRI and MPM, we primarily compare our approach with the original NRI and MPM. We also include dNRI [12], which is specifically designed for dynamically evolving interaction graphs. As these works analyzed the performance in the offline setup, we evaluated their models in the online setup. We follow their default implementation but replace the encoder with the trainable adjacency matrix. More details on the training setup are available in the supplementary material.
4.1 Inferring Relation in Evolving Interaction Graph
We explore relation inference accuracy in the synthetic systems incorporating evolving interaction graphs. These systems involve constant parameters and no switching in the dynamics. Figure demonstrates the relation accuracy over time and predicted trajectories in the springs system. The relation accuracy is evaluated based on the number of true positives and true negatives excluding self-interaction (i.e., total 10x9 relations). Our approach is able to quickly recover the relation accuracy when the model encounters a new interaction graph. For example, at the bottom of Figure 2(a), the target interaction matrix suddenly changes at the 15k-th iteration. While it fails to adapt in single iteration, the target and predicted matrices are aligned well after 100 iterations. In contrast, the baseline (MPM) accuracy slowly increases throughout the entire training iterations, showing a significant gap in the average relation accuracy. Figure 2(b) compares the target and predicted trajectories at the 15k-th iteration (i.e., right after the new interaction) and 18k- 1-th iteration (i..., right before another interaction), indicating the trajectory quality also improves along with the accuracy.
Table 2 showcases the average accuracy and average MSE loss during entire iterations in the springs and charged systems. As the interaction graph changes over time, simply reporting the final accuracy only represents how well the model adapts to the last graph. Accordingly, we report the average

---

Interaction Skiters.
Prediction (2 and Target (R) 10k
1.0-
0.8
0.6
MPN
Gurew 1931/6 Dec
0.4
24k
Training Peritions
Prediction (L] and Target (R) 18K-1
Torget
Prodiction
Rot.
15c-1
15k
15 100
15k 18k-1
Figure 2: Prediction results of ORI with MPMr decoder and the baseline MPM in the springs system. (a) the relation accuracy the two models throughout the training (top) and visualization of the target and predicted adjacency matrix in our model (bottom). (b) target and predicted trajectories in our model.
Table 2: Comparison with offline learning models in springs and charged systems with evolving interactions. Acc and mse stand for the relation accuracy and mse on the predicted trajectory averaged over the entire training iterations. The number following mse (e.g., mse 10) denotes the mse at the 10-th prediction time step.
Springs
Charged
Method
Ace (%)
mic
mse
mse 20
mae 30
Acc (%)
mse
mse 10
mse
mse
dNRI [12]
51.1
4.346-5
8.07e-
2.400-3
5.340-3
50,3
2.02c-3
5.00e-3
9,44c-3
5.40e-2
NRI [1]]
57.4
1.70e.4
1,36e
4.49e
1.1le.2
$2.0
3.80e
1.13e2
2.40e
4.63e
MPM [23]
61.6
2.76c-4
1.10c3
4.100-
9.15c-3
51.7
6.16c-3
1.21e-2
2.60e-2
4.86c-2
NRIm
74.5
1.45o4
8.19c4
2.71633
7.320-3
88.6
6,60c-3
1.6le2
3.796-2
7.33e-2
Ours NRIr
94.2
1.24o4
6.02o-
1.7603
3.44o-3
90.9
4.87e-3
1.47e22
3.540-2
6.86-2
Ours w/ MPMr
95.2
2.80e.4
4.54e.4
1.43e-3
3.16e-3
95.3
6.98e.3
1.35e-2
3.06e-2
5.91e:2
Ours w/ MPMa
96.4
2.59c-4
3.74e4
1.17e-
2.62o-3
87.1
6.85e-3
1.42-2
3.25e-2 6.26e-2
accuracy over entire iterations to understand the model's accuracy on the multiple graphs and how fast adapts the change the graphs. ORT with four different decoders consistently outperform the existing encoder and decoder-based methods with respect to the accuracy. Note that TMM is applied to both existing methods and ours for the fair comparison.
crucial to emphasize that, in the charged system, our results with almost 40% higher accuracy does not exhibit lower trajectory errors. For example, NRI reaching an accuracy of 52.0% still shows the lower MSE of 3.80 10-3 4.63 10-2 over all the prediction steps than our results. This implies the comparison solely using the MSE loss may not indicate the quality of inferred relations at all. The lower MSE in existing methods is probably achieved by the larger capacity than our methods due to the encoder, overfitting the trajectory modeling. not the relation inference. However, in spring systems, the methods with higher accuracy exhibits lower trajectory errors.
4.2 Inferring Relation in Evolving Interaction Graph and Dynamics
ORI is evaluated on the two additional evolving scenarios in the springs (spr) and charged (cha) systems, where 1) interaction graph and parameter evolve and 2) interaction graph and the dynamics itself evolve. In addition,
analyze the relation learning rate to understand how the relation accuracy responds depending on the learning rate. The yellow, red, and blue curves correspond to ORI with AdaRelation and ORI with constant learning rates (lower and upper bound). Figure 3 presents the relation accuracy and relation learning rate over 30k training iterations in both the evolution scenarios. Although the models tend to slowly converge compared to the springs systems, they still adapt to the systems with evolving parameters or switching dynamics, reaching the 1.0 accuracy given enough training iterations.
interesting observation lies in changes of the relation learning rate. For example, in Figure 3 (left bottom), the learning rate in AdaRelation (yellow) automatically increases at the 12k-th
7

---

Interaction
spr
Relatin Accunayy
1.0
Reatonfcuuscy
1.0
0.
0.
0.6
0.8
0.4
1.2K
15k
18k
24k
27k
308
21K
Retarion Loarning Rate
16
150
-00
100
20
15200on
relationn
10
6k
21k
horations
Training Nteraioons
(4)
Figure Prediction results of ORI with NRIr decoder in the charged system with evolving interaction and parameters (a) and ORI with MPMr decoder in the springs and charged systems with evolving interaction and dynamics (b). 1tst row compares the relation accuracy between constant learning rates and AdaRelation. 2-nd row shows changes in the relation learning rate throughout the training.
1.0
0.8
0.M
0.05
(756)
Garew
(250)
Relation Accuret
0.9
Ours
0.5
(125)
0.13
and wrrrng retooneetttooo
0.04
0.7
HPH
0.4
0.12
0.03
SAI
0.02
0.6
cNRI
0.7
0.01
0.07
0.0
10k
6%
12%
Training Berations
(1)
(C)
Figure 4: Comparison between ORI and existing methods with respect the relation accuracy (a), variance in the adjacency matrix (b), and variance in the predicted trajectory (c) depending on encoder complexity. The number in the MPM (-) represents the dimension of hidden states in the encoder.
iteration, decreases over time, and then increases again at the 15k-th iteration. This means that AdaRelation notices change in the interaction graph of the systems in unsupervised manner and hence increases the learning rate for a while. This ensures not only the fast adaptation to a new environment but also the stability (i.e., less fluctuation in the accuracy) after the relation accuracy converges. Moreover, Figure 3 (right bottom) demonstrates that AdaRelation controls the learning rate depending on the dynamics as well. The models with a high learning rate are stable enough in the springs systems. However, the fluctuation in the accuracy emerges in the charged systems, particularly when the accuracy almost reaches 1. AdaRelation effectively suppresses such fluctuation without sacrificing the convergence speed. For example, between the 18k-th and k iterations, the accuracy AdaRelation converges as fast as the high learning rate's one while having much less fluctuation in the later iterations. Thus, AdaRelation effectively enhances the convergence speed, stability, and overall accuracy evolving multi-agent interacting systems. The related accuracy and ablation studies are available in the supplementary material.
4.3 Discussion on Performance Gain ORT
We discuss how existing methods fail to clarify the benefit of ORI over them. Since they share the same trajectory predictors, the primary reason should be studied with respect to the encoder side.
Lightweight encoders in existing works. One of the key features in ORI is the encoder-less design, having much fewer trainable parameters, excluding the trajectory predictor, than existing works. To clarify whether the performance gain is from the less trainable parameters, we demonstrate how existing methods perform when their encoder is significantly lightened while having the same decoder. Figure 4(a) showcases the relation accuracy depending on the encoder complexity in the spring system with evolving interaction. However, the performance gain from their lightweight encoders not significant, indeed still much lower than ORI. Accordingly, we explore the following two additional experiments.

---

R
Targetes Durs
MPN Doc
Taiget
HAM
Framee (Taining
(78)
(80)
(82)
(84)
1770
Figure 5: Prediction results of ORI with MPMr decoder and MPM in CMU MoCap dataset. 1-st represents the last frame in the predicted and target trajectory from ORL 2-nd and 3-rd rows visualize the top-30 stongest interaction edges in the corresponding frame from ORI and MPM. Note that MPM allocate higher relation strengths in the front foot while ORI focuses the foot behind.
Changes in inferred relations. Figure 4(b) describes changes in the adjacency matrix (Di(s) throughout training in the models with different encoder complexity. Note that Dr()) estimates how much the current adjacency matrix differs from the past one (i.e., ~ [[) T(L w)) [l1). First, the range of Dice) in ours is consistent throughout the entire iterations, effectively updating the adjacency matrix from the early stage of training. In contrast, MPMs even with the smaller encoder, such as MPM (64) and MPM (32), exhibit sudden increases in DI(1) after few thousands iterations. This means that the predicted adjacency matrix is not responding much to the observed trajectory, implying their encoders completely fail to discover the relationship between the observed trajectory and the adjacency matrix. The larger models, MPM (256) and MPM (128), show worse results such slow increases in Dia)) and almost no changes in the first 10k training iterations. That is, the lightweight encoder still shows the totally different behavior ORI and does not effectively update the adjacency matrix in the early stage.
Output variance given true and wrong relations. Following the above paragraph, we study how the slow optimization of the encoder influences the trajectory predictor. Figure 4(c) displays the gap in MSE losses between the output with true interaction graph and one with completely wrong interaction graph over the training iterations. This essentially represents how sensitive the trajectory predictor is to the input interaction graph. ORI demonstrates a clearly larger gap in the MSE losses than all the other MPM models. Apart from ORI, the smaller encoders increase the MSE gap in the trajectory predictor. That is, the failure in the encoder degrades the trajectory predictor, which potentially influences the encoder again. In summary, allocating the trainable adjacency matrix in ORI ensures the stable update in the embedding adjacency matrix to the trajectory predictor and enhances its output variance depending on the interaction graph.
4.4 Real-world Application
ORI is assessed the real-world human motion dataset (CMU MoCap) [22] against existing offline methods. Figure 5 showcases the target and predicted trajectories on walking motion from ORI (I-st row) and the top-30 strongest relations between joints in a skeleton model for the corresponding frame (2-nd row). Additionally, the figure incorporates the inferred relations from MPM (3-rd row). The visual comparison illustrates that ORI's predicted joint trajectories closely align with the target, yet ORI exhibits higher MSE loss compared MPM (see the supplementary material). However, similar with the observation in the charged systems (Table 2). ORI appears to offer more interpretable relation inference on the joints. For example, 3-rd row of the figure, MPM simplifies relational inference by cyclically focusing on right foot, left foot, and right foot again. In contrast, ORI introduces an additional layer of shifts the relation, emphasizing primary connections between left foot, right knee, and right foot. This additional detail improves the interpretability of the walking motion.

---

Conclusion
Summary. We introduced Online Relational Inference (ORT), novel framework for online re- lational inference in evolving multi-agent interacting systems. ORI employs an adaptive learning rate technique, AdaRelation, allowing adapt dynamically changing environments through online backpropagation. Our approach also includes the Trajectory Mirror (TM) data augmentation method to enhance generalization. This model-agnostic framework seamlessly integrates with various neural relational inference architectures, providing robust solution for real-time applications in com- plex, evolving systems. Future work will focus on enhancing the adaptability in more fast-evolving interaction and exploring its applicability to wider range of multi-agent systems.
Limitation and future work. Our current experiments do not evaluate ORI in non-ideal environ- ments, incorporating directed interaction graphs or/and variable number of nodes. This limits the potential of ORI in relatively ideal environments. We expect that ORI can be extended scenarios when agents are added or deleted, provided we are aware of which agent is added and deleted by adding and deleting corresponding row and column in the adjacency matrix. While the experiments in the paper do not explicitly include directed interactions, ORI does not assume any symmetry (undirected graph) in the adjacency matrix. other words, there is no technical limitation to apply ORT in directed interaction.
Acknowledgements
This work is partly supported by the Office of Naval Research under Grant Number N00014-20-1- 2432 and National Science Foundation under Grant Number 2328962. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Office of Naval Research, National Science Foundation, or the U.S. Government.
References
[1] Alvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter Battaglia. Learning to simulate complex physics with graph networks. In International conference machine learning, pages 8459-8468. PMLR, 2020.
[2] Tobias Pfaff, Meire Fortunato, Alvaro Sanchez- Gonzalez, and Peter W Battaglia. Learning besh-based simulation with graph networks. arXiv preprint arXiv:2010.03409, 2020.
[3] Chuang Gan, Jeremy Sehwartz, Seth Alter, Damian Mrowca, Martin Schrimpf, James Traer, Julian De Freitas, Jonas Kubilius, Abhishek Bhandwaldar, Nick Haber, et al. Threedworld: A platform for interactive multi-modal physical simulation. arXiv preprint arXiv:2007.04954. 2020.
[4] Jeongwhan Choi, Hwangyong Choi, Jechyun Hwang, and Noseong Park. Graph neural con- trolled differential equations for traffic forecasting. In Proceedings of the AAAI Conference Artificial Intelligence, volume 36, pages 6367-6374, 2022.
[5] Maosen Li, Siheng Chen, Yanning Shen, Genjia Liu, Ivor Tsang, and Ya Zhang, Online multi- agent forecasting with interpretable collaborative graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2022.
[6] Zijie Huang, Yizhou Sun, and Wei Wang. Coupled graph ode for learning interacting system dynamics. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pages 705-715, 2021.
[7] Chengxi Zang and Fei Wang. Neural dynamics complex networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 892-902 2020.
[8] Zijie Huang, Jeehyun Hwang, Junkai Zhang, Jinwoo Baik, Weitong Zhang, Dominik Wodarz, Yizhou Sun, Quanquan Gu, and Wei Wang. Causal graph ode: Continuous treatment effect modeling in multi-agent dynamical systems, arXiv preprint arXiv:2403.00 000100878 2024.
10

---

[9] Beomseok Kang, Harshit Kumar, Minah Lee, Biswadeep Chakraborty, and Saibal Mukhopad- hyay. Learning locally interacting discrete dynamical systems: Towards data-efficient and scalable prediction. arXiv preprint arXvv:200.006460, 2024.
[10] Adam Santoro, David Raposo, David G Barrett, Mateusz Malinowski. Razvan Pascanu, Peter Battaglia, and Timothy Lillicrap. A simple neural network module for relational reasoning. Advances in neural information processing systems, 30, 2017.
[11] Thomas Kipf, Ethan Fetaya, Kuan-Chieh Wang, Max Welling, and Richard Zemel. Neural relational inference for interacting systems. In International conference on machine learning, pages 2688 2697. PMLR, 2018.
[12] Colin Graber and Alexander G Schwing. Dynamic neural relational inference. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 8513-8522, 2020.
[13] Victoria M Dax, Jiachen Li, Enna Sachdeva, Nakul Agarwal, and Mykel Kochenderfer. Disentangled neural relational inference for interpretable motion prediction. IEEE Robotics and Automation Letters, 2023.
[14] Enna Sachdeva and Chiho Choi. Dider: Discovering interpretable dynamically evolving relations. IEEE Robotics and Automation Letters, 7(4):1 1823-11830, 2022.
[15] Jiachen Li, Fan Yang. Masayoshi Tomizuka, and Chiho Choi. Evolvegraph: Multi-agent trajectory prediction with dynamic relational reasoning. Advances neurul information processing systems, 33:19783-19794 2020.
[16] Sindy Löwe, David Madras, Richard Zemel, and Max Welling. Amortized causal discovery: Learning to infer causal graphs from time-series data. In Conference on Causal Learning and Reasoning, pages 509 525. PMLR, 2022.
[17] Longyuan Li, Jian Yao, Li Wenliang, Tong He,, Tianjun Xiao, Junchi Yan, David Wipf, and Zheng Zhang. Grin: Generative relation and intention network for multi-agent trajectory prediction. Advances in Neural Information Processing Systems, 14-27107-27118 2021.
[18] Xiao Luo, Haixin Wang. Zijie Huang, Huiyu Jiang, Abhijeet Gangan, Song Jiang, and Yizhou Sun. Care: Modeling interacting dynamics under temporal environmental variation. Advances in Neural Information Processing Systems, 36, 2024.
[19] Suresh Bishnoi, Jayadeva Jayadeva, Sayan Ranu, and NM Anoop Krishnan. Brognet: Momentum-consert
graph neural stochastic differential equation for learning brownian dynamics. In The Twelfth International Conference on Learning Representations, 2023.
[20] Zijie Huang, Yizhou Sun, and Wei Wang. Generalizing graph ode for learning complex system dynamics across environments. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 798-809, 2023.
[21] Doyen Sahoo, Quang Pham, Jing Lu, and Steven CH Hoi. Online deep learning: learning deep neural networks the fly. In Proceedings of the 27th International Joint Conference Artificial Intelligence, pages 1660-2666, 2018
[22] CMU. Carnegie-mellon motion capture database, 2003. URL http::///ommee cs. cs. cmu, edu.
[23] Siyuan Chen, Jiahai Wang, and Guoging Li. Neural relational inference with efficient mes- sage passing mechanisms. Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 7055-7063, 2021.
[24] Dong Gong, Frederic Z Zhang, Javen Qinfeng Shi, and Anton Van Den Hengel. Memory- augmented dynamic neural relational inference. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11843-11852, 2021.
[25] Armand Comas, Yilun Du, Christian Fernandez Lopez, Sandesh Ghimire, Mario Sznaier, Joshua B Tenenbaum, and Octavia Camps. Inferring relational potentials in interacting systems. In International Conference on Machine Learning, pages 6364-6383, PMLR, 2023.
11

---

[26] Xiao Luo. Jingyang Yuan, Zijie Huang, Huiyu Jiang, Yifang Qin, Wei Ju, Ming Zhang, and Yizhou Sun. Hope: High-order graph ode for modeling interacting dynamics. In International Conference Machine Learning, pages 23124-23132 23139. PMIR, 2023.
[27] James Morrill, Patrick Kidger, Lingyi Yang, and Terry Lyons. Neural controlled differential equations for online prediction tasks. arXiv preprint arXiv:2106.1 1028, 2021.
[28] Martin Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th international conference machine learning (ieml-03), pages 928 938993, 2003.
[29] Elad Hazan et al. Introduction to online convex optimization. Foundations and Trends@ in Optimization, 2(3-4):157-325, 2016.
[30] Andri Ashfahani and Mahardhika Pratama. Autonomous deep learning: Continual learning approach for dynamic environments. In Proceedings of the 2019 SIAM international conference on data mining. pages 666-674. SIAM, 2019.
[31] Stefan Duffner and Christophe Garcia. An online backpropagation algorithm with validation error-based adaptive learning rate. In International Conference Artificial Neural Networks, pages 249-258. Springer, 2007.
[32] Rui Zhang, Zong-Ben Xu, Guang-Bin Bin Huang, and Dianhui Wang. Global convergence of online bp training with dynamic learning rate. IEEE Transactions on neural networks and learning systems, 3302):33--341, 2012.
12

---

A Training Setup
Training setup. For the synthetic datasets, the model observed the first 30 time steps (3:)30:)) and then predict the next 30 time steps (Z1++300. The next prediction window is defined on 21+-0-1+-0--
Similarly, for the CMU MoCap data, the model observed the first 10 time steps and then predict the next 10 time steps, We trained the models with single GTX 2080Ti GPU.
Implementation details. We directly use the implemented decoders from NRI [11] and MPM [23] for ORI with NRI and ORI with MPM. The hidden size on the decoders are set to 256 as default. The learning rate for the decoders is le-4.
The initial learning rates (n)) for the adjaceney matrix is 100 for ORT with NRT decoders and 20 for ORI with MPM decoders. The lower and upper bound for learning rates in AdaRelation are Mnin = 100 Mmaax = 200 for ORI with NRI decoders, Munin 20 7/max 50 for ORI with MPM decoders. The threshold and adaptation step size a in AdaRelation are set to 0.05 and 1. For the CMU MoCap dataset, we observe that the gradient on the adjacency matrix is relative small; in order to ensure more dynamic updates in the adjacent matrix, we largely increase the r7(0) to 100k.
B Additional Discussion
Computational complexity. In terms of trainable parameters, NRT has 721.4k for encoder and 727.3k for decoder; MPM has 1724.9k for encoder and 1071.7k for decoder; dNRI has 883.7k for encoder and 269.8k for decoder. In terms of FLOPs per iteration, NRI shows 177. 17788MFLOPs for encoder and 5040.5MFLOPs for decoder, MPM shows 3.9GFLOPs for encoder and 10.9GFLOPs for decoder. It indicates that the decoder consumes more computation than the encoder even though they are with the similar level of trainable parameters.
dhomu |11. The norm of gradient indicates that how the Intuition behind the norm of gradient dT())
trajectory error (m))) changes by the adjacency matrix (I)()). Ideally, we expect this norm being high enough so that the model learns the strong correlation between the trajectory of agents and their relation. In other words, the low norm of gradient means that, the model returns the similar trajectory regardless of the relation (i.e., adjacency matrix), which is undesirable.
Intuition behind the deviation D1(0-- The deviation is function of ((t())((t-))lb where both I(I) and T([ -ww w) are the learned adjacency matrix, not the actual one. Hence, the significant change in the actual adjacency matrix used for generating the observed time series, may not necessarily lead to large values of the deviation. This depends on how quickly ORT learns the new adjacency matrix as discussed below.
Consider a scenario in which the actual adjacency matrix significantly changes between the time steps t w and t. Assume ORI has learned the actual adjacency matrix in the time step w. Now, in the time step t, ORI can respond in two possible ways.
First, ORI may quickly identify the new adjacency matrix at the time step t. In this case, [tt) and T(L w) will be related to the new and previous actual adjacency matrices, respectively. Assuming the two actual adjacency matrices are significantly different, the deviation between two learned adjacency matrices will also be large. Hence, based on equation (3), the learning rate will decrease. This will make the learned adjacency matrix stable, thereby helping ORI to stay at the new learned adjacency matrix at the time f, which is desirable as that is also the actual adjacency matrix.
Let us now consider the second case where ORI does not quickly learn the new actual adjacency matrix and hence, the learned adjacency matrix at time t stays close to the one learned at time step t w. In other words. [[tt) I(t w)]lı remains low even if the actual adjacency matrices have changed. In this case, following equation (2), the learning rate increases to rapidly update the learned adjacency matrix, which is desirable quickly move the learned matrix to the actual one.
In summary, the equation (2) and (3) appropriately update the learning rate when actual adjacency matrix changes, even without any knowledge/sunpervision of that actual matrices.
13

---

45
55
55
of liteattnnst
0.9
0.8
50% Kcs.
-5
27%
50
0.4
0.40 0.45 0.50
Training Ierratooss
Disatmaantty
(9)
Figure 6: Correlation between the dissimilarity and the number of iterations required to reach 90%
accuracy since the interaction graph evolves in ORI with MPMr decoder in the springs system.
Resseh Accussy
1.0
0.8
0.8
Case
Cioe3
Cored
Trarning Bensioon
Figure 7: ORI with MPMr decoder in five different cases with irregular evolution in interaction. The system is based on springs system with 10 agents and consists of three Ik iterations, four 2k iterations, and three 3k iterations.
C Additional Experimental Results
Correlation between dissimilarity in graph and relational accuracy. Even though we randomly evolved the interaction graphs, there exists a slight similarity between the graphs. A dissimilarity defined on two interaction graphs to understand how it influences relational accuracy. We first sum the element-win tinenttwwiee difference between two interaction graphs and then divide by the number of elements. This dissimilarity is compared with the number of iterations required to reach 90% accuracy since the interaction graph evolves. For example, ORI with MPMr decoder in the springs system (Figure in the main paper), in the early stage of training, we do not observe much correlation between the dissimilarity, as the model is not yet matured. However, after approximately 18k iterations, more dissimilar interaction graphs require more training iterations to reach the accuracy 90% (see Figure 6). Note that, higher dissimilar interaction graphs indicate the higher accuracy drop when the interaction graph evolves. This leads to the lower initial accuracy and hence requires less iterations. (see the second column Figure 6(a)). However, we consider the variation of the required training iterations still marginal (e.g., 45-55 iterations in Figure 6(b)).
Irregular evolution in relations. We consider five different cases with irregular evolutions in interaction follows. The relational accuracy iterations is described in Figure 7.
Case 1: interaction graph changes after 1k, 1k, 1k, 2k, 2k, 2k, 2k, 3k, 3k, 3k iterations.
Case 2: interaction graph changes after 3k, 3k, 3k, 2k, 2k, 2k, 2k, 1k, 1k, 1k iterations.
Case 3: interaction graph changes after 1k, 3k, 2k, 2k, 1k, 3k, 2k, 3k, 1k, 2k iterations.
Case 4: interaction graph changes after 2k, 3k, 1k. 3k, 2k, 2k, 1k, 2k, 3k, 1k iterations.
Case 5: interaction graph changes after 3k, 1k, 1k, 2k, 3k, 2k. 2k, 1k, 3k, 2k iterations.
From Case 1 to Case 5, the accuracy is 93.9%, 92.2%, 93.8%, 93.1 %, and 92.5%. Overall, the variation in the accuracy marginal, and hence, the performance ORI not significantly influenced by the irregular evolutions. However, since we only considered 1k, 2k, and 3k iterations, more extreme scenarios, such as few-thousands iterations would be an interesting future study.
MSE in CMU MoCap. Table 3 summarizes the MSE loss on the predicted trajectory in the CMU MoCap dataset. Overall, the difference in the MSE losses are not significant. However, the original MPM method works better than ORI with NRI and ORI with MPM.

---

Table 3: Comparison of MSE loss on the predicted trajectory in the CMU MoCap dataset.
CMU MoCap
Method
msc 1
msc
mse
NRI [1]]
1.23c-4
6.64e:3
2.32c-3
MPM [23]
1.17e-4
5.28e-3
1.49e-3
Ours w/ NRIr
1.12c-4
6,57c-4
1.86c-3
Ours w/ MPMr
1.31e-4 7.23e.4 2.22e-3
Table Comparison with prior online learning method in springs systems with 20 agents.
Method
Relative MSE compared [11]
Relative MSE compared to dNRI [12]
Prediction step
2
10
5
10
CoGNN-AR [5]
0.339
0.345
0.617
0.680
0.635
0.731
0,714
1.194
1,373
1.312
CoGNN-LSTM [5]
0.268
0.259
0.317
0.340
0.371
0.577
0.536
0.613
0.686
0.766
CoGNN-TC [5]
0.286
0.310
0.300
0.340
0.352
0.615
0.643
0.581
0.686
0.727
Ours w/ NRIm
0.565
1.655
0.672
0.578
0.563
0.565
0.631
0.863
1.048
1.151
MPMr
1,028
1.681
0.359
0.253
0.228
1.028
0.641
0.461
0.459
0.466
Ours w/ MPMa
1.653
1.610
0.279
0.181
0.165
1.653
0.614
0.359
0.329
0.337
Comparison with prior online work. The primary related work for online learning in multi-agent interacting systems is [5]. They also considered springs systems with 20 agents, but their code, implementation details, and relation accuracy are not available, making direct comparison difficult. Instead, provide indirect comparison using similar data (springs system with 20 agents and evolving interaction). This work compared their method with NRI [1 1] and dNRI, so we normalize their results with their NRI and dNRI, and ours with our NRI and dNRL, and then compare these results as relative MSE (Table 4). We observe that in the very early prediction steps (e.g., steps and 2), their method performs better, but ours achieves a clearly lower relative MSE loss the later time steps.
Ablation study. We perform ablation studies on AdaRelation and Trajectory Mirror in the three different evolving systems. For AdaRelation, the models with constant learning rates, lower or upper-bound learning rate in AdaRelation, are compared. Table 5, Table 6, and Table summarize the results. We bold the highest accuracy (or others with less than 0.5% difference). Overall, the large accuracy gain is resulted from Trajectory Mirror, and AdaRelation further enhances the accuracy.
15

---

Table 5;
Ablation study in springs and charged systems with evolving interaction.
Springs with evolving interaction
Charged with evolving interaction
Method
Trajectory Mirror
AdaRelation
Acc (%)
Trajectory Mirror
AdaRelation
Acc (%)
w/
constant (100)
90.2
constant (100)
Ours w/ NRIr
constant (200)
94.4
constant (200)
87.0
Ours w/ NRIr
94.2
90.9
w/ MPMr
80.5
90.2
Ours w/ MPMr
4
constant (20)
93.1
constant (20)
92.2
Ours w/ MPMr
constant (50)
95.5
constant (50)
94.1
Ours w/ MPMr
95.2
95.3
Table Ablation study in springs and charged systems with evolving interaction and parameter.
Springs with evolving interaction and parameter
Charged with evolving interaction and parameter
Method
Trajectory Mirror
AdaRelation
Acc (%)
Trajectory Mitror
AdaRelation
Acc (%)
Ours w/ NRIr
constant (100)
94.9
constant (100)
83.4
Ours NRIr
constant (200)
95.0
constant (200)
73.8
Ours w/ NRI
94.7
84.8
Ours w/ MPMr
85.1
79.6
Ours MPMr
constant (20)
93.4
(20)
92.5
Ours w/ MPMr
constant (50)
95.4
constant (50)
88.8
Ours MPMr
92.1
Table 7: Ablation study in springs and charged systems with evolving interaction and dynamics.
Evolving interaction and dynamics (springs and charged)
Method
Trajectory Mirror
AdaRelation
Acc
Ours w/ MPMr
54.5
Ours w/ MPMr
constant (20)
86.5
Ours w/ MPMr
90.7
Ours w/ MPMr
91.3
16

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes].
Justification: We clearly mentioned proposed techniques in the abstract and introduction. Furthermore, we summarized the results we showed through experiments abstract and introduction.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes].
Justification: We added limitations in the conclusion section.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: We do not claim any theoretical contribution.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We included all the information about models and learning settings in the supplementary to ensure reproducibility.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Anonymized code is available at http://aaooyyoooossssperrrecccce 14488-25FC
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: Necessary details about the experiments to understand the results are given in the main paper, while full details are provided in the supplementary.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We reported the mean results across multiple experimental conditions.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
17

---

Justification: Details about computational resources are provided in the supplementary.
9. Code Of Ethies
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics httpa: httpar/ppp..cpppp///tttiiicceeee
Answer: [Yes]
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no direct path to any negative applications.
Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All existing assets used the paper are properly credited.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Code/model are provided with proper documentation.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
18


Reviews:
[{"review_id": "vqSOdIYeGd", "reviewer": "Reviewer_fSnF", "summary": "This paper focuses on online relational inference (ORI) for dynamical systems. It points out from the optimization perspective that in the existing encoder-decoder framework, the encoder responds slowly to streaming data when inferring the evolving interaction graphs. It proposes to learn the adjacency matrix directly via a model-agnostic online learning scheme. The key design is an adaptive learning rate scheduling strategy based on the important observation that the predicted trajectory is sensitive to the change of the adjacency matrix. Delicately designed experiments show how ORI responds timely to the change of interaction graphs and dynamics, and reveal the limitations of existing methods.", "questions": "Q1. The setting for graph evolution can be too idealistic. The graph regularly evolves every 3k iterations, while the duration can be irregular in real-world scenarios. Will that affect the performance of the proposed method?\n\nQ2. Do you instantiate a trainable adjacency matrix for each sample/simulation? Can ORI achieve inductive learning, i.e., generalizing to unseen samples?\n\nQ3. How can you extend ORI to handle multiple types of interaction relations?\n\nQ4. In the testing stage, how many iterations are required for updating the adjacency matrix at each time step? Although ORI is encoder-free, it requires at least one extra backward pass of the decoder to update the adjacency matrix. Can compare the running time of ORI with the encoder-based methods, e.g., dNRI, NRI, MPM? I believe responding in real-time is a desirable property for ORI.", "limitations": "N/A", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1. This paper makes the first attempt to tackle online relational inference under the streaming data setting.\n\nS2. The observation that the predicted trajectory is sensitive to the change of the adjacency matrix is important for relational inference. It directly inspires the simple yet effective design of the online learning strategy.\n\nS3. The carefully designed experiments help the readers understand from a mechanistic perspective how some key factors limit the inferring accuracy of existing methods, and how ORI properly overcomes these issues by its delicate design. Both the results and the analyzing procedure will be beneficial for later research.", "weaknesses": "W1. This paper considers the adjacency matrix as a trainable parameter. This is reasonable when the set of nodes does not change. However, with node addition and deletion, the meaning of each element in the adjacency matrix becomes inconsistent. It is unclear if ORI can handle this scenario.\n\nW2. The trajectory mirror is indeed a data augmentation trick, although it may not be widely adopted in relational inference. I am not sure if this can be highlighted as a contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1. [Addition and Deletion of Node]** Thank you for pointing it out. We agree that the current ORI is studied when the number of nodes is constant. We will clarify this assumption in the revised draft, and add this as a limitation of the current study. \nWe expect that ORI can be extended to scenarios when nodes are added or deleted, provided we are aware of which node is added and deleted by adding and deleting corresponding row and column in the adjacency matrix. We will update the paper to include this as a future work. \n\n**W2. [Trajectory Mirror as Contribution]** We believe that the trajectory mirror is as an effective, useful, and most importantly, simple approach to improve the accuracy. For example, Table 7 in the supplementary material, ORI with/without the trajectory mirror shows 91.3%/54.5%. Also, as the reviewer mentioned, it has not been explored before in the problem of relational inference. \n\n**Q1. [Irregular Evolution in Relations]** We consider five different cases with irregular evolutions in interaction. The system is based on springs system with 10 agents and consists of three 1k iterations, four 2k iterations, and three 3k iterations, instead of ten 3k iterations. Please see Figure 1 in the attached pdf.\n\nCase1 (Accuracy: 93.9%): interaction graph changes after 1k, 1k, 1k, 2k, 2k, 2k, 2k, 3k, 3k, 3k iterations. \n\nCase2 (Accuracy: 92.2%): interaction graph changes after 3k, 3k, 3k, 2k, 2k, 2k, 2k, 1k, 1k, 1k iterations. \n\nCase3 (Accuracy: 93.8%): interaction graph changes after 1k, 3k, 2k, 2k, 1k, 3k, 2k, 3k, 1k, 2k iterations. \n\nCase4 (Accuracy: 93.1%): interaction graph changes after 2k, 3k, 1k, 3k, 2k, 2k, 1k, 2k, 3k, 1k iterations. \n\nCase5 (Accuracy: 92.5%): interaction graph changes after 3k, 1k, 1k, 2k, 3k, 2k, 2k, 1k, 3k, 2k iterations. \n\n\nOverall, the variation in the accuracy is marginal (92.2%~93.8%), and hence, the performance of ORI is not significantly influenced by the irregular evolutions. However, since we only considered 1k, 2k, and 3k iterations, more extreme scenarios, such as few ~ thousands iterations would be an interesting future study.\n\n\n**Q2. [Instantiation of Adjacency Matrix]** We instantiate our adjacency matrix only at the initialization stage. The same adjacency matrix is used throughout the entire samples and simulations without assuming that we know when the interaction evolves. Since our online setup incorporates the streaming of unseen trajectories driven by unseen interaction graphs, system parameters, and even unseen dynamics, ORI is generalizable to unseen samples. We will clarify this in the experiment section.\n\n**Q3. [Multiple Types of Relations]** ORI is already capable of handling multiple types of interactions through a multi-channel adjacency matrix (as we mentioned in line 166, I(t) \\in R^N x N x m; m is the channel dimension), where each channel represents a specific type of interaction. For instance, in the charged system, the two channels of the adjacency matrix represent attraction and repulsion, respectively. ORI determines whether there is an attraction or repulsion on an edge. Although these examples involve a small number of relation types, there is no technical limitation preventing ORI from managing scenarios where edges represent a weighted combination of multiple forces or interactions, with the possibility of zero weights for any force on a specific edge.\n\n\n**Q4. [Iterations Required to Update Adjacency Matrix]** First of all, we assume an online setup where there does not exist separate training and testing stages. Regardless of training and testing, we update the adjacency matrix only once at each time step (i.e., ORI does not perform iterative optimization). \n\n**[Computational Complexity]** We compare the overall computational complexity, including running time, in ORI with NRIr decoder and ORI with MPMr decoder with the encoder-based methods (e.g., NRI, MPM, and dNRI). In terms of trainable parameters, NRI has 721.4k for encoder and 727.3k for decoder; MPM - 1724.9k for encoder and 1071.7k for decoder; dNRI - 883.7k for encoder and 269.8k for decoder. In terms of FLOPs per iteration, NRI shows 177.8MFLOPs for encoder and 5040.5MFLOPs for decoder; MPM – 3.9GFLOPs for encoder and 10.9GFLOPs for decoder. It indicates that the decoder consumes more computation than the encoder even though they are with the similar level of trainable parameters. For the running time in a single 2080Ti GTX GPU, NRI shows 0.194sec/iter; MPM – 0.183sec/iter; dNRI – 0.096sec/iter. ORI with NRIr decoder shows 0.235sec/iter; ORI with MPMr decoder – 0.192sec/iter. While ORI shows slightly higher running time than the encoder-based models, theoretically, as ORI does not have an encoder network and apply the gradient descent to much smaller adjacency matrix, its overall FLOPs is approximated to the only decoder’s FLOPs. Also, since the FLOPs are mostly allocated at the decoder, their running time will be largely depending on the decoder design. We observe that the latency in ORI occurs at the unoptimized code for two separate gradient descent (one for the decoder and another one for the adjacency matrix). We are working on optimizing the run-time of that step in ORI.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nYqe5RKwHI", "reviewer": "Reviewer_mc8m", "summary": "The paper introduces a novel framework called Online Relational Inference (ORI) designed to identify hidden interaction graphs in evolving multi-agent systems using streaming data. ORI employs online backpropagation and treats the adjacency matrix as a trainable parameter, optimized through an adaptive learning rate technique called AdaRelation. Additionally, a data augmentation method named Trajectory Mirror (TM) is introduced to improve generalization by exposing the model to varied trajectory patterns. Experimental results were carried on synthetic and real-world datasets.", "questions": "Here are my questions:\n\n1. In line 30 \"They generally perform training offline assuming the well-aligned distribution in training and testing data.\" Actually, I don't think this really matters, as the previous methods are unsupervised on the relations.\n\n2. Line 40 \"... primarily due to the slow optimization of the encoder.\" I could not reach this statement from the results in table 2. BTW, table 2 is too far from this sentence.\n\n3. As stated: the method \"employs the historical adjacency matrix to indirectly estimate the decoder’s sensitivity over the adjacency matrix and determine whether the learning needs to be accelerated.\" Will the errors be accumulated as well?\n\n4. Line 145 \"This means the learning is performed from the loss generated by the decoder... \" Please check the ELBO loss in NRI. It contains a part called KL-term, which works on the output of encoder.\n\n5. It would be great if the authors can elaborate more about the deviation in Eq. 1. What might be the intuition behind the deviation? How did this value change during training? Or possibly, it would be better if the equation comes with a proof on it actually works.\n\n6. As the $I(t)$ is initially filled with 0.5. Then how to get the adjacency matrix from it to feed to the GNN?\n\n7. What is the meaning behind the norm of gradient $\\|\\|\\frac{dL_{mse}}{dI(t)}\\|\\|_1$?\n\n8. Can you explain more about the biased data mentioned in line 200? How are these kinds of data characterized?\n\n9. Honestly speaking, I do not get the point of Trajectory Mirror. The operations of Trajectory Mirror blurred the dynamics of the system.\n\n10. In the experiments, how do the interaction graphs evolve? Does the performance of the proposed method correlated with the similarity between the graphs before and after change?\n\n11. Does the method scale well to larger dynamical systems?", "limitations": "Although the authors state the limitation in just one sentence in the conclusion, I would like to encourage the authors discuss the limitations with more details. Such as the scalability, the adaption to directed graphs, and so on.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The proposed method is the first addressing the dynamics in relational inference with online-learning approach.\n- ORI is the first model-agnostic online relational inference framework for multi-agent systems.", "weaknesses": "- The reliance on an encoder-less design may not fully leverage the potential benefits of more sophisticated encoder architectures. Such as how to perform supervised learning?\n- Several statements in the paper need to be explained, please refer to questions.\n- The model does not explicitly address the inference of directed interactions between components. \n- The evaluation metrics like AVG. Accuracy in Table 2 confused me a lot. Usually we report the final accuracy instead of the AVG. Accuracy over the entire training period.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1. [Encoder-less Design for Supervised Learning?]** This paper focuses on a fully unsupervised setup for learning the relation graph. By ‘supervised learning’ in this comment, we assume the reviewer is referring to a scenario where the true relation graph is available for the training set. We agree that having a sophisticated encoder would be beneficial in such cases, but it is not the focus of this paper.\n\n**W3. [Directed Interaction]** While the experiments in the paper do not explicitly include directed interactions, ORI does not assume any symmetry (undirected graph) in the adjacency matrix. In other words, there is no technical limitation to apply ORI in directed interaction.\n\n**W4. [Why avg Accuracy]** The interaction graph changes over time. Simply reporting the final accuracy only represents how well the model adapts to the last graph. Accordingly, we report the average accuracy over entire iterations to understand the model’s accuracy on the multiple graphs and how fast it adapts to the change in the graphs. \n\n**Q1. [Alignment in Train and Test Data]** As mentioned in line 31, the distribution in training and testing data here means not only the relations but also the governing dynamics. Temporal evolution in the dynamics, for example, if the training data is a spring system and testing data is a charged system, can significantly degrade the performance on interaction learning as studied in [1].\n\n[1] \"Care: Modeling interacting dynamics under temporal environmental variation.\" Neurips, 2024.\n\n**Q2. [Poor Performance on Existing NRI models]** We understand that reaching this statement solely from Table 2 is not obvious. We will re-write this statement to “... significantly degrades the accuracy on relational inference since their decoder quickly learns the trajectory prediction even before the encoder generates reasonable interaction graphs.\n\n**Q3. [Error Accumulation in AdaRelation]** No. The statement refers the equation (2) (i.e., the deviation D_I(t)) which measures the norm of the difference between the current adjacency matrix and the past one. The deviation is newly measured each time step, and hence the error is not accumulated.\n\n**Q4. [ELBO Loss in NRI]** We agree that NRI has a KL-divergence loss to optimize its VAE-based model. “The learning” in line 145 we meant was supervised learning from external information, such as trajectories. We will change the statement to “The only supervision is defined by the predicted trajectories from the decoder...”.\n\n**Q6. [Initial Adjacency Matrix filled with 0.5]** Our adjacency involves continuous values as mentioned in line 166 (I_i,j(t)∈[0,1]). Hence, initial adjacency matrix filled with 0.5 is directly provided to the GNN without binarization.\n\n**Q7. [Intuition behind the Norm of Gradient |dL/dI|]** The norm of gradient indicates that how the trajectory error (∆L) changes by the adjacency matrix (∆I). Ideally, we expect this norm being high enough so that the model learns the strong correlation between the trajectory of agents and their relation. In other words, the low norm of gradient means that, the model returns the similar trajectory regardless of the relation (i.e., adjacency matrix), which is undesirable. \n\n**Q5. [Intuition behind the Deviation D_I(t)]** Our intuition in the deviation was to compare the current adjacency matrix (i.e., I(t)) with the past one (i.e., I(t-w)) to estimate whether the learning is too fast or too slow. For example, if the difference between them is large, we decrease the learning rate to stabilize the learning. Conversely, it the difference is small, we increase the learning rate to speed up the learning. We briefly provide mathematical sketch as follows: 1) expand equation (2) into ||I(t) - I(t-1) + I(t-1) - ... - I(t-w)||, 2) apply triangular inequality to get ||I(t) - I(t-1)|| + ... + ||I(t-w+1)-I(t-w)||, 3) use equation (1). This creates the inequality between the deviation and the norm of gradient, connecting the intuition behind the deviation and the norm of gradient. Also, please see Figure 3 in the attached pdf.\n\n**Q8. [Biased Learning without Trajectory Mirror]** Our data is the trajectory consists of agent's position and velocity. Let's consider a simple scenario where their position and velocity remain positive values for a while. Then, the decoder will be overfitted to positive-valued trajectories, challenging to predict negative-valued trajectory. Trajectory Mirror exposes the model to the various rotation of systems to enhance the generalizability and adaptation speed in streaming systems (e.g., 36.8% acc. increase; Table 7 in the supplementary).\n\n**Q9. [Details in Trajectory Mirror]**  Trajectory Mirror is basically a Euclidean transformation. Multi-agent systems with pairwise interactions remain invariant under Euclidean transformations when the total interaction on each agent is a sum of quasi-linear functions (e.g., agent's relative positions). This implies that Euclidean transformations, such as Trajectory Mirror, do not alter the interaction dynamics of the system, which hold true for a wide range of natural and engineered multi-agent systems [2].\n\n[1]. \"Translational and rotational invariance in networked dynamical systems.\" IEEE Transactions on Control of Network Systems, 2017.\n\n[2]. \"Consensus and cooperation in networked multi-agent systems,” Proceedings of the IEEE, 2007.\n\n\n**Q10. [How Target Relation Evolves?]** As mentioned in line 221, the target interaction graphs randomly evolve. Each simulation follows a Bernoulli process with p=0.5 for each edge in the graphs. Please see Figure 2 in the attached pdf to check the correlation between accuracy and similarity between the graphs.\n\n**Q11. [Scalability of ORI]** Please see Table 1 in the attached pdf. ORI models still outperform the existing methods in spring systems with 15 and 20 agents.\n\n**Limitations** We will elaborate the limitation of ORI as the reviewer suggested.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DxwLtGjB5j", "reviewer": "Reviewer_arYB", "summary": "The paper introduces the Online Relational Inference (ORI) framework to identify hidden interaction graphs in evolving multi-agent systems using streaming data. The framework employs online backpropagation, updating the model with each new data point, thus adapting to dynamic environments in real-time. ORI features a trainable adjacency matrix optimized through an adaptive learning rate technique, which adjusts based on the historical sensitivity of the decoder to changes in the interaction graph. Experimental results on synthetic datasets and the CMU MoCap dataset demonstrate ORI's effectiveness in improving relational inference accuracy and adaptability.", "questions": "See above.", "limitations": "NA", "rating": 5, "confidence": 1, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper is well-written.\n2. The studied problem is interesting.\n3. The experimental validation on both synthetic and real-world datasets is sufficient.", "weaknesses": "1. The efficiency analysis is not sufficient. I suggest authors should include the comparison of training time.\n2. The compared baselines are weak and limited. Authors should include stronger baselines published in 2023-2024.\n3. The technical contribution is a little weak. The Eqn. 1 seems to be a natural way to optimize the graph structure. Moroever, I'm not sure whether there is overfitting when minimizing Eqn. 1. \n4. The limitation of the work is not sufficiently discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1. [Efficiency Analysis]** Thank you for the suggestion. We agree that more efficiency analysis on ORI will provide important information to readers. We compare the overall computational complexity, including the number of trainable parameters, FLOPs, and running time, in ORI with NRIr decoder and ORI with MPMr decoder with the encoder-based methods (e.g., NRI, MPM, and dNRI). \nIn terms of trainable parameters, NRI has 721.4k for encoder and 727.3k for decoder; MPM - 1724.9k for encoder and 1071.7k for decoder; dNRI - 883.7k for encoder and 269.8k for decoder. In terms of FLOPs per iteration, NRI shows 177.8MFLOPs for encoder and 5040.5MFLOPs for decoder; MPM – 3.9GFLOPs for encoder and 10.9GFLOPs for decoder. Note, FLOPs are estimated by fvcore package, but dNRI was not compatible. While the decoder and encoder have the similar level of trainable parameters (e.g., NRI and MPM), the decoder often shows much higher FLOPs due to the repeated trajectory prediction while the encoder predicts a single interaction graph per trajectory. \n\nFor the running time in a single 2080Ti GTX GPU, NRI shows 0.194sec/iter; MPM – 0.183sec/iter; dNRI – 0.096sec/iter. ORI with NRIr decoder shows 0.235sec/iter; ORI with MPMr decoder – 0.192sec/iter. Although ORI shows slightly higher running time than the encoder-based models, theoretically, as ORI does not have an encoder network and apply the gradient descent to much smaller adjacency matrix, its FLOPs is approximated to the only decoder’s FLOPs. In addition, note that, their running time will be largely depending on the decoder design as the decoder is the major computational bottleneck. \nWe observe that the latency in ORI occurs at the unoptimized code for two separate gradient descent (one for the decoder and another one for the adjacency matrix). We are working on optimizing the run-time of that step in ORI.\n\n**W2. [Recent Baselines]** The reviewer's concern is valid. We experimented another very recent baseline “GDP” [1] in springs and charged systems (same setup in Table 2 in main paper). GDP shows 68.9% accuracy in springs systems and 51.4% accuracy in charged systems. While the accuracy in the springs system is much higher than other encoder-based methods, it is still much lower than ORI-based methods (~96.4%). The performance gap is even higher in the changed system. Please see the result in Table 2 in the attached pdf. We consider exploring more recent baselines and applying ORI on top of these methods will be an interesting future work.\n\n[1] Pan, Liming, Cheng Shi, and Ivan Dokmanic. \"A Graph Dynamics Prior for Relational Inference.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 38. No. 13. 2024.\n\n**W3. [Technical Contribution]** we consider ORI made several novel contributions for evolving multi-agent systems. First, we introduced a new research direction on “online relational inference for evolving multi-agent systems”. As Reviewer fSnF and Reviewer mc8m mentioned in Strengths, the proposed method is the first attempt to tackle online relational inference with a model-agnostic framework, unexplored despite wide practical applications. \n\nIn addition, ORI incorporates novel adaptive learning rate technique, AdaRelation, and simple yet effective data augmentation technique, Trajectory Mirror. These two novel techniques are strategically combined with online backpropagation in the adjacency matrix-based encoder-free design, significantly enhancing the accuracy in various evolving multi-agent systems. Moreover, ORI integrates seamlessly with other NRI networks, boosting flexibility.\nWe believe our technical contributions will introduce an impactful research direction, providing useful information to the related community for later research, as stated by Reviewer fSnF (Strength 3). \n\n**W4. [Limitations]** While ORI is the first attempt to learn the relational inference in online learning scenario, there have been many different problem setups in offline learning scenario. For example, more challenging environments will incorporate directed interaction graphs or/and variable number of nodes. Our current experiments do not evaluate ORI in such non-ideal environments, limiting the potential of ORI in relatively ideal environments. We will add discussion and details on the limitation of ORI in the final paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank the reviewers fSnF, mc8m, arYB for their positive feedback: carefully designed experiments and analysis beneficial for later research (fSnF), the first model-agnostic online relational inference framework for multi-agent systems (mc8m), well-written with sufficient experimental validation on both synthetic and real-world datasets (arYB).

The attached pdf has three figures and two tables (reviewer fSnF: Figure 1, reviewer mcm8: Figure 2, Figure 3, Table 1, reviewer arYB: Table 2).

We hope we have addressed the reviewers’ concerns and questions regarding the paper and that they will reconsider their rating based on these discussion.

Meta-review: The paper presents an interesting setting for dynamic neural relational inference where the updates are made in a streaming fashion. It proposes an adaptive learning rate updating rule for the trainable adjacency matrix with encouraging empirical results. Please add clarifications and motivations made during the discussion to the final version, especially with Reviewer mc8m.
Decision: Accept (poster)