Learning Heterogeneous Interaction Strengths by Trajectory Prediction with Graph Neural Network
Abstract
Dynamical systems with interacting agents are universal in nature, commonly modeled by a graph of relationships between their constituents.
Recently, various works have been presented to tackle the problem of inferring those relationships from the system trajectories via deep neural networks, but most of the studies assume binary or discrete types of interactions for simplicity.
In the real world, the interaction kernels often involve continuous interaction strengths, which cannot be accurately approximated by discrete relations.
In this work, we propose the relational attentive inference network (RAIN) to infer continuously weighted interaction graphs without any ground-truth interaction strengths.
Our model employs a novel pairwise attention (PA) mechanism to reﬁne the trajectory representations and a graph transformer to extract heterogeneous interaction weights for each pair of agents.
We show that our RAIN model with the PA mechanism accurately infers continuous interaction strengths for simulated physical systems in an unsupervised manner.
Further, RAIN with PA successfully predicts trajectories from motion capture data with an interpretable interaction graph, demonstrating the virtue of modeling unknown dynamics with continuous weights.
Introduction
Dynamical systems with interactions provide a fundamental model for a myriad of academic ﬁelds, yet ﬁnding out the form and strength of interactions remains an open problem due to its inherent degeneracy and complexity.
Although it is crucial to identify the interaction graph of a complex system for understanding its dynamics, disentangling individual interactions from trajectory data without any ground-truth labels is a notoriously hard inverse problem.
There has been a long history and substantial amount of work on both inferring the network topology and the nonlinear interactions between interacting constituents from data [3, 5, 4, 22, 10, 8], along with the development of various measures to capture the relation between constituents (e g., Pearson correlations, mutual information, transfer entropy, Granger causality and variants thereof [21]).
Many of these inferences focus on speciﬁc systems with the necessity for a model prior, such as domain knowledge of the agent characteristics, proper basis construction, and detailed assumptions on the system dynamics.
Recently, by phenomenal advances in machine learning, adopting a neural network as a key component of the interaction inference has gained attention from researchers [24, 14, 26].
The key strength of these approaches comes from the fact that a neural network enables relatively free-form modeling of the system.
One inﬂuential work in this direction, neural relational inference (NRI) [14], explicitly infers edges by predicting the future trajectories of the given system.
But previous studies for extracting interaction graphs with neural networks [14, 25, 9] mainly focused on inferring edges with discrete edge types, which means that they are incapable of distinguishing the edges of different
[Figure]
interaction strengths with the same type.
Considering the common occurrence of such heterogeneous interaction strengths throughout diverse systems, the assumption of discrete edge types severely limits the expressibility of the model.
A weighted interaction graph can be expressed in the form of a connectivity matrix, which is a conventional adjacency matrix with continuous-valued entries that indicate the interaction strengths.
One recent study [19] tackles this problem of inferring the connectivity matrix by ﬁrst training the neural network and then additionally applying a ‘graph translator’ to extract continuous graph properties, which needs a ground-truth label to train.
Also, various attempts for continuous strength inference from machine learning researchers [11, 12, 2] often lack veriﬁcations through the dataset with ground-truth interaction strength.
Several studies from physicists [27, 17] employed perturbation analysis and response dynamics to infer continuous interaction strengths, and reported a good agreement with the ground-truth strength.
However, these correlation-based methods usually require more than 10 5 to 10 7 data points for the inference, which is a feasible size for an entire dataset but not for a single instance, and thus the direct application to experimental data is difﬁcult.
In the current work, we propose a neural network called Relational Attentive Inference Network (RAIN) to address the problem of inferring weighted interaction graphs from multivariate trajectory data in an unsupervised manner.
RAIN infers the interaction strength between two agents from previous trajectories by learning the attentive weight while simultaneously learning the unknown dynamics of the system and thus is able to precisely predict the future trajectories.
Our model employs the attention mechanism twice: once for the construction of pairwise trajectory embedding and once for the actual graph weight extraction.
Differing from previous approaches such as the graph attention network (GAT) [24], RAIN aims to infer the absolute interaction strength that governs the system dynamics.
By comparing the inferred interaction strengths of simulated physical systems with groundtruth values that are not provided at the training stage, we verify that RAIN is capable of inferring both system dynamics and weighted interaction graphs solely from multivariate data.
We further show that RAIN outperforms discrete baselines on real-world motion capture data, representing a system in which we cannot be certain whether a continuous form of interaction strengths even exists.
In this way, we demonstrate that the rich ﬂexibility and expressibility of the continuous modeling of interaction strengths are crucial for the accurate prediction of the future dynamics of an unknown empirical system.
Model description
Our RAIN model as shown in Fig.
1 comprises three parts trained jointly: an encoder that compresses time series data, a graph extraction module that infers the interaction weight between every pair of agents, and a decoder that predicts the future trajectories of each agent.
Note that RAIN does not
[Figure]
×
[Figure]
〈 , 〉 ××
[Figure]
× × × 〈 , 〉 〈 , 〉 〈 , 〉 〈 , 〉 〈 , 〉 t= t= t= t= t= t=
[Figure]
require a ground-truth interaction graph for training; instead, it produces an interaction graph as a byproduct of future trajectory prediction.
In the following, we formalize our model and describe each component in detail.
2.1 Encoder and pairwise attention
The long short-term memory (LSTM) encoder is composed of a single layer of gated recurrent unit (GRU) [6] with hidden state dimension d lstm = 128 .
The encoder receives T enc steps of trajectories x 1 , x 2 , . . .
, x T enc , each consisting of R state variables of N agents, and produces corresponding T enc hidden states, h 1 , h 2 , . .
, h T enc .
The last hidden state h T enc could preserve the information from the entire trajectory in theory.
But we found that a naive ﬁnal hidden state is insufﬁcient to fully capture the interaction strength in large-scale systems.
In this study, we propose a pairwise attention (PA) mechanism to effectively infer the interaction strength between two trajectories.
The intuition behind
PA is straightforward: a single hidden state cannot be a suitable choice for extracting the interaction strengths from every possible pair.
Thus, we calculate the attention between same-time hidden states to assign weights to their contribution, as depicted in Fig.
We use a slightly modiﬁed transformer [23] with m = 4 heads and a head dimension of d h = d lstm /m
=
32 to emphasize strong interaction.
See [23] for a detailed description of the transformer architecture.
Formally, the LSTM hidden states are processed into Key ( K ) , Query ( Q ) , and Value ( V ) matrices as follows,
[Equation]
where the symbol ⊕ indicates the concatenation of the matrices at the last dimension, f LSTM is the
GRU layer, x ti are the state variables of the i th agent at time t , and f pair ,X is a stack of multilayer perceptron (MLP) layers for K pair , Q pair , and V pair .
The superscripts t and m on X indicate the time and head number, respectively.
The attention-weighted hidden state ˜h is expressed as follows,
[Equation]
where α t,m pair ,ij is an attentive weight between agents i and j , and ˜h mi is a weighted hidden state for the m th attention head.
RAIN passes ˜h to the graph extraction module.
By employing the PA mechanism, we can reﬁne the LSTM hidden states by focusing on the speciﬁc time of interactions, which is unique to each agent pair.
We highlight that comparing the same-time hidden states only, rather than considering the full attention as in conventional settings, signiﬁcantly reduces the time complexity of the inference while achieving the goal of the extraction of temporal information.
Also, since the transformer can handle asymmetric relationships by differentiating key and query, PA can capture directional connections with ease.
2.2 Graph extraction module
The graph extraction module of RAIN is similar to the GAT [24].
The difference between ours and the original GAT is that RAIN has no prior knowledge of the underlying graph and thus the α graph attention value should infer the presence of the edge itself as well as its strength, while GAT aims to ﬁnd the relative importance between ﬁxed graph edges.
Also, the conventional inner-product attention of GAT is replaced with a transformer architecture in RAIN to achieve a more ﬂexible representation.
We use attention-weighted hidden states from the PA mechanism to apply the transformer as follows,
[Equation]
[Equation]
(8) where f graph ,X is a stack of MLP layers for K graph , Q graph , and V graph .
Note that instead of softmax which normalizes the attention weight, sigmoid activation σ is used for graph attention to obtain the absolute interaction strength.
This is because our main goal is to infer the ground-truth interaction strength, not the relative strength for a single instance.
Extracted graph attention α graph ,ij becomes a weight for the decoder module.
2.3 Decoder
The decoder shares the same GRU layer with the encoder module and the employ new value function,
V dec ,i , for the message aggregation.
For agent i , RAIN concatenates attention-weighted value vectors from other agents along with its own value vector as follows,
[Equation]
RAIN ﬁnally produces a prediction of the state variables of the next step using a stack of MLP layers f dec , f µ , and f σ ,
[Equation]
[Equation]
The outputs of f µ and f σ are r -dimensional vectors, each representing the means and variances of the difference of state variables.
RAIN samples the next state from a Gaussian distribution with f µ and f σ and adds the values to the previous state.
The decoder iterates this for T dec steps to predict the future states.
For training, we employ negative log-likelihood (NLL) loss for a Gaussian distribution, L NLL =
T dec (cid:88)
[Equation]
where ∆ y t,qi , µ t,qi , and σ t,qi are the true future state difference, predicted mean, and predicted variance of the state variable q of agent i at time t , respectively.
Experiments
We demonstrate the capability of RAIN by performing inference tasks with various model systems ranging from simulated physical systems to real motion capture data from a walking human.
All of
[Table]
the models and baselines are implemented with PyTorch and optimized with Adam [13].
Data and models are available at github.com/nokpil/RAIN.
See the Supplementary Materials for details on the data generation, model architectures, and training details.
3.1 Physical simulations
We simulated the trajectories of two physical systems: a spring-ball system and phase-coupled
Kuramoto oscillators [16].
Unlike previous studies [14, 25] in which every interaction strength w in the system is discrete and constant (commonly w = 1 ), we consider a more general setting where interaction strengths are drawn from a continuum and are thus heterogeneous.
Assuming that two agents i and j are interacting, the interaction strength for each system would be a spring constant k ij for a spring-ball system and a coupling weight w ij for a Kuramoto model.
Although our model can handle asymmetric interaction strength—thanks to the asymmetric nature of the transformer—we take the symmetric form of connectivity matrix k ij = k ji for the simulated systems.
For the spring-ball and Kuramoto systems, we ﬁrst select the edges between n nodes with probability p (excluding self-connections) to construct an interaction graph, and then assign a randomly sampled interaction strength to each edge from a uniform distribution U [0 , 1] while expressing non-assigned edges with zero interaction strength.
We generate 10k training samples and 2k validation samples for all simulated tasks.
The state variables consisting of trajectories are x, y, v x , v y (positions and velocities) for the spring-ball system.
For the Kuramoto oscillators, a concatenated vector of dφdt , sin φ , and intrinsic frequency ω are used to construct the trajectories, where φ is the phase of an oscillator.
For evaluation, we measure how accurately the model predicts future trajectories by mean squared error (MSE) and how well the model retrieves the original connectivity matrix by two forms of
Pearson correlation.
We ﬁrst gather every retrieved interaction strength a and corresponding true interaction strength k from all 2k validation samples and calculate the correlation in total ( ρ tot ).
Secondly, we independently calculate the correlation for each validation sample and take the average of the 2k samples ( ρ sample ).
One may interpret ρ tot as the overall performance of the model, while ρ sample indicates the expected correlation for a single instance at a test time.
We excluded diagonal trivial zeroes (due to no self-loop interaction) while calculating correlation, so the reported value is strictly lower compared to the case where every value in the connectivity matrix is used.
Correlations from the baseline models with discrete edge types are obtained by assigning continuous weights to each edge type according to every possible permutation and choosing the best one.
More precisely, we assign n weights of 0 , 1 n − 1 , 2 n − 1 , . .
, 1 to n edge types with n !
different assignments and report the highest correlation.
[Figure]
3.1.1 Results
Table 1 shows the correlations between the true interaction graph and the predicted interaction strengths from various models.
Following [14], we measure the correlation between trajectory feature vectors as Corr.
(Path) and the correlation between trained LSTM feature vectors as Corr.
(LSTM).
NRI(2) and NRI(4) indicate the NRI from [14] with 2 and 4 edges types, respectively.
We check the effect of PA by testing the RAIN model both with and without the PA mechanism.
In Table 1, we can clearly see that RAIN with PA signiﬁcantly outperforms every other model at inferring interaction strengths.
Note that the high correlations from the NRI models for the spring-ball system can only be achieved if and only if we know the ground-truth interaction strength and choose the best permutation.
Considering the huge performance difference between RAIN without and with PA, our model and PA mechanism are both crucial for accurate interaction strength retrieval.
We show
MSE results for predicting 50 future states in Table 2, Here, again following [14], SingleLSTM runs a single LSTM for each object separately, while JointLSTM concatenates all state vectors (thus may handle a ﬁxed number of agents only) and trains a single LSTM to jointly predict all future states.
See the Supplementary Materials for further analysis, including full histograms of the correlation distributions.
Figures 3 and 4 show the results of future prediction and interaction strength inference forinteracting agents, where 5 trajectories are drawn for visualization.
In Fig.
3, we can observe that all models, including NRI(2), succeed in capturing the existence of interaction between agents.
But it is apparent that the NRI(2) model fails to capture the interactions with small weights, while RAIN without PA yields in numerous false positives with spurious weights.
Such weakness of each model is reﬂected in relatively large prediction errors compared to RAIN with PA, especially for ball 4 (green) and ball 5 (purple).
The power of the PA mechanism becomes more evident with the Kuramoto oscillators (Fig. 4), where only RAIN with PA succeeds in retrieving meaningful interaction weights.
To sum up, we can conclude that the PA mechanism clearly helps the reﬁnement of hidden states and thus yields better results with a greater correlation with the ground-truth interaction weights.
3.1.2 Impact of weak interaction
By inferring interaction with a continuous strength as RAIN does, we can capture an entire spectrum of interactions with a single model.
Particularly, we ﬁnd that our model is able to detect weak interactions that are often ignored by the discrete NRI models due to their limited capacity.
In Figure 5, we emphasize the signiﬁcance of the inference of weak interaction by constructing the connectivity matrix in a form of a multilayer network [15] with 2 layers.
Here, we prepare two densely connected layers of springs where their spring constants are sampled from [0 . 5 , 1] , uniformly.
Between the two layers, we set a single inter-layer link (connecting ball 3 and ball 8 ) with a spring constant of 0 . 3 , which is much smaller than the intra-layer interaction weights.
Since the synchronization of a multilayer network largely depends on the strength and structure of inter-layer coupling [18, 7], capturing this weak but signiﬁcant connection between the two layers is critical for trajectory prediction.
As Fig.
5 shows, RAIN with PA accurately captures this weak interaction, while NRI(2) misses it and thus its predicted trajectory considerably deviates from the true one. [Figure]
3.2 Motion capture data
Next, we test our model in inferring interactions between the joints of a walking human data from the
CMU motion database [1].
Different from the physical simulations, this real-world system does not possess any well-known dynamics with a continuous interaction strength.
Following [14] and [9], we use the data from subject # 35 with 31 joints.
We split the training and validation data sets into a 4 :ratio.
We use the same protocol as with the physical systems: provide 50 time steps of trajectories and let the model predict 50 unseen time steps afterward.
For the motion data, we train the model to predict the raw value of the future state instead of the difference from the current state.
Figure 6A shows the errors from each model’s prediction, where RAIN with PA achieved the lowest
MSE.
An example visualization of the prediction is shown in Fig.
6B. Without dynamically reevaluating the interaction graph at every time step as reported in [14] and [9], our model precisely predicts the future states from static but continuous interaction weights.
Also, as shown in Fig.
6C, our model produces a smooth and block-wise connectivity matrix which is much easier to interpret compared to that from the discrete NRI(2).
This is because there is no restriction or natural meaning for each edge type in the discrete model, and thus the manual sparsity prior is needed to handle the ‘no interaction’ edges if an abundance is expected.
On the other hand, the attention value of
RAIN is expected to convey the strength of interaction directly.
For instance, it is clear from RAIN’s connectivity matrix that the dynamics of the feet are less correlated with the rest of the joints and also that the spine moves along with the right leg, arm, and hand, both of which agree with the qualitative movements of a human while walking.
Conclusion
In this work, we introduced RAIN, a model for inferring continuous weighted interaction graphs from trajectory data in an unsupervised manner.
With the PA mechanism that computes the attention between same-time LSTM hidden states between agents, we can sharply reﬁne the representation for the interaction weight extraction.
Our model successfully inferred the absolute interaction weights from simulated physical systems and further showed great prediction performance with an empirical system, demonstrating the advantage of continuous weight modeling in relational learning.
Notably,
RAIN needs only 50 time steps of data to infer the interaction weights, and thus requires less data by several orders of magnitude than correlation-based theoretical methods [27, 17].
Also, we found that the reﬁnement of the LSTM hidden state with PA is critical for meaningful performance.
Since the PA mechanism is generally applicable to a neural model where its trajectory is represented by a recurrent neural network, one may expect an increase in performance by employing PA in other relational models without increasing the inference time signiﬁcantly.
In the real world, interactions between agents in complex systems possess a broad range of characteristics.
For instance, they can [Table] be either positive (excitatory or encouraging) or negative (inhibitory or suppressing), time-delayed with heterogeneous time scales, and noisy both inherently and externally.
Complex systems in nature generally contain every aspect of these characteristics, such as neural signals in a human brain.
Reinforcing the current RAIN architecture to handle data of such complex nature with a single model would be a promising future direction to explore.
Data availability
Simulation code and data ﬁles are available at https://github.com/nokpil/RAIN.  Acknowledgement
This research was supported by the Basic Science Research Program through the National Research
Foundation of Korea NRF-2022R1A2B5B02001752 (HJ) and National Research Foundation of
Korea Grant funded by the Korean Government NRF-2018S1A3A2075175 (SH).
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information on how to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] or [N/A] .
You are strongly encouraged to include a justiﬁcation to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .
• Did you include the license to the code and datasets?
[No] The code and the data are proprietary.
• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] See Abstract and Introduction.
(b) Did you describe the limitations of your work?
[Yes] See Conclusion.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[N/A] (b) Did you include complete proofs of all theoretical results?
[N/A] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See Data availability.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See Supplemental material.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[N/A] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Supplemental material.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] See the manuscript where we cite [1] for its dataset.
(b) Did you mention the license of the assets?
[N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
[Yes] See Data availability.
(d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
A Appendix A.1 RAIN implementation
For RAIN, we globally adopt Mish [20] as a non-linear activation function which has a functional form as follows.
[Equation]
Hereafter, we denote the batch dimension as B , the number of input time step as T , the number of agent as N , the number of state variables as S , the agent number as N .
Typically, B = 128 , T = 50 ,
S =
(Kuramoto), 4 (Spring), 6 (motion), and N = 5 , 10 , 31 (motion).
A.1.1 Encoder
Intially, the model recieves the input data with the size of B × T × N × S at every batch.
We ﬁrst encoding each state with MLP with a 2-layer MLP with hidden dimension of 64 and output dimension of 128 .
The encoded state is then feeded to a sigle GRU layer with hidden dimension of 256 , which produces the LSTM hidden states with the size of B × N × 256 for both future predictions and the grpah extraction.
For the initial hidden state, we also train a single-layer MLP with output dimension of 256 , which recieves the raw state variables (at time 0 ) and produces the initial hidden states for our
GRU layer.
A.1.2 Pairwise attention For
pairwise attention (PA), we use transformer [23] with 4 heads with 32 dimensions each, total of 128 dimension.
Hence, the key (L), query (Q), and value (V) is calculated via a linear layer with input dimension of 256 and output dimension of 128 (to match with the transformer dimension, 32 × 4 = 128 ).
After we perform PA by calculating the same-time attention and weighted average, the reﬁned hidden states goes to the graph extraction module.
Since PA produces a single hidden states for every pair of agents, we now have the output with the size of B × N × N × 128 , at this stage.
A.1.3 Graph extraction module
For graph extraction, we employ a 3-layer MLP with hidden dimension of 32 , 16 and output dimension of 1 .
This module makes the reﬁned hidden states into the size of B × N × N × 1 , which indicates the
N × N weighted interaction graphs that our module has inferred.
Note that the RAIN model without
PA uses two concatenated individual LSTM hidden states (from agent i and j ) for the graph extraction, and thus the input dimension of the graph extraction module becomes 2 × ( LSTM dimension ) =(and the same as RAIN with PA afterwards).
Finally, we add diagonal mask ﬁlled with −to the result and apply a sigmoid function.
This effectively reduces every diagonal entries into( σ ( − 10000) ≈ 0 ) and normalize the scale of the interaction strength into [0 , 1] .
A.1.4 Decoder
In decoding stage, we ﬁrst feed the current trajectory data to the LSTM encoder and gets the hidden states for current time step.
We then calcualte the another value function with a linear layer, V dec , with output dimension of 128 .
For a prediction of single agent, we concatenate two vectors; its own value and weighted average of all other value, according to the weights from the graph extraction module.
Hence, the input dimension of the decoder module should be 2 × 128 =
(since two value functions are concatenated).
The decoder of RAIN is consists of three MLPs: one primary 2-layer MLP with hidden and output dimension of 256 for hidden states encoding, and two secondary 2-layer MLPs with hidden dimension of 256 and output dimension of S .
After we passes the concatenated values to the primary MLP, we further feeds the results to two secondary MLPs for yielding means and variances of the desired output.
A.1.5 Training details
For training, we iterate the decoder 50 times while sampling the trajectory data from generated means and variances, and compared it with state difference to calculate NLL loss.
We perform gradient descent at each iteration step, let gradient ﬂows through backward time and yield 50 losses in total.
We average all the NLL losses and perform backward propagation.
A.2 Baseline implementation
In the current work, we trained various baselines to compare the performance with our model.
More precisely, we trained SingleLSTM, JointLSTM, NRI(2), NRI(4), and both RAIN with and without
PA.
The SingleLSTM model consists of a single 2-lyaer LSTM with hidden dimension of 256 .
Similar to RAIN, we train the model by feeding 50 previous states and aim to predict 50 future states, but individually.
Hence, there are no information communicated between agetns.
Furthermore, we used the hidden states of the second layer of LSTM in SingleLSTM for calcualting Corr (LSTM), by measuring the Pearson correlation between two agent’s hidden states and again maesure those correlations to the ground-truth interaction strengths.
For the JointLSTM model, we ﬁrst concatenate all states from the agents, which yields the vector with dimension of N × S .
A single LSTM is trained to recieve this concatenated vector, and produce a single (again, concatenated) vector to predict the future states of all agents at once.
This is the most naive form of neural network that (in theory) enables the information communication between agents.
Other trainig protocols are same as RAIN.
For NRI(2) and NRI(4), we faithfully followed the original training protocol of [14], includingsteps of burning-in process.
We set the hidden state dimension as 256 and also used learning rate scheduling as mentioned in the original paper.
We did not employ dynamical graph re-evaluation, since we wanted to compare the performance between static graphs.
A.3 Dataset details A.3.1 Spring-ball system
Spring-ball systems are consist of N balls and connecting springs between them.
The interaction between balls are Hookean force, F ij = − k ij x ij where k ij is a spring constant and x ij is a relative position vector between two balls.
In this study, we aim to infer k ij without knowing any prior information about the dynamics.
All of the N balls are initially start with random 2D positions and velocities that is drawn from the normal distribution N (0 , 1) .
Here, velocities are further normalized to have a vector norm of 1 .
The balls are kinetically moving in a square box with sides of length 5 , centered at the origin.
The collision between every object is perfectly elastic (In practice, the size of the ball is ignored and the collision between balls never happens).
We simulate each trajectory 1000 times steps with time interval dt = 0 .
005 , and it is further subsampled every 10 steps to construct total 100 steps of [Figure] training data; 50 steps for input and 50 steps for validation.
In the simulation, the spring constant is drawn uniformly from [0 , 1] multiplied by a constant factor 0 .
05 for stability.
A.3.2 Phase-coupled oscillators
For Kuramoto oscillators, each ball its intrinsic frequency w i and interacting with each other by following differential equation:
[Equation]
with coupling constants k ij , drawn uniformly from [0 , 2] .
We simulate the system by solving (17) with fourth-order Runge-Kutta integrator with a step size dt = 0 .
Intrinsic frequency w i ∈ N and initial phase φ t =
i is drawn uniformly from [1 , 10] and [0 , 2 π ) , respectively.
A.3.3 Motion capture data
As mentioned in the main manuscript, we used the data of subject # 35 from CMU motion databse [1].
The trial number spans from 1 to 16 , and 28 to 34 , following [9].
We used data processing codes from [9] to preprocess the amc ﬁles into the trainig and validation dataset.
By split the data by a length of 100 ( 50 input steps and 50 future steps), we got 63 sequences of train data and 23 sequences of validation data.
A.4 Further analysis
The effect of PA mechanism is further illustrated by plotting distributions of correlations in Figure 7.
Here,