Learning interaction rules from multi-animal trajectories via augmented behavioral models
Authors: Keisuke Fujii
Naoya Takeishi
Kazushi Tsutsui
Emyo Fujioka
Nozomi Nishiumi
Ryooya Tanaka
Mika Fukushiro
Kaoru Ide
Hiroyoshi Kohno
Ken Yoda
Susumu Takahashi
Shizuko Hiryu
Yoshinobu Kawahara
Venue: NeurIPS 2021 Poster (2021)
Keywords: multi-agent trajectory
Granger causality
neural networks
interpretability

Abstract: Extracting the interaction rules of biological agents from movement sequences pose challenges in various domains. Granger causality is a practical framework for analyzing the interactions from observed time-series data; however, this framework ignores the structures and assumptions of the generative process in animal behaviors, which may lead to interpretational problems and sometimes erroneous assessments of causality. In this paper, we propose a new framework for learning Granger causality from multi-animal trajectories via augmented theory-based behavioral models with interpretable data-driven models. We adopt an approach for augmenting incomplete multi-agent behavioral models described by time-varying dynamical systems with neural networks. For efficient and interpretable learning, our model leverages theory-based architectures separating navigation and motion processes, and the theory-guided regularization for reliable behavioral modeling. This can provide interpretable signs of Granger-causal effects over time, i.e., when specific others cause the approach or separation. In experiments using synthetic datasets, our method achieved better performance than various baselines. We then analyzed multi-animal datasets of mice, flies, birds, and bats, which verified our method and obtained novel biological insights.

Learning interaction rules from multi-animal trajectories via augmented behavioral models
Keisuke Fujii"
Naoya Takeishi University of Applied Sciences and Arts Western Switzerland RIKEN Center for Advanced Intelligence Project
Nagoya University
RIKEN Center for Advanced Intelligence Project JST PRESTO
Kazushi Tsutsui
Emyo Fujioka Doshisha University
Nozomi Nishiumi National Institute for Basic Biology
Ryoya Tanaka Nagoya University
Nagoya University
Mika Fukushiro Doshisha University
Kaoru Ide Doshisha University
Hiroyoshi Kohno Tokai University
Ken Yoda
Nagoya University
Susumu Takahashi Doshisha University
Shizuko Hiryu Doshisha University
Yoshinobu Kawahara
Kyushu University
RIKEN Center for Advanced Intelligence Project
Abstract
Extracting the interaction rules of biological agents from movement sequences pose challenges in various domains. Granger causality is a practical framework for analyzing the interactions from observed time-series data; however, this frame- work ignores the structures and assumptions of the generative process in animal behaviors, which may lead interpretational problems and sometimes erroneous assessments of causality. In this paper, we propose a new framework for learning Granger causality from multi-animal trajectories via augmented theory-based be- havioral models with interpretable data-driven models. We adopt an approach for augmenting incomplete multi-agent behavioral models described by varying dynamical systems with neural networks. For efficient and interpretable learning, our model leverages theory-based architectures separating navigation and motion processes, and the theory-guided regularization for reliable behavioral modeling. This can provide interpretable signs Granger-causal effects over time, i.e., when specific others cause the approach or separation. In experiments using synthetic datasets, our method achieved better performance than various baselines. We then analyzed
datasets of mice, flies, birds, and bats, which verified our method and obtained novel biological insights.
Introduction
Extracting the interaction rules of real-world agents from data is fundamental problem in variety of scientific and engineering fields. For example, animals, vehicles, and pedestrians observe other's states and execute their actions in complex situations. Discovering the directed interaction rules of such agents from observed data will contribute to the understanding of the principles of biological
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

agents' behaviors. Among methods analyzing directed interactions within multivariate time series, Granger causality (GC) [23 is practical framework for exploratory analysis 49 in various fields,
such as neuroscience 655 and economics
3 (see Section 2). Recent methodological developments
have focused on inferring GC under nonlinear dynamies (e.g., 73 81
55
However, the structure of the generative process in biological multi-agent trajectories, which include navigational and motion processes (54) regarded time-varying dynamical systems (see Section 3.T. is not fully utilized in existing base models of GC including vector autoregressive [27] and
recent neural models 773 31 81 Ignoring the structures of such processes in animal behaviors will lead to interpretational problems and sometimes erroneous assessments of causality. That is, incorporating the structures into the base model for inferring GC, e.g., augmenting (inherently) incomplete behavioral models with interpretable data-driven models (see Section (.2). can solve these problems. Furthermore, since data-driven models sometimes detect false causality that is counterintuitive to the user of the analysis, e.g., introducing architectures and regularization to utilize will be effective for a reliable base model of GC
scientific knowledge (see Sections 3.2 and 2
method.
In this paper, we propose framework for learning GC from biological multi-agent trajectories via augmented behavioral models (ABM) using interpretable data-driven neural models. We adopt an approach for augmenting incomplete multi-agent behavioral models described by time-varying dynamical systems with neural networks (see Section 3.2). The ABM leverages theory-based archi- tectures separating navigation and motion processes based on a well-known conceptual behavioral model [54), and the theory-gure -hidee regularization (see Section 4.2) for interpretable and reliable behavioral modeling. This framework can provide interpretable signs of Granger-causal effects over time, e.g., when specific others cause the approach or separation.
The main contributions of this paper are as follows. (1) We propose framework for learning Granger causality via ABM, which can extract interaction rules from real-world multi-agent and multi-dimensional trajectory data. (2) Methodologically, we realized the theory-guided regularization for reliable biological behavioral modeling for the first time. The -huide regularization can leverage scientific knowledge such that "when this situation occurs, it would be like this" (i.e., domain experts often know an input and output pair of the prediction model). Existing methods in Granger causality did not consider the utilization of such knowledge. (3) Biologically, our methodological contributions lies in the reformulation of a well-known conceptual behavioral model [54], which did not have a numerically computable form, such that we can compute and quantitatively evaluate (4) In the experiments, our method achieved better performance than various baselines using synthetic datasets, and obtained new biological insights and verified our method using multiple datasets of mice, birds, bats, and flies. In the remainder of this paper, we describe the background of GC in Section 2 Next, we formulate our ABM Section 3 and the learning and inference methods in Section
2 Granger Causality
GC [3 is one of the most popular and practical approaches to infer directed causal relations from observational multivariate time series data. Although the classical GC is defined by linear models, here introduce more recent definition of [73 for non- mon-linear GC. Consider p stationary time- series
x = {x ... across timesteps = {1, [l.,, T) and a non- non-linear autoregressive function 9j. such that
x = g, (a x²2)) + +
(1)
where 2/ = (-.,, xz)) denotes the present and past of series and el represents independent noise. We then consider that variable x' does not Granger-cause variable x', denoted as x' -s x', if and only if 9,(() is constant in x'z,- Granger causal relations are equivalent to causal relations in the underlying directed acyclic graph if all relevant variables are observed and no instantancous (i.e., connections between variables at the same timestep) connections exist [59]. Many methods for Granger causal discovery, including vector autoregressive [27] and recent deep learning- can be encapsulated by the following framework. First, define function approaches 77 31 81.
fe (c.g., an multilayer perceptrons (MLP) in [73), linear model in (27)). which learns to predict the next time-step of the test sequence æ. Then, we fit fe to x by minimizing some loss (e.g., mean squared error) L: 0, = argmin, L(x, fa). Finally, we apply some fixed function h (e.g., thresholding) (c.g., (55) to the learned parameters to produce a Granger causal graph estimate for x: Go h(0.).
2

---

Furthermore, we need to differentiate between positive and negative Granger- causal effects (e.g. approaching and separating), Based on the definition of 45)) we define the effect sign as follows: if (-)()) is increasing in all a'ge= then we say that variable a² has positive effect on x² if gj (:) is decreasing in a'ze then æ² has a negative effect on æ Note that æf can contribute both positively and negatively to the future of x² at different delays.
Overall, the causality measures, however elaborate in construction, are simply statistics estimated from a model [7T]. If the model inadequately represents the system properties of interest, subsequent analyses based on the model will fail to address the question of interest. The inability of the model to represent key features of interest can cause interpretational problems and sometimes erroneous assessments of causality. Therefore, in our case, incorporating the structures of the generative process for animal behaviors (i.e., Eq. 2 in a numerically computable form will be required. We thus propose the ABM based on well-known conceptual model (54) in biological sciences in the next section.
3 Augmented behavioral model
Our motivation for developing interpretable behavior models is to obtain new insights from the results of Granger causality. In this section, we firstly formulate a well-known conceptual behavioral model [54] so that can be computable. Second, we propose (multi-an mnulli-animal) ABMs with theory- based architectures based on scientific knowledge. Further, we discuss the relation to the existing explainable neural models [ The diagram of our method is described in Appendix C
3.1 Formulation of a conceptual behavioral model
In movement ecology, which is a branch of biology concerning the spatial and temporal patterns of behaviors of organisms, coherent framework [54] has been conceptualized to explore the causes, mechanisms, and patterns of movement. For example, two alternative structural representations [54] were proposed to model new position p++1 from its current location p: (for details, see Appendix AA: the motion-driven case p = (f /N fN (#, rr, Drr,, Wr, Pt). Tr, Wr, Pt)) + Et, and ) + Et, where w, the internal state, Ω is the motion capacity, 4 is the navigation capacity, and Tı is the environmental factors (these are conceptual parameters). fas. fN, and fu are conceptual functions to represent actions of the motion (or planning), navigation, and movement progression processes, respectively.
the navigation-driven case Pi+1 = fu (JN (b, fas (22, Wt, Pt), rr, Wt, Pt))
For efficient learning of the weights in the model (i.e., coefficient of Granger causality) in this paper, we consider simple case with homogeneous navigation and motion capacities, and internal states. Moreover, to make the contribution of fM. fNN and fu interpretable after training from the data for extracting unknown interaction rules (and for obtaining scientific new insights), one of the simplified processes for agent is represented by
a fi (r), x)). fhe (r) 2f), r/ at) + +
where x dR includes location p' and velocity for the agent i. We here consider r' R(p-1)d, including p- - other agents' d,-dimensional information. This formulation does not assume either driven navigation-driven case. Such behaviors have been conventionally modeled by mathematical equations such as force- and rule-based models (e.g., reviewed by (77 (7)). Recently, these models have become more sophisticated by incorporating the models into hand-crafted functions representing anticipation (e.g.. (29 51) and navigation (e.g., 761).
However, these conventional and recent models are sometimes too simplistic and customized for the specific animals, respectively; thus it is sometimes difficult to define the dynamics of general biological multi-agent systems (i.c., multiple species of animals). Therefore, methods for learning parameters and interaction rules of behavioral models are needed. There have been some researches to estimate specific parameters (and their distributions) of the interpretable behavior models (c.g.. 84 85 [5]), and others to model the parameters and rules in purely data-driven manners (i.e., sometimes uninterpretable) only for accurate prediction (e.g., [16 28 ). In the proposed framework, we consider flexible data-driven interpretable models to focus on inferring GC for exploratory analysis from the observed data without specific knowledge of the species and obtaining additional data.
Recently, some attempts have been made to explore flexible and interpretable models bridging theory- based and data-driven approaches. For example, a paradigm called theory-guided data science has
3

---

been proposed (30,, which leverages the wealth of scientific knowledge for improving the effective- ness of data-driven models in enabling scientific discovery. For example, scientific knowledge can be used as architectures regularization terms in learning algorithms in physical and biological sciences (e.g., [6] 21). In biological multi-agent systems, an approach extract interpretable dynamical information based on physics- based knowledge [18] from multi-agent interaction data, and another approach made a particular module such as observation (e.g., [9] interpretable in mostly black-box neural models. However, these data-driven models did not sufficiently utilize the above scientific knowledge of multi-animal interactions. In the next subsection, to make the model (e.g., of GC) flexible and interpretable, we propose ABM with theory- based architectures.
3.2 Augmented behavioral model with theory-based architectures
In this subsection, we propose a ABM using interpretable neural models with theory-based archi- tectures for learning GC from multi-animal trajectories. In general, it is scientifically beneficial if a model mimics the genteranngg process well, e.g., because existing scientific insights can be leveraged or revalidated. our case of GC, additionally, is expected to eliminate obvious erroneous causality by utilizing existing knowledge, we thus propose a theory-based ABM for learning GC.
Generally, scientific knowledge can be used to influence the architecture of data-driven scientific models. Most design considerations are mainly motivated to simplify the learning procedure, mini- mize the training loss, and ensure robust generalization performance [00] In some cases, domain knowledge can be used designing neural models by decomposing the overall problem into modular sub-problems. For example, in our problem, to describe the overall process of multi-animal behaviors, modular neural models can be learned for different sub-processes, such as the navigation, planning, and movement processes U/N/iia and Jü. respectively) described in Section 3.1 This will help in using the power of learning frameworks while following a high-level organization in the architecture that motivated by domain knowledge (30). Specifically, accurately model the relationships between agents (finally interpreted as causal relationships) with limited information in usual GC settings, we explicitly formulate the J' and ft and estimate f'v and f'M from data.
In summary, our base ABM can be expressed as
Σ brikk
(hi
Fir (hi hi + e
where h2 E RR is a vector concatenating the self state z/-x Rd and all others' state ri- R(0- 1)d, and O denotes a element-wise multiplication. K is the order of the autoregressive model. Fx .P dh + Rdxd, are matrix-valued functions that represent navigation and motion functions, which are implemented by MLPs. For brevity, omit the intercept term here and in the following equations. The value of the element of r is [-1, 1] is like a switching function value, i.e., a positive or negative sign to represent the approach and separation from others. The value of the element of Fit is positive value or zero, which changes continuously and represents coefficients of time-varying dynamics, Relationships between agents x² "e",....""" x and their variability throughout (...t.k (F*+)(((())))) O F (hi-
time can be examined by inspecting coefficient matrices &
We separate Vo into Frith prit.k( (hi_.) and Fiit p (hi_x) for two reasons: interpretability and efficient use of scientific knowledge. The interpretability of two coefficients F* and F contributes to the understanding of navigation and motion planning processes of animals (i.e., signs and amplitudes in the GC effects), respectively. The efficient use of scientific knowledge in the learning of model enables us to incorporate the knowledge into the model. The effectiveness was shown in the ablation studies in the experiments. Specifie forms of Eq. (3) are described in Appendices E.2 and The formulation of the model via linear combinations of the interpretable feature h; for an explainable neural model is related to the self-explanatory neural network (SENN) 2
3.3 Relation self-explanatory neural network
SENN 2 was introduced as a class of intrinsically interpretable models motivated by explicitness, faithfulness, and stability properties. A SENN with a link function g(-) and interpretable basis concepts h(x)): RP + IR" follows the form
f(x) ((((()))(()))))))))))))))))))(() (()))))))...))))h()))
(4)

---

B for each agent is summarized as the where a E Rº are predictors; and 8(-) is a neural network with outputs (here, we consider the simple case of = 1 and d, 1). We refer to ((x) as generalized coefficients for data point æ and use them to explain contributions of individual basis concepts to predictions. In the case of g(.) being sum and concepts being raw inputs, Eq. (4) simplifies to f((x) = ((rr)(a))a
this paper, we regard the movement function fi as g(-) and the function of I'v and f'r as for the following interpretable modeling of fū f'x
and fär- Appendix B presents additional properties SENNs need to satisfy and the learning algorithm, as defined by [2.. Note that our model does not always satisfy the requirements of SENN [2 45] due to the modeling of time-varying dynamies (see Appendix B). SENN was first applied to GC 45 via generalized vector autoregression model (GVAR): Ex, where Ve, IP + IPPP is neural network parameterized by (x)) is a matrix whose components correspond to the generalized coefficients for lag at timestep 1. The component (i, of ee +b) corresponds to the influence of a on at-- However, the SENN model did not use scientific knowledge of multi-element interactions and may cause interpretational problems and sometimes erroncous assessments of causality.
3t 1= Me, (Zt-
Og. Vo
4 Learning with theory-guided regularization and inference
Here, describe the learning method of the ABM including theory- regularization. We first overview the learning method and define the objective function. We then explain the theory-guided regularization for incorporating scientific knowledge into the learning of the model. Finally, describe the inference of GC by our method. Again, the overview of our method is described in Appendix C
4.1 Overview
To mitigate the inference in multivariate time series, Eq. following expression:
K
2z >
[ F (hl FlitA plitt((
(hL-2)) h hi ki.......
FR (KP_ he- Ec. (S)
where 2, and E, concatenate the original variables for all p agents (various Fs and Psis are learned parameters). We train our model by minimizing the following penalized loss function with the mini-batch gradient descent
Σ (Epred((s, 2x) + AL ACsparotty (7.) YCo v v( t=
Σ K=
BLooooth P D.),
(6)
where (3rx)=m1 is single observed time series of length T with d-dimensions and p-agents; à; is the one-step forecast for the t-th time point based on Eg.
(5) 1 RddxKdn is defined as concatenated 1 . in a row; 8 is coefficient determined by the following theory-guided regiiddddrgglannne regularization; and X, 3, 7 2 0 are regularization parameters. The loss function in Eq. 66) consists of four terms: (i) the mean squared error (MSE) prediction loss, (ii) -induiit-induiingg penalty term, (iii) theory-guided regularization, and (iv) the smoothing penalty term. The sparsity-inducing term Esparsity is an appropriate penalty on the norm of L Among possible various regularization terms, in our implementation, we employ the clastic-net-style penalty
matrix of [ [ 0 Vt J......... 2 V
,..x''...
Csparesty! (L)) = -ース (a (a + (1 - a) (.lF.)) with a = 0.5, based on [45)
term 86 56)
Note that other penalties be also easily adapted to model. The smoothing penalty term, given by femoothh ( 1411 1),) -- | V I1 is the average norm of the difference between generalized coefficient matrices for two consecutive time points. This penalty term encourages smoothness in the evolution of coefficients with respect to time [45]. To avoid overfitting and model selection problems, eliminate unused factors based prior knowledge (for details, see Appendices E.2 and G.2
4.2
Theory-guided regularization
The third term in Eq. (6) is the theory-guided regularization for reliable Granger causal discovery by leveraging regularization with scientific knowledge. Here we utilize theory-based and data-driven prediction results and impose penalties in the appropriate situations as described below. Again, let é; be the prediction from the data. In addition to the data, prepare some input-output
5

---

pairs (it-kSi, 22) based on scientific knowledge. We call them pairs of theory-guided feature and prediction, respectively. In this case, we assume that the theory-guided cause or weight of the ABM V." is uniquely determined. When the difference between åt and Th is below a certain threshold, we assume that the cause (weight) of á; is equivalent to the cause of att.
In animal behaviors, the theory-guided prediction utilizes the intuitive prior knowledge such that the agents go straight from the current state if there is no interaction. In this case, T, includes the same velocity as the previous step and the corresponding positions after going straight. The penalty is expressed as LTa ,,, V --R exp(1læ; 221l//o 2
where Vi E (gddx((x11)) (p-1)d,, is the weight matrix regarding others' information (i.c., climinating the information of the agents themselves from 1 and a is parameter regarding the threshold. Note that here the matrix L TG corresponding to TG is a zero matrix representing no interaction with others (i.e., 1 - V 1|? = =1
Next, we consider the general cases. All possible combinations of the pairs are denoted as the direct product Ho := L x M mp E M), where L = fl..... .) PP] and M = (-1,0.11 if we consider the sign of Granger causal effects (otherwise, M = (0.1)) However, if we consider the pairs (it- uniquely determined, it will be considerably fewer number of combina- tions by avoiding underdetermined problems. We denote the set of the uniquely-deterined combinations Hi C Ho. We then impose penalties the weights: LTG , v
x M {(l m mp) |[ E L A mi E
M A
HH](T- Exma....,,ee (exp(Ilae, ã. lo) V "r..m.......... 11%-) where V WTG f.I.ma.......
R ((--))dr is the weight matrix regarding others' information in Lr In animal behaviors, due to unknown terms, such as inertia and other biological factors, the aecory-guided prediction utilizes the only intuitive prior knowledge such that the agents go straight from the current state if there are no interactions (i.e., [Hi] = 1).
4.3 Inference of Granger causality
Once L is trained, we quantify strengths of Granger-causal relationships between variables by aggregating matrices V across all K, K,d, d, f into summary statistics. Although most neural GC methods [73 [55 31 45] did not provide an obvious way for handling multi-dimensional time series (i.e., d > 1). our main problems include two- or three-dimensional positional and velocity data for each animal. Therefore, we compute the norm with respect to spatial dimensions d, d,, and the
sign of the GC separately. That is, we aggregate the obtained generalized coefficients into matrix S E IRpXP as follows:
median (Fi.s) @=I....d ....
)
Si.5 = signmax
signmax K+ISIST IskgK
max K-ISIST
max (1SKSK
(7)
where j. -R)xxxxxxd is computed by reshaping and concatenating L over K+1S15 T. (L) 2. ||r is the Frobenius norm of the matrix (V.r)).. Rddxd, for each i, k. The signmax is an original function to output the sign of the larger value of the absolute value of the maximum and minimum values (e.g., signmax ((1, 2,,2, -3}) = -1). If we do not consider the sign of Granger causal effects, we ignore the coefficient of the signed function. If we investigate the GC effects we eliminate max function among l. Note that we only consider off-diagonal elements of adjacency matrices and ignore self-causal relationships. Intuitively, St,j are statisties that quantify the strength of the Granger-causal effect of æ² on x using magnitudes of generalized coefficients. We expect Si, to be close to 0 for non-causal relationships and St >> 0 x. + x². Note that in practice S not binary-valued, as opposed to the ground truth, which we want to infer, because the outputs of Mis are not shrunk to exact zeros. Therefore, we need procedure deciding for which variable pairs Si are significantly different from 0. To infer binary matrix of GC relationships, we a heuristic threshold. For the detail, see Appendix D.I
time,
Related work
Methods for nonlinear GC. Initial work for nonlinear GC methods focused on time-varying dynamic
Bayesian networks 77) regularized logistic regression with time-varying coefficients (34), and kernel-
based regression models [46] 699 (9]] Recent approaches to inferring Granger-causal relationships
leverage the expressive power of neural networks [50] 79
73
55
31
43
81 and are often based on

---

regularized autoregressive models. Methods using sparse-input MLPs and long short-term memory model nonlinear autoregressive relationships have been proposed 733,, followed by more sample efficient economy statistical recurrent unit (eSRU) architecture [31. Other researchers proposed a temporal causal discovery framework that leverages attention- based convolutional neural networks [55] and a framework to interpret signs of GC effects and their variability through time building on SENN [2]. However, the structure of time-varying dynamical systems in multi-animal trajectories
was not fully utilized in the above models.
Information-theoreiic analysis for multi-animal motions. In this topic, most researchers have adopted transfer entropy (TE) and its variants and have analyzed them in terms of e.g., information
cascades rather than causal discovery among animals. In the pioneering work, [78] analyzed infor-
mation cascades among artificial collective motions using (conditional) TE 40 41. [64] applied
variants of conditional mutual information to identify dynamical coupling between the trajectories of foraging meerkats. TE has been used to study the response of schools of zebrafish to robotic replica of the animal ( [36, to infer leadership in pairs of bats (58] and simulated zebrafish [10] and
to identify interactions in swarm of insects (Chironomus riparius) [42]. Local TE (or pointwise TE)
has been used to detect local dependencies at specific time points in a swarm of soldier crabs 75. teams of simulated RoboCup agents [Н], and school of fish [3] Since biological collective
67 40
motions are intrinsically time-varying dynamical systems, we compared our methods and local TE in
experiments.
Other Biological multi-agent motion analysis. Previous studies have investigated leader-follower relationships. For example, the existences of the leadership have been investigated via the correlation
in movement with time delay (e.g.. [53 (6]) and via global physical (e.g., 4 and statistical
properties [57]. Meanwhile, methods for data-driven biological multi-agent motion modeling have
been intensively investigated for pedestrian (e.g.. 25)). vehicles (e.g., 63] 72), animals [16] 28), and athletes (e.g.. 883 37 In most of these methods, the agents are assumed to have the full observation of other agents to achieve accurate prediction. In contrast, some researches have modeled
partial observation in real-world multi-agent systems (26 38 24 18 19 22]. However, the above
approaches required a large amount of training data and would not be suitable for application to the multi-anima trajectory datasets that are measured in small quantities.
6 Experiments
The purpose of our experiments is to validate the proposed methods for application real-world multi- animal movement trajectories, which have usually a smaller amount of sequences and no ground truth of the interaction rules. Thus, for verification of our methods, we first compared their performances to infer the Granger causality to those in various baselines using two synthetic datasets with ground truth: nonlinear oscillator (Kuramoto model) and boid model simulation datasets. We used the same ABM as applied to real-world multi-animal trajectory datasets: mice, birds, bats, and flies. To demonstrate the applicability to the multi-element dynamics other than multi-animal trajectories, we validated our method using the Kuramoto dataset (the results are shown in Appendix F). Each method was trained only on one sequence according to most neural GC frameworks (73) (1]5) The hyperparameters of the models were determined by validation datasets in the synthetic data experiments (for the details, see Appendices D and The common training details, (binary) inference methods, computational resources, and the amount of computation are described in Appendix D Our code is available at https://gii thub. cww/ http:/eissue199811///000
6.1 Synthetic datasets
For verification of our method, we compared the performances to infer the GC those in various baselines using two synthetic datasets with ground truth. To compare with various baselines of GC methods, we tackled problems where the true causality is not changed over time. Here, we compared our methods (ABM) to 5 baseline methods: economic statistical recurrent unit (eSRU) BI]; amortized causal discovery (ACD) 43 GVAR (45) (this is the most appropriate baseline); and simple baselines such as linear GC and local TE modified from 8I 4 Except for ACD [3]. most baselines did not provide an obvious way for handling multi- dimeiiidmensional time series, whereas our main problems include two- or three-dimensiona trajectories for each animal. Therefore, we modified the baselines except for ACD to compute norms with respect to spatial dimensions (2 or 3) for comparability with the proposed method. Note that the interpretations of the relationships estimated by ACD and Local TE are different from other methods, thus the sign of the relationship could not be investigated (we denote such by N/A in Table I.
7

---

We investigated the continuously-valued inference results: the norms of relevant weights, scores, and strengths of GC relationships. We compared these scores against the true structures using areas under receiver operating characteristic (AUROC) and precision-recall (AUPRC) curves. We also evaluated thresholded inference results: accuracy (Ace) and balanced accuracy (BA) scores. For the inference methods of binary matrix of GC relationships, see Appendix D.1 For all evaluation metrics, we only considered off-diagonal elements of adjacency matrices, ignoring self-causal relationships.
Boid model. Here, we evaluated the interpretability and validity of our method on the sim-
ulation data using the boid model, which contains five agents movement trajectories (for details, see Appendix G). In this experiment, we set the boids (agents) directed prefer- ences: we randomly set the ground truth relationships 1,0, and -1 as the rules of attrac- tion, no interaction, and repulsion, respectively. Figure [ illustrates that e.g., boid #5 was attracted to boid #1 (i.e., true relationship: 1) and boid #3 avoided boid #1 (true rela- tionship: 1)). In this figure, our method detected the changes in the signed relation- ships whereas the GVAR [45 did not.
gpan 0
10
DUTE true
The performances were evaluated using St,, in Eq. (7) throughout time because our method and ground truth were sensitive the sign as shown in Figure [T] Table ! (up- per) shows that our method achieved better performance than various baselines. The ablation studies shown in [ reveal that the main two contributions of this work, the theory-guided regularization LTG and learning navigation function F% and motion function F separately, im- proved the performance greatly. These sug- gest that the utilization of scientific knowl- edge via the regularization and architee- tures efficiently worked in the limited data situations. Similarly, the results of the Ku- ramoto dataset are shown in Appendix [ indicating that our method achieved much better performance than these baselines. Therefore, method can effectively infer the GC in multi-agent (or multi-element) systems with partially known structures.
90
100
150
pvar
positive: attraction
negative: repulsion
50 100 130
Figure 1: Example results the boid model. Left: five boids (agents) movements. Trajectories are histories of the move- ment. Right: the results of our method (blue) and GVAR 4S] (black dash) for the relationship between the cause (boid #1) and effects (other boids; i.c., St, for i = 1 and j = 2..... 5). The hinary relationships are descrihed in the upper of the plot. 1, O, and -1 indicate attraction, no interaction, and repulsion, respectively. Note that the magnitudes of method and GVAR (45) were normalized with their maximal values (thus, the values were not be comparable among methods and red ground truth). For the detail, see the main text.
Boid model
Bal. Acc.
AUPRC
BApos
BAneg 0.530 + 0.165
Linear GC
0.487 + 0.028
0.591 + 0.169
0.55 + 0.150
Local TE
0,634 1 0.130
0.580 + 0.141
N/A
N/A
cSRU B1
0.500 + 0.000
0.452 + 0.166
0.495 + 0.102
0.508 + 0.153
ACD 43
0.411 + 0.099
0.497 + 0.199
N/A 0.524 0.199
N/A
GVAR [55
0.441 1 0.090
0.327 0.119
0.579 1 0.126
ABM
LTG
0.500 1 0.021
0.417 1.115
0.513 1 0.096
0.619 1 0.157
ABM Fy
0.542 + 0.063
0.385 4 0.122
0.544 +: 0.160
0.508 + 0.147
ABM LrG
0.683 + 0.124
0.638 0.096
0.716 - 0.172
0.700 = 0.143
ABM (ours)
0.767 1 0.146
0.819 + 0.126
0.724 0.189
0.760 = 0.160
Table 1: Performance comparison the boid model.
6.2 Multi-animal trajectory datasets
We here analyzed biological multi-agent trajectory datasets of bats, birds, mice, and flies and obtained new biological insights using our framework (for the results of flies, see Appendix (H.. We used the same ABM as used the boid dataset. In real-world data, since there was no ground truth, we used the hyperparameters of the boid simulation dataset. As possible verification method, our method can be verified by investigating whether the GC result follows the hypothesis using mice and flies (Appendix H datasets, which were controlled based on scientific knowledge. Next, we show that
8

---

A
mice
C
bats
Our method
GVAR
effect
1.5
same oage ffferent cage
leave cave
attraction
20
Ouratonn
0.5
interaction
aitraction
repulsion
attraction
repulsion
forward
repulsion
B
birds
ID1 intraction
duraton
0.3-
repuision
return
20
repuision
0.2
ID3 altrcciion
reputtion
0.1
8/11
8/15
8/19
8/24
9/04
9/14
10 15
cause
Figure Analyzed results for multiple species multi-animal trajectories. The results and details are given in the main text and Appendix [I] Asterisks mean the statistically significant difference between groups (p < 0.05). (A) Results of three mice data grown in the different (red) and same (blue) cages, The vertical axis indicates the duration [sec] of their attraction and repulsion during 10-second bins of three interactions. Our method significantly extracted distinctive differences between cages in both movements. (B) Results the longitudinal two or three birds data. The horizontal axis indicates the measurement date. The GPS trajectorics identified three young brown boobies (red, green, and blue) were analyzed (missing values indicates no measurement). The vertical axis indicates the normalized duration of positive (attraction: solid line) and negative (repulsion: negative) GCs for each hird (i.e., worked as the cause of another one two birds). Error bar is the standard error among the segment during the movement. (C) Results of the observational 27 bats. The horizontal and vertical axes are the agents of the cause and effect in GC inferred by our method, respectively. The agents were sorted in the order they framed out by leaving and returning to the cave (the groups of leaving and returning were separated). The colors the signed maximal values of the absolute GC coefficients inferred by method, i.c, red, green, and blue indicate attraction (I), no interaction (0), and repulsion (-1), respectively.
our methods as analytical tools can obtain new insights from birds and bats datasets based on the quantitative results. Compared with the methodologies mentioned Section 5 (i.e., uninterpretable information -hheorrticcccccccccce approaches or using non-causal features), our method has advantages for providing local interactions: interpretable signs of Granger-causal effects over time (i.c., our findings are all new).
Mice for verification. As application effectiveness of our method using three mice raised in different environments. The hypothesis is that, is well known (e.g.. (52), when grown in different cages, they are more socially novel to others, thus more frequently attractive and repulsive movements will be observed. In this experiment, we regarded the same/different cage as group pseudo-label, and confirmed that our method could extract its features in an unsupervised manner. We analyzed the trajectories of three mice in the same and different cages measured at 30 Hz for min each (see also Appendix HH. As shown in FigureZA, our method extracted a significantly larger duration in the different cage than that in the same cages for both movements (p- 0.05; p statistical p-value), whereas GVAR 45 did not in repulsive movements (p > 0.05) but did in attractive movements (p < 0.05) and extracted too much interaction. The main reason for the too much interaction in GVAR was the overdetection of the attraction and repulsion as shown in Figure rright (black break line). The statistical analysis and videos are presented in Appendix H and the supplementary materials. Our methods characterized the movement behaviors before the contacts with others, which have been previously evaluated (e.g.. 774).
dypothesis-drivee study, for example, show the
Growing birds. Animals grow while interacting with other individuals, but the directed interaction between young individuals has not been fully investigated as longitudinal (i.e., long period) studies. Here, as an example, we analyzed the flight GPS (two-dimensional) trajectories of three juvenile brown boobies Sula leucogaster over six times for 34 days (11.91 + 0.09 [h] for each day). which were recorded at Hz. We segmented two or three bird trajectories within km and during moving (over km/s) each other, which interactions were considered to exist, and obtained 25 sequences of length 367 + 278 [s] (for details, see Appendix H. Results of inferring GC in Figure 2 show that on the first measurement day, the most frequent directed interactions were observed between ID) 1 and 2 (particularly two individuals had more repulsions and ID 1 had fewer attractions). On the

---

other hand, in the second and subsequent measurements, it was observed that the most interacting individuals changed every measurement day. One possible factor of the decrease in the duration of interactions (especially repulsion) may be the habituation with the same individuals. Measurements and analyses over longer periods will reveal the acquisition of social behavior in young individuals.
Wild bats. As an example of an exploratory analysis, we applied our method to three-dimensional trajectories of eastern bent-wing bats Miniopterus fuliginosus that left cave (some bats returned to the cave). Although some multi-animal studies have investigated leader-follower relationships (see also Section F). those in wild bats are unknown. We used two sequences with 7 and 27 bats of length 237 and 296 frames, respectively, which were obtained via digitizing the videos at 30 Hz. Details of the dataset are given Appendix H As a result, among 138 interactions of all 34 individuals within the leaving and returning groups, there were 46 interactions where the locationally- (i.e., flying forward) bats repelled the following bats in the same direction, 27 interactions where the leading bats were attracted from the following bats, 65 ones with no interactions (the results of the following bats were not discussed because was obvious; see also example results of 27 bats in Figure 2C). Since bats can echo-locate other bats in all directions up to a range of approximately 20 bats in the
m [44 6.. the locationally-leading bats can be influenced by the same direction (if no perception, they cannot be influenced). The results suggest that the groups of flying bats would not show simple leader-follower relationships.
7 Conclusions
We proposed a framework for learning GC from multi-animal trajectories via a theory-basee ABM with interpretable neural models. In our framework, as shown in Figure [ the duration of interaction and non-interaction, attraction and repulsion, their amplitudes (or strength), and their timings can be interpretable. In the experiments, our method can analyze the biological movement sequence of mice, birds, and bats, and obtained novel biological insights. One possible future research direction is to incorporate other scientific knowledge into the models such as body inertia (or visuo-motor delay). Real-world animals have certain visuo-motor delays, but they also predict the others' movements (i.e., the visuo-motor delays may be smaller). This is an inherently ill-posed and challenging problem, which will be our future work.
For socictal impact, our method can be utilized as real-world multi-agent analyses to estimate interaction rules such as in animals, pedestrians, vehicles, and athletes in sports. On the other hand, there are some concerns in our method from the perspectives of negative impact when applied to human data. One is privacy problem by the tracking of groups of individuals to detect their activities and potential interactions over time. This topic has been discussed such as in [60]. Although we did not apply method to human data, solutions for such problem will improve the applicability of the proposed method in our society.
Acknowledgments
This work was supported by JSPS KAKENHI (Grant Numbers 19H04941, 20H04075, 16H06541, 2528 1056, 21H05296, 18H03786, 21H05295, 19H04939, JP18H03287, 19H04940, and 21H05300), IST PRESTO (JPMUPR20CA), and JST CREST (JPMJCR1913). For obtaining flies data, we would like to thank Ryota Nishimura at Nagoya University.
References
[1] A. Alahi, K. Goel, V. Ramanathan, A. Robicquet. L. Fei-Fei, and S. Savarese. Social Istm: Human trajectory prediction in crowded spaces. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 961-971 2016.
[2] D. Alvarez-Melis and T. Jaakkola. Towards robust interpretability with self-cxplaining neural networks. In Advances in Neural Information Processing Systems, pages 77844 2018
[3] M. O. Appiah. Investigating the multivariate Granger causality between energy consumption, economic growth and CO2 emissions in Ghana. Energy Policy, 112:198 208,2088822 2018
[4] A. Attanasi, A. Cavagna, L. Del Castello, I. Giardina, T. S. Grigera. A. Jelic, S. Melillo, L. Parisi, O. Pohl, E. Shen, et al. Information transfer and behavioural inertia in starling flocks. Nature Physics, 10(9):691-696 2014.
10

---

[5] M. Bansal, A. Krizhevsky, and A. Ogale. Chauffeurnet: Learning to drive by imitating the best and synthesizing the worst. arXiv preprint arXiv:1812.00079. 2018.
[6] T. Beleyur and H. R. Goerlitz. Modeling active sensing reveals echo detection even in large groups of bats. Proceedings of the National Academy of Sciences, 166652)26662-266668 2019.
[7] K. Branson, A. A. Robie, J. Bender, P. Perona, and M. H. Dickinson. High tigh-throughhut ethomics in large groups of drosophila. Nature Methods, 6(6):451-457, 2009.
[8] C. H. Brighton and G. K. Taylor. Hawks steer attacks using guidance system tuned for close pursuit of erratically manoeuvring targets. Nature communications, 10(1):1-10, 2019.
[9] S. Butail, F. Ladu, D. Spinello, and M. Porfiri. Information flow in animal-robot interactions. Entropy. 16(3):1315 1330, 2014.
[10] S. Butail, V. Mwaffo, and M. Porfiri. Model-free -tfoomattion-thoorttii approach to infer leadership in pairs zebrafish. Physical Review 93(4)004241,, 2016.
[11] O. M. Cliff, J. T. Lizier, X. R. Wang, P. Wang, O. Obst, and M. Prokopenko. Quantifying long-range interactions and coherent structure in multi-agent dynamics. Artificial Life, 23(1):34557, 2017.
[12] D. Couzin, Krause, R. James, G. D. Ruxton, and R. Franks. Collective memory and spatial sorting in animal groups. Journal of Theoretical Biology, 218(1):1-11, 2002.
[13] E. Crosato, L. Jiang. V. Lecheval, J. T. Lizier, X. R. Wang, P. Tichit, G. Theraulaz, and Prokopenko. Informative and misinformative interactions in a school fish. Swarm Intelligence, 12(4):283-305, 2018.
[14] E. Demir and Dickson. fruitless splicing specifies male courtship behavior in drosophila. Cell, 121(5):785 794,,9992222 2005.
[15] R. Escobedo, V. Lecheval, Papaspyros. F. Bonnet, F. Mondada, Sire, and G. Theraulaz. A data-driven method for reconstructing and modelling social interactions in moving animal groups Philosophical Transactions of the Royal Society B, (75(1807):20190)
2020.
[16] E. Eyjolfsdottir, K. Branson, Y. Yue, and P. Perona. Learning recurrent representations for hierarchical behavior modeling. In International Conference on Learning Representations. 2017.
[17] K. Fujii and Y. Kawahara. Dynamic mode decomposition in vector-valued reproducing kernel hilbert spaces for extracting dynamical structure among observables. Neural Networks, 11 117:94-103, 2019.
[18] K. Fujii, N. Takeishi, M. Hojo, Y. Inaha, and Y. Kawahara. Physically-imleppretable classification of network dynamics for complex collective motions. Scientific Reports, 10(3005), 2020.
[19] K. Fujii, N. Takeishi, Y. Kawahara, and K. Takeda. Policy learning with partial observation and mechanical constraints for multi-person modeling, arXiv preprint vrXiv:2007.03/55 2020.
[20] E. Fujioka, M. Fukushiro. K. Ushio, K. Kohyama, H. Habe, and S. Hiryu. Three- dimensiona trajectory construction and observation of group behavior of wild bats during cave emergence. Journal Robotics and Mechatronies, 33(3)5556-563, 2021.
[21] T. Golany, K. Radinsky, and D. Freedman. Simgans: Simulator- based generative adversarial networks for eog synthesis to improve deep ecg classification. International Conference on Machine Learning, pages 3597-3606. PMLR, 2020.
[22] C. Graber and A. G. Schwing. Dynamic neural relational inference. Proceedings of the IEEEJCVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
[23] C. W. Granger. Investigating causal relations by econometric models and cross-spectral methods. Econo- metricu: journal of the Econometric Society, pages 424-438, 1969.
[24] L. Guangyu, J. Bo, Z. Hao, C. Zhengping, and L. Yan. Generative attention networks for multi-agent behavioral modeling. In Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020.
[25] A. Gupta, J. Johnson, L. Fei-Fei, S. Savarese, and A. Alahi. Social gan: Socially acceptable trajectorics with generative adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2255 2265-2264, 2018.
[26] Y. Hoshen. Vain: Attentional multi- predictive modeling. In Advances in Neural Information Processing Systems 30, pages 2701-2711, 2017.

---

[27] A. Hyvärinen, K. Zhang, S. Shimizu, and P. O. Hoyer. Estimation of a structural vector autoregression model using non-gaussianity. Journal of Machine Learning Research, 11(5), 2010.
[28] M. J. Johnson, D. K. Duvenaud, A. Wiltschko, R. P. Adams, and S. R. Datta, Composing graphical models with neural networks for structured representations and fast inference. In Advances in Neural Information Processing Systems 29, pages 2946-2954, 2016.
[29] I. Karamouzas, B. Skinner, and S. J. Guy. Universal power law governing pedestrian interactions. Physical review letters, 113(23))238701, 2014.
[30] A. Karpatne, G. Atluri, J. H. Faghmous, M. Steinbach, A. Banerjee, A. Ganguly, S. Shekhar, N. Samatova, and V. Kumar. Theory-guider data science: A new paradigm for scientific discovery from data. IEEE Transactions on Knowledge and Data Engineering. 29(10)2311 2211182233122211 2017.
[31] S. Khanna and V. Y. Tan. Economy statistical recurrent units for inferring nonlinear granger causality. In International Conference on Learning Representations. 2019.
[32] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015.
[33] T. Kipf, E, Fetaya K.-C. Wang, M. Welling, and R. Zemel. Neural relational inference for interacting systems. In International Conference on Machine Learning, pages 2688-2697, 2018.
[34] M. Kolar, L. Song, A. Ahmed, E. P. Xing, et al. Estimating time-varying networks. Annals of Applied Stutistics, (1):94-122, 2010.
[35] Y. Kuramoto. Self-entrainment of a population of coupled non-linear oscillators. International Symposium on Mathematical Problems in Theoretical Physics, pages 420-422. Springer, 1975.
[36] F. Ladu, V. Mwaffo, J. Li, S. Macri, and M. Porfiti. Acute caffeine administration affects zebrafish response to a robotic stimulus. Behavioural Brain Research, 289:48-54, 2015.
[37] H. M. Lc, Y. Yue, P. Carr, and P. Lucey. Coordinated multi-agent imitation learning. In Proceedings of the 34th International Conference on Machine Learning-Voluur Volume 70, pages 1995-2003, 2017.
[38] E. Leurent and J. Mereat. Social attention for autonomous decision-making in dense traffic. arXiv preprint arXiv: /911. 12250, 2019.
[39] N. Lim, F. d' Alché Buc, C. Auliac, and G. Michailidis. valuued kernel-based vector autoregressive models for network inference. Machine Learning, 99(3):489 513, 2015.
[40] J. T. Lizier, M. Prokopenko, and A. Y. Zomaya. Local information transfer spatiotemporal filter for complex systems. Physical Review 77(2)0261 10, 2008
[41] J. T. Lizier. M. Prokopenko. and A. Y. Zomaya. Information modification and particle collisions in distributed computation. Chaos: An Interdisciplinary Journal of Nonlinear Science, 20(3):037109, 2010.
[42] W. M. Lord, J. Sun, N. T. Ouellette, and E. M. Bollt. Inference of causal information flow in collective animal behavior. IEEE Transactions Molecular, Biological and Multi-Scal Communications, 201):107- 116, 2016.
[43] S. Löwe. D. Madras, R. Zemel, and M. Welling. Amortized causal discovery: Learning to infer causal graphs from time-series data. arXiv preprini arXiv:2006. 10833, 2020.
[44] Y. Maitani, K. Hase, K. I. Kobayasi, and S. Hiryu. Adaptive frequency shifts of echolocation sounds in miniopterus fuliginosus according the rrequencymmddulate pattern of jamming sounds. Journal of Experimental Biology, 221(23), 2018.
[45] R. Marcinkevics and J. E. Vogt. Interpretable models for granger causality using self-explaining neural networks. In 1st NeurIPS workshop on Interpretable Inductive Biases and Physically Structured Learning (2020), 2020.
[46] D. Marinazzo, M. Pellicoro, and S. Stramaglia. Kernel-granger causality and the analysis of dynamiccal networks. Physical review 7765)056215, 2008
[47] F. Martinez-Gil, M. Lozano, I. García- Fermándz,, and F. Fernández. Modeling, evaluation, and scale on artificial pedestrians: literature review. ACM Computing Surveys (CSUR), 50(5):1-35, 2017.
[48] A. Mathis, P. Mamidanna, K. M. Cury, T. Abe, V. Murthy, M. W. Mathis, and M. Bethge. Deeplabeut: markerless pose estimation of user-defined body parts with deep learning. Nature Neuroscience, 2018.
12

---

[49] J. M. McCracken. Exploratory causal analysis with time series data, Synthesis Lectures on Data Mining and Knowledge Discovery, 8(1):1-147, 2016.
[50] A. Montalto, S. Stramaglia, L., Faes, G. Tessitore, R. Prevete, and D. Marinazzo. Neural networks with non-uniform embedding and explicit validation phase to assess granger causality. Neural Networks, 71:159-171, 2015.
[51] H. Murakami, T. Nizato, and Y.-P. Gunji. Emergence of coherent and cohesive swarm based mutual anticipation. Scientific reports, 7:46447, 2017.
[52] J. Nadler, S. Moy. G. Dold, D. Trang. N. Simmons, A. Perez. N. Young. R. Barbaro, J. Piven, T. Magnuson. et al. Automated apparatus for quantitation of social approach behaviors in mice. Genes, Brain and Behavior, 3(5):303-3144 2004.
[53] M. Nagy, Z. Åkos, Biro, and T. Viesek. Hierarchical group dynamics in pigeon flocks. Nature, 464(7290):890 893, 2010.
[54] R. Nathan, W. M. Getz, E. Revilla, M. Holyoak, R. Kadmon, D. Saltz, and P. E. Smouse. A movement ecology paradigm for unifying organismal movement research, Proceedings of the National Academy of 2008.
Sciences, 105(49):19052-1900)
[55] M. Nauta, D. Bucur, and C. Seifert. Causal discovery with attention-based convolutional neural networks. Machine Learning and Knowledge Extraction, 1(1):312-340, 2019.
[56] W. B. Nicholson, D. S. Matteson, and J. Bien, Varx-l: Structured regularization for large vector autoregres- sions with exogenous variables. International Journal of Forecasting, 33(3):627-651, 2017.
[57] T. Niizato, K. Sakamoto, Y.-i. Mototake, H. Murakami, T. Tomaru, T. Hoshika, and T. Fukushima. Finding continuity and discontinuity in fish schools via integrated information theory. PloS One, 15(2)=e0229573, 2020.
[58] N. Orange and N. Abaid. A transfer entropy analysis of leader-follower interactions in flying bats. The European Physical Journal Special Topics, 224(17):327--32) 3292329339999..... 2015.
[59] J. Peters, D. Janzing, and B. Schölkopf. Elements of causal inference: foundations and learning algorithms. The MIT Press, 2017.
[60] V. Primault, A. Boutet, S. B. Mokhtar, and L. Brunie. The long road to computational location privacy: A survey. IEEE Communications Surveys & Tutorials, 21(3):772-2293,, 2018
[61] M. Raissi, P. Perdikaris, and G. E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686-707, 2019.
[62] C. W. Reynolds. Flocks, herds and schools: A distributed behavioral model. Proceedings of the 14th annual Conference on Computer Graphics and Interactive Techniques, pages 25-34, 1987.
[63] N. Rhinchart, R. McAllister, K. Kitani, and S. Levine. Precog: Prediction conditioned goals in visual multi-agent settings. In Proceedings of the IEEE International Conference on Computer Vision, pages 2821 2921-2830, 2019.
[64] T. O. Richardson, N. Perony, C. J. Tessone, C. A. Bousquet, M. B. Manser, and F. Schweitzer. Dynamical coupling during collective animal motion. arXiv preprint /3Xi::/311.1411 2013.
[65] A. Roebrocck, E. Formisano, and R. Goebel. Mapping directed influence over the brain using Granger causality and fMRI. Neurolmage, 25( 25(1):230-24 242, 2005.
[66] D. Sankey, L. O' O'Bryan, S. Garnier, G. Cowlishaw, P. Hopkins, M. Holton, 1. Fürtbauer, and A. King. Consensus of travel direction is achieved by simple copying, not voting, in free-ranging goats. Royal Society Open Science, 8(2):201 128, 2021.
[67] T. Schreiber. Mcasuring information transfer. Physical Review Letters, 85(2):411, 2000.
[68] J. C. Simon and M. H. Dickinson. A new chamber for studying the behavior of drosophila. Plos One, 5(133, 2010.
[69] V. Sindhwani. H. Q Minh, and A. C. Lozano. Scalable matrix-valued kernel learning for ligeesiiensi nonlinear multivariate regression and granger causality. In Proceedings of the Twenty-Ninth Ninth Conference on Uncertainty in Artificial Intelligence, pages 186-595,20 2013.
13

---

[70] L. Song, M. Kolar, and E. Xing. Time-varying dynamic bayesian networks. Advances in Neural Information Processing Systems, 22:1732-1740, 2009.
[71] P. A. Stokes and P. L. Purdon. A study of problems encountered in granger causality analysis from neuroscience perspective. Proceedings of the National Academy of Sciences, (1404))87033-E7000 2017.
[72] C. Tang and R. R. Salakhutdinov. Multiple futures prediction. In Advances in Neural Information Processing Systems 32, pages 15398-15408, 2019.
[73] A. Tank, I. Covert, N. Foti, A. Shojaie, and F. Fox. Neural granger causality for nonlinear time series. arXiv preprint arXiv: /802. 05842, 2018.
[74] P. K. Thanos, C. Restif, J. R. O' O'Rourke, C. Y. Lam, and D. Metaxas. Mouse social interaction test (most): quantitative computer automated analysis of behavior. Journal of Neural Transmission, 124(1):3-11, 2017.
[75] T. Tomaru, H. Murakami, T. Nizzto,, Y. Nishiyama, K. Sonoda, T. Moriyama, and Y.-P. Gunji. Information transfer a swarm of soldier crabs. Artificial Life and Robotics, 21(2):177-180, 2016.
[76] K. Tsutsui, M. Shinya, and K. Kudo. Human navigational strategy for intercepting an erratically moving target in chase and escape interactions. Journal of motor behavior, 52(6):500-700, 2020.
[77] T. Viesek and A. Zafeiris. Collective motion. Physics Reports, 517(3-4):71-44)) 2012.
[78] X. R. Wang, J. M. Miller, T. Lizier, M. Prokopenko, and L. F. Rossi. Quantifying and tracing information cascades in swarms. PloS One, 707)4400084, 2012.
[79] Y. Wang. K. Lin, Y. Qi, Q. Lian, S. Feng. Z. Wu, and G. Pan. Estimating brain connectivity with varying- length time lags using recurrent neural network. IEEE Transactions on Biomedical Engincering, 65(9):1953 1963, 2018.
[80] I. Winkler, D. Panknin, D. Bartz, K.-R. Müller, and S. Haufe. Validity of time reversal for testing granger causality, IEEE Transactions on Signal Processing, (4((1))2744- 2016.
[81] T. Wu, Breuel, M. Skuhersky, and J. Kautz. Discovering nonlinear relations with minimum predictive information regularization. arXiv preprint arXiv:2001, 200.008800..8885.... 2020.
[82] K. Yoda, M. Murakoshi, K. Tsutsui, and H. Kohno. Social interactions of juvenile brown boobies observed with animal-borne video cameras. PLoS One, 6(5)e11902, 2011.
[83] S. Zheng, Y. Yue, and J. Hobbs. Generating long-term trajectories using deep hierarchical networks. In Advances Neural Information Processing Systems 29, pages 1543-1551, 2016.
[84] A. Zienkiewicz, D. Barton, M. Porfiri, and M. Di Bemardo. Leadership emergence in data-driven model of zebrafish shoals with speed modulation. The Eumpean Physical Journal Special Topics, 22)(33) 3360, 2015.
[85] A. K. Zienkiewicz, F. Ladu, D. A. Barton, M. Porfiri, and M. Di Bernardo. Data-driven modelling of social forces and collective behaviour in zebrafish. Journal of Theoretical Biology, 443:39-51, 2018.
[86] H. 7au and Hastie. Regularization and variable selection via clastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(2)3001-30,, 2005.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you deseribe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [N/A]
14

---

3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (c.g., with respect the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes]
4. If you using existing assets (e.g., code, data, models) curating/releasing assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
15


Reviews:
[{"review_id": "xAWZY1Qdn5T", "reviewer": "Reviewer_hrQV", "summary": "The manuscript proposes a hybrid method for learning Granger causality that combines data-driven learning methods and theory-based behavioral models for the goal of modeling multi-agent behavior. \n\nThe main motivation is that current models either do not consider the data generation process behind the observed time series or if they do, they are tailored to specific animal species, thus having generalization limitations.\n\nMore specifically, a Self-Explainable Neural Network (SENN) -like model is used where scientific knowledge on multi-agent interactions is integrated. In this regard, key contributions of the proposed method include: a theory-guided loss function to regularize training and a motion and navigation function Experiments on synthetic and realistic data show the effectiveness of the proposed method.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The proposed method is sound and different design decisions behind are properly motivated. The content of the manuscript is relatively clear and has a good flow.\n \nIn addition, the proposed method was tested on real-world datasets, where it proved effective at validating observations already made in other fields.\n \nMy main criticisms with the manuscript are the following:\n\n\n- C1: The text in l.118-119 gives the impression that models that learn parameters or rules in a purely data-driven manner are inherently not interpretable. I suggest rephrasing this sentence to avoid this confusion since  this this statement is clearly not accurate.\n \n \n- C2: The introduction of the manuscript and presentation of the proposed method were properly motivated and detailed. While these are desirable properties for a manuscript, those two parts are quite extended when compared to the experimental part of the manuscript. Relevant aspects of the conducted experiments and results are delegated to the Appendix. For instance, the ablation study in Appendix G shows the effect that the proposed theory-guided regularization $L_{TG}$ and learning navigation $F^k_N$ and motion $F^k_M$ functions (the main contributions of the manuscript) have on performance. This is a critical experiment that should be in the main body of the paper.\nDue to delegations like this, the manuscript does not feel as a self contained document.\n  \n- C3: In l.311-313  it is stated that for the baselines it is not obvious how to handle multi-dimensional time series\nIn the one hand, the capability of handling multi-dimensional data it is a strength of the your method. I would suggest emphasizing this strength in the manuscript. \nOn the other hand, the observation (l.311-313) makes me wonder whether the selected baselines are the most appropriate ones for a fair comparison with respect to the proposed method.\n\n- C4: For the experiments in the real multi-animal trajectory datasets, in l.358-359 it is stated that since there is no ground truth for real-world data, the hyperparameters of the boid simulation dataset are re-used.\nI was wondering, how applicable is the re-use of these  parameters when going from a simplistic synthetic setting to a more realistic one? Are there potential side-effects that could be expected? Did you observe any problem, possibly caused by this factor, during the execution of your experiments on real-world data?\n\n- C5: In several parts of the manuscript it is stated that through the proposed method novel biological insights were obtained. I was wondering were these novel insights validated by experts in the respective fields?\n\n**Post Rebuttal Update**\nI thank the author for addressing my review on their rebuttal. Having read all the reviews and the feedback provided by the authors, this is my current position with respect to the manuscript:\n\nOverall the authors addressed to a good extent most of the weak points and criticisms raised on my review, which will be included in a revised version of the submitted manuscripts.\n\nI agree with reviewer C9GF on that some parts of the manuscript could use some extra attention in order to improve their presentation.\n\nThe ethical reviewers share my opinion on that while the proposed method poses some potential risks, this are not strong enough to reject it. More importantly, these are aspects that could be easily addressed with a proper discussion either in the manuscript or in its appendix.\n\nOn the optimistic side, I believe the manuscript has the potential to be better given that all the amendments promised in the rebuttal are applied in the revised version. On the pessimistic side, I am not sure whether it would be possible to include all the possible amendments given the space limitations imposed in the manuscript.\n\nKeeping the previous points, I am in favor of increasing initial rating towards \"6: Marginally above the acceptance threshold\" but will not push for the acceptance of the manuscript. In this regard, I am more in favor of trying to improve the current weak aspects of the manuscript rather than trying to get it accepted in its current form. (This observation seems to be shared by C9GF)\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "\nRegarding potential negative applications of the proposed method. The proposed method has a clear negative behind tracking groups of individuals over time, be it in video recordings or via their activities and potential interactions in social networks. In this regards, the manuscript limits itself to provide a very superficial pointer towards fairness issues when operating on top of data from individuals.\nWhile I do not find a potential dual use issue a barrier to stop the publication of a good idea, I do believe that these issues should be at least acknowledged in a proper manner. I would encourage the authors, in the appendix,  to discuss at length potential problems regarding the tracking of collective activities in social networks.", "review_text": "", "extra_scores": {"needs_ethics_review": "Yes", "ethics_review_area": ["Inappropriate Potential Applications & Impact  (e.g., human rights concerns)"], "time_spent_reviewing": "4"}, "extra_text": {"ethical_concerns": "\nRegarding potential negative applications of the proposed method. The proposed method has a clear negative behind tracking groups of individuals over time, be it in video recordings or via their activities and potential interactions in social networks. In this regards, the manuscript limits itself to provide a very superficial pointer towards fairness issues when operating on top of data from individuals.\nWhile I do not find a potential dual use issue a barrier to stop the publication of a good idea, I do believe that these issues should be at least acknowledged in a proper manner. I would encourage the authors, in the appendix,  to discuss at length potential problems regarding the tracking of collective activities in social networks."}}, {"review_id": "QaHBXgNA8wq", "reviewer": "Reviewer_gmyy", "summary": "In this paper, the authors propose a method for learning rules of interaction between individuals from　trajectory data of animals. Their method is based on the Granger causality methodology for time series data but differs from previous studies in that it introduces a new network architecture and a regularization term based on the theory of animal behavior, which enables more accurate learning. The method has been validated using synthetic datasets and then applied to real animal behavior data to gain new knowledge about animal behavior.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Strengths\n\n* In-depth and extensive experiments. After validating the method on two types of synthetic data, the authors apply the method to real animal movement trajectory data. In the real data experiments, the authors first check whether the results of the proposed method are consistent with existing knowledge on mice data, and then apply the method to birds and bats data to gain new knowledge. All of these experiments have a clear purpose and work well to present the advantages and features of the proposed method.\n\n* The datasets used in the experiment. The animal trajectory datasets are collected for this paper, which is rare in the machine learning community. It is interesting to see the new insights derived by the proposed method from such data.\n\n* This paper is well-written; the problems, the backgrounds, and the contributions are clearly stated with appropriate citations. \n\n### Weaknesses\n\n* Small novelty of the method. It is claimed that the novelty of the method is the introduction of a network architecture and a regularization term based on the theory of animal behavior to the existing Granger causality estimation method, but this in itself is a straightforward combination of previous methods and has little originality. \n\n* Validation has only been done on datasets where the problem size is relatively small. It would be of great interest to analyze whether it is possible to successfully find Granger causality on larger datasets, what the computation time would be, and whether any other problems would arise.\n\n### Other comments\n* Are the discoveries about growing birds and wild bats new discoveries or are they a validation of previous discoveries?  In particular, for wild bats, it would be good to compare and discuss with the contents of [57].\n\n* This paper may possibly be preferred more at data mining conferences such as KDD than at machine learning conferences such as NeurIPS. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations are not written by the authors, but it is not clear whether the method can be applied to large datasets. There seems to be no negative impact on society.", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "DYFNc_dXRPF", "reviewer": "Reviewer_C9GF", "summary": "The proposed paper introduces a novel approach for learning the Granger causality based on augmented behavioral models. The objective of the discussed method is to extract interaction rules from real-world multi-agent trajectory data with neural networks. The main contributions of this work are a framework for learning Granger causality, methods for efficient and interpretable learning, and the decomposition of motion and navigation. The method is evaluated based on a number of experiments and based on datasets of mice, flies, birds, and bats.   \n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, this is an interesting paper that has most things going for it. The proposed framework is technically-sound, the experiments are well thought-out, and the results are promising. Furthermore, the topic the paper addresses is important and I appreciate the interdisciplinary character of this work. However, the paper is somewhat dense and many sections are not easy to comprehend. I have the impression that more care could be taken in providing additional details and explanations across Section 3 and 4. Especially, the motivation for developing interpretable behavior models comes too short and interpretability of the proposed framework is not assessed. Moreover, several sentences show grammar and spelling mistakes that need to be taken care of (examples provided below). So in conclusion, I think this paper could be accepted if proper care is taken in preparing the final version of this work. However, the paper would also benefit from a more thorough revision outside the review cycle from NeurIPS. Therefore, I am not fully too enthusiastic about accepting this work. \n\nSpecific comments:  \n\n- L1: Is 'moving sequences' really the correct meaning here? Or should it be 'movement sequences'? Similar in L407 it doesn't seem 'moving sequence' is the right grammar.\n\n- L27, L83: For the less movement-ecology-versed reader it would be helpful to provide more information about what is meant by 'generative process in biological multi-agent trajectories' or the 'generative process in animal behaviors. \n\n- L88: What is meant by 'we incorporate the structures into the base model of a CG framework'? Sentences like this are too vague and need to be clarified. What is meant by 'structures'?\n\n- In Sections 3 and 4, I am missing a clearer description of the model and what components are actually being learned. Perhaps an overview figure would help to make this more clear. \n\n- L70 'an multi layer perceptrons' --> 'a ... perceptron'\n\n- L71 and L72: 'some loss', 'some fixed function'. This sounds too vague and it would be helpful to be more specific here. \n\n- L103: A clearer description as to why only one time series is considered needs to be provided here. It is also not clear based on the description provided in Line 653 as it is mostly a copy from the main text.\n\n- L105: While I understand the motivation to make the contributions of f_M, f_N, and f_U interpretable -- which is in fact an interesting objective -- it appears somewhat out of context and more details are necessary. As is the paragraph (L103-112) is too difficult to understand. Similar in Line 120. What is the goal for making f_* interpretable? What this goal accomplished?\n\n- L131-132, L134-135: These sentences read too repetitive. \n\n- In Section 4.1. it is not clear what is actually learned. What is the model architecture that is being trained?\n\n- L346: 'were sensitive the sign as shown' --> rephrase.\n\n-L649: If i am not mistaken u_t is not being referenced anywhere. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations of the method are not discussed. In the Conclusion the authors suggest that their method could have negative impact, but the discussion remains rather vague. Both should be prepared appropriately prior to publishing this work. ", "review_text": "", "extra_scores": {"ethical_concerns": "I do not have any ethical or diversity concerns toward the topic this paper addresses or the paper in general. \n", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "9s1-G1GQl8q", "reviewer": "Reviewer_cyHu", "summary": "This paper proposes a framework for learning Granger causality with the goal of finding interaction rules from trajectory data of multiple animals. The framework has an \"augmented behavior model\" based on a conceptual behavior model studied in [53]. The experiments are on a range of datasets, from synthetic to real data of mice, birds, and bats. The main baseline of comparison is GVAR [44], which this method compares against for synthetic data and on real mice data. Based on the experiments, the interaction rules found are in the form of attraction, no interaction, or repulsion between different agents.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1.  I appreciate that the authors tested on a range of datasets. However, I feel the results method could be more clear. For example, in Figure 2A, the authors say that \"our method extracted a significantly larger duration in the same cage than that in the different cages\", but the figure shows that the duration of same cage is lower for both attractive and repulsion, so this point is not clear to me. The y-axis between this method and GVAR is also significantly different - can the authors comment on this? Also, what does the \"*\" mean in Figure 2A? Additionally, the authors mention that the model is interpretable (ex: lines 8, 10), but there are many ways to interpret whether a model is interpretable and I'm not sure where interpretability is demonstrated in the experiments (is it by the duration of interaction & non-interaction?). Could the authors clarify this point? \n2. Overall, I found the methods section difficult to understand. Perhaps a diagram or pseudo-code of the algorithm (ex, as provided by GVAR [44] in Algorithm 1) might enable the authors to more clearly communicate their setup to a broader range of audiences who may not be familiar with this sub-field. \n3.  Line 291: \"for application to multi-animal trajectory datasets that can only be measured in small quantities\". Why can multi-animal trajectory data only be in small quantities? Some labs are releasing large amounts of trajectory data collected over time. I agree that it is difficult and time-consuming to measure this but I don't agree that these datasets can only be in small quantities. \n\nPost Rebuttal Update:\n\nI've read the other reviews and the author response, and my current rating is raised to borderline, learning towards accept (assuming the authors will apply all the changes promised in the rebuttal). This is because the authors have clarified some of my points of confusion with the paper (methods as well as experiments). I still feel the paper could be presented more clearly, but the authors have mentioned adding a figure or a pseudo-code of their algorithm, which I feel would improve clarity of the methods section substantially. I agree with reviewer hrQV that if all the amendments in the rebuttal are applied to the paper, the revision would be stronger.\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I didn't see where the authors mentioned the limitations of their method - would be great if the authors could point me in the right direction. There is a paragraph in line 411 addressing societal impacts. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a hybrid method for learning Granger causality that combines data-driven learning methods and theory-based behavioral models for the goal of modeling multi-agent behavior. The paper addresses an important problem and can stimulate further research in this direction.
Initial reviews were split on the technical contribution and also the technique being a combination of existing methods. 
However, after the discussion and rebuttal the reviewers have turned more positive about the ideas presented in the paper. 
The AC echoes the view point in this case that combining existing methods is often challenging and can reveal important scientific insights.
Overall after the discussion all reviewers are in agreement about the value of the contribution and recommend acceptance. 
Decision: Accept (Poster)