Learning interaction rules from multi-animal trajectories via augmented behavioral models

1 Introduction . Extracting the interaction rules of real-world agents from data is a fundamental problem in a variety of scientific and engineering fields . For example , animals , vehicles , and pedestrians observe other ’ s states and execute their actions in complex situations . Discovering the directed interaction rules of such agents from observed data will contribute to the understanding of the principles of biological ∗fujii @ i.nagoya-u.ac.jp 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . agents ’ behaviors . Among methods analyzing directed interactions within multivariate time series , Granger causality ( GC ) [ 23 ] is a practical framework for exploratory analysis [ 49 ] in various fields , such as neuroscience [ 65 ] and economics [ 3 ] ( see Section 2 ) . Recent methodological developments have focused on inferring GC under nonlinear dynamics ( e.g. , [ 73 , 31 , 81 , 55 , 43 ] ) . However , the structure of the generative process in biological multi-agent trajectories , which include navigational and motion processes [ 54 ] regarded as time-varying dynamical systems ( see Section 3.1 ) , is not fully utilized in existing base models of GC including vector autoregressive [ 27 ] and recent neural models [ 73 , 31 , 81 ] . Ignoring the structures of such processes in animal behaviors will lead to interpretational problems and sometimes erroneous assessments of causality . That is , incorporating the structures into the base model for inferring GC , e.g. , augmenting ( inherently ) incomplete behavioral models with interpretable data-driven models ( see Section 3.2 ) , can solve these problems . Furthermore , since data-driven models sometimes detect false causality that is counterintuitive to the user of the analysis , e.g. , introducing architectures and regularization to utilize scientific knowledge ( see Sections 3.2 and 4.2 ) will be effective for a reliable base model of a GC method . In this paper , we propose a framework for learning GC from biological multi-agent trajectories via augmented behavioral models ( ABM ) using interpretable data-driven neural models . We adopt an approach for augmenting incomplete multi-agent behavioral models described by time-varying dynamical systems with neural networks ( see Section 3.2 ) . The ABM leverages theory-based architectures separating navigation and motion processes based on a well-known conceptual behavioral model [ 54 ] , and the theory-guided regularization ( see Section 4.2 ) for interpretable and reliable behavioral modeling . This framework can provide interpretable signs of Granger-causal effects over time , e.g. , when specific others cause the approach or separation . The main contributions of this paper are as follows . ( 1 ) We propose a framework for learning Granger causality via ABM , which can extract interaction rules from real-world multi-agent and multi-dimensional trajectory data . ( 2 ) Methodologically , we realized the theory-guided regularization for reliable biological behavioral modeling for the first time . The theory-guided regularization can leverage scientific knowledge such that “ when this situation occurs , it would be like this ” ( i.e. , domain experts often know an input and output pair of the prediction model ) . Existing methods in Granger causality did not consider the utilization of such knowledge . ( 3 ) Biologically , our methodological contributions lies in the reformulation of a well-known conceptual behavioral model [ 54 ] , which did not have a numerically computable form , such that we can compute and quantitatively evaluate it . ( 4 ) In the experiments , our method achieved better performance than various baselines using synthetic datasets , and obtained new biological insights and verified our method using multiple datasets of mice , birds , bats , and flies . In the remainder of this paper , we describe the background of GC in Section 2 . Next , we formulate our ABM in Section 3 , and the learning and inference methods in Section 4 . 2 Granger Causality . GC [ 23 ] is one of the most popular and practical approaches to infer directed causal relations from observational multivariate time series data . Although the classical GC is defined by linear models , here we introduce a more recent definition of [ 73 ] for non-linear GC . Consider p stationary time-series x = { x1 , ... xp } across timesteps t = { 1 , ... , T } and a non-linear autoregressive function gj , such that xjt+1 = gj ( x 1 ≤t , ... , x p ≤t ) + ε j t , ( 1 ) where xj≤t = ( ... , x j t−1 , x j t ) denotes the present and past of series j and ε j t represents independent noise . We then consider that variable xi does not Granger-cause variable xj , denoted as xi 9 xj , if and only if gj ( · ) is constant in xi≤t . Granger causal relations are equivalent to causal relations in the underlying directed acyclic graph if all relevant variables are observed and no instantaneous ( i.e. , connections between two variables at the same timestep ) connections exist [ 59 ] . Many methods for Granger causal discovery , including vector autoregressive [ 27 ] and recent deep learning-based approaches [ 73 , 31 , 81 ] , can be encapsulated by the following framework . First , we define a function fθ ( e.g. , an multilayer perceptrons ( MLP ) in [ 73 ] , a linear model in [ 27 ] ) , which learns to predict the next time-step of the test sequence x . Then , we fit fθ to x by minimizing some loss ( e.g. , mean squared error ) L : θ ? = argminθ L ( x , fθ ) . Finally , we apply some fixed function h ( e.g. , thresholding ) ( e.g. , [ 45 ] ) to the learned parameters to produce a Granger causal graph estimate for x : Ĝx = h ( θ ? ) . Furthermore , we need to differentiate between positive and negative Granger-causal effects ( e.g , approaching and separating ) . Based on the definition of [ 45 ] , we define the effect sign as follows : if gj ( · ) is increasing in all xi≤t , then we say that variable xi has a positive effect on xj , if gj ( · ) is decreasing in xi≤t , then x i has a negative effect on xj . Note that xi can contribute both positively and negatively to the future of xj at different delays . Overall , the causality measures , however elaborate in construction , are simply statistics estimated from a model [ 71 ] . If the model inadequately represents the system properties of interest , subsequent analyses based on the model will fail to address the question of interest . The inability of the model to represent key features of interest can cause interpretational problems and sometimes erroneous assessments of causality . Therefore , in our case , incorporating the structures of the generative process for animal behaviors ( i.e. , Eq . ( 2 ) ) in a numerically computable form will be required . We thus propose the ABM based on a well-known conceptual model [ 54 ] in biological sciences in the next section . 3 Augmented behavioral model . Our motivation for developing interpretable behavior models is to obtain new insights from the results of Granger causality . In this section , we firstly formulate a well-known conceptual behavioral model [ 54 ] so that it can be computable . Second , we propose ( multi-animal ) ABMs with theorybased architectures based on scientific knowledge . Further , we discuss the relation to the existing explainable neural models [ 2 ] . The diagram of our method is described in Appendix C . 3.1 Formulation of a conceptual behavioral model . In movement ecology , which is a branch of biology concerning the spatial and temporal patterns of behaviors of organisms , a coherent framework [ 54 ] has been conceptualized to explore the causes , mechanisms , and patterns of movement . For example , two alternative structural representations [ 54 ] were proposed to model a new position pt+1 from its current location pt ( for details , see Appendix A ) : the motion-driven case pt+1 = fU ( fM ( Ω , fN ( Φ , rt , wt , pt ) , rt , wt , pt ) ) + εt , and the navigation-driven case pt+1 = fU ( fN ( Φ , fM ( Ω , rt , wt , pt ) , rt , wt , pt ) ) + εt , where wt is the internal state , Ω is the motion capacity , Φ is the navigation capacity , and rt is the environmental factors ( these are conceptual parameters ) . fM , fN , and fU are conceptual functions to represent actions of the motion ( or planning ) , navigation , and movement progression processes , respectively . For efficient learning of the weights in the model ( i.e. , coefficient of Granger causality ) in this paper , we consider a simple case with homogeneous navigation and motion capacities , and internal states . Moreover , to make the contribution of fM , fN , and fU interpretable after training from the data for extracting unknown interaction rules ( and for obtaining scientific new insights ) , one of the simplified processes for agent i is represented by xit+1 = f i U ( f i N ( r i t , x i t ) , f i M ( r i t , x i t ) , r i t , x i t ) + ε i t , ( 2 ) where xi ∈ Rd includes location pi and velocity for the agent i . We here consider ri ∈ R ( p−1 ) dr including p− 1 other agents ’ dr-dimensional information . This formulation does not assume either motion-driven or navigation-driven case . Such behaviors have been conventionally modeled by mathematical equations such as force- and rule-based models ( e.g. , reviewed by [ 77 , 47 ] ) . Recently , these models have become more sophisticated by incorporating the models into hand-crafted functions representing anticipation ( e.g. , [ 29 , 51 ] ) and navigation ( e.g. , [ 8 , 76 ] ) . However , these conventional and recent models are sometimes too simplistic and customized for the specific animals , respectively ; thus it is sometimes difficult to define the dynamics of general biological multi-agent systems ( i.e. , multiple species of animals ) . Therefore , methods for learning parameters and interaction rules of behavioral models are needed . There have been some researches to estimate specific parameters ( and their distributions ) of the interpretable behavior models ( e.g. , [ 84 , 85 , 15 ] ) , and others to model the parameters and rules in purely data-driven manners ( i.e. , sometimes uninterpretable ) only for accurate prediction ( e.g. , [ 16 , 28 ] ) . In the proposed framework , we consider flexible data-driven interpretable models to focus on inferring GC for exploratory analysis from the observed data without specific knowledge of the species and obtaining additional data . Recently , some attempts have been made to explore flexible and interpretable models bridging theorybased and data-driven approaches . For example , a paradigm called theory-guided data science has been proposed [ 30 ] , which leverages the wealth of scientific knowledge for improving the effectiveness of data-driven models in enabling scientific discovery . For example , scientific knowledge can be used as architectures or regularization terms in learning algorithms in physical and biological sciences ( e.g. , [ 61 , 21 ] ) . In biological multi-agent systems , an approach extract interpretable dynamical information based on physics-based knowledge [ 18 ] from multi-agent interaction data , and another approach made a particular module such as observation ( e.g. , [ 19 ] ) interpretable in mostly black-box neural models . However , these data-driven models did not sufficiently utilize the above scientific knowledge of multi-animal interactions . In the next subsection , to make the model ( e.g. , of GC ) flexible and interpretable , we propose an ABM with theory-based architectures .

Summary: The proposed paper introduces a novel approach for learning the Granger causality based on augmented behavioral models. The objective of the discussed method is to extract interaction rules from real-world multi-agent trajectory data with neural networks. The main contributions of this work are a framework for learning Granger causality, methods for efficient and interpretable learning, and the decomposition of motion and navigation. The method is evaluated based on a number of experiments and based on datasets of mice, flies, birds, and bats.