Causal Navigation by Continuous-time Neural Networks

1 Introduction Input NCP ODE-RNN LSTM CT-GRU Unlike machine learning systems , natural learning systems excel at generalizing learned skills beyond the original data distribution ( Hasani et al. , 2020 , Hassabis et al. , 2017 , Sarma et al. , 2018 ) . This is due to the mechanisms they deploy during their learning process , such as active use of closed-loop interventions , accounting for distribution shifts , and the temporal structures ( de Haan et al. , 2019 , Schölkopf , 2019 ) . These factors are largely disregarded or are engineered away in the development of modern ML systems . To take the first steps towards resolving these issues , we can investigate the spectrum of causal modeling ( Peters et al. , 2017 ) . At one end of the spectrum , there are physical models of agents ∗Equal Contributions . 1CSAIL MIT , 2IST Austria , Correspondence to : rhasani @ mit.edu Code and data are available at : https : //github.com/mit-drl/deepdrone 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . and environments described by differential equations . These physical models allow us to investigate interventions , predict future events using past information , and can describe the statistical dependencies in the system . On the other end of the spectrum , statistical models allow us to construct dependencies and make predictions given independent and identically distributed ( i.i.d . ) ( see Fig . 2 ) . While physical models provide complete descriptions , it is intractable to define the differential equation systems that effectively model high-dimensional and complex sensory data . For long , causal modeling frameworks aims at bridging this gap to extract statistical dependencies while constructing causal graphs ( Pearl , 2009 , Spirtes et al. , 2000 ) or structural causal models ( Schölkopf , 2019 ) to intervene and explain relationships . In this paper , we aim to use continuous-time ( CT ) neural networks ( Chen et al. , 2018b , Funahashi and Nakamura , 1993 , Hasani et al. , 2021b ) equipped with causal structures in order to get closer to the properties of physical models . In particular , we look into different representations of CT networks to see under what conditions and formulation they can form a causal model . We discover that the class of liquid time-constant networks ( LTCs ) ( Hasani et al. , 2021b ) which are expressive continuous-time models constructed by bilinear approximation of neural ordinary differential equations ( Chen et al. , 2018b ) , satisfy the properties of a causal model . The uniqueness of their solution and their ability to capture both internal and external interventions make their forward- and backward- mode causal . Therefore , they can impose inductive biases on their architectures to learn causal representations . We analyze how certain continuous-time ( CT ) neural networks are dynamical causal models . Furthermore , to justify the theoretical results , we empirically validate that these properties scale to high-dimensional visual input data in complex learning environments . We propose a series of control and navigation tasks for an end-to-end autonomous flight across various levels of complexity and temporal reasoning . We observe that traditional deep-learning models are capable of solving this task on offline , passive datasets but fail when deployed in closed-loop , active testing settings ( de Haan et al. , 2019 , Wen et al. , 2020 ) . On the other hand , we find that only LTC-based models are able to complete the tasks in closed-loop interaction with the environments . Specifically , consider a visual navigation task wherein a drone agent should learn to fly from point A to a target fixated at a point B , given only a sequence of raw visual inputs . The mapping between incoming pixels to the temporal and physical structure of the navigation is causal if we can explain and intervene at each step to observe how navigation decisions are decided based on the input pixels . We observe that in a neighborhood environment , agents based on LTCs ( such as Neural Circuit Policies ( Lechner et al. , 2020a ) ) learn to stably ( Lechner et al. , 2020b ) attend to the target at point B throughout the task ’ s time horizon ( Fig . 1 ) . Therefore , causes of navigation to point B at a next time step is the agent ’ s learned attention profile on the input images . These causal mappings are not present in other deep models ( Fig . 1 ) . Here , we show how LTCs ’ ability to capture causal structures directly from visual inputs results in improved robustness as well as interpretable decision making . Summary of Contributions . i ) We show theoretical evidence for the capability of CT neural networks in learning causal structures ; ii ) we perform extensive experiments supporting the effectiveness of causal CT models in visual drone navigation tasks with varying memory-horizons ; and iii ) we conduct robustness analysis of CT models in closed loop-testing within real-world scenarios . 2 Related Works . In this section , we describe research works that are closely related to the core findings of the paper . Causal Learning – The dominant approach toward learning causal models are graphical methods ( Ruggeri et al. , 2007 , Russell and Norvig , 2002 ) , which try to model cause-effect relationships as a directed graph ( Pearl , 2009 , Pearl et al. , 2009 ) . Bayesian networks further combine graphical models with Bayesian methods ( Kemp et al. , 2010 , Ruggeri et al. , 2007 ) to decompose the learning problem into a set of Bayesian inference sub-problems . ( Weichwald et al. , 2020 ) showed the effectiveness of such an approach for learning causal structures in nonlinear time-series via a set of linear models . Continuous-time Bayesian networks further adapted the idea to modeling cause-effect relationships in time-continuous processes ( Gopalratnam et al. , 2005 , Nodelman et al. , 2002 , 2003 , Nodelman , 2007 ) . A different approach for causal modeling of time-continuous processes is to learn ODEs , which under certain conditions imply a structured causal model ( Rubenstein et al. , 2016 ) . zIn this work , we describe a class of continuous models that has the ability to account for interventions and therefore captures the causal structures from data . Continuous-time Models – CT models manifest a large range of benefits compared to discretized deep models . They can perform adaptive computations through continuous vector fields realized by advanced ODE solvers ( Chen et al. , 2018b ) . They are strong in modeling time-series data and realize memory and parameter efficiency ( Chen et al. , 2018b , Lechner et al. , 2020a ) . A large number of alternative approaches have tried to improve and stabilize their training , namely the adjoint method ( Gholami et al. , 2019 ) , use neural ODEs in complex time-series prediction tasks ( Hasani et al. , 2021a , Lechner and Hasani , 2020 , Lechner et al. , 2019 , Rubanova et al. , 2019b ) , characterize them better for inference and density estimation tasks ( Dupont et al. , 2019 , Durkan et al. , 2019 , Hanshu et al. , 2020 , Holl et al. , 2020 , Jia and Benson , 2019 , Liebenwein et al. , 2021 , Massaroli et al. , 2020 , Quaglino et al. , 2020 ) , and to verify them ( Gruenbacher et al. , 2021 , Grunbacher et al. , 2021 ) . In this work , we prove an important property of the CT network : We show that a bilinear approximation of Neural ODEs can give rise to expressive causal models . Imitation Learning ( IL ) – Imitation learning describes the task of learning an observation-action mapping from human demonstrations ( Schaal , 1999 ) . This objective can either be achieved via behavior cloning , which directly learns from observation-action pairs ( Lechner et al. , 2019 , 2021 ) , or indirectly via inverse reinforcement learning ( Brunnbauer et al. , 2021 , Ng and Russell , 2000 ) which first constructs a reward function from an optimal policy . The most dominant behavior cloning paradigm is based on the DAgger framework ( Ross et al. , 2011 ) , which iterates over the steps of expert data collection , supervised learning , and cloned policy evaluation . State-aware IL ( Schroecker and Isbell , 2017 ) further adds a secondary objective to the learning task to bias the policy towards states where more training data is available . Recently , IL methods have been adapted to domains where the environment dynamics of the expert and learned policy mismatch ( Desai et al. , 2020 ) . More recent imitation learning one-shot methods pre-train policies via meta-learning to adapt to a task , such that task-specific behavior can be cloned with as little as a single demonstration ( Duan et al. , 2017 , Yu et al. , 2018 ) . Alternatively , generative adversarial imitation learning phrases the behavior cloning problem as a min-max optimization problem between a generator policy and discriminator classifier ( Ho and Ermon , 2016 ) . Baram et al . ( 2017 ) extended the method by making the human expert policy end-to-end differentiable . The method has been further adapted to imperfect ( Wu et al. , 2019 ) and incomplete demonstrations ( Sun and Ma , 2019 ) . Visual Navigation – Cognitive mapping and planning ( Gupta et al. , 2017a , b ) addresses the problem of learning to navigate from visual input streams by constructing a map of the environment and plan the agent ’ s actions to achieve a given goal . Chen et al . ( 2018a ) adapted the approach for the goal of exploring and mapping the environment . An alternative method represents the map in the form of a graph ( Savinov et al. , 2018 ) . Neural SLAM ( simultaneous location and mapping ) shares many characteristics of cognitive mapping and planning ( Chaplot et al. , 2019 ) but takes one step further in separating mapping , pose estimation , and goal-oriented navigation . Target driven navigation , where the agent must find a target in an unknown environment , has seen successes using both semantic segmentation ( Mousavian et al. , 2019 ) , and neural SLAM ( Chaplot et al. , 2020 ) . Moreover , visual navigation for learning-to-drive context , have extensively studied the causal confusion problem ( de Haan et al. , 2019 ) , and the generalization of the imitation learning problems , through using modules to extract useful priors from pixel inputs ( Chen et al. , 2020 , Filos et al. , 2020 , Park et al. , 2020 , Rhinehart et al. , 2019 , Sauer et al. , 2018 ) . These methods can benefit from the LTC-based networks designed in our present study , to enhance their knowledge distillation pipelines . 3 Problem Setup . In this section , we describe necessary concepts to formally derive the main results of our report . 3.1 Causal Structures . In a structural causal model ( SCM ) , given a set of observable random variables X1 , X2 , . . . , Xn , as vertices of a directed acyclic graph ( DAG ) , we can compute each variable from the following assignment ( Schölkopf , 2019 ) : Xi : = fi ( PAi , Ui ) , i = 1 , . . . , n. ( 1 ) Here , fi is a deterministic function of the parents of the event , Xi , in the graph ( PAi ) and of the stochastic variable , Ui . One can intuitively think of the causal structure framework as a function estimation problem rather than in terms of probability distributions ( Spirtes et al. , 2000 ) . Direct causation is implied by direct edges in the graph through the assignment described in Eq . 1 . The stochastic variables U1 , . . . , Un ensure that a joint distribution P ( Xi|PAi ) is constructed as a general objective ( Pearl , 2014 , Schölkopf , 2019 ) . The SCM framework enables us to explore through the known physical mechanisms and functions to build flexible probabilistic models with interventions , replacing the `` slippery epistemic probabilities , P ( Xi , PAi ) , with which we had been working so long '' in the study of machine learning systems ( Pearl , 2009 , Schölkopf , 2019 ) . Interventions can be formalized by the SCM framework , as operations that alter a subset of properties of Eq . 1 . For instance , modifying Ui , or replacing fi ( and as a result Xi ) ( Karimi et al. , 2020 ) . Moreover , by assuming joint independence of Uis , we can construct causal conditionals known as causal ( disentangled ) factorization , as follows ( Schölkopf , 2019 ) : p ( X1 , . . . , Xn ) = n∏ i=1 p ( Xi|PAi ) . ( 2 ) Eq . 2 stands for the causal mechanisms by which we can model all statistical dependencies of given observables . Accordingly , Causal learning involves the identification of the causal conditionals , the function fi , and the distribution of Uis in assignment Eq . 1 .

Summary: This paper explores the capabilities of continuous-time networks to form casualty structures, as well as their applications to visual drone navigation. The authors find that models based on the liquid time-constant networks (LTCs) uniquely provide causal structures. The theoretical discoveries are then verified in closed-loop visual navigation experiments. LTC-based policies significantly outperform regular continuous time models and discrete RNNs, all trained using imitation learning.