Causal Navigation by Continuous-time Neural Networks
Authors: Charles J Vorbach
Ramin Hasani
Alexander Amini
Mathias Lechner
Daniela Rus
Venue: NeurIPS 2021 Poster (2021)
Keywords: Continuous-time neural networks
Causality
neural ODEs
continuous-depth models
visual navigation

Abstract: Imitation learning enables high-fidelity, vision-based learning of policies within rich, photorealistic environments. However, such techniques often rely on traditional discrete-time neural models and face difficulties in generalizing to domain shifts by failing to account for the causal relationships between the agent and the environment. In this paper, we propose a theoretical and experimental framework for learning causal representations using continuous-time neural networks, specifically over their discrete-time counterparts. We evaluate our method in the context of visual-control learning of drones over a series of complex tasks, ranging from short- and long-term navigation, to chasing static and dynamic objects through photorealistic environments. Our results demonstrate that causal continuous-time deep models can perform robust navigation tasks, where advanced recurrent models fail. These models learn complex causal control representations directly from raw visual inputs and scale to solve a variety of tasks using imitation learning.

Causal Navigation by Continuous-time Neural Networks
Charles Vorbach" " Ramin Hasani Alexander Amini', Mathias Lechner?, Daniela Rus'
Abstract
Imitation learning enables high-fidelity, vision-based learning of policies within rich, photorealistic environments. However, such techniques often rely on tradi- tional discrete-time neural models and face difficulties in generalizing to domain shifts by failing to account for the causal relationships between the agent and the environment. In this paper, propose a theoretical and experimental framework for learning causal representations using continuous- cittuuuus--imeeeee neural networks, specifi- cally over their discrete-time counterparts. We evaluate our method the context of visual-control learning of drones over series of complex tasks, ranging from short- and long-term navigation, to chasing static and dynamic objects through photorealistic environments. Our results demonstrate that causal continuous-tin time deep models can perform robust navigation tasks, where advanced recurrent models fail. These models learn complex causal control representations directly from raw visual inputs and scale to solve variety of tasks using imitation learning.
Introduction
Input
NCP
ODE-RNN
LSTM
CT-GRU
Unlike machine learning sys- tems, natural learning systems excel at generalizing learned skills beyond the original data distribution (Hasani et al., 2020, Hassabis al., 2017, Sarma et al.. 2018). This is due to the mechanisms they deploy during their learning process, such as active use of closed-loop inter- ventions, accounting for distri- bution shifts, and the temporal structures (de Haan et al., 2019, Schölkopf, 2019). These factors are largely disregarded or are en- gineered away in the develop- ment of modern ML systems.
Time
Figure 1: Causal navigation from raw visual inputs. Given sequence raw RGB inputs (left) drone is trained to navigate towards the red-cube target. We visualize the saliency maps (right) for each model. Neural circuit policies (Lechner et al., 2020a) (a specific representation of CT-RNNs) can learn causal relation- ships (i.e., attend to the red-cube) directly from data while other models fail to do so. ODE-RNNS (Rubanova et al., 2019b), LSTM (Hochreiter and Schmidhuber, 1997) and CT- Gated Recurrent Units (Mozer et al., 2017). Saliency maps are computed by the visual backprop algorithm (Bojarski et al., 2016).
To take the first steps towards re- solving these issues, we can in- vestigate the spectrum of causal modeling (Peters et al., 2017). At one end of the spectrum, there are physical models of agents
'Equal Contributions. CSSAIL MIT, ²IST Austria, Correspondence to: rhasani @mit edu Code and data available https://githie www.mmitmdr//eeeeceeeeooeo drr.tttddddddeeedneeeeeee
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

and environments described by differential equations. These physical models allow us to inves- tigate interventions, predict future events using past information, and can describe the statistical dependencies in the system.
On the other end of the spectrum, statistical models allow us to construct dependencies and make predic- tions given independent and identically distributed (i.i.d)) (see Fig. 2). While physical models provide complete descriptions, it is intractable to define the differential equation systems that effectively model high-dimensions and complex sensory data. For long, causal modeling frameworks aims at bridging this gap to extract statistical dependencies while con- structing causal graphs (Pearl, 2009, Spirtes et al., 2000) or structural causal models(Schölkkof,, 2019) to intervene and explain relationships.
Taxonomy of Models
Physical
Structural causal
Causal graphs
Statistical models
Insights about system
Learn from data
Predict in IID Interventions
Answer counter factual
In this paper, we aim to use continuous-time (CT) neural networks (Chen et al., 201 Funahashi and Nakamura, 1993, Hasani al., 2021b) equipped with causal structures in order to get closer to the properties of physical models. In particular, we look into different representations of CT networks to see under what conditions and formulation they can form causal model. We discover that the class of liquid time-constant networks (LTCss) (Hasani et al., 2021b) which are expressive continuous-ting models constructed by bilinear approximation of neural ordinary differential equations (Chen et al., 201 2018b), satisfy the properties of causal model. The uniqueness of their solution and their ability to capture both internal and external interventions make their forward- and backward- mode causal. Therefore, they can impose inductive biases on their architectures to learn causal representations.
Taxonomy of models in the spec- trum of causality. Figure data is taken from (Peters al., 2017).
Figure 2:
We analyze how certain continuous citiiuuous-tmeeeeee (CT) neural networks are dynamical causal models. Fur- thermore, justify the theoretical results, we empirically validate that these properties scale to high-ding -igh-dimensional visual input data in complex learning environments. We propose series of control and navigation tasks for an end-to-end autonomous flight across various levels of complexity and temporal reasoning. We observe that traditional deep-learning models are capable of solving this task on offline, passive datasets but fail when deployed in closed-loop, active testing settings (de Haan et al., 2019, Wen et al., 2020). On the other hand, we find that only LTC-based models are able to complete the tasks in closed-loop interaction with the environments.
Specifically, consider a visual navigation task wherein a drone agent should learn to fly from point A to target fixated a point B, given only a sequence of raw visual inputs. The mapping between incoming pixels to the temporal and physical structure of the navigation is causal if we can explain and intervene at each step observe how navigation decisions are decided based on the input pixels, We observe that in a neighborhood environment, agents based on LTCs (such as Neural Circuit Policies (Lechner al., 2020a)) learn to stably (Lechner al., 2020b) attend to the target at point B throughout the task's time horizon (Fig. 1). Therefore, causes of navigation to point B at next time step is the agent's 's learned attention profile on the input images. These causal mappings are not present in other deep models (Fig. 1). Here, we show how LTCs' ability to capture causal structures directly from visual inputs results in improved robustness as well as interpretable decision making.
Summary of Contributions. i) We show theoretical evidence for the capability of CT neural net- works in learning causal structures; ii) we perform extensive experiments supporting the effectiveness of causal CT models in visual drone navigation tasks with varying seemory-horizons and iii) we conduct robustness analysis of CT models in closed loop-testing within real-world scenarios.
2 Related Works
In this section, we describe research works that are closely related to the core findings of the paper.
Causal Learning- - The dominant approach toward learning causal models are graphical methods (Ruggeri et al., 2007, Russell and Norvig, 2002), which try to model cause-effect relationships as directed graph (Pearl, 2009, Pearl et al., 2009). Bayesian networks further combine graphical models

---

with Bayesian methods (Kemp et al., 2010, Ruggeri et al.. 2007) to decompose the learning problem into a set of Bayesian inference sub-problems. (Weichwald et al., 2020) showed the effectiveness of such an approach for learning causal structures in nonlinear time-series via set of linear models. Continuous-time Bayesian networks further adapted the idea to modeling cause-effect relationships in time-continuous processes (Gopalratnam et al., 2005, Nodelman et al., 2002, 2003, Nodelman, 2007). A different approach for causal modeling of time-continuous processes is to learn ODEs, which under certain conditions imply a structured causal model (Rubenstein et al., 2016). zIn this work, we describe class of continuous models that has the ability to account for interventions and therefore captures the causal structures from data.
Continuous-time Models - CT models manifest a large range of benefits compared to discretized deep models. They can perform adaptive computations through continuous vector fields realized by advanced ODE solvers (Chen et al., 201 They are strong in modeling time-series data and realize memory and parameter efficiency (Chen et al., 201 8b, Lechner et al., 2020a). A large number of alternative approaches have tried to improve and stabilize their training. namely the adjoint method (Gholami et al., 2019), use neural ODEs in complex time-series prediction tasks (Hasani et al., 202 la, Lechner and Hasani, 2020, Lechner et al., 2019, Rubanova et al., 2019b). characterize them better for inference and density estimation tasks (Dupont et al., 2019, Durkan et al., 2019, Hanshu et al., 2020, Holl et al., 2020, Jia and Benson, 2019, Liebenwein et al., 2021, Massaroli et al., 2020, Quaglino et al., 2020), and to verify them (Gruenbacher et al., 2021, Grunbacher et al., 2021). In this work, prove an important property of the CT network: We show that a bilinear approximation of Neural ODEs can give rise expressive causal models.
Imitation Learning (IL)- - Imitation learning describes the task of learning an observation-action mapping from human demonstrations (Schaal, 1999). This objective can either be achieved via behavior cloning, which directly learns from observation-action pairs (Lechner et al., 2019, 2021), I), or indirectly via inverse reinforcement learning (Brunnbauer et al., 2021, Ng and Russell, 2000) which first constructs a reward function from an optimal policy. The most dominant behavior cloning paradigm is based on the DAgger framework (Ross et al., 2011), which iterates over the steps of expert data collection, supervised learning. and cloned policy evaluation. State-aware IL (Schroecker and Isbell, 2017) further adds secondary objective to the learning task to bias the policy towards states where more training data is available. Recently, IL methods have been adapted to domains where the environment dynamics of the expert and learned policy mismatch (Desai et al., 2020).
More recent imitation learning one-shot methods pre-train policies via meta-learning to adapt to a task, such that task-specific behavior can be cloned with as little a single demonstration (Duan et al., 2017, Yu et al., 2018). 8). Alternatvelyy, generative adversarial imitation learning phrases the behavior cloning problem as a min-max optimization problem between a generator policy and discriminator classifier (Ho and Ermon, 2016). Baram et al. (2017) extended the method by making the human expert policy end-to-end differentiable. The method has been further adapted to imperfect (Wu et al., 2019) and incomplete demonstrations (Sun and Ma, 2019).
Visual Navigation Cognitive mapping and planning (Gupta et al., 2017a,b) addresses the problem of learning to navigate from visual input streams by constructing a map of the environment and plan the agent's s actions to achieve given goal. Chen et al. (2018a) adapted the approach for the goal of exploring and mapping the environment. An alternative method represents the map in the form of graph (Savinov et al., 2018). Neural SLAM (simultaneous location and mapping) shares many characteristics of cognitive mapping and planning (Chaplot et al., 2019) but takes one step further in separating mapping, pose estimation, and goal-oriented navigation. Target driven navigation, where the agent must find a target in an unknown environment, has seen successes using both semantic segmentation (Mousavian et al., 2019), and neural SLAM (Chaplot et al., 2020).
Moreover, visual navigation for caaning-to-drits context, have extensively studied the causal con- fusion problem (de Haan et al., 2019), and the generalization of the imitation learning problems, through using modules to extract useful priors from pixel inputs (Chen et al., 2020, Filos et al., 2020, Park et al., 2020, Rhinehart et al., 2019, Sauer et al., 2018). These methods can benefit from the baS-based networks designed in our present study, to enhance their knowledge distillation pipelines.
3 Problem Setup
In this section, we describe necessary concepts to formally derive the main results of our report.
3

---

3.1 Causal Structures
In a structural causal model (SCM), given a of observable random variables X1, X2.X2,... as vertices of directed acyclic graph (DAG), we can compute each variable from the following assignment (Schölkopf. 2019):
X2....,
....... X,,
(1) Here, fi is a deterministic function of the parents of the event, X, in the graph (PA,) and of the stochastic variable, U.. One can intuitively think of the causal structure framework as a function estimation problem rather than terms of probability distributions (Spirtes et al., 2000). Direct causation is implied by direct edges in the graph through the assignment described in Eq. 1. The stochastic variables UI....,A Un ensure that a joint distribution P(X; PA:) constructed as general objective (Pearl, 2014, Schölkopf, 2019).
: (PA,,, i=
The SCM framework enables us to explore through the known physical mechanisms and functions to build flexible probabilistic models with interventions, replacing the "slippery epistemic probabilities, P(X,, PA.). with which we had been working so long" in the study of machine learning systems (Pearl, 2009, Schölkopf, 2019). Interventions can be formalized by the SCM framework, as operations that alter subset of properties of Eq. 1. For instance, modifying U. or replacing fs (and as a result X,) (Karimi et al., 2020). Moreover, by assuming joint independence of U,s, we can construct causal conditionals known as causal (disentangled) factorization, as follows (Schölkopf, 2019):
p(XI, X..) II T(X(PA.))
Eq. stands for the causal mechanisms by which we can model all statistical dependencies of given observables. Accordingly, Causal learning involves the identification of the causal conditionals, the function fi, and the distribution of U.S in assignment Eq. 1.
3.2 Differential Equations Can form Causal Structures
Physical dynamics can be modeled by a set of differential equations (DEs). DEs allow us to predict the future evolution of a dynamical system and describe its behavior as result of interventions. Their coupled time-evolution enables us to define averaging mechanisms for computing statistical dependencies (Peters et al., 2017). A system of differential equations enhances our understanding of the underlying physical phenomenon, explains its behavior, and dissects its causal structure.
For instance, consider the following system of DEs: di g(x,, x E Rd, with initial values at To. where g is a nonlinear function. The Picard-Lindelöf theorem (Nevanlinna, 1989) states that a differential equation of the form above would have unique solution as long as y is Lipschitz. Therefore, if we unroll the system to infinitesimal differentials using the explicit Euler method, we get: x(t + ōt) = x(t) + dt (x). This representation under the uniqueness condition shows that the near future events of are predicted using its past information, thus, forming causal structure.
Thus, DE system is a causal structure that allows us to process the effect of interventions on the system. On the other side of the spectrum of causal modeling (Peters et al., 2017), pure statistical models allow us to learn structures from data with little insight about causation and associations between epiphenomena. Since causality aims to bridge this gap, in this paper, we propose to construct causal models with continuous-time neural networks.
3.3 Continuous-time Neural Networks
CT networks are a class of deep learning models with their hidden states being represented by ordinary differential equations (ODEs) (Funahashi and Nakamura, 1993). The hidden state .x(t) of neural network f is computed by the solution of the initial value problem below (Chen et al., 2018b):
dz -(((,, E dt
where f' is parametrized by 0. CT models enable approximation of a class of functions which we did not know how to generate otherwise (Chen et al., 2018b, Hasani et al., 2021b). They can be used in both inference and density estimation with constructing continuous flow to model data,

---

efficiently (Dupont et al., 2019, Grathwohl et al., 2018, Rubanova et al., 2019a). CT models can be formulated in different representations. For instance, to achieve stability, CT recurrent neural networks (CT-RNNss) were introduced in the following form (Funahashi and Nakamura, 1993):
dx(() dt
x(() ((x()),,0).
x(t) derives the system to equilibrium with a time-constant т. To increase expressiv- ity (Raghu et al., 2017), liquid time-constant networks (LTCs) with the following representation can be used (Hasani et al., 2021b):
where the term
dx(() - - +++++ ((x(0, [((0), 10),4,0) @ x(1) + ((((1), ((),4,0) @ A. dt
(5)
In Eq. 5, (1) is the hidden state of an LTC layer with D cells, (())))( is the input to the system, 7(xx1) is the fixed internal time-constant vector), (Dx1)) is an output control bias vector, and @ is the Hadamard product. Here, the ODE system follows bilinear dynamical system (Penny et al., 2005) approximation of Eq. 3 to construct an input-dependent nonlinearity in the time-constant of the differential equation. This was shown to significantly enhance the expressive power of CT models in robotics and time-series prediction tasks (Hasani al., 2021b, Lechner et al., 2020a). In the following, we show how CT models can be designed as causal structures to perform more interpretable real-world applications.
Results
In this section, we first explain how continuous time model identified by Eq. 3, standalone, cannot satisfy the causal structure properties (Peters et al., 2017) even under Lipschitzness of its network. We then show that the class of liquid time-constant networks can enable causal modeling.
4.1 Causal Modeling with Continuous-tine Networks
Neural ODEs cannot account for external interventions. Let f be the nonlinearity of continuous- time neural network. Then the learning system defined by Eq. 3, form temporal causation, however cannot account for external interventions (change of the environment conditions), and therefore does not form a causal structure even when f is Lipschitz-cootinnouus.
To describe this in detail, unfold the ODE by infinitesimal differentials as follows:
x(t + 5t) x(t) + fx,
(6)
If is Lipschitz continuous, based on Picard-Lindelöf's existence theorem, the trajectories this ODE system are unique and thus invertible. This means that, at least locally, the future events of the system can be predicted by its past values. Since the transformation invertible, this setting is also true if we run the ODE backward in time (e.g., during the training process).
When the ODE system is trained by maximum likelihood estimation, given an initial weight distri- bution, the statistical dependencies between the system's 's variables might emerge from data. The resulting statistical model can predict in i.i.d. setting and learn from data, but cannot predict under distribution shift or implicit/explici interventions, simply because the system's 's semantics does not have input-dependent terms. Thus, the system cannot answer counterfactual questions (Mooij et al., 2013).² Although Neural ODEs in their generic representation cannot account for interventions, their other forms can help design a causal model.
LTCs are Dynamic Causal Models. LTCs described by Eq. 5 resemble the representation of simple Dynamic Causal Model (DCM) (Friston et al., 2003) with bilinear Taylor approximation (Penny et al., 2005). DCMs aim to extracting the causal architecture of a given dynamical systems. DCMs represent the dynamics of the hidden nodes of a graphical model by ODEs, and allow for feedback connectivity structures unlike Bayesian Networks (Friston et al., 2003). A simple DCM
A counterfactual question describe causal relationship of the form: "If X had not occurre, Y would not have occurred (Molnar, 2020)"
5

---

can be designed by a second-orde order approximation (bilinear) of a given system such as dx/dt. F(x(t), , 8), as follows (Friston et al., 2003):
dx/dt dt (A I(t)B)x(t) +C((t)
OF (x
FF C OF
A
B
1=0"
(x(t)
aI( 2==
where I(t) is the inputs to the system, and x(t) is the nodes' hidden states. A key property of DCM representation is its ability to capture both internal and external causes on the system (interventions). Matrix A is fixed internal coupling of the system. Matrix B controls the impact of the inputs on the coupling sensitivity among the network's nodes (controlling internal interventions). Matrix embodies the external inputs' influence the state of the system (controlling external interventions).
DCMs have shown promise in learning the causal structure of the brain regions complex time- series data (Breakspear, 2017, Ju and Bassett, 2020, Penny et al., 2005). DCMs can be extended to universal framework for causal function approximation by neural networks through the LTC neural representations (Hasani et al., 2021b).
Proposition 1. Let f be the nonlinearity of a given LTC network identified by Eq. 5. Then the learning system identified by Eq. 5 can account for internal and external interventions by its weight parameters (W(PxD) WDDxm), ((Dx1)) }, for D LTC cells, and input size m and D and therefore. forms dynamical causal model, if is Lipschitz- Lipschitz-continuuus.
Proof. To prove this, we need to show two properties for the LTC networks: 1) Uniqueness of their solution 2) Existence of internal and external intervention coefficients:
Uniqueness. By using the Picard-Lindelöf theorem (Nevanlinna, 1989), it was previously shown that for an F RDD + R bounded C² -amppnnn the differential equation:
j = (1/T + F(*))æ + AF(x),
has unique solution on [0, oc). (Full proof in Lemma of (Hasani al., 2021b)).
Interventions Coefficients. Let f be a Lipschitz-continuous activation function such as tanh, then [(((t), 2, 0) lanh(W,xx WI + b). If we set x 0. in Eq. then the external intervention coefficients C Eq. 7 for LTCs can be obtained by:
0F
AA
W(1
==0
The corresponding internal intervention coefficients B of Eq. 7, for LTC networks becomes:
F²F
w( [2W. (A x) (A-x)+ +1]
(x(1)a((1)
This shows that by manipulating matrices W, W,,, and A one control internal and external interventions to an LTC system which gives the statement of the proposition.
Proposition 1 shows that LTCs causal models in their forward pass, and by manipulation of their parameters, one can gain insights into the underlying systems. We next show that LTCs' backward path also gives rise to causal model. This implies that LTCs trained from demonstration via reverse-mode automatic differentiation (Rumelhart et al., 1986) can give rise to a causal model.
4.2 Training LTCs via Gradient Descent Yields Causal Models
The uniqueness of the solution of LTCs allows any ODE solver to reproduce a forward pass trajectory with backward computations from the end point of the forward pass. This property enables us to use the backpropagation algorithm, either by using the adjoint sensitivity (Pontryagin, 2018) method or backpropagation through time (BPTT) to train an LTC system. Formally, for a forward pass trajectory of the system states x(t) of an LTC from to to tn.
compute scalar value loss, L at t, by:
In dx
L(x(t,)) L (x((t)
(8)
to
dt

---

Suppose we use the adjoint method for training the network, then the augmented state is computed
di = =aa (t) BLTC (Chen et al., 2018b). To by a(t) = ãx(ej. whose kinctics are determined by da
Ot
dx(()
compute the loss gradients in respect to the parameters, we run the adjoint dynamics backwards from gradients dx(t,)
db and states x(t,,,, while solving a third integral in reverse as follows: di db
- a² (1) BLTC dr. All integrals can be solved in reverse-mode by call to the ODE solver.
Based on Proposition 1, for an LTC network at each training iteration, the internal and external interventions not only help the system learn statistical dependencies from data but also facilitate the learning of the causal mapping. These results bridge the gap between pure physical models and causal structural models to obtain better learning systems.
Real-world implications. Consider a simple drone navigation task in which the objective is for the drone to navigate from position A to target position B while simultaneously avoiding obstacles. In this setting, the agent is asked to infer high-level control signals directly from visual inputs to reach the target goal. Even the simplest vision- deep model can accomplish this task in passive open-loop settings (Lechner et al., 2020a). However, completion in a closed-loop environment while inferring the true causal structure of the task is significantly more challenging. For instance, where should the agent attend to for taking the next action? With what mechanisms does the system infer the objective of the task? And accordingly, how robust the decisions of the learned policy?
In the next section, we perform an extensive experimental evaluation to find answers to the above questions. We also aim to validate our theoretical results on the capability of LTC models to capture the true causal structure of a nigh-dimensional visual reasoning task, where other models fail.
5 Experiments
We designed photorealistic visual navigation tasks with varying memory horizons including (1) navigating to a static target, (2) chasing a moving target, and (3) hiking with guide markers (Fig. 3).
Experimental setup. We designed tasks in Microsoft's AirSim (Madaan et al., 2020) and Unreal Engine. To create data for imitation learning, we use greedy path search with Euclidean heuris- tic over unoccupied voxels to obtain knot points for cubic spline interpolation, which is then followed via pure pursuit controller. This strategy is modified for each task (details in the supplements).
Visual navigation static targets
B Chasing moving target C
Hiking
Tarpet
start poslt
Task porsoeative ronment
Baselines. We evaluate NCP networks (Lechner et al., 2020a) against a set of baseline models. This includes ODE- RNNs (Rubanova et al., 2019b) which the recurrent network version of Neu- ral ODEs (Chen et al., 2018b), long short- term memory networks (LSTMs) (Hochreiter and Schmidhuber, 1997), and CT-GRU networks (Mozer et al., 2017), which are the continuous equivalent of GRUs (Chung et al., 2014). These baselines are chosen to validate our theoretical results: Not all CT models causal models. Similarly, discretized RNN models, such as LSTMs, perform well on tasks with long-term dependencies, yet they are not causal models. In Section 4, we showed that NCPs, which are sparse neural networks built based on LTC neurons (Hasani et al., 2021b) are dynamical causal models. Therefore, they can learn the true causation of given task. In our experiments, camera images are perceived by convolutional layers and are fed into the RNN networks which act as a controller. For a fair comparison, the number of trainable parameters of all models is within the same range, and they are all trained by Adam optimizer (Kingma and Ba, 2014) with a cosine similarity loss (See more details in the supplements).
Visual drone navigation tasks. A) Navigation to static target, B) Chasing a moving target, C) Hiking with a set of markers in the environment
Figure 3:
5.1 Navigation to Static Target with Occlusion
In this task, the drone navigates to target marker that less than 25 meters away and visible to it. We place red cube a random unoccupied voxel in the environment to function as the target
7

---

NCP
ODE-RNN
LSTM
CT-GRU
Figure 4: Navigation to static target in closed-loop environments. NCPs are the only models that can capture the causal structure of the tasks directly from visual data.
marker. We constrain the target marker to appear in the drone's viewing frustum for most cases. Though there may be occlusion of the target upon random respond of the drone in the environment (more details in the supplements). Occlusions create temporal dependencies. We tested static CNN networks and observed that they fail to perform scenarios in which the target is occluded. Table shows that in both neighborhood and forest environments, all agents learn the task with a reasonable validation loss in passive imitation learning setting. However, once these agents are deployed with closed-looy control, we observed that the success rates for LSTM and ODE-RNNs drop to only 24% and 18% of 50 attempted runs. CT-GRU managed complete this task in 40% of the runs, whereas NCPs completed the tasks in 48% of the runs. Why is this the case?
To understand these results better, the Visual-Backprop algorithm (Bojarski et al., 2016) to compute the saliency maps of the learned features in the input space. Saliency maps would show us where the attention of the network was when taking the next navigation decision. As shown Fig. 4, observe that NCP has learned to attend to the static target within its field of view to make future deci- sion. This attention profile was not present in the saliency maps of the other agents. LSTM agents are sensitive to lighting conditions compared to the CT models. This experiment supports our theoretical results on the ability of LTC-based models to learn causal representations.
used
Table I: Validation short-term navigation. Co- sine similarity loss [-1, 1] (smaller better), n=5.
Algorithms
Environments
RedWood Forest
Neighborhood
LSTM ODE-RNN CT-GRU
-0,823 + 0.006
-0,838 + 0.019
-0.815 = 0.043
-0.855 0.019
-0.855 = 0.001
-0.877 0.002
NCP (ours)
-0.859 + 0.035
-0.855 0.008
NCP
ODE-RNN
LSTM
CT-GRU
Figure 5: Chasing moving target in closed-loop environments. NCPs are the only models that can capture the causal structure of the tasks directly from visual data.

---

Table 2: Chasing objects, Validation performance Table 3: Hiking, Validation performance with with co-sine similarity loss (smaller is better), n=5 sine similarity loss (smaller is better), n=5
Algorithms
Environments
Algorithms
Environments
RedWood Forest
Neighborhood
RedWood Forest
Neighborhood
LSTM
-0,943 1 0.028
-0.947
0,008
LSTM
-0.273 1 0.388
-0.781 1 0.030
ODE-RNN
-0.967 1 0.009
-0,953
0.011
ODE-RNN
-0.89 0.026
-0.710 1 0.003
CT-GRU
-0,958 1 0.017
-0.979
0.003
CT-GRU
-0.359 1 0.073
-0.725 1 0.086
NCP (ours)
-0.936 + 0.022
-0.975
0.012
NCP (ours)
-0.676 + 0.192
-0.711 1 0.013
5.2
Chasing Moving Target
In this task, the drone follows a target marker along smooth spline path. Using generate and test method, we create a path for the drone to follow by using a random walk with momentum to find knot points for fitting a spline (details in the supplements). Table 2 shows that all agents were able learn to follow their targets in passive open-loop case. However, similar to the previous experiment, we witnessed that all models can successfully complete the task closed-loop setting where interventions play big role. NCPs were 78% successful at completing their task, while LSTM in 66%, EDE-RNNSs 52%, and CT-GRU in 38% were successful. Once again, we looked into the attention maps of the models, illustrated in Fig. 5. We see that CT-GRU networks did not learn to attend the target they follow. LSTMs again show sensitivity to lighting conditions. ODE-RNNs keep close distance to the target, but they occasionally lose the target. In contrast, NCPs have learned to attend to the target and follow them as they move the environment.
5.3 Hiking Through an Environment
In this task, the drone follows multiple target markers which are placed the surface of obstacles within the environment (Fig. 3C) (Experimental details in Supplements). This task is significantly more complex than the previous tasks, especially when agents deployed directly in the environ- ment. This is because the agents have to learn to follow a much longer time-horizon task, by visual cues, in hierarchical fashion.
Interestingly, we see most agents learn reasonable degree of validation loss during the learning process as depicted by Table 3. Even ODE-RNNs realize excellent performance in the passive setting. However, when deployed in the environment, none of the models other than NCP could perform task completely in 50 runs. NCPs could perform 30% successfully thanks to their causal structure.
How to improve the performance of models in the hiking task? In order to improve the perfor- mance on a purely visually-navion navigated hiking task, an agent must be supplied with long-tem terg-termmae memory component (much longer than that of LSTMs). All ODE-based models including NCPs require gradient wrapper to be able to perform well on tasks with very long-term dependencies. For instance, mixed memory architectures such ODE-LSTM (Lechner and Hasani, 2020) can use NCPs as their continuous-time memory wrapped together with gradient propagation mechanisms enabled by an LSTM network, to perform better on the hiking task.
CNN network's performance in all scenarios. In Table 4, we summarized the success rate of CNNs in all tasks when deployed in closed-loop. As expected, observe that when temporal dependencies in the tasks appear (such Occlusion, or Hiking) as well as when the input images are highly perturbed (such heavy rain and Fog) the performance CNNs drastically decreases. This observation validates that having a memory component, in all tasks is essential.
Table 4: Closed-loop evaluation trained policies on various navigation and interaction tasks. Agents and policies are reinitialized randomly at the beginning of each trial (n=50). Values correspond to success rates (higher is better).
Static Target
Chasing
Hiking
Model
Clear
Fog
Light Rain
Heavy Rain
Occlusion
Clear
Fog
Light Rain
Heavy Rain
Clear
CNN
36%
6%
32%
2%
4%
50%
42%
54%
28%
0%
24%
22%
22%
20%
66%
62%
56%
44%
2%
ODE-RNN
18%
10%
18%
29%
24%
52%
42%
62%
44%
4%
CT-GRU
40%
8%
60%
32%
28%
38%
36%
48%
42%
0%
NCP (ours)
52%
52%
30%

---

6 Discussions, Scope and Conclusions
We provided theoretical evidence for the capability of different representations of continuous time neural networks in learning causal structures. We then performed a set of experiments to confirm the effectiveness of continuous-timg causal models in high-dimensional and visual drone navigation tasks with different semoryy-hrizoone time-constant networks has the great potential of learning causal structures in closed-loop reasoning tasks where other advanced RNN models cannot perform well.
compared to other methods. We conclude that the class of liquid
Model performances drop significantly in closed-loop. Table shows a summary of success episodes when the agents were deployed in various navigation tasks. As the memory-horizon and the complexity of the task increases, models that are not causal struggle with close-loop interactions with the environment. Continuous-tiin models with causal structures, such NCPs, considerably improve the real-world performance of agents.
Clear
Rain
Fog
Robustness to environmental perturbations. We next investigated how robust are these mod- els under heavy environmental perturbations such as rain. We used the AirSim weather API to include heavy rain and fog in closed-loop tests of the static target and chasing a moving target task (Fig. 6).
We observed that under high-intensstt rain, all models have performance drop, but NCPs show more resiliency to these perturbations. High-intensity rain and fog (Table 4) confuses LSTM and ODE RNNs the most where they are not able to complete the task.
Figure 6: Sample input images from the forest environment. While agents are trained on offline, passive datasets from the Clear environment, test- ing performed in an active control setup under various levels of environmental perturbations.
When shall we use temporal convolutions and attention-basee methods? Both temporal con- volution (Bai et al., 20 and attention- architectures (Vaswani et al., 2017) are reasonable choices for many spatiotemporal tasks (Kaufmann al., 2020, Lee et al., 2020). However, they require the user to explicitly set the temporal horizon of the observation memory, whereas RNNs learn the temporal memory component, implicitly. Consequently, RNNs are usually the preferred choice when the temporal horizon is not known or varies, which is the case in our experiments.
Handling Occlusions and temporal dependencies. All tasks discussed require temporal depen- dencies because of environmental line-of-sight occlusions between the drone and the target. We specifically observed that NCPs are capable of handling more complex runs (where occlusions maximal) and other models fail. We have also tested these scenarios with (single frame) CNNs and observed that the model could not complete the task successfully in all cases where the marker is occluded. CNNs achieved only a 14% success rate in static target runs with an occlusion which is at least 50% lower than the results of the other recurrent models shown in Table 4.
If temporal samples arrive at a constant rate, why a continuous-timg time model performs better? The use of continuous models (implemented by ODEs) for continuous problems can potentially lead to the design of better causal mechanisms (getting closer to physical modeling in the taxonomy of causal models). Moreover, many recent works show advanced CT models can outperform advanced discretized RNNs even when the incoming samples are equidistant (Erichson et al., 2021, Rusch and Mishra, 2021). Also, CT-models realize novel category of functions which was not possible to realize otherwise (i.e., the vector fields realized by complex ODE solvers cannot be achieved by discretized models) (Chen et al., 2018b).
is not trivial to show causal properties for gated RNNs. In gated recurrent networks (c.g., LSTMs), the gating mechanisms can be counted an implicit intervention to the state representation. However, a direct theoretical framework to prove this is not feasible as there is no direct relation between dynamic causal models and LSTMs, for instance. Besides, our empirical study in visual navigation tasks suggests that the gating mechanism does not help causality (See for instance the attention maps presented in Figures. 1, 4, and 5.)
In summary, we showed that special presentation of CT models realize causal models and can significantly enhance decision making in real-world applications.
10

---

Acknowledgments
C.V., R.H. A.A. and D.R. are partially supported by Boeing and MIT. A.A. is supported by the National Science Foundation (NSF) Graduate Research Fellowship Program. M.L. is supported in part by the Austrian Science Fund (FWF) under grant Z211-N22 (Wittgenstein Award), Research was sponsored by the United States Air Force Research Laboratory and the United States Air Force Artificial Intelligence Accelerator and was accomplished under Cooperative Agreement Number FA8750-199 1--30--99--111111 2000 The views and conclusions contained in this document those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the United States Air Force the U.S. Government. The U.S. Govermment is authorized to reproduce and distribute reprints for Govermment purposes notwithstanding any copyright notation herein.
Funding Transparency Statement
Authors declare no competing interests. Funding in direct support of this work: The Bocing Company through the Explainable Control Project, National Science Foundation (NSF) Graduate Research Fellowship Program, Austrian Science Fund (FWF) under grant Z21 1-N23 (Wittgenstein Award), the United States Air Foree Research Laboratory and the United States Air Force Artificial Intelligence Accelerator and was accomplished under Cooperative Agreement Number F-88750-19-2-1000.
References
Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.0127/ 2018.
Nir Baram, Oron Anschel, Itai Caspi, and Shie Mannor. End-to-end differentiable adversarial imitation learning. In International Conference on Machine Learning, pages 390-399. PMLR. 2017.
Mariusz Bojarski, Anna Choromanska, Krzysztof Choromanski, Bernhard Fimerr, Larry Jackel, Urs Muller, and Karol Zieba. Visualbackprop: efficient visualization of cnns. arXiv preprint arXiv:1611.05415 2016.
Michael Breakspear. Dynamic models of large-scale brain activity. Nature neuroscience, 20(3): 340-352, 2017.
Axel Brunnbauer, Luigi Berducci, Andreas Brandstätter, Mathias Lechner, Ramin Hasani, Daniela Rus, and Radu Grosu. Model-based versus model-free deep reinforcement learning for autonomous racing cars. arXiv preprint arXiv:2 103.0499, 2021.
Devendra Singh Chaplot, Dhiraj Gandhi, Saurabh Gupta, Abhinav Gupta, and Ruslan Salakhutdi- nov. Learning to explore using active neural slam. In International Conference on Learning Representations, 2019.
Devendra Singh Chaplot, Ruslan Salakhutdinov, Abhinav Gupta, and Saurabh Gupta. Neural topological slam for visual navigation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 12875-12884, 2020.
Dian Chen, Brady Zhou, Vladlen Koltun, and Philipp Krähenbühl. Learning by cheating. In Conference on Robot Learning, pages 66-75. PMLR, 2020.
Tao Chen, Saurabh Gupta, and Abhinav Gupta. Learning exploration policies for navigation. In International Conference on Learning Representations, 2018a
Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary dif- ferential equations. In Advances in neural information processing systems, pages 6571-6583, 2018b.
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks sequence modeling. arXiv preprint
2014.
11

---

Pim de Haan, Dinesh Jayaraman, and Sergey Levine. Causal confusion in imitation learning, arXiv preprint arXiv: 11979, 2019.
Siddharth Desai, Ishan Durugkar, Haresh Karnan, Garrett Warnell, Josiah Hanna, Peter Stone, and Al Sony. An imitation from observation approach to transfer learning with dynamics mismatch. Advances in Neural Information Processing Systems, 33, 2020.
Yan Duan, Marcin Andrychowicz, Bradly C Stadie, Jonathan Ho, Jonas Schneider, Ilya Sutskever, Pieter Abbeel, and Wojeiech Zaremba. One-shot imitation learning. arXiv preprint arXiv:1703.07326, 2017.
Emilien Dupont, Armaud Doucet, and Yee Whye Tch. Augmented neural odes. In Advances in Neural Information Processing Systems, pages 3134-3144, 2019.
Conor Durkan, Artur Bekasov, Iain Murray, and George Papamakarios. Neural spline flows. In Advances in Neural Information Processing Systems, pages 7509-7520, 2019.
N. Benjamin Erichson, Omri Azencot, Alejandro Queinuga, Liam Hodgkinson, and Michael W. Mahoney. Lipschitz recurrent neural networks. In International Conference on Learning Represen- tations, 2021. URL https:// wtw.://www....tt.ooooooaaoaa/aaa
Angelos Filos, Panagiotis Tigkas, Rowan McAllister, Nicholas Rhinehart, Sergey Levine, and Yarin Gal. Can autonomous vehicles identify, recover from, and adapt distribution shifts? In International Conference on Machine Learning, pages 3145-3153, PMLR, 2020.
Karl J Friston, Lee Harrison, and Will Penny. Dynamic causal modelling. Neuroimage, 19(4): 1273-1302, 2003.
Ken-ichi Funahashi and Yuichi Nakamura. Approximation of dynamical systems by continuous time recurrent neural networks. Neural networks, 6(6):801-806, 1993.
Amir Gholami, Kurt Keutzer, and George Biros. Anode: Unconditionally accurate memory-efficient gradients for neural odes. arXiv preprint 19X.::990.110296 2019.
Karthik Gopalratnam, Henry Kautz, and Daniel S Weld. Extending continuous time bayesian networks. In AAAI, pages 981-986 2005.
Will Grathwohl, Ricky TQ Chen, Jesse Bettencourt, Ilya Sutskever, and David Duvenaud. Ffjord: Free-form continuous dynamics for scalable reversible generative models. arXiv preprint arXiv:1810.01366, 2018.
Sophie Gruenbacher, Mathias Lechner, Ramin Hasani, Daniela Rus, Thomas A Henzinger, Scott Smolka, and Radu Grosu, Gotube: Scalable stochastic verification of continuous-depth models. arXiv preprint arXiv: arXhv:2107.08467, 2021.
Sophie Grunbacher, Ramin Hasani, Mathias Lechner, Jacek Cyranka, Scott A. Smolka, and Radu Grosu. On the verification of neural odes with stochastic guarantees. Proceedings of the AAAI Conference on Artificial Intelligence, 35(13): 15555-1115553313 May 2021.
Saurabh Gupta, James Davidson, Sergey Levine, Rahul Sukthankar, and Jitendra Malik. Cognitive mapping and planning for visual navigation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2616 2625, 2017a.
Saurabh Gupta, David Fouhey, Sergey Levine, and Jitendra Malik. Unifying map and landmark based representations for visual navigation. arXiv preprint arXiv: 171.:0112..08225, 2017b.
YAN Hanshu, DU Jiawei, TAN Vincent, and FENG Jiashi. On robustness of neural ordinary differential equations. In International Conference on Learning Representations, 2020.
Ramin Hasani, Mathias Lechner, Alexander Amini, Daniela Rus, and Radu Grosu. The natural lottery ticket winner: Reinforcement learning with ordinary neural circuits. Proceedings of the International Conference Machine Learning, 2020.
12

---

Ramin Hasani, Mathias Lechner, Alexander Amini, Lucas Liebenwein, Max Tschaikowski, Gerald Teschl, and Daniela Rus. Closed-fore continuous-depth models. arXiv preprint arXiv:2106.13898 2021a.
Ramin Hasani, Mathias Lechner, Alexander Amini, Daniela Rus, and Radu Grosu. Liquid time- constant networks. Proceedings of the AAAI Conference on Artificial Intelligence 35(9):6657-7666, May 2021b.
Demis Hassabis, Dharshan Kumaran, Christopher Summerfield, and Matthew Botvinick. Neuroscience-inspired artificial intelligence. Neuron, 95(2):245-258, 2017.
Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. arXiv preprint
1oXY::6606.03466, 2016.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735-1780, 1997.
Philipp Holl, Vladlen Koltun, and Nils Thuerey. Learning to control pdes with differentiable physics. arXiv preprini arXiv:2001. 007457, 2020.
Junteng Jia and Austin R Benson. Neural jump stochastic differential equations. In Advances in Neural Information Processing Systems, pages 9843-9854, 2019.
Harang Ju and Danielle S Bassett. Dynamic representations in networked neural systems. Nature Neuroscience, 23(8):908-911 2020.
Amir-Hossein Karimi, Bernhard Schölkopf, and Isabel Valera. Algorithmic recourse: from counter- factual explanations to interventions. arXiv preprint arXivv2002..62278, 2020.
Elia Kaufmann, Antonio Loquercio, René Ranftl, Matthias Müller, Vladlen Koltun, and Davide Scaramuzza. Deep drone acrobatics. arXiv preprint arXiv:2006.0574 2020.
Charles Kemp, Noah D Goodman, and Joshua B Tenenbaum. Learning to learn causal models. Cognitive Science, (4(7))1185-122)) 2010.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1rXiv:1412.6988, 2014.
Mathias Lechner and Ramin Hasani. Learning long-term dependencies in irreuularly-sampled time series. arXiv preprint arXiv:2006.00 04418, 2020.
Mathias Lechner, Ramin Hasani, Manuel Zimmer, Thomas A Henzinger, and Radu Grosu. Designing worm-inspired neural networks for interpretable robotic control. In International Conference on Robotics and Automation (ICRA), pages 87-94, 2019.
Mathias Lechner, Ramin Hasani, Alexander Amini, Thomas A Henzinger, Daniela Rus, and Radu Grosu. Neural circuit policies enabling auditable autonomy. Nature Machine Intelligence, 2(10): 642 652, 2020a.
Mathias Lechner, Ramin Hasani, Daniela Rus, and Radu Grosu. Gershgorin loss stabilizes the recurrent neural network compartment of an end-to-end robot learning scheme. 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 5446-5452. IEEE, 2020b.
Mathias Lechner, Ramin Hasani, Radu Grosu, Daniela Rus, and Thomas A Henzinger. Adversarial training is not ready for robot learning. arXiv preprint arXiv:2103.08187, 2021.
Joonho Lee, Jemin Hwangbo, Lorenz Wellhausen, Vladien Koltun, and Marco Hutter. Learning quadrupedal locomotion over challenging terrain. Science robotics, 5(47), 2020.
Lucas Liebenwein, Ramin Hasani, Alexander Amini, and Daniela Rus. Sparse flows: Pruning continuous-depth models. arXiv preprint arXiv:2106. 12718, 2021.
Ratnesh Madaan, Nicholas Gyde, Sai Vemprala, Matthew Brown, Keiko Nagami, Tim Taubner, Eric Cristofalo, Davide Scaramuzza, Mae Schwager, and Ashish Kapoor. Airsim drone racing lab. In NeurIPS 2019 Competition and Demonstration Track, pages 177-191. PMLR, 2020.
13

---

Stefano Massaroli, Michael Poli, Jinkyoo Park, Atsushi Yamashita, and Hajime Asma. Dissecting neural odes. In 34th Conference on Neural Information Processing Systems, NeurIPS 2020. The Neural Information Processing Systems, 2020.
Christoph Molnar.
Interpretable machine learning. Lulu. com, 2020.
Joris M Mooij, Dominik Janzing, and Bernhard Schölkopf. From ordinary differential equations to structural causal models: the deterministic case. In Proceedings of the Twenty-Ninh Conference on Uncertainty in Artificial Intelligence, pages 440-448, 2013.
Arsalan Mousavian, Alexander Toshev, Marek Fišer, Jana Košecká, Ayzaan Wahid, and James Davidson. Visual representations for semantic target driven navigation. In 2019 International Conference Robotics and Automation (ICRA), pages 8846-8852. IEEE, 2019.
Michael C Mozer, Denis Kazakov, and Robert Lindsey. Discrete event, continuous time mns. arXiv preprint arXiv:1710.04110 2017.
Olavi Nevanlinna. Remarks on picard-lindello iteration. BIT Numerical Muthematics, 29()):535-56,,
1989.
Andrew Y Ng and Stuart Russell. Algorithms for inverse reinforcement learning. In Proc. 17th International Conf. Machine Learning. Citeseer, 2000.
Uri Nodelman, Christian Shelton, and Daphne Koller. Learning continuous time bayesian networks. In Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence, pages 451-458, 2002.
Uri Nodelman, Christian R Shelton, and Daphne Koller. Learning continuous time bayesian networks. In Proceedings of the Nineteenth International Conference on Uncertainty in Artificial Intelligence, 2003.
Uri D Nodelman. Continuous time Bayesian nefworks. PhD thesis, Stanford University, 2007.
Seong Hyeon Park, Gyubok Lee, Jimin Seo, Manoj Bhat, Minseok Kang, Jonathan Francis, Ashwin Jadhav, Paul Pu Liang, and Louis-Philippe Morency. Diverse and admissible trajectory forecasting through multimodal context understanding. In European Conference on Computer Vision, pages 282-298. Springer, 2020.
Judea Pearl. Causality. Cambridge university press, 2009.
Judea Pearl. Probabilistic reasoning in intelligent systems: networks of plausible inference. Elsevier, 2014.
Judea Pearl et al. Causal inference in statistics: An overview. Statistics surveys, 3:96-146 2009. 2009
Will Penny, Zoubin Ghahramani, and Karl Friston. Bilinear dynamical systems. Philosophical Transactions of the Royal Society B: Biological Sciences, 360(1457):983 2005.
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: foundations and learning algorithms. The MIT Press, 2017.
Lev Semenovich Pontryagin. Mathematical theory of optimal processes. Routledge, 2018.
Alessio Quaglino, Marco Gallieri, Jonathan Masci, and Jan KoutnÅk. Snode: Spectral discretization of neural odes for system identification. In International Conference on Learning Representations, 2020.
Maithra Raghu, Ben Poole, Jon Kleinberg, Surya Ganguli, and Jascha Sohl-Dickstein. On the expressive power deep neural networks. In international conference on machine learning, pages 2847-2854, PMLR, 2017.
Nicholas Rhinehart, Rowan McAllister, and Sergey Levine. Deep imitative models for flexible inference, planning, and control. In International Conference Learning Representations, 2019.
14

---

Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and stafistics, pages 627-635. JMLR Workshop and Conference Proceedings, 2011.
Yulia Rubanova, Ricky T. Q. Chen, and David K Duvenaud. Latent ordinary differential equations for ireeularly-sampled time series, In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-But Buc, E. Fox, and R. Gareett, editors, Advances in Neural Information Processing Systems, volume 32, pages 332005330. Curran Associates, Inc., 2019a.
Yulia Rubanova, Ricky TQ Chen, and David Duvenaud. Latent odes for prrguularlyssamlle time series. arXiv preprint arXiv: 1907.v/907.0000 /000/097700977 2019b.
Paul K Rubenstein, Stephan Bongers, Bernhard Schölkopf, and Joris M Mooij. From deterministic odes to dynamic structural causal models. arXiv preprint 1eXiv:/608.000 08028, 2016.
F Ruggeri, F Faltin, and R Kenett. Bayesian networks. encyclopedia of statistics in quality and reliability, 2007.
2007.
David Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back-propagating errors. Nature, 323(6088):533-536, 1986.
Konstantin Rusch and Siddhartha Mishra. Coupled oscillatory recurrent neural network (co(
An accurate and (gradient) stable architecture for learning long time dependencies. In International Conference on Learning Representations, 2021. URL https:/// http://///oeeeeeeeeeeeeeeeoeooee F3s69X2W01a.
Stuart Russell and Peter Norvig. Artificial intelligence: a modern approach. Pearson, 2002.
Gopal P Sarma, Chee Wai Lee, Tom Portegys, Vahid Ghayoomie, Travis Jacobs, Bradly Alicea, Matteo Cantarelli, Michael Currie, Richard C Gerkin, Shane Gingell, et al. Openworm: overview and recent advances in integrative biological simulation of caenorhabditis elegans. Philosophical Trunsactions of the Royal Society B,
2018.
Axel Sauer, Nikolay Savinov, and Andreas Geiger. Conditional affordance learning for driving in urban environments. Conference on Robot Learning, pages 237-252, PMLR, 2018.
Nikolay Savinov, Alexey Dosovitskiy, and Vladlen Koltun. Semi paramettii topological memory for navigation. International Conference on Learning Representations, 2018
Stefan Schaal. Is imitation learning the route humanoid robots? Trends in cognitive sciences, 3(6):
233- 242, 1999.
Bernhard Schölkopf. Causality for machine learning. arXiv preprint arXi:: 2019.
Yannick Schroecker and Charles Isbell. State aware imitation learning. In Proceedings of the 31st
International Conference Neural Information Processing Systems, pages 2915-2924, 2017.
Peter Spirtes, Clark N Glymour, Richard Scheines, and David Heckerman. Causation, prediction, and search. MIT press, 2000.
Mingfei Sun and Xiaojuan Ma. Adversarial imitation learning from incomplete demonstrations. arXiv preprint 1rXiv:1905.722 12310, 2019.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 1998-6008, 2017.
Sebastian Weichwald, Martin E Jakobsen, Phillip Mogensen, Lasse Petersen, Nikolaj Thams, and Gherardo Varando. Causal structure learning from time series: Large regression coefficients may predict causal links better in practice than small p-values. In NeurIPS 2019 Competition and Demonstration Track, pages 27-36. PMLR, 2020.
15

---

Chuan Wen, Jierui Lin, Trevor Darrell, Dinesh Jayaraman, and Yang Gao. Fighting copycat agents in behavioral cloning from observation histories. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 2564-2573 Curran Associates, Inc., 2020. URL, http::///o/o nwurips. ccpperrr///////////p/e// 2020/file/ 1b1
www.f6777744fff- Paper pdf.
Yueh-Hua Wu, Nontawat Charoenphakdee, Han Bao, Voot Tangkaratt, and Masashi Sugiyama. Imita- tion learning from imperfect demonstration. In International Conference on Machine Learning. pages 6818-6827. PMLR, 2019.
Tianhe Yu, Pieter Abbeel, Sergey Levine, and Chelsea Finn. One-shot hierarchical imitation learning of compound visuomotor tasks. arXiv preprini orxiv: 1043, 2018.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material URL)? [Yes]
(b) Did you specify all the training details (c.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (c.g., with respect the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes]
4. If you using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any assets either in the supplemental material or URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you't 're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
16


Reviews:
[{"review_id": "fKyAK3HZj8o", "reviewer": "Reviewer_JrdD", "summary": "The manuscript studied a formulation of continuous-time neural networks and discussed its ability to learn causal structures in the environment. \n\nThe manuscript made claims on the following contributions:\n- Providing theoretical background and insights on how continuous-time neural networks learn causal structures\n- Experiments on visual drone navigation tasks\n- Analysis of robustness in closed-loop control settings", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Section 2, Re: Imitation, Visual Navigation: There is some missing discussion here, in the context of visuomotor control for navigation. The learning-to-drive community is actively attempting to address issues in imitation learning, such as causal confusion [10] and generalisability, through attempting to approximate informative priors [DATF, DIM, RIP], coupling policies with classical controllers [DIM, RIP], knowledge distillation [LBC], affordance learning [CAL], and others. It would be more appropriate for the present manuscript to contextualise its contributions with respect to these advances.\n\nL65, 256-257: The manuscript makes a general assertion that 'learning from offline demonstrations (for deployment to online settings)' (let's call this condition #1) is easier than 'learning from online observations and feedback (for deployment to online settings)' (condition #2) and, implicitly, 'learning from offline demonstrations, online observations, and online feedback (for deployment to online settings)' (condition #3). This is misleading and, relative to the justification provided in the manuscript, fundamentally imprecise; perhaps more realistic scenarios, such as those mentioned in my other comments should be considered by the manuscript. A major issue with i.i.d. imitation (besides limited behaviour/event mode coverage, from distribution shift and mismatched support) is that there is limited or non-existent observability over the environment and system dynamics that might result in particular outcomes. Furthermore, it is more difficult to resolve multi-agent causal relationships (e.g., traffic light causes pedestrian to walk, traffic light causes car to stop, pedestrian causes car to stop, car causes pedestrian to stop, etc.) in an imitative learning paradigm. In these settings, online optimisation (where characterisation of environment and/or system dynamics is available) can actually make for an easier objective. \n\nSection 4.1: The observation that continuous-time inductive biases provide models with robustness to causal confusion and distribution shift is not new, nor are couplings between structured physical models and neural systems. Both have been incorporated in other approaches for visual navigation and trajectory forecasting. See [DATF] (re: Verlet Integration), [DIM], and [RIP].\n\nL184, 185: \"Results\", \"results\" --> \"Approach\", \"approach\"\n\nSection 5: While additional tasks are always nice, I would like to see comparison of the work presented in this manuscript with other approaches on more well-established benchmarks. This is more than just a request to perform more experiments: the precise problems that this manuscript is attempting to alleviate (causal confusion, generalisability) are exactly the challenges that such tasks as CARLA NoCrash, CARNOVEL, and CARLA Challenge are designed to provide assessments for. Moreover, there already exist several contemporary works that have used these benchmarks to study these problems, thereby eliminating the need to only use simple RNN-class baselines.\n\nL42: Missing word: \"(i.i.d) (see Fig. 2)\" --> \"(i.i.d) observations (see Fig. 2)\"\n\n--------\n\nReferences\n\n[DATF] Park, Seong Hyeon, Gyubok Lee, Jimin Seo, Manoj Bhat, Minseok Kang, Jonathan Francis, Ashwin Jadhav, Paul Pu Liang, and Louis-Philippe Morency. \"Diverse and admissible trajectory forecasting through multimodal context understanding.\" In European Conference on Computer Vision, pp. 282-298. Springer, Cham, 2020.\n\n[DIM] Rhinehart, Nicholas, Rowan McAllister, and Sergey Levine. \"Deep Imitative Models for Flexible Inference, Planning, and Control.\" In International Conference on Learning Representations. 2019.\n\n[RIP] Filos, Angelos, Panagiotis Tigkas, Rowan McAllister, Nicholas Rhinehart, Sergey Levine, and Yarin Gal. \"Can autonomous vehicles identify, recover from, and adapt to distribution shifts?.\" In International Conference on Machine Learning, pp. 3145-3153. PMLR, 2020.\n\n[LBC] Chen, Dian, Brady Zhou, Vladlen Koltun, and Philipp Krähenbühl. \"Learning by cheating.\" In Conference on Robot Learning, pp. 66-75. PMLR, 2020.\n\n[CAL] Sauer, Axel, Nikolay Savinov, and Andreas Geiger. \"Conditional affordance learning for driving in urban environments.\" In Conference on Robot Learning, pp. 237-252. PMLR, 2018.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "Ib3Tv77qMBU", "reviewer": "Reviewer_G1FC", "summary": "This submission proposes a theoretical and experimental framework for learning causal representations using continuous-time neural networks. The proposed method is evaluated on a visual drone navigation tasks of chasing static and dynamic objects in the AirSim simulation. Experiments show that the proposed method outperforms LSTMs, recurrent neural network version of Neural ODEs and continuous-time GRUs.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n- The paper is well motivated. Learning causal relationships is an important machine learning problem. Learning visual navigation policies is a good testbed for this problem.\n\n- The paper is written very well. It is easy to follow and the method is described well.\n\n- The method is novel to the best of my knowledge. It leverages continuous-time neural networks to learn casual relationships using a class of liquid time-constant networks. \n\n- The experiments is well designed. The authors test the method on a series of visual drone navigation tasks with increasing difficulty and implement a good variety of baselines.\n\n- The experiments show the effectiveness of the method empirically. The attention visualization is also informative. Supplementary material provides good video visualizations\n\n- The supplementary material also provides code for reproducibility.\n\n\nWeaknesses:\n\n- The proposed method performs worse than the baselines for a few settings. The submission will benefit from some experiments for a different task in a different simulator to establish the consistency of results.\n\n- It would be good to add map-based methods mentioned in the visual navigation paragraph in the related work as baselines. Some of these methods such as CMP or Neural SLAM are shown to perform much better than LSTM policies.\n\n- All the methods perform poorly on the Hiking task. It would be good to discuss the limitations of NCP and author's thoughts on what needs to be improved to tackle complex tasks such as Hiking.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "IHd2y1uwrB", "reviewer": "Reviewer_cZAE", "summary": "This paper explores the capabilities of continuous-time networks to form casualty structures, as well as their applications to visual drone navigation. The authors find that models based on the liquid time-constant networks (LTCs) uniquely provide causal structures. The theoretical discoveries are then verified in closed-loop visual navigation experiments. LTC-based policies significantly outperform regular continuous time models and discrete RNNs, all trained using imitation learning.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "=== Strengths ===\n\n+ This paper provides theoretical analysis on whether various continuous-time networks can form causal structures. The authors show that LTCs are causal models whereas other continuous models such as regular neural ODEs are not.\n\n+ LTC-based policies significantly outperform baselines in closed-loop experiments. This shows great potential of LTC-based models being a generic policy architecture in robotics or RL.\n\n+ I appreciate the saliency analysis across models as it futhers shows the causal structure captured by the training data\n\n=== Weaknesses ===\n\n- Experiments in table 4 shows LTC’s effectiveness as it outperforms LSTM as a discrete RNN baseline, and ODE-RNN as a CT baseline. I am curious to see a vanilla CNN baseline and how it performs in the same tasks, as prior work has shown regular RNNs can suffer from the causal confusion problem in the imitation learning setting [1,2].\n\n=== Other comments ===\n\n- LTC (or NCP) as a generic policy architecture provides causal structures as the authors show in the imitation learning experiments. I am curious how it would perform against other architectures in the RL setting.`\n\n\\\n[1] Casual Confusion in Imitation Learning, de Haan et al., 2019\n\n[2] Fighting Copycat Agents in Behavioral Cloning from Observation Histories, Wen et al., 2020\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Discussed in the paper.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces a method for visual navigation using continuous time (CT) neural networks in the continuous liquid time neural networks (LTC). It claims, that this variant is able to learn causal structures, compared to baseline CT models as well as standard deep models like LSTMs, and evaluates these claims on drone navigation tasks.

The paper received 3 expert reviews, which were on the fence. After the discussion phase, two of the reviewers were clearly positive of the paper, and the remaining reviewers was borderline slightly negative.

Several weaknesses were pointed out by the critical reviewer:
- the limitations of positioning as imitation learning
- novelty of the claim that continuous-time formulations increase robustness wrt to causal confusion 
- Simplicity of the tasks

The authors could provide answers to most of these issues, but the remaining weakness was the simplicity of the experiments.

The AC's reading of the paper was positive. He agrees on the simplicity of the tasks, but judges that this paper has sufficient merits and novelty to be of interest to the community.

The paper was then discussed between AC and SAC, who confirms the decision.
Decision: Accept (Poster)