Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This submission proposes a theoretical and experimental framework for learning causal representations using continuous-time neural networks. The proposed method is evaluated on a visual drone navigation tasks of chasing static and dynamic objects in the AirSim simulation. Experiments show that the proposed method outperforms LSTMs, recurrent neural network version of Neural ODEs and continuous-time GRUs.

main_review: Strengths:

- The paper is well motivated. Learning causal relationships is an important machine learning problem. Learning visual navigation policies is a good testbed for this problem.

- The paper is written very well. It is easy to follow and the method is described well.

- The method is novel to the best of my knowledge. It leverages continuous-time neural networks to learn casual relationships using a class of liquid time-constant networks. 

- The experiments is well designed. The authors test the method on a series of visual drone navigation tasks with increasing difficulty and implement a good variety of baselines.

- The experiments show the effectiveness of the method empirically. The attention visualization is also informative. Supplementary material provides good video visualizations

- The supplementary material also provides code for reproducibility.


Weaknesses:

- The proposed method performs worse than the baselines for a few settings. The submission will benefit from some experiments for a different task in a different simulator to establish the consistency of results.

- It would be good to add map-based methods mentioned in the visual navigation paragraph in the related work as baselines. Some of these methods such as CMP or Neural SLAM are shown to perform much better than LSTM policies.

- All the methods perform poorly on the Hiking task. It would be good to discuss the limitations of NCP and author's thoughts on what needs to be improved to tackle complex tasks such as Hiking.
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The manuscript studied a formulation of continuous-time neural networks and discussed its ability to learn causal structures in the environment. 

The manuscript made claims on the following contributions:
- Providing theoretical background and insights on how continuous-time neural networks learn causal structures
- Experiments on visual drone navigation tasks
- Analysis of robustness in closed-loop control settings
main_review: Section 2, Re: Imitation, Visual Navigation: There is some missing discussion here, in the context of visuomotor control for navigation. The learning-to-drive community is actively attempting to address issues in imitation learning, such as causal confusion [10] and generalisability, through attempting to approximate informative priors [DATF, DIM, RIP], coupling policies with classical controllers [DIM, RIP], knowledge distillation [LBC], affordance learning [CAL], and others. It would be more appropriate for the present manuscript to contextualise its contributions with respect to these advances.

L65, 256-257: The manuscript makes a general assertion that 'learning from offline demonstrations (for deployment to online settings)' (let's call this condition #1) is easier than 'learning from online observations and feedback (for deployment to online settings)' (condition #2) and, implicitly, 'learning from offline demonstrations, online observations, and online feedback (for deployment to online settings)' (condition #3). This is misleading and, relative to the justification provided in the manuscript, fundamentally imprecise; perhaps more realistic scenarios, such as those mentioned in my other comments should be considered by the manuscript. A major issue with i.i.d. imitation (besides limited behaviour/event mode coverage, from distribution shift and mismatched support) is that there is limited or non-existent observability over the environment and system dynamics that might result in particular outcomes. Furthermore, it is more difficult to resolve multi-agent causal relationships (e.g., traffic light causes pedestrian to walk, traffic light causes car to stop, pedestrian causes car to stop, car causes pedestrian to stop, etc.) in an imitative learning paradigm. In these settings, online optimisation (where characterisation of environment and/or system dynamics is available) can actually make for an easier objective. 

Section 4.1: The observation that continuous-time inductive biases provide models with robustness to causal confusion and distribution shift is not new, nor are couplings between structured physical models and neural systems. Both have been incorporated in other approaches for visual navigation and trajectory forecasting. See [DATF] (re: Verlet Integration), [DIM], and [RIP].

L184, 185: "Results", "results" --> "Approach", "approach"

Section 5: While additional tasks are always nice, I would like to see comparison of the work presented in this manuscript with other approaches on more well-established benchmarks. This is more than just a request to perform more experiments: the precise problems that this manuscript is attempting to alleviate (causal confusion, generalisability) are exactly the challenges that such tasks as CARLA NoCrash, CARNOVEL, and CARLA Challenge are designed to provide assessments for. Moreover, there already exist several contemporary works that have used these benchmarks to study these problems, thereby eliminating the need to only use simple RNN-class baselines.

L42: Missing word: "(i.i.d) (see Fig. 2)" --> "(i.i.d) observations (see Fig. 2)"

--------

References

[DATF] Park, Seong Hyeon, Gyubok Lee, Jimin Seo, Manoj Bhat, Minseok Kang, Jonathan Francis, Ashwin Jadhav, Paul Pu Liang, and Louis-Philippe Morency. "Diverse and admissible trajectory forecasting through multimodal context understanding." In European Conference on Computer Vision, pp. 282-298. Springer, Cham, 2020.

[DIM] Rhinehart, Nicholas, Rowan McAllister, and Sergey Levine. "Deep Imitative Models for Flexible Inference, Planning, and Control." In International Conference on Learning Representations. 2019.

[RIP] Filos, Angelos, Panagiotis Tigkas, Rowan McAllister, Nicholas Rhinehart, Sergey Levine, and Yarin Gal. "Can autonomous vehicles identify, recover from, and adapt to distribution shifts?." In International Conference on Machine Learning, pp. 3145-3153. PMLR, 2020.

[LBC] Chen, Dian, Brady Zhou, Vladlen Koltun, and Philipp Krähenbühl. "Learning by cheating." In Conference on Robot Learning, pp. 66-75. PMLR, 2020.

[CAL] Sauer, Axel, Nikolay Savinov, and Andreas Geiger. "Conditional affordance learning for driving in urban environments." In Conference on Robot Learning, pp. 237-252. PMLR, 2018.
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper explores the capabilities of continuous-time networks to form casualty structures, as well as their applications to visual drone navigation. The authors find that models based on the liquid time-constant networks (LTCs) uniquely provide causal structures. The theoretical discoveries are then verified in closed-loop visual navigation experiments. LTC-based policies significantly outperform regular continuous time models and discrete RNNs, all trained using imitation learning.

main_review: === Strengths ===

+ This paper provides theoretical analysis on whether various continuous-time networks can form causal structures. The authors show that LTCs are causal models whereas other continuous models such as regular neural ODEs are not.

+ LTC-based policies significantly outperform baselines in closed-loop experiments. This shows great potential of LTC-based models being a generic policy architecture in robotics or RL.

+ I appreciate the saliency analysis across models as it futhers shows the causal structure captured by the training data

=== Weaknesses ===

- Experiments in table 4 shows LTC’s effectiveness as it outperforms LSTM as a discrete RNN baseline, and ODE-RNN as a CT baseline. I am curious to see a vanilla CNN baseline and how it performs in the same tasks, as prior work has shown regular RNNs can suffer from the causal confusion problem in the imitation learning setting [1,2].

=== Other comments ===

- LTC (or NCP) as a generic policy architecture provides causal structures as the authors show in the imitation learning experiments. I am curious how it would perform against other architectures in the RL setting.`

\
[1] Casual Confusion in Imitation Learning, de Haan et al., 2019

[2] Fighting Copycat Agents in Behavioral Cloning from Observation Histories, Wen et al., 2020

limitations_and_societal_impact: Discussed in the paper.
review_title: null

Meta-review: This paper introduces a method for visual navigation using continuous time (CT) neural networks in the continuous liquid time neural networks (LTC). It claims, that this variant is able to learn causal structures, compared to baseline CT models as well as standard deep models like LSTMs, and evaluates these claims on drone navigation tasks.

The paper received 3 expert reviews, which were on the fence. After the discussion phase, two of the reviewers were clearly positive of the paper, and the remaining reviewers was borderline slightly negative.

Several weaknesses were pointed out by the critical reviewer:
- the limitations of positioning as imitation learning
- novelty of the claim that continuous-time formulations increase robustness wrt to causal confusion 
- Simplicity of the tasks

The authors could provide answers to most of these issues, but the remaining weakness was the simplicity of the experiments.

The AC's reading of the paper was positive. He agrees on the simplicity of the tasks, but judges that this paper has sufficient merits and novelty to be of interest to the community.

The paper was then discussed between AC and SAC, who confirms the decision.
Decision: Accept (Poster)