Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: The paper proposes an interesting, and to the best of my knowledge novel, pipeline for learning a semantic map of the environment with respect to navigability, and simultaneously uses it for further exploring the environment.

The pipeline can be summarized as follows: Navigate somewhere using some heuristic. When navigation "works", as well as when encountering something "negative", back-project that into past frames, and label the corresponding pixels as such: either positive or negative. This generates a collection of partially densely labelled images, on which a segmentation network can be learned that learns which part of the RGBD input are navigable and which should be avoided. For navigation, navigability of the current frame is predicted, and that prediction is down-projected into an "affordance map" that is used for navigation. One experiment confirms the usefulness of such an affordance map.


I am marking weak reject currently because of the following concerns, which might be me just missing something. On the one hand, I am glad to see something that is not just blind "end to end RL with exploration bonus", sounds reasonable, and works well. On the other hand, I do have several major concerns about the method, outlined as follows:

1. How can this approach work for moving obstacles? Let's say a monster walks from point A to point B, and collides with the agent at point B. Then, point B is marked as a hazard, but in the previous frames, the monster is not located at point B, and thus an image region that does not contain the monster is marked as hazard. Am I missing something here?
2. The method does not seem practical for actual mobile robots, only for in-game or in-simulation agents. The reason being that in order to learn "robot should not bump into baby", the robot actually needs to bump into multiple babies in order to collect data about that hazard. To be fair, blind "PPO+exploration bonus" suffers from the same problem, but in this paper, the whole motivation is about mobile robots (at least that was my impression after reading it).

Furthermore, I do not think I would be able to reproduce any of the experiments, as many details are missing. Will code be released?


###### Post-rebuttal update

I am happy with the author's response to my concerns, and they have included corresponding discussions in their paper. Thus, I am improving my rating to recommend acceptance of this paper to ICRL2020.
review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: This paper presents an approach for navigating and exploring in environments with dynamic and environmental hazards that combines geometric and semantic affordance information in a map used for path planning.

Overall this paper is fairly well written.  Results in a VizDoom testbed show favorable performance compared to both frontier and RL baselines, and the author's approach is more sample-efﬁcient and generalizable than RL-based approaches.

I wouldn't consider any particular aspect of this paper to be that novel, but it is a nice combination of leveraging active self-supervised learning to generate spatial affordance information for fusion with a geometric planner.

As humans show the best performance on the tasks, it might be worth considering learning a policy from human demonstrations through an imitation learning approach.
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: The paper proposes to learn affordance maps: a method to judge whether a certain location is accessible. This is done by distilling a series of "trial and error" runs and the relation of a pixel in the image/depth plane to a corrdinate into a model.

I like the idea and think the paper should be accepted. The idea to use trial and error (something I prefer to self-supervision, which is used differently in many contexts, I believe) to obtain a data set for learning a model is nice and very practical.

Some concerns that I think should be adressed.

- The term information gain is used wrongly. The entropy of class labels is not infogain. Infogain is the expected KL of the model posterior from the model prior. Please correct this. See [1, 2].
- *Learning* a model of the environment and using it for navigation/exploratin has also been tackled recently by [1]. I think the authors should draw connetions to that work.
- Self-supervision has recently been proposed by Lecun as a subsitute (of sorts) for unsupervised learning. What he means is that a part of the data is used to predict another part of the data. I have no hard feelings about the term, personally preferring unsupervised, but the authors should be aware of the name clash.

I wonder how the authors envision to extend this method to real scenarios. The "trial and error" method is clearly not viable for robotics setups, as hazards are costly. It would be nice if the authors could give there perspective on things.

[1] Depeweg et al, "Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning", Proceedings of the 35th International Conference on Machine Learning
[2] Mirchev et al, "Approximate Bayesian Inference in Spatial Environments" in proceedings of Robotics: Science and Systems XV.
review_title: Official Blind Review #1

Meta-review: This paper presents a framework for navigation that leverages learning spatial affordance maps (ie what parts of a scene are navigable) via a self-supervision approach in order to deal with environments with dynamics and hazards. They evaluate on procedurally generated VizDoom levels and find improvements over frontier and RL baseline agents.

Reviewers all agreed on the quality of the paper and strength of the results. Authors were highly responsive to constructive criticism and the engagement/discussion appears to have improved the paper overall. After seeing the rebuttal and revisions, I believe this paper will be a useful contribution to the field and I’m happy to recommend accept.

Decision: Accept (Poster)