Paper:
The ability to autonomously explore and navigate a physical space is a fundamental requirement for virtually any mobile autonomous agent, from household robotic vacuums to autonomous vehicles. Traditional SLAM-based approaches for exploration and navigation largely focus on leveraging scene geometry, but fail to model dynamic objects (such as other agents) or semantic constraints (such as wet floors or doorways). Learning-based RL agents are an attractive alternative because they can incorporate both semantic and geometric information, but are notoriously sample inefficient, difficult to generalize to novel settings, and are difficult to interpret. In this paper, we combine the best of both worlds with a modular approach that learns a spatial representation of a scene that is trained to be effective when coupled with traditional geometric planners. Specifically, we design an agent that learns to predict a spatial affordance map that elucidates what parts of a scene are navigable through active self-supervised experience gathering. In contrast to most simulation environments that assume a static world, we evaluate our approach in the VizDoom simulator, using large-scale randomly-generated maps containing a variety of dynamic actors and hazards. We show that learned affordance maps can be used to augment traditional approaches for both exploration and navigation, providing significant improvements in performance.

Review:
The paper proposes an interesting, and to the best of my knowledge novel, pipeline for learning a semantic map of the environment with respect to navigability, and simultaneously uses it for further exploring the environment.
The pipeline can be summarized as follows: Navigate somewhere using some heuristic. When navigation "works", as well as when encountering something "negative", back-project that into past frames, and label the corresponding pixels as such: either positive or negative. This generates a collection of partially densely labelled images, on which a segmentation network can be learned that learns which part of the RGBD input are navigable and which should be avoided. For navigation, navigability of the current frame is predicted, and that prediction is down-projected into an "affordance map" that is used for navigation. One experiment confirms the usefulness of such an affordance map.
I am marking weak reject currently because of the following concerns, which might be me just missing something. On the one hand, I am glad to see something that is not just blind "end to end RL with exploration bonus", sounds reasonable, and works well. On the other hand, I do have several major concerns about the method, outlined as follows:
1. How can this approach work for moving obstacles? Let's say a monster walks from point A to point B, and collides with the agent at point B. Then, point B is marked as a hazard, but in the previous frames, the monster is not located at point B, and thus an image region that does not contain the monster is marked as hazard. Am I missing something here?
2. The method does not seem practical for actual mobile robots, only for in-game or in-simulation agents. The reason being that in order to learn "robot should not bump into baby", the robot actually needs to bump into multiple babies in order to collect data about that hazard. To be fair, blind "PPO+exploration bonus" suffers from the same problem, but in this paper, the whole motivation is about mobile robots (at least that was my impression after reading it).
Furthermore, I do not think I would be able to reproduce any of the experiments, as many details are missing. Will code be released?
###### Post-rebuttal update
I am happy with the author's response to my concerns, and they have included corresponding discussions in their paper. Thus, I am improving my rating to recommend acceptance of this paper to ICRL2020.