Venue: CoRL 2021 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: Full disclosure; I was made aware of this paper by [name removed] tweeting about it in April: [link removed] and I’m also aware that this work was presented at an ICLR workshop this year: [link removed]. So at best, this is a single-blind review.

The paper presents a novel method for goal-driven visual outdoor navigation. The method is pretrained on a large dataset to propose goals for navigation and the online algorithm creates a graph of the environment and learns compressed visual embeddings that are robust to weather, light conditions, and distractor objects.

The work is overall very impressive and I respect the sheer amount of field experiments that were carried out for this work. However, I think that the work currently suffers from some clarity issues to the point of being impossible to reproduce without contacting the authors.


strengths_and_weaknesses: ### Strengths

- The content that made it into the paper is well-written and well-presented. The text itself is clear, the Figures are illustrative, the pseudocode blocks are mostly great, and the experimental section is excellent.
- As mentioned above, the experimental time commitment in this work is staggering. The authors mentioned over 100 hours of real-world experiments and there was probably also robot maintenance and intermittent tuning of the method, so I have to commend the authors for that sheer effort!
- I really appreciated that the authors clearly stated their research questions at the beginning of the experiments section.
- The choice of recent baselines and amount thereof is great.

### Weaknesses

- Practicality: The authors mentioned that this was only possible by having a large offline dataset of the robot and also all the data is collected on the same robot platform. That raises many questions that weren't addressed in the paper:
  1. You emphasized that the training and testing environments weren't the same. But how disjoint/out-of-distribution is the data? What's the closest distance between any possible sample from the train and test dataset. I feel like if they were recorded in the same neighborhoods, the test environment is very much in-distribution and some of the claims of generality need to be toned down a bit.
  2. I'm unclear what the importance of the perspective is in this method - can the data be recorded from a different perspective (height above ground) or with a different visual sensor? What about a different FOV?
  3. Part of your motivation is that this method uses "only" real-world data and doesn't need to be pretrained in simulation (see related works section line 66 and following) but also your "...method enables the robot to explore an environment from scratch in just 20 minutes, using prior experience from other environments". What about pretraining your method in simulation? Have you looked into that? Would any of this work if the dataset was simulated? Because I feel like a large dataset for pretraining would be easier to come by if it were simulated.
  4. You also mention in the main body of the paper that you're making the dataset publicly available (in the appendix?) (line 109) but I see no evidence of that. The dataset could have been uploaded to some anonymous google drive folder or cloud storage to be given to the reviewers for anonymous review. 
- Reproducibility: The present text raises many questions about how to reimplement this:
	5. You mention in the main body of the paper that the data is collected over 18 months through random walk but in the appendix, you further specify that the data is not random but obtained from a heuristic that resets after collision and also something about bumpiness and self-labeling. The reader is left utterly lacking any understanding of this dataset. (a) How many data points are there? (b) What is the distribution of the data, i.e. are there any over- or underexplored areas, what is the distribution of "bumpinesses" and collisions in your data? (c) did you preprocess the data in any way? (d) Did you filter the data to exclude certain episodes or events?
	6. How exactly is the pretraining done? (a) I.e. what parts are pretrained and for how long? (b) How is the prior $r(\cdot)$ calculated?
	7. In Algorithm 1, there are a few unclear things - some of which are explained in the appendix. (a) the function LeastExploredNeighbor is taking a hyperparameter $\delta_2$ and the purpose of this HP is unclear. This is explained in the appendix but from my best understanding, the appendix is there to provide additional examples, and additional clarification but the main body of the paper should be able to be clear and self-contained. (b) Same for the function SubgoalNavigate and the hyperparameter $H$. (c) What is $o_n$ in line 11? (d) If $r(\cdot)$ is the prior over goals that returns the value of a normal distribution (i.e. a scalar, line 128), then how is a goal sampled from that in line 9? 


summary_of_recommendation: I think this paper has a lot of potential and I would like to see this accepted but to recommend acceptance, I first need some crucial details clarified (see weaknesses section above).
issues: See section "Weaknesses" above. I numbered all my issues with this paper so the authors can respond in turn. Some additional questions, mostly out of interest:

8. What's the formula for SCT in table 1?
9. You mentioned moving objects (people) but I didn't see any in your Figures and video. How does your model work when there's foot traffic, qualitatively?
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper addresses the problem of finding a goal image specified by a user in a possibly unknown environment using a single ground mobile robot. The paper proposes a method that uses learned goal-conditioned distance model with a latent variable model representing visual goals, trained on a variety of environments, to explore the environment, and uses a topological graph to navigate towards the found goal. The paper shows experimental results with a real ground robot in known and unknown environments to validate the proposed method.
strengths_and_weaknesses: The paper presented an interesting learning system which improves the state of the art in image-goal navigation by handling new environments and at the same time using prior experience in other environments. The paper is generally well written, with provided motivations behind the choices made. The paper is technically sound with a rather complete experimental evaluation with a real robot that includes comparisons with other 6 methods in different settings and an ablation study. It is quite impressive the dataset that was collected over a span of 18 months.

The main comments about the paper revolve around the actual practicality of the proposed method and the lack of comparison with alternative non-learning-based approaches. Some of the questions include: in outdoor, what are some of the actual uses of the proposed method which does not use any other sensor other than camera, as GPS is typically reliable, especially considering the types of environments considered in the paper? While different environments have been considered, the environments share similar characteristics, i.e., some human-made structures with large passages; how does the method generalize to totally different environments, e.g., in forest? What happens when the goal is much further away than 80m? What is the motivation to use a pure learning-based approach for the full pipeline, when for example visual SLAM could be used to create a map; the robot could follow a coverage pattern; and eventually find the goal avoiding to have paths that look suboptimal? Discussions on these aspects would improve the quality and the impact of the paper.

A minor comment regards the dataset used, which could be better described in terms of how the 20 hours of offline data.

summary_of_recommendation: While the paper could discuss the practicality and compare with more traditional exploration systems, the paper presents an interesting method for image-goal navigation in unseen environments with a rather complete evaluation that includes a real robot.
issues: The revised paper could discuss the questions raised in the "Strengths and Weaknesses" box.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: The paper proposes an approach to learn a latent representation of relative goals to be able to produce actions and distance estimates from a current image and a goal image. This model, trained on offline data, is used to build a topological map of a new environment, explore rapidly this new environment, and later navigate in this environment. Performances evaluations and comparisons with a large set of baselines are provided on a real robot in several outdoors environments, showing improved exploration and navigation speed. 

strengths_and_weaknesses: The paper is very well written, the motivation and relation to the state of the art are clear, the technical description is sufficient in the paper and well complemented in the appendix and supplementary video. At a high level, the proposed approach illustrates the well-known fact that goal-based exploration is much more efficient than random action sampling, but its implementation as a vision-based latent goal model seems an interesting contribution and its evaluation on real robots in challenging outdoor environments is quite impressive.

I don't see any major problem with the paper, I have only a few questions that could be clarified :
- What is the reward function used in the PPO baseline? Only sparse at the goal or using a more informative reward?
- Could you illustrate the interest of fine-tuning the model? Is it important in the performances? And does the model specialize in the environment or integrates knowledge and remains valid for other environments (i.e. no forgetting)?
- Line 195: you mention a relabeling scheme that is not described in sec 3.2 nor in the appendix.
summary_of_recommendation: The paper proposes an interesting new approach, with a solid experimental evaluation.
issues: See the minor weaknesses mentioned above.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: This paper presents a robot learning system for goal-directed exploration in novel environments where the robot is tasked to navigate to a goal region in the environment while provided with a goal image taken at the goal region. The system, called Rapid Exploration Controllers for Outcome-driven Navigation (RECON) consists of two parts: An autoencoder that serves as the policy and distance estimator, given the goal image and the current image of the robot, and a topological map that is constructed online to guide the robot towards exploring the environment efficiently. The system is evaluated and compared to 6 baseline methods on a set of previously unseen real-world scenarios.
strengths_and_weaknesses: Overall, the method is interesting, albeit somewhat incremental, and the motivations behind the two parts of the system (i.e. the autoencoder and the topological map) are clear. The paper does a good job in specifying the problem it attempts to address. A big plus of the system is the fact that it doesn't require any geometric map of the environment, which is often unavailable or hard to obtain for complex outdoor scenarios. The experimental results indicate that the system is more efficient in exploring previously unseen environments and reaching a goal within the environment compared to the baseline methods. I also appreciate the fact that the system was tested on a physical robot, which indicates that the system is effective in real-world scenarios.

The main problem I have with the paper is that some concepts and ideas are not very well explained which makes the paper hard to follow at times, particularly in Section 4.2: 

First, the graph (i.e. the "topological memory") and what it represents needs to be explained, idealy as early as Section 3 instead of handwavy introducing it in Section 4.2. Additionally, I was initially under the impression that the nodes in the graph represent states and the edges represent their connectivity, but after examining the algorithms, it seems that the nodes represent images and the edges represent distances. This needs to be mentioned.

Second, in Section 4.2 (i) (Feasible Goal), it is not clear to me how a large probability of the goal embedding z_g^t under the prior implies that the model believes it can reach the goal?

Third, in Section Section 4.2 (ii) the paper mentions that the robot determines if it is at the frontier of the graph by checking it's distance to the least explored node within a distance threshold. How does this work? Line 8 in Algorithm 1 seems to suggest that the mean of the decoder is used to measure the distance between o_t (the current observation) and the observation associated to a node (I suppose this is what o_n denotes in Algorithm 1?). However, I'm just guessing here because this is not explained in the paper. 

Fourth, how and when exactly is the graph constructed? Appendix B1 mentions that at the end of each trajectory, the subroutine ExpandGraph is called to do this. So I guess this routine should be called in the for loop in Algorithm 3?

Additionally, the paper should mention the structure of the trajectory dataset that was used to train the model in Section 4.1. Equation 2 reveals that a datapoint consists of a observation-goal observation-action-distance quadruple, but this is nowhere specified in the main text. 

UPDATE: 
Based on the author's feedback and the revisions that were carried out, I'm changing my recommendation from "Weak Accept" to "Strong Accept".
summary_of_recommendation: The paper proposes a system for for visual-guided goal discovery that seems to work well in practice. However, some revisions are neccessary to improve the clarity of the paper.
issues: The points above regarding unclear explanations and missing notations need to be addressed.
review_title: null

Meta-review: In this paper, a robotic learning system for goal-oriented exploration in a novel environment is proposed. A compact representation of goal images is learned and a topological memory is constructed online incrementally during the exploration. All reviewers recognize the field experiments conducted. The concerns are mostly focused on the practicality of the proposed system and some more detailed explanation are lacking. 

In the rebuttal session, the authors have responded to the reviewers' concerns and more details are provided. And the reviewers have reached a consensus of accepting this paper. 
Decision: Accept (Oral)