Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper focuses on the problem of last-mile navigation on image-goal navigation, which aims at reaching the goal after discovering it. It is also the key problem of many visual navigation methods but have barely been researched. The proposed method SLING is proved to raise the SOTA on success rate. Physical experiments are carried to prove the effectiveness on real environment.
strengths_and_weaknesses: Strengths:
1)	The problem of last-mile navigation have barely been researched, despite it is proved to be the reason of many failures.
2)	This paper proposes a simple and interpretable module that can be used with many baselines. The experimental results are rich and impressive.
3)	Physical experiments are carried to demonstrate the effectiveness of transferring to real environment.

Weaknesses:
1)	There is no experiment that compares or combine SLING with VGM [1] which is the SOTA ImageNav method.
2)	It would be better to discuss some failure cases caused by adding SLING as a plug-in module, so it is more clear how SLING affects the baselines it is added to.

References:
[1] O. Kwon, N. Kim, Y. Choi, H. Yoo, J. Park, and S. Oh. Visual graph memory with unsupervised representation for visual navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15890–15899, 2021.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: The paper raises an interesting problem for deeper research, and proposes an effective method to tackle the problem with adequate experiments.
issues: Add experiments on VGM.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper tackles the problem of last-mile embodied visual navigation, in which the agent has to navigate to the goal after the goal image is visible in the current view. Once the image is visible, the correspondences between the current and the goal image is used to recover the relative rotation and translation of the two images. This, combined with the depth, is used to estimate the heading and distance to the goal, which is then used with a local metric map to avoid obstacles to head to the given goal. The paper also proposes ways to switch between goal discovery and last mile navigation phases, enabling the proposed method to easily be combined with existing goal discovery methods. The method is evaluated thoroughly on Gibson and MP3D datasets with numerous state of the art baselines establishing that the proposed method performs much better than the baselines and gets consistent improvement. 
strengths_and_weaknesses: 1. The paper is well motivated. It picks the task of last mile navigation which empirically causes a non-trivial loss in performance in the proposed end-to-end navigation methods. 
2. The paper is well written, has illustrative figures, and is easy to follow.
3. The experiments and ablations are quite thoroughly done on both Gibson, MP3D and real-world scenes. The baselines look relevant and include state-of-the-art methods including end-to-end RL, Imitation Learning, and Neural Modular Policies. The improvements convincingly show that the proposed method SLING adds non-trivial gains when combined with the existing goal discovery methods.  
4. A limitation of the proposed method is that it has several error-prone steps and hyperparameters which make it very complicated to implement and deploy. It would be good to simplify some of the steps by subsuming them in a learned policy, as discussed below in the suggested baseline.
5. Another limitation of the paper is that the paper compares with no alternative baseline methods for last mile navigation itself. One baseline which would be good to have is to learn a policy to navigate to the goal when initialized with the last mile navigation task, i.e., goal in sight. 
6. Another baseline that would be instructive to have is Neural Topological SLAM [3], which also seems to use the geometric information which should implicitly lead to good last mile navigation performance. 
7. In L273-274, the authors say that the gains become sharper for the hard tasks. For completeness, could the authors describe what is the difference in these tasks compared to the easier benchmark in the real world? If the task just entails more distant goals, then I would assume that this is the job of goal discovery, and not last mile navigation, and hence would require an explanation on why the performance of the proposed method is significantly better in this regime.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The paper focuses on the problem of last-mile navigation and gives a compelling empirical case to show that the proposed method SLING can lead to consistent improvement across the board when combined with existing goal discovery methods. I'm leaning towards an acceptance because the paper has strong empirical evidence to support the importance of separately solving the last mile navigation task, and then combining it with existing goal discovery methods. 
issues: From the comments above:

1. Lacks an alternative baseline method comparison for last mile navigation itself. One baseline which would be good to have is to learn a policy to navigate to the goal when initialized with the last mile navigation task, i.e., goal in sight. 
2. Another baseline that would be instructive to have is Neural Topological SLAM [3], which also seems to use the geometric information which should implicitly lead to good last mile navigation performance. 
3. In L273-274, the authors say that the gains become sharper for the hard tasks. For completeness, could the authors describe what is the difference in these tasks compared to the easier benchmark in the real world? If the task just entails more distant goals, then I would assume that this is the job of goal discovery, and not last mile navigation, and hence would require an explanation on why the performance of the proposed method is significantly better in this regime.
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper addresses the challenge of last-mile goal-reaching in visual navigation tasks, i.e. the last stage of navigating to an image-goal in an unknown environment, when the goal is in the line-of-sight of the agent. The authors propose a simple controller that extracts visual keypoints in the agent’s observations and establishes correspondence to the goal image — converting the last-mile image-goal problem into a point-goal problem. Extensive quantitative analysis suggests that this indeed improves performance on benchmark tasks, and the learned policies generalize to real-world indoor environments.

strengths_and_weaknesses: Strengths
---
- The paper is very well-written and the observation of converting the last mile to image-goal navigation tasks to a point-goal navigation task, albeit simple, seems well motivated. I also found the method explanation to be easy to follow.
- The authors do a good job at statistically significant evaluations (both in sim and real) of the tasks and baselines presented. The experiments compare against SOTA visual navigation techniques and include useful ablations that make it very easy to understand which design decisions were most relevant.

*** 
Weaknesses
---
- [Brittle Components] While the proposed method makes complete sense on paper, I am very skeptical of the {feature extraction + correspondence matching + lifting + SFM + geometric estimation} pipeline being broadly usable beyond simulated Habitat tasks and simple indoor settings. The biggest motivation for learning-based approaches to navigation is to reduce the “hard” reliance of the geometric pipelines on factors such as camera intrinsic parameters, local odometry and pose estimation, etc. While re-combining learning and geometry is generally a good idea, the specific mechanism proposed in the paper relies a bit too heavily on the brittle parts of geometric method and it seems unlikely to broadly useful.
- [Interpreting Results: Upper Bound] I had some trouble interpreting the results at different environment complexities and what the contribution of the proposed method (SLING) really is — having some sort of an oracular upper bound baseline would be really useful here. The improvements in many cases seem absurdly high, and it is not clear why such large improvements are caused by SLING (esp. for real-world experiments); the paper does not discuss this too well. Two baselines I strongly recommend adding, that shed light on different aspects of the system, can be:
	-  An oracle baseline for the exploit-only phase would be a useful addition to have (the paper currently contains Oracle-GD, not in exploit case). This could serve as an _upper bound_ to evaluate how good the detection and PointGoal performance really is.
	- An evaluation with increased tolerance (for any baseline: NRNS/OVRL) that directly gets at the hypothesis presented to motivate the two-stage process (Line 73).
	- Another important metric to include (in Table 1) would be the “distance of trajectory endpoint to goal” for the non-SLING baselines. This would really help understand the failure modes better — are the failures happening mostly due to long-horizon issues in the last mile, sooner, or other reasons.
- [Understanding Oracle-GD Improvements] I don’t follow the argument about result improvements with Oracle-GD (Line 245). The *NRNS+Oracle-GD* baseline has access to the ground-truth environment state, then why does adding SLING to this baseline lead to such a massive jump in performance? Particularly, the goal-discovery phases of the two methods are identical, and post discovery, SLING only adds information about the estimated goal location but this is already available (noise-free) to the method — why then does the performance change at all? I may be misinterpreting how this baseline works, but this needs to be clarified in the text better.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: In my review, I raised a major concern regarding the significance of the findings (owing to brittle components), and suggestions regarding presentation of experimental results. I cannot recommend acceptance of the paper in its current form, but I am very willing to engage with the authors in the discussion phase and reconsider my recommendation.

---
**Update:** After the round of discussions with the authors, my concerns have been mostly addressed, as long as the authors actually incorporate these (and other changes/additional expts requested by other reviewers) changes in the final version. I'm on the borderline between a WA/WR and cannot enthusiastically recommend acceptance, but I have no major actionable concerns with the paper after the recent discussion.
issues: - [Points Raised Above] I raised some concerns in the above section and some clarification in response to those points would really help me re-evaluate the submission score.
	- Adding new baselines for upper bound and a better metric
	- Understanding Oracle-GD baseline
review_title: null

Meta-review: metareview: Concluding comments and observations:

The authors work introduces a navigation module structure and associated switches between goal discovery and "last mile navigation" for image-based navigation once a goal has been identified as being sufficiently close/prevalent in observed image sensor data. 

The following are some of the key strengths and weaknesses in my view:

Strengths:
-  the last mile navigation phase vs exploration phase decomposition approach appears promising
-  the paper's core method is very simple to understand and implement and easy to combine with existing approaches
-  the paper and its revision provide an extensive experimental evaluation

Weaknesses:
-  the simplicity of the proposed method also means that there is little theoretical or deep methodological advance reported in this work - the focus is mostly on an apparently simple and effective approach and its empirical evaluation.

The reviewers were of similar opinions with 2 x weak accept and 1x reviewer who declared border line weak accept/weak reject in the discussion. I am recommending the work to be accepted. 

I could appreciate how this paper stays true to a "simple method" and just focuses on the empirical evaluation thereof - I could see this method being appreciated and implemented in industrial / real world systems if it can be confirmed to increase performance as indicated. From a learning theory/deeper methodology contribution point of view it is however not so clear if there is a lasting contribution there.
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)