Original: Last-Mile Embodied Visual Navigation
Revised: Last-Mile Embodied Visual Navigation

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract: Realistic long-horizon tasks like image-goal navigation involve exploratory and exploitative phases.
Revised: Abstract: Realistic long-horizon tasks like image-goal navigation involve exploratory and exploitative phases.

Original: Assigned with an image of the goal, an embodied agent must explore to discover the goal , i.e ., search efﬁciently using learnedpriors.
Revised: Assigned with an image of the goal, an embodied agent must explore to discover the goal , i.e ., search efﬁciently using learned priors.

Original: Once the goal is discovered, the agent must accurately calibrate the lastmile of navigation to the goal.
Revised: Once the goal is discovered, the agent must accurately calibrate the lastmile of navigation to the goal.

Original: As with any robust system, switches between ex- ploratory goal discovery and exploitative last-mile navigation enable better recovery from errors.
Revised: As with any robust system, switches between exploratory goal discovery and exploitative last-mile navigation enable better recovery from errors.

Original: Following these intuitive guide rails, we propose SLING toimprove performance of existing image-goal navigation systems.
Revised: Following these intuitive guide rails, we propose SLING to improve the performance of existing image-goal navigation systems.

Original: Entirely complementing prior methods, we focus on last-mile navigation, and leverage the underlying geometric structure of the problem with neural descriptors.
Revised: Entirely complementing prior methods, we focus on last-mile navigation and leverage the underlying geometric structure of the problem with neural descriptors.

Original: With simple but effective switches, we can easily connect SLING with heuristic, reinforcement learning, and neural modular policies.
Revised: With simple but effective switches, we can easily connect SLING with heuristic, reinforcement learning, and neural modular policies.

Original: On a standardized image-goal navigation benchmark [1], we improve performance across policies, scenes, and episodecomplexity, raising the state-of-the-art from 45% to 55% success rate.
Revised: On a standardized image-goal navigation benchmark [1], we improve performance across policies, scenes, and episode complexity, raising the state-of-the-art from 45% to 55% success rate.

Original: Beyondphotorealistic simulation, we conduct real-robot experiments in three physicalscenes and ﬁnd these improvements to transfer well to real environments. 
Revised: Beyond photorealistic simulation, we conduct real-robot experiments in three physical scenes and ﬁnd these improvements to transfer well to real environments. Code and results: https://jbwasse2.github.io/portfolio/

Original:  Introduction
Revised:  Introduction

Original: Imagine you are at a friend’s home
Revised: Imagine you are at a friend’s home

Original: and you want to ﬁnd the couch you have seen in your friend’sphoto.
Revised: and you want to ﬁnd the couch you have seen in your friend’s photo.

Original: At ﬁrst, you use semantic priors i e . priors about the semantic structure of the world, tonavigate to the living room (a likely place for the couch).
Revised: At ﬁrst, you use semantic priors i.e . priors about the semantic structure of the world, to navigate to the living room (a likely place for the couch).

Original: But as soon as you get the ﬁrst glimpseof the couch, you implicitly estimate the relative position of the couch, use intuitive geometry, andnavigate towards it.
Revised: But as soon as you get the ﬁrst glimpse of the couch, you implicitly estimate the relative position of the couch, use intuitive geometry and navigate towards it.

Original: We term the latter problem, of navigating to a visible object or region, as lastmile navigation.
Revised: We term the latter problem, of navigating to a visible object or region, as lastmile navigation.

Original: The ﬁeld of visual navigation hashad a rich history.
Revised: The ﬁeld of visual navigation has a rich history.

Original: Early approaches used hand-designed featureswith geometry for mapping followed by standard planning algorithms.
Revised: Early approaches used hand-designed features with geometry for mapping followed by standard planning algorithms.

Original: But such an approach failsto capture the necessary semantic priors that could be learned from data.
Revised: But such an approach fails to capture the necessary semantic priors that could be learned from data.

Original: Therefore, in recent years,we have seen more efforts and signiﬁcant advances in capturing these priors for semantic navigationtasks such as image-goal [2, 3, 4, 1, 5, 6] and object-goal navigation [7, 8, 9, 10].
Revised: Therefore, in recent years, we have seen more efforts and signiﬁcant advances in capturing these priors for semantic navigation tasks such as image-goal [2, 3, 4, 1, 5, 6] and object-goal navigation [7, 8, 9, 10].

Original: The core ideais to train a navigation policy using reinforcement or imitation learning and capture semantics.
Revised: The core idea is to train a navigation policy using reinforcement or imitation learning and capture semantics.

Original: Butin an effort to capture the semantic priors, these approaches almost entirely bypass the underlyinggeometric structure of the problem, speciﬁcally when the object or view of interest has already beendiscovered.
Revised: But in an effort to capture the semantic priors, these approaches almost entirely bypass the underlying geometric structure of the problem, speciﬁcally when the object or view of interest has already been discovered.

Original: One can argue that last mile navigation can indeed be learned from data itself.
Revised: One can argue that last-mile navigation can indeed be learned from data itself.

Original: We agree that,in principle, it can be.
Revised: We agree that, in principle, it can be.

Original: However, we argue and demonstrate that an unstructured local policy forlast-mile navigation is either (a) sample inefﬁcient (billions of frames in an RL framework [11])or (b) biased and generalize poorly when learned from ofﬂine-demonstrations (due to distributionalshift [12, 13]).
Revised: However, we argue and demonstrate that an unstructured local policy for lastmile navigation is either (a) sample inefﬁcient (billions of frames in an RL framework [11]) or (b) biased and generalize poorly when learned from ofﬂine demonstrations (due to distributional shift [12, 13]).

Original: Therefore, our solution is to revisit the basics!
Revised: Therefore, our solution is to revisit the basics!

Original: We propose S witchable L ast-Mile
Revised: We propose S witchable L ast-Mile

Original: [Figure]
Revised: [Figure]

Original: I mage-Goal N avi g ation (SLING) – a simple yet very effective geometric navigation module andassociated switches.
Revised: I mage-Goal N avi g ation (SLING) – a simple yet very effective geometric navigation system and associated switches.

Original: Our approach can be combined with any off-the-shelf learned policy that usessemantic priors to explore the scene.
Revised: Our approach can be combined with any off-the-shelf learned policy that uses semantic priors to explore the scene.

Original: As soon as the object or view of interest is detected, the
Revised: As soon as the object or view of interest is detected, the

Original: SLING switches to the geometric navigation module.
Revised: SLING switches to the geometric navigation system.

Original: We observe that SLING provides signiﬁcantperformance gains across baselines, simulation datasets, episode difﬁculty, and real-world scenes.
Revised: We observe that SLING provides signiﬁcant performance gains across baselines, simulation datasets, episode difﬁculty, and real-world scenes.

Original: Our key contributions are: (1) A general-purpose last-mile navigation module and switches, thatwe connect with ﬁve diverse goal discovery methods, leading to improvements across the board.(2) A new state-of the-art of 55.4% success i.e .
Revised: Our key contributions are: (1) A general-purpose last-mile navigation system and switches, that we connect with ﬁve diverse goal discovery methods, leading to improvements across the board. (2) A new state-of the-art of 54.8% success i.e .

Original: a huge jump of 22.4% vs .
Revised: a huge jump of 21.8% vs .

Original: published work [5] and9.8% vs .
Revised: published work [5] and 9.2% vs .

Original: a concurrent preprint [6], on the most widely-tested fold (Gibson-curved) of the AI Habitatimage-goal navigation benchmark [1]; (3) Extensive robot experiments of image-goal navigationin challenging settings with improved performance over a neural, modular policy [1] trained onreal-world data [14].
Revised: a concurrent preprint [6], on the most widely-tested fold (Gibson-curved) of the AI Habitat image-goal navigation benchmark [1]; (3) Extensive robot experiments of image-goal navigation in challenging settings with improved performance over a neural, modular policy [1] trained on real-world data [14].

Original:  Related Work
Revised:  Related Work

Original: Prior work in visual navigation and geometric 3D vision is pertinent to SLING.
Revised: Prior work in visual navigation and geometric 3D vision is pertinent to SLING.

Original: Embodied navigation.
Revised: Embodied navigation.

Original: Anderson et al .
Revised: Anderson et al .

Original: [15] formalized different goal deﬁnitions and metrics forevaluation of embodied agents.
Revised: [15] formalized different goal deﬁnitions and metrics for the evaluation of embodied agents.

Original: In point-goal navigation, relative coordinates of the goal are available (either at all steps [16, 11, 17, 18] or just at the start of an episode [9, 19, 20]).
Revised: In point-goal navigation, relative coordinates of the goal are available (either at all steps [16, 11, 17, 18, 19] or just at the start of an episode [9, 20, 21]).

Original: Successfulnavigation to a point-goal could be done without semantic scene understanding, as seen by competitive depth-only agents [16, 11].
Revised: Successful navigation to a point-goal could be done without semantic scene understanding, as seen by competitive depth-only agents [16, 11].

Original: Semantic navigation entails identifying the goal through an image(image-goal [1, 2]), acoustic cues (audio-goal [21, 22]), or a category label (object-goal [8, 9]).
Revised: Semantic navigation entails identifying the goal through an image (image-goal [1, 2, 22]), acoustic cues (audio-goal [23, 24]), or a category label (object-goal [8, 9]).

Revised: Several extensions of navigation include language-conditioned navigation following [25, 26, 27, 28], social navigation [29, 30, 31, 32, 33], and multi-agent tasks [34, 35, 36, 37, 38, 39].

Revised: However, each of these build-off single-agent navigation and beneﬁt from associated advancements.

Revised: For more embodied tasks and paradigms, we refer the reader to a recent survey [40].

Original: Inthis work, we focus on image-goal navigation in visually rich environments.
Revised: In this work, we focus on image-goal navigation in visually rich environments.

Original: Image-goal navigation.
Revised: Image-goal navigation.

Original: Chaplot et al .
Revised: Chaplot et al .

Original: [3] introduced a modular and hierarchical method for navigating to an image-goal that utilizes a topological map memory.
Revised: [3] introduced a modular and hierarchical method for navigating to an image-goal that utilizes a topological map memory.

Original: Kwon et al .
Revised: Kwon et al .

Original: [23] introduced amemory representation based on image similarity, which in turn is learned in an unsupervised fashion from unlabeled data and  agent’s observed images.
Revised: [41] introduced a memory representation based on image similarity, which in turn is learned in an unsupervised fashion from unlabeled data and the agent’s observed images.

Original: Following up on [3], NRNS [1] improvesthe topological-graph-based architecture and open-sourced a public dataset and IL and RL baselines [11, 3], within AI Habitat.
Revised: Following up on [3], NRNS [1] improves the topological-graph-based architecture and open-sourced a public dataset and IL and RL baselines [11, 3] within AI Habitat.

Original: This dataset has been adopted for standardized evaluation [5, 6].
Revised: This dataset has been adopted for standardized evaluation [5, 6].

Original: ZER [5] focuses on transferring an image-goal navigation policy to other navigation tasks.
Revised: ZER [5] focuses on transferring an image-goal navigation policy to other navigation tasks.

Original: In aconcurrent preprint, Yadav et al .
Revised: In a concurrent preprint, Yadav et al .

Original: [6] utilize self-supervised pretraining [24] to improve end-to-endvisual RL policy [11] for the image-goal navigation benchmark.
Revised: [6] utilize self-supervised pretraining [42] to improve an end-to-end visual RL policy [11] for the image-goal navigation benchmark.

Original: Our contributions are orthogonalto the above, and can be easily combined with them, as we demonstrate in Sec. 4.
Revised: Our contributions are orthogonal to the above and can be easily combined with them, as we demonstrate in Sec. 4.

Original: Last-mile navigation.
Revised: Beyond simulation, SLING ﬁnds relevance to the rich literature of navigating to an image-goal on physical robots. Meng et al . [4] utilize a neural reachability estimator and a local controller based on a Riemannian Motion Policy framework to navigate to image-goals. Hirose et al .

Revised: [43] train a deep model predictive control policy to follow a trajectory given by a sequence of images while being robust to variations in the environment.

Revised: Even in outdoor settings, meticulous studies have shown great promise, based on negative mining, graph pruning, and waypoint prediction [44] and utilizing geographic hints for kilometer-long navigations [45].

Revised: Complementing this body of work, SLING tackles image-goal navigation in challenging indoor settings, without needing any prior data of the test environment (similar to [1, 6, 3]) i.e . during evaluation no access to information (trajectories,

Revised: GPS, or top-down maps) in the test scenes is assumed.

Revised: Last-mile navigation.

Original: The works included above focus primarily on goal discovery.
Revised: The works included above focus primarily on goal discovery.

Original: In contrast,recent works have also identiﬁed ‘last-mile’ errors that occur when the goal is in sight of or closeto the agent.
Revised: In contrast, recent works have also identiﬁed ‘last-mile’ errors that occur when the goal is in sight of or close to the agent.

Original: For multi-object navigation, Wani et al .
Revised: For multi-object navigation, Wani et al .

Original: [25] observed a two-fold improvement whenallowing an error budget for the ﬁnal ‘found’ or ‘stop’ actions.
Revised: [46, 47] observed a two-fold improvement when allowing an error budget for the ﬁnal ‘found’ or ‘stop’ actions.

Original: Chattopadhyay et al .
Revised: Chattopadhyay et al .

Original: [26] foundthe last step of navigation to be brittle i.e . small perturbations lead to severe failures.
Revised: [48] found the last step of navigation to be brittle i.e . small perturbations lead to severe failures.

Original: Ye et al .
Revised: Ye et al .

Original: [10]identiﬁed last-mile errors as a prominent error mode (10% of the failures) in object-goal navigation.
Revised: [10] identiﬁed last-mile errors as a prominent error mode (10% of the failures) in object-goal navigation.

Original: However, none of these works address the problem with the last-mile of navigation.
Revised: However, none of these works address the problem with the last-mile of navigation.

Revised: From a study inspired by [46], we infer that better (or more tolerant to error) last-mile navigation can indeed lead to better performance in the image-goal navigation task (details in Appendix H).

Original: Connections to 3D vision.
Revised: Connections to 3D vision.

Original: The objective of our last-mile navigation module is to predict the relativecamera pose between two images i.e . agent’s view and image-goal.
Revised: The objective of our last-mile navigation system is to predict the relative camera pose between two images i.e . agent’s view and image-goal.

Original: To this end, pose estimationof a calibrated camera from 3D-to-2D point correspondences connects our embodied navigationtask to geometric 3D computer vision.
Revised: To this end, pose estimation of a calibrated camera from 3D-to-2D point correspondences connects our embodied navigation task to geometric 3D computer vision.

Original: The Perspective-n-Point (PnP) formulation, with extensiveresearch and efﬁcient solvers [27, 28, 29], ﬁts this use-case perfectly.
Revised: The Perspective-n-Point (PnP) formulation, with extensive research and efﬁcient solvers [49, 50, 51], ﬁts this use case perfectly.

Original: To ﬁnd an accurate PnPsolution, locating correspondences between  local features of the two images is critical.
Revised: To ﬁnd an accurate PnP solution, locating correspondences between the local features of the two images is critical.

Original: We utilize SuperGlue et al . [30] that is based on correspondences learned via attention graph neural nets andpartial assignments.
Revised: We utilize SuperGlue [52] which is based on correspondences learned via attention graph neural nets and partial assignments.

Original: We defer details of PnP and ﬁnding correspondences to Sec. 3.3, to make theapproach self-sufﬁcient.
Revised: We defer details of PnP and ﬁnding correspondences to Sec. 3.3, to make the approach self-sufﬁcient.

Original: Notably, different from related  3D vision works, we apply PnP solvers tosequential decision making in embodied settings, particularly, image-goal navigation.
Revised: Notably, different from related works in 3D vision [53, 54, 55], we apply SLING to sequential decision-making in embodied settings, particularly, image-goal navigation.

Revised: To take policies to the real world, we utilized robust SLAM methods [56, 57] for local odometry and pose estimation, which has also been found reliable by prior works in sim-to-real [58, 59, 60].

Original:  SLING
Revised:  SLING

Original: In this section, we begin with an overview of the task and the entire pipeline of SLING.
Revised: In this section, we begin with an overview of the task and the entire pipeline of SLING.

Original: We thendiscuss the implementations for goal discovery, our proposed module for last-mile navigation, andswitches to easily combine it with prior works.
Revised: We then discuss the implementations for goal discovery, our proposed system for last-mile navigation, and switches to easily combine it with prior works.

Original: While we explain key design choices in the mainpaper, a supplementary description and a list of hyperparameters, for effective reproducibility, isdeferred tothe Appendix A.
Revised: While we explain key design choices in the main paper, a supplementary description and a list of hyperparameters, for effective reproducibility, is deferred to Appendix A.

Original: 3.1 Overview
Revised: 3.1 Overview

Original: We follow the image-goal navigation task benchmark by Hahn et al .
Revised: We follow the image-goal navigation task benchmark by Hahn et al .

Original: [1] (similar to  prior formulations [2, 3]).
Revised: [1] (similar to the prior formulations [2, 3]).

Original: The agent observes an RGB image I a , a depth map D a , and the image-goal I g .
Revised: The agent observes an RGB image I a , a depth map D a , and the image-goal I g .

Original: Theagent can sample actions from A = { move forward, turn right, turn left, stop } .
Revised: The agent can sample actions from A = { move forward right left stop } .

Original: The stopaction terminates the episode.
Revised: The stop action terminates the episode.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 1a, we divide image-goal navigation into – a goal discovery and a last-mile navigation phase.
Revised: 1a, we divide image-goal navigation into – a goal discovery and a last-mile navigation phase.

Original: In the goal discovery phase, the agent is responsible for discovering the goal 
Revised: In the goal discovery phase, the agent is responsible for discovering the goal i.e .

Original: i.e .navigating close enough for the goal to occupy a large portion of the egocentric observation (‘goaldiscovered’ image).
Revised: navigating close enough for the goal to occupy a large portion of the egocentric observation (‘goal discovered’ image).

Original: Fig.
Revised: Fig.

Original: 1b shows how the control ﬂows between modules.
Revised: 1b shows how the control ﬂows between our system.

Original: If the explore ) exploitswitch isn’t triggered, learning-based exploration will continue.
Revised: If the explore (cid:41) exploit switch isn’t triggered, learning-based exploration will continue.

Original: Otherwise, if the explore ) exploitswitch triggers, the agent’s observations now overlap with the image-goal and the control ﬂowsto the last-mile navigation module.
Revised: Otherwise, if the explore (cid:41) exploit switch triggers, the agent’s observations now overlap with the image-goal and the control ﬂows to the last-mile navigation system.

Original: We ﬁnd that a one-sided ﬂow (as attempted in [1, 3]) fromexplore ) exploit is too optimistic.
Revised: We ﬁnd that a one-sided ﬂow (as attempted in [1, 3]) from explore (cid:41) exploit is too optimistic.

Original: Therefore, we introduce symmetric switches, including one thatﬂows control back to goal discovery. 
Revised: Therefore, we introduce symmetric switches, including one that ﬂows control back to goal discovery. [Figure]

Original: 3.2 Goal Discovery
Revised: 3.2 Goal Discovery

Original: We can combine our versatile last-mile navigation module and switching mechanism with any priormethod.
Revised: We can combine our versatile last-mile navigation system and switching mechanism with any prior method.

Revised: These prior methods are previously suggested solutions to image-goal navigation.

Original: We demonstrate this with ﬁve diverse goal discovery (GD) implementations.
Revised: We demonstrate this with ﬁve diverse goal discovery (GD) implementations.

Original: Straight [31].
Revised: Straight [61].

Original: A simple, heuristic exploration where the agent moves forward and unblocks itself,if stuck, by turning right (similar to an effective exploration baseline in [31]).
Revised: A simple, heuristic exploration where the agent moves forward and unblocks itself, if stuck, by turning right (similar to an effective exploration baseline in [61]).

Original: Distance Prediction Network (NRNS-GD) [1].
Revised: Distance Prediction Network (NRNS-GD) [1].

Original: Exploratory navigation is done by proposing waypoints in navigable areas (determined utilizing the agent’s depth mask), history is maintained usinga topological map, and processed using graph neural nets.
Revised: Exploratory navigation is done by proposing waypoints in navigable areas (determined utilizing the agent’s depth mask), history is maintained using a topological map, and processed using graph neural nets.

Original: The minimum cost waypoint is chosenutilizing outputs from a distance prediction network.
Revised: The minimum cost waypoint is chosen utilizing outputs from a distance prediction network.

Original: More details  inthe Appendix B and [1].[Figure]
Revised: More details are given in Appendix B and [1].

Original: Decentralized Distributed PPO (DDPPO-GD) [11].
Revised: Decentralized Distributed PPO (DDPPO-GD) [11].

Original: An implementation of PPO [32] for photorealistic simulators where rendering is the computational bottleneck.
Revised: An implementation of PPO [62] for photorealistic simulators where rendering is the computational bottleneck.

Original: This has been a standardend-to-end deep RL baseline in prior works, across tasks [18, 1, 5, 6, 33].
Revised: This has been a standard end-to-end deep RL baseline in prior works, across tasks [18, 1, 5, 6, 63].

Original: Ofﬂine Visual Representation Learning (OVRL-GD) [6].
Revised: Ofﬂine Visual Representation Learning (OVRL-GD) [6].

Original: A DDPPO network, with its visualencoder pretrained using self-supervised pretext tasks [24] on images obtained from 3D scans [34].
Revised: A DDPPO network, with its visual encoder pretrained using self-supervised pretext tasks [42] on images obtained from 3D scans [64].

Original: Environment-State Distance Prediction (Oracle-GD).
Revised: Environment-State Distance Prediction (Oracle-GD).

Original: To quantify the effect of errors comingfrom the goal discovery phase, we devise an upper bound.
Revised: To quantify the effect of errors coming from the goal discovery phase, we devise an upper bound.

Original: This is an oracle variant of NRNS-GDthat directly accesses the environment state to obtain distances  for  goal discovery phase.
Revised: This is a privileged variant of NRNS-GD that accesses the ground-truth distances from the environment, exclusively for the goal discovery phase.

Original: For detailsof its construction,  see Appendix B.
Revised: For ﬁne details of its construction, particularly, how we curtail this to be an oracle explorer and not an oracle policy, see Appendix B.

Original: 3.3 Last-Mile Navigation
Revised: 3.3 Last-Mile Navigation

Original: The proposed last-mile navigation module transforms the agent’s observations and image-goal toactions that takes the agent closer to the goal.
Revised: The proposed last-mile navigation module transforms the agent’s observations and image-goal into actions that take the agent closer to the goal.

Original: The steps are shown in Fig.
Revised: The steps are shown in Fig.

Original: 2 and detailed next.
Revised: 2 and detailed next.

Original: Neural Feature Extractor.
Revised: Neural Feature Extractor.

Original: We ﬁrst transform  agent’s RGB
Revised: We ﬁrst transform the agent’s RGB

Original: I a to local features ( ˆX a , F a ) , whereˆX a 2 R n a ⇥ 2 are the positions and F a 2 R n a ⇥ k are the visual descriptors in the agent’s image.
Revised: I a to local features ( ˆX a , F a ) , where ˆX a ∈ R n a × 2 are the positions and F a ∈ R n a × k are the visual descriptors in the agent’s image.

Original: Here, n a is the number of detected local features and k is the length of each descriptor.
Revised: Here, n a is the number of detected local features and k is the length of each descriptor.

Original: Similarly, I g leads to features ( ˆX g , F g ) , where ˆX g 2 R n g ⇥ 2 and F g 2 R n g ⇥ k with n g local features inthe image-goal.
Revised: Similarly, I g leads to features ( ˆX g , F g ) , where ˆX g ∈ R n g × 2 and F g ∈ R n g × k with n g local features in the imagegoal.

Original: Following DeTone et al .
Revised: Following DeTone et al .

Original: [35], we adopt an interest-point detector, pretrained onsynthetic data followed by cross-domain homography adaptation (here, k = 256 ).
Revised: [65], we adopt an interest-point detector, pretrained on synthetic data followed by cross-domain homography adaptation (here, k = 256 ).

Original: Matching Module.
Revised: Matching Module.

Original: From extracted features ( ˆX a , F a ) and ( ˆX g , F g ) , we predict matched subsets
Revised: From extracted features ( ˆX a , F a ) and ( ˆX g , F g ) , we predict matched subsets

Original: X a 2 R n ⇥ 2 and X g 2 R n ⇥ 2 .
Revised: X a ∈ R n × 2 and X g ∈ R n × 2 .

Original: The matching is optimized to have X a and X g correspond to the samepoint.
Revised: The matching is optimized to have X a and X g correspond to the same point.

Original: We utilize an attention-based graph neural net  that tackles partial matches and occlusions wellusing an optimal transport formulation, following Sarlin et al .
Revised: We utilize an attention-based graph neural net (GNN) that tackles partial matches and occlusions well using an optimal transport formulation, following Sarlin et al .

Revised: The above neural feature extractor and GNN-based matcher help enjoy beneﬁts of learning-based methods, particularly, those pretrained on large ofﬂine visual data without needing online, end-to-end ﬁnetuning.

Revised: The geometric components, relying on these neural features, are described next.

Original: Lifting Points from 2D ) 3D.
Revised: Lifting Points from 2D (cid:41) 3D.

Original: Next, the agent’s 2D local features are lifted to 3D with respect to theagent’s coordinate frame i.e . P a 2 R n ⇥ 3 .
Revised: Next, the agent’s 2D local features are lifted to 3D with respect to the agent’s coordinate frame i.e . P a ∈ R n × 3 .

Original: This is done by utilizing the camera intrinsic matrix K(particularly, principle point p x , p y and focal lengths f x , f y ) and the corresponding depth values foreach position in X a , say d a 2 R n .
Revised: This is done by utilizing the camera intrinsic matrix K (particularly, principle point p x , p y and focal lengths f x , f y ) and the corresponding depth values for each position in X a , say d a ∈ R n .

Original: The i th row of P a is calculated as
Revised: The i th row of P a is calculated as

Original: [Equation]
Revised: [Equation]

Original: where X a ( i, 1) and X a ( i, 2) correspond to the x and y coordinate of i th feature in X a , respectively.
Revised: where X a ( i, 1) and X a ( i, 2) correspond to the x and y coordinate of i th feature in X a , respectively.

Original: Formally, d a ( i ) :
Revised: Formally, d a ( i ) :

Original: =
Revised: =

Original: D a ( X a ( i, 1) , X a ( i, 2)) .
Revised: D a ( X a ( i, 1) , X a ( i, 2)) .

Original: Perspective-n-Point.
Revised: Perspective-n-Point.

Original: The objective of the next step i.e .
Revised: The objective of the next step i.e .

Original: Perspective-n-Point (PnP) is to ﬁnd the rotation and translation between  agent and goal camera pose that minimized reprojection error.
Revised: Perspective-n-Point (PnP) is to ﬁnd the rotation and translation between the agent and goal camera pose that minimized reprojection error.

Original: Concretely, for a given rotation matrix R 2 R 3 ⇥ 3 and translation vector t 2 R 3 , the 3D positions P a oflocal features can be reprojected from the coordinate system of the agent to that of the goal camera:[Equation] Reprojection
Revised: Concretely, for a given rotation matrix R ∈ R 3 × 3 and translation vector t ∈ R 3 , the 3D positions P a of local features can be reprojected from the coordinate system of the agent to that of the goal camera:

Original: [Equation]
Revised: [Equation]

Original: where ˜X g are the reprojected positions.
Revised: where ˜X g are the reprojected positions.

Original: Minimizing the reprojection error e , via ePnP [29] and
Revised: Minimizing the reprojection error e , via ePnP [51] and

Original: RANSAC [27] (to handles outliers), we obtain the predicted rotation and translation.
Revised: RANSAC [49] (to handle outliers), we obtain the predicted rotation and translation.

Original: The reprojection is visualized in Fig.
Revised: The reprojection is visualized in Fig.

Original: 2, where the agent’s amber point is lifted and reprojected in goal cameracoordinates.
Revised: 2, where the agent’s amber point is lifted and reprojected in goal camera coordinates.

Original: The reprojection is different from its correspondence in the goal image.
Revised: The reprojection is different from its correspondence in the goal image.

Original: Estimating Distance and Heading to Goal.
Revised: Estimating Distance and Heading to Goal.

Original: The predicted translation t can help calculate thedistance ⇢
Revised: The predicted translation t can help calculate the distance

Original:  = k t k 2 from the agent to the goal.
Revised: ρ = (cid:107) t (cid:107) 2 from the agent to the goal.

Original: Similarly, the heading  from the agent to the goalcan be obtained from the dot product of the unit vectors along the optical axis (of the agent’s view)and t .
Revised: Similarly, the heading φ from the agent to the goal can be obtained from the dot product of the unit vectors along the optical axis (of the agent’s view) and t .

Original: Concretely,  = sgn ( t [1]) ⇤ arccos ( t · o a / k t k 2 k o a k 2 ) .
Revised: Concretely, φ = sgn ( t [1]) ∗ arccos ( t · o a / (cid:107) t (cid:107) 2 (cid:107) o a (cid:107) 2 ) .

Original: The sign comes from t [1] whichpoints along the axis perpendicular to the agent’s optical axis, but parallel to the ground.
Revised: The sign comes from t [1] which points along the axis perpendicular to the agent’s optical axis but parallel to the ground.

Original: The sign isparticularly important when calculating  heading as it distinguishes between the agent turning rightor left.
Revised: The sign is particularly important when calculating the heading as it distinguishes between the agent turning right or left.

Original: Local policy.
Revised: Local policy.

Original: Finally, the distance ⇢ and heading  between the agent’s current position to theestimated goal are utilized to estimate actions in the action space
Revised: Finally, the distance ρ and heading φ between the agent’s current position to the estimated goal are utilized to estimate actions in the action space

Original: A to reach the goal.
Revised: A to reach the goal.

Original: Followingaccuracte implementations [36, 1], we adopt a local metric map to allow the agent to heuristicallyavoid obstacles and move towards the goal.
Revised: Following accurate implementations [66, 1], we adopt a local metric map to allow the agent to heuristically avoid obstacles and move towards the goal.

Original: For further details, see Appendix A.
Revised: For further details, see Appendix A.

Original: 3.4 Switches
Revised: 3.4 Switches

Original: We deﬁne simple but effective switches between the two phases of goal discovery ( explore ) and lastmile navigation ( exploit ).
Revised: We deﬁne simple but effective switches between the two phases of goal discovery ( explore ) and lastmile navigation ( exploit ).

Original: The explore ) exploit switch is triggered if the number of correspondencesn > n th , where n th is a set threshold.
Revised: The explore (cid:41) exploit switch is triggered if the number of correspondences n > n th , where n th is a set threshold.

Original: This indicates that  agent’s image has signiﬁcant overlapwith the image-goal, so control can ﬂow to the last-mile navigation phase.
Revised: This indicates that the agent’s image has signiﬁcant overlap with the image-goal, so control can ﬂow to the last-mile navigation phase.

Original: We ﬁnd that this simpleswitch performs better than training a speciﬁc deep net to achieve the same (variations attemptedin [1, 3, 4]).
Revised: We ﬁnd that this simple switch performs better than training a speciﬁc deep net to achieve the same (variations attempted in [1, 3, 4]).

Original: For exploit ) explore , if the optimization for R , t (see Eq.
Revised: For exploit (cid:41) explore , if the optimization for R , t (see Eq.

Original: (2)) fails or if the predicteddistance is greater than d th (tuned to 4m), the agent returns to  goal discovery phase.
Revised: (2)) fails or if the predicted distance is greater than d th (tuned to 4m), the agent returns to the goal discovery phase.

Original:  Experiments
Revised:  Experiments

Original: We report results for image-goal navigation both in photorealistic simulation and real-world scenes.
Revised: We report results for image-goal navigation both in photorealistic simulation and real-world scenes.

Original: 4.1 Data and Evaluation
Revised: 4.1 Data and Evaluation

Original: We evaluate image-goal navigation policies on the benchmark introduced by Hahn et al .
Revised: We evaluate image-goal navigation policies on the benchmark introduced by Hahn et al .

Original: [1] andfollow their evaluation protocol and folds.
Revised: [1] and follow their evaluation protocol and folds.

Original: The benchmark consists of numerous folds: { Gibson [37], MP3D [38] } ⇥ { straight, curved } ⇥ { easy, medium, hard } .
Revised: The benchmark consists of numerous folds: { Gibson [67], MP3D [68] }×{ straight, curved }×{ easy, medium, hard } .

Original: For a direct comparison to prior work [3,1, 5, 6] that reports primarily on ‘Gibson-curved’ fold, we follow the same in the main paper (anddefer results on more folds to the Appendix C).
Revised: For a direct comparison to prior work [3, 1, 5, 6] that reports primarily on ‘Gibson-curved’ fold, we follow the same in the main paper.

Original: Particularly for MP3D, following [5], we also show(zero-shot) cross-domain transfer of the models that are available only in Gibson.
Revised: Consistent performance trends are seen in ‘Gibson-straight’ and in the MP3D folds as well.

Original: Performanceon image-goal navigation is chieﬂy evaluated via two metrics – percentage of successful episodes( success ) and success weighted by inverse path length ( SPL ) [15].
Revised: These results are deferred to Appendix C and Appendix F. Performance on image-goal navigation is chieﬂy evaluated via two metrics – percentage of successful episodes ( success ) and success weighted by inverse path length ( SPL ) [15].

Revised: For top-performing baselines, we also include the average distance to the goal at the end of the episode in

Revised: Appendix G.

Original: The objective of the image-goalnavigation task is to execute stop within 1 m of the goal location.
Revised: The objective of the image-goal navigation task is to execute stop within 1 m of the goal location.

Original: The agent is allowed 500 steps.
Revised: The agent is allowed 500 steps.

Revised: [Table]

Original: 4.2 Methods
Revised: 4.2 Methods

Original: We compare our last-mile navigation with several standardized baselines [39, 11, 1].
Revised: We compare our last-mile navigation with several standardized baselines [69, 11, 1].

Original: Note thatﬁeld-of-view, rotation amplitude, etc . vary across baselines and we adopt the respective settings forfair comparison (speciﬁc details  areincluded in Appendix A).
Revised: Note that ﬁeld-of-view, rotation amplitude, etc . vary across baselines and we adopt the respective settings for fair comparison (implementation details of SLING are in Appendix A).

Original: Prior methods use a mix of sensorsincluding RGB, depth, and agent pose, but no dense displacement vector to the goal.
Revised: Prior methods use a mix of sensors including RGB, depth, and agent pose, but no dense displacement vector to the goal.

Original: Behavior Cloning with Spatial Memory. Applies imitation learning (IL) wherein I a and I g arerepresented with ResNet18 [40]. Moreover, using the depth map D a , the observations are represented as a spatial metric map (found to be effective across embodied AI tasks [36, 25]).
Revised: While we did include the most relevant baselines in Sec. 3.2, we also compare SLING to several other image-goal solvers.

Original:  Behavior Cloning with GRU. Another IL baseline where the observations at each time step areencoded identical to the above. However, instead of the spatial metric map, a GRU [41] is employed have been effective for semantic navigation [42, 43, 21, 44]) [Table]
Revised: This includes imitation learning baselines such as Behavior Cloning (BC) w/ Spatial Memory [69] and BC w/ Gated Recurrent Unit [69, 70].

Original: Zero Experience Replay (ZER) [5] . A recent plug-and-play RL policy trained using rewards obtained from moving closer to the goal and to [-look towards it, as well as view augmentation. Theauthors shared metrics over the folds of the benchmark [1], particularly Gibson-curved and crossdomain transfer results on MP3D, which we include in Tab. 1. 
-] DDPPO [11], NRNS [1], OVRL [6].
Revised: We also compare to [-look towards it, as well as view augmentation. Theauthors shared metrics over the folds of the benchmark [1], particularly Gibson-curved and crossdomain transfer results on MP3D, which we include in Tab. 1. 
-]established reinforcement learning baselines – DDPPO [11] and Ofﬂine Visual Representation Learning (OVRL) [6].

Original: These methods have been described as part of goal discovery,see Sec. 3.2.
Revised: OVRL also makes use of pretraining using a self-supervised objective.

Original: For DDPPO, we report results from Hahn et al . [1], trained for 100M steps (10x morecompute than NRNS).
Revised: Finally, we compare to related modular baselines include NRNS [1] and Zero Experience Replay (ZER) [5].

Original: For NRNS, we report the reproducible metrics from their ofﬁcial implementation (differs slightly from the paper [1]). At time of submission, OVRL is the best-performingmethod on Gibson-curved fold of the benchmark [1]. For OVRL, we requested their checkpoints,and re-evaluated them to report detailed metrics across easy-medium-hard folds.
Revised: We defer a detailed discussion of these baselines to Appendix B.

Original: SLING & Ablations.
Revised: SLING & Ablations.

Original: For a comprehensive empirical study, we combine SLING with Straight-GD,
Revised: For a comprehensive empirical study, we combine SLING with Straight-GD,

Original: NRNS-GD, DDPPO-GD, OVRL-GD, and Oracle-GD (see Sec.
Revised: NRNS-GD, DDPPO-GD, OVRL-GD, and Oracle-GD (see Sec.

Original: 3.2 for details).
Revised: 3.2 for details).

Revised: We also introduce a neural baseline, DDPPO-LMN, a DDPPO model trained to perform last-mile navigation.

Original: Further, we includeclear ablations to show the efﬁcacy of the components of our method and robustness to realistic poseand depth sensor noise:
Revised: Further, we include clear ablations to show the efﬁcacy of the components of our method and robustness to realistic pose and depth sensor noise:

Original: • w/ MLP switch: instead of SLING’s explore ) exploit switch (that utilizes geometric structure), if aMLP 2 detects similarity between the agent and goal images (as in [1]).• w/o Recovery: if the exploit ) explore switch is removed i
Revised: • w/ MLP switch: instead of SLING’s explore (cid:41) exploit switch (that utilizes geometric structure), if a MLP 1 detects similarity between the agent and goal images (as in [1]). • w/o Recovery: if the exploit (cid:41) explore switch is removed i.e . one-sided ﬂow of control.

Original: e . one-sided ﬂow of control.• w/o Neural Features: if the neural features [35] are replaced with traditional features [45].
Revised: • w/o Neural Features: if the neural features [65] are replaced with traditional features [71].

Original: • w/ Pose Noise: add noise to pose that emulates real-world sensors [36, 46] (same as [3, 1])
Revised: • w/ Pose Noise: add noise to pose that emulates real-world sensors [66, 72] (same as [3, 1]).

Original: • w/ Depth Noise: imperfect depth by adopting  Redwood Noisy Depth model [47] in AI Habitat.
Revised: • w/ Depth Noise: imperfect depth by adopting the Redwood Noisy Depth model [73] in AI Habitat.

Original: • w/ Oracle-GD: an upper bound, using ground-truth distances from the simulator (see Sec. 3.2).
Revised: • w/ Oracle-GD: privileged baseline where NRNS-GD can access ground-truth distances to move the agent closer to the goal during exploration (see Sec. 3.2 and Appendix B).

Revised: • w/ Oracle-LM-Pose: privileged last-mile system with perfect displacement from agent to goal • w/ Oracle-LP: privileged baseline where local policy can teleport agent to the goal prediction

Original: 4.3 Quantitative Results
Revised: 4.3 Quantitative Results

Original: In the following, we include takeaways based on the results in Tab. 1 and Tab. 2.
Revised: In the following, we include takeaways based on the results in Tab. 1 and Tab. 2.

Original: State-of-the-art performance.
Revised: State-of-the-art performance.

Original: As Tab. 1 details, SLING + OVRL-GD outperforms a suite of IL,
Revised: As Tab. 1 details, SLING + OVRL-GD outperforms a suite of IL,

Original: RL, and neural modular baselines.
Revised: RL, and neural modular baselines.

Original: The gibson-curved fold is widely adopted in prior works andhence the focus in the main paper.
Revised: The Gibson-curved fold is widely adopted in prior works and hence the focus of the main paper.

Original: With a 55.4% overall success and 37.4 SPL we are the bestperforming method on the benchmark, improving success rate by 22.4% vs .
Revised: With a 54.8% overall success and 37.3 SPL we are the bestperforming method on the benchmark, improving success rate by 21.8% vs .

Original: ZER and 9.8% vs .
Revised: ZER and 9.2% vs .

Original: OVRL (‘overall success’ column of rows 5,6, and 10).
Revised: OVRL (‘overall success’ column of rows 5, 6, & 11).

Revised: In Appendix I, we also demonstrate state-ofthe-art performance when panoramic images are used.

Original: [Table]
Revised: [Table]

Original: SLING works across methods.
Revised: SLING works across methods.

Original: Using switches, we add our last-mile navigation module to
Revised: Using switches, we add our last-mile navigation system to

Original: DDPPO [11], NRNS [1] [Table] and OVRL [6] [Table]
Revised: DDPPO [11], NRNS [1], and OVRL [6], and observe gains across the board.

Original: and observe gains across the the board.

Original: Shown in Tab. 1,
Revised: As shown in Tab. 1,

Original: SLING improves the success rate of DDPPO by 22.2%, NRNS by 21.8%, and OVRL by 9.8% (rows3 & 8, 4 & 9, 6 & 10, respectively).
Revised: SLING improves the success rate of DDPPO by 22.2%, NRNS by 21.8%, and OVRL by 9.2% (rows 3 & 9, 4 & 10, 6 & 11, respectively).

Original: Quite surprisingly, SLING even with simple straight exploration,can outperform deep IL and RL baselines (rows 1, 2,  & 3).
Revised: Quite surprisingly, SLING even with simple straight exploration, can outperform deep IL, RL, and modular baselines. (rows 1, 2, 3, 4, & 8).

Original: SLING succeeds across scene datasets Similar improvements are also seen in MP3D scenes –adding SLING to NRNS improves success by 17.1% (rows 14 & 15). In a cross-domain transfersetting, SLING improves the overall success of OVRL by 14.8% (rows 12 & 13).
Revised: SLING outperforms neural policies for last-mile navigation. SLING surpasses DDPPO trained over 400M steps for last-mile navigation by 10.5% on success rate (rows 7 & 11). SLING succeeds across scene datasets.

Original: Despite it’scross-domain setting, SLING + OVRL-GD performs best in overall SPL (18.8%) and over severalfolds of MP3D-curved.
Revised: Similar improvements are also seen in MP3D scenes – adding SLING to OVRL improves success by 5.1%. Further details and results can be found in

Original: NRNS lags behind in planning efﬁcient routes, as evidenced by lower SPLcompared to other methods (see rows 4, 9, and 14).
Revised: Appendix F.

Original: SLING is resilient to sensor noise.
Revised: SLING is resilient to sensor noise.

Original: As shown in rows 6 & 7 of Tab. 2, minor drops in performance are observed despite challenging noise in pose and depth sensors – SPL successively reduces15.1 ) 14.3 ) 14.0% (rows 2 ) 6 ) 7).
Revised: As shown in rows 6 & 7 of Tab. 2, minor drops in performance are observed despite challenging noise in pose and depth sensors – SPL successively reduces 15.1 (cid:41) 14.3 (cid:41) 14.0% (rows 2 (cid:41) 6 (cid:41) 7).

Original: Geometric switches are better.
Revised: Geometric switches are better.

Original: Performance reduces if we swap out SLING’s explore ) exploitswitch with the MLP switch of NRNS [1].
Revised: Performance reduces if we swap out SLING’s explore (cid:41) exploit switch with the MLP switch of NRNS [1].

Original: The effect is exasperated when SLING’s exploit ) exploreswitch is also removed, leading to a drop of 12% (rows 2 and 4).
Revised: The effect is exasperated when SLING’s exploit (cid:41) explore switch is also removed, leading to a drop of 12% (Tab. 2, rows 2 & 4).

Original: The neural features utilized in SLING are useful, as seen by comparing rows 2 and 5.
Revised: The neural features utilized in SLING are useful, as seen by comparing rows 2 and 5.

Original: Further, over a set of 6500 image pairs,we evaluate the accuracy of switches.
Revised: Further, over a set of 6500 image pairs, we evaluate the accuracy of switches.

Original: SLING’s explore ) exploit switch is 92.0% accurate and MLPswitch [1] is only at 82.1%.
Revised: SLING’s explore (cid:41) exploit switch is 92.0% accurate and MLP switch [1] is only at 82.1%.

Original: Also, SLING exploit ) explore switch is 84.1% accurate while NRNSdoesn’t have such a recovery switch (details of this study in Appendix D).
Revised: Also, SLING exploit (cid:41) explore switch is 84.1% accurate while NRNS doesn’t have such a recovery switch (details of this study in Appendix D).

Revised: Large potential for last-mile navigation.

Revised: When Oracle-LM-Pose and Oracle-LP are used there is a 10.2% overall improvement in success from 43.5 to 53.7% (Tab. 2, rows 2 & 10).

Revised: Notably, in easy episodes, oracle performance is an ambitious upper bound with an increase in success of 13.9% (58.7 (cid:41) 72.6%).

Revised: For the hard ( i.e . longer) episodes, the oracle components have a relatively lower impact.

Revised: This is quite intuitive as goal discovery errors are a more prominent error mode in long-horizon episodes instead of last-mile navigation.

Original: Improvements with Oracle-GD.
Revised: Improvements with Oracle-GD.

Original: Even if we assume a perfect variant of goal discovery modulefrom [1], we observe that performance saturates at 67.7% success (row 8, Tab. 2).
Revised: Even if we assume a perfect variant of goal discovery system from [1], we observe that performance saturates at 67.7% success (row 11, Tab. 2).

Original: Comparing rows8 and 9, SLING can boost this asymptotic success rate by 18.5% (67.7 ) 86.2%).
Revised: Comparing rows 11 and 12, SLING can boost this asymptotic success rate by 18.5% (67.7 (cid:41) 86.2%).

Original: Analysis: Why is SLING more robust?
Revised: Analysis: Why is SLING more robust?

Original: In Fig.
Revised: In Fig.

Original: 3a, we visualize the frequency distribution of heading (from the agent to the target) in expert demonstrations [1] (‘train GT’) and that observed at inference (‘test GT’).
Revised: 3a, we visualize the frequency distribution of heading (from the agent to the target) in expert demonstrations [1] (‘train GT’) and that observed at inference (‘test GT’).

Original: With no geometric structure, NRNS picks up the bias in training data, particularly,towards the heading of 0 (optimal trajectories entail mostly moving forward).
Revised: With no geometric structure, NRNS picks up the bias in training data, particularly, towards the heading of 0 (optimal trajectories entail mostly moving forward).

Original: Concretely, 72.2% ofthe training data is within [  15 ° , 15 ° ] .
Revised: Concretely, 72.2% of the training data is within [ − 15 ° , 15 ° ] .

Original: This drops to 39.4% at test time, when the last-milenavigation phase is reached (using the best-performing Oracle-GD). Quantiﬁed with (ﬁrst) Wasserteindistance, W Test GT , NRNS ) = 0 .
Revised: This drops to 39.4% at test time when the last-mile

Original: 0134 vs .

Original: W ( Test GT , SLING )

Original: = 0 .

Original: 0034 , demonstrating SLINGcan better match the distribution at inference.

Original: 4.4 Physical Robot Experiments

Original: We test the navigation policies on a TerraSentia [48] wheeled robot, equipped with an Intel ®

Original: RealSense ™ D435i depth camera (further hardware details in Appendix E).

Original: The robot is initialized inan unseen indoor environment and provided an RGB image-goal.

Original: We ran a total of 120 trajectories,

Original: [Figure]
Revised: [Figure]

Original: P r o b a b ili t y D e n s i t y
Revised: P r o b a b ili t y D e n s i t y

Original: F un c t i o n ( f r o m
Revised: F un c t i o n ( f r o m

Original: K D E )
Revised: K D E )

Original: [Figure]
Revised: Train GT Test GT

Original: NRNS SLING 
Revised: NRNS SLING 1m

Revised: Start

Original: [Figure]
Revised: [Figure]

Revised: Figures 5, 6, 7)

Revised: 4.4 Physical Robot Experiments

Revised: We test the navigation policies on a TerraSentia [74] wheeled robot, equipped with an Intel ®

Revised: RealSense ™ D435i depth camera (further hardware details in Appendix E).

Revised: The robot is initialized in an unseen indoor environment and provided an RGB image-goal.

Original:  requiring 30 human hours of effort, across three scenes and two levels of difﬁculty.
Revised: We ran a total oftrajectories, requiring 30 human hours of effort, across three scenes and two levels of difﬁculty.

Revised: Following the previously collected simulation dataset [1], easy goals are 1.5-3m from the starting location and hard goals are 5-10m from the starting location.

Original: Particularly, wetest within an ofﬁce and the common areas in two department buildings, over easy and hard episodes(following deﬁnitions from [1]).
Revised: Particularly, we test within an ofﬁce and the common areas in two department buildings, over easy and hard episodes (following deﬁnitions from [1]).

Original: The physical setup (ofﬁce) is shown in Fig.
Revised: The physical setup (ofﬁce) is shown in Fig.

Original: 3b.
Revised: 3b.

Original: As in simulation,the agent is successful if it executes stop action within 1m of the goal.
Revised: As in simulation, the agent is successful if it executes stop action within 1m of the goal.

Original: Examples of the image-goalutilized in physical robot experiments and precautions taken are included in Appendix E.
Revised: Examples of the image-goal utilized in physical robot experiments and precautions taken are included in Appendix E.

Original: [Figure] As shown in Tab. 3, for sim-to-real experiments, webase the goal discovery module on the NRNS model.
Revised: As shown in Tab. 3, for sim-to-real experiments, we base the goal discovery system on the NRNS model.

Original: We choose NRNS as the authors published an instantiation trained exclusively on real-world trajectories, particularly, RealEstate10K [14] (house tours videos from 
Revised: We choose NRNS as the authors published an instantiation trained exclusively on real-world trajectories, particularly, RealEstate10K [14] (house tours videos from YouTube).

Original: YouTube).
Revised: [Table]

Original: In preliminary experiments, we veriﬁed thatthis NRNS instantiation outperformed its simulation counterpart.
Revised: In preliminary experiments, we veriﬁed that this NRNS instantiation outperformed its simulation counterpart.

Original: For a direct comparison, in SLING+ NRNS-GD, we utilize the same goal discovery module but add our switching and last-mile navigation module (SLING) around it.
Revised: For a direct comparison, in SLING + NRNS-GD, we utilize the same goal discovery system but add our switching and last-mile navigation system (SLING) around it.

Original: With SLING, we improve performance from 40.0% success to56.6%.
Revised: With SLING, we improve performance from 40.0% success to 56.6%.

Original: The gains become more prominent as the task horizon increases, leading to an improvementin success rate from 3.3% to 20.0%.
Revised: The gains become more prominent as the task horizon increases, leading to an improvement in success rate from 3.3% to 20.0%.

Original: Here, we include limitations of SLING. First , we rely on correspondences i.e . mistakes in keypoint feature extraction or matching failures directly lead to errors in predicted actions (neural features [35] reduce this effect). Second , as we add structure to the last-mile navigation problem, wealso add design parameters like distance threshold d th and correspondence threshold n th .
Revised: The large gains in hard episodes (which are exploration heavy) are accounted to SLING’s better explore (cid:41) exploit switch and SLING’s last-mile navigation system that is not biased to zero heading (particularly important for curved and long episodes).

Original: The latterwe tuned depending on the size of the image (640 ⇥ 480 in NRNS and 128 ⇥ 128 in OVRL).

Original: Third ,currently in SLING we utilize only the agent’s current observation for estimating distance and heading.

Original: Using temporal smoothening could make our prediction more robust.

Original: Following the baselines proposed in the benchmark [1], we also assume access to depth and posesensors.

Original: Studies in [5] also show improvements when using depth and pose sensors.

Original: For physicalexperiments, the robot comes equipped with an inexpensive depth camera, and uses SLAM [49] forpose estimations.

Original: While we demonstrate robustness to sensor noise (Tab. 2), in future work, wecould try relaxing this assumption with a depth prediction module.

Original:  Conclusion
Revised:  Conclusion

Original: In this work, we identify and leverage the geometric structure of last-mile navigation for the challenging image-goal navigation task [1].
Revised: In this work, we identify and leverage the geometric structure of last-mile navigation for the challenging image-goal navigation task [1].

Original: With analysis of data distributions, we demonstrate thatlearning from expert demonstrations may lead to developing a bias.
Revised: With analysis of data distributions, we demonstrate that learning from expert demonstrations may lead to developing a bias.

Original: Being entirely complementaryto prior work, we demonstrate that adding SLING leads to improvements across data splits, episodecomplexity, and goal discovery policies, establishing the new state-of-the-art for image-goal navigation [1].
Revised: Being entirely complementary to prior work, we demonstrate that adding SLING leads to improvements across data splits, episode complexity, and goal discovery policies, establishing the new state-of-the-art for image-goal navigation [1].

Original: We also transfer policies trained in simulation to real-world scenes and demonstratesigniﬁcant gains in performance.
Revised: We also transfer policies trained in simulation to real-world scenes and demonstrate signiﬁcant gains in performance.

Original: Further improvements in  switching mechanism, neural keypointfeatures, visual representations from view augmentations, etc . complement our proposed approachto help improve performance in future work.
Revised: Further improvements in the switching mechanism, neural keypoint features, visual representations from view augmentations, etc . complement our proposed approach to help improve performance in future work.

Revised: Like any method, SLING has several aspects where follow-up works can improve on.

Revised: We list them explicitly: (1) Our method is limited by mistakes in matching correspondences.

Revised: (2) We add additional parameters that need to be tuned.

Revised: (3) We make a single prediction for last-mile navigation.

Revised: We assume access to depth and pose information.

Revised: More details of these aspect as well as a discussion on pose errors, depth noise, and the nuanced image-goal navigation deﬁnition in Appendix J.

Revised: Acknowledgments

Revised: We thank the reviewers for suggesting additional experiments to make the work stronger.

Revised: JW and

Revised: GC are supported by ONR MURI N00014-19-1-2373.

Revised: We are grateful to Akihiro Higuti and Mateus

Revised: Valverde for physical robot help, Dhruv Batra for helping broaden the scope, Jae Yong Lee for help with geometric vision formulation, Meera Hahn for assistance in reproducing NRNS results, and

Revised: Shubham Tulsiani for helping ground the work better to 3D vision.

Revised: A big thanks to our friends who gave feedback to improve the submission draft – Homanga Bharadhwaj, Raunaq Bhirangi,

Revised: Xiaoming Zhao and Zhenggang Tang,